Handling of tape and backtape in survey data
Hello everyone - I have a holiday question for the group. We have for years done our survey shots both directions, obtaining a forward and backward reading for both compass and inclinometer. The two readings are averaged by Survex and Therion in an effort to reduce errors. The tape reading, however, has only been a single number. Now that the use of DistoX2 devices is becoming common, it is easy to get a distance measurement both forward and backward. I have been ignoring the backward distance reading in the field. It does not get into my notes. I might need to change that practice. I looked at the Survex manual yesterday and see that it supports both TAPE and BACKTAPE (also called LENGTH and BACKLENGTH) in the survey data. That brings up two questions: 1) Does Therion also recognize the BACKTAPE data type? 2) What does Survex do if both TAPE and BACKTAPE are given? Does it average the two readings? Does Therion do the same thing? I understand that Therion will use Survex to reduce the centerline data - if Survex is installed. In case Survex is NOT installed, then Therion reduces the centerline itself. As a result they might handle this situation differently. As I write this, another related question occurred to me. When either Therion or Survex averages a forward and backward reading, how many significant digits does it carry in the calculation? Can the significant digits be changed? And the same question applies to loop closure calculations. How many significant digits are carried through the calculations? Thanks! ======== Bill Gee
Hi Bill, et al, I’ve always thought that backsites didn’t need specific handling as effectively they form a small loop so would be handled by the loop averaging. I don’t often do backsites, and get reasonable accuracy without (as good as 0.5%, depending on conditions). But when I do I always do a full reading, indeed it would be hard not to with a DistoX-PDA “total station” as all the data is collected automatically. In fact for this setup, there is no way to mark a backsite as a backsite (or if there is, I’ve never used it!) and instead Toporobot just generates a new station for the backsite. I then manually “equate” these in the .th file, creating a loop, together with any actual loops. I usually do this after compiling the survey for the first time, as it’s fun to see on the 3d how close any two measured stations appear, and hence get a visual feel for the accuracy and where potential blunders might (eg mislabelling a station) But now I’m wondering how Survex handles all these loops because if every leg has a backsite, then the number of loops increases, crumbs, very quickly. Hmm, I can't even begin to work out how to count that mathematically. It’s very much smaller than nCr because the loops can only be made up of adjacent legs, eg for 5 legs, the loops generated between any three are only 1-2-3, 2-3-4, and 3-4-5; nCr also counts 1-2-4, 1-2-5, etc, which are not loops. So while nCr quickly becomes uncomputable for even modest sizes of n, I don’t know if the number of loops also becomes uncomputable? What does Survex actually do for all these loops? Best regards Ben Sent from my iPhone
On 25 Dec 2021, at 14:51, Bill Gee <bgee@campercaver.net> wrote:
Hello everyone -
I have a holiday question for the group. We have for years done our survey shots both directions, obtaining a forward and backward reading for both compass and inclinometer. The two readings are averaged by Survex and Therion in an effort to reduce errors. The tape reading, however, has only been a single number.
Now that the use of DistoX2 devices is becoming common, it is easy to get a distance measurement both forward and backward. I have been ignoring the backward distance reading in the field. It does not get into my notes. I might need to change that practice.
I looked at the Survex manual yesterday and see that it supports both TAPE and BACKTAPE (also called LENGTH and BACKLENGTH) in the survey data. That brings up two questions:
1) Does Therion also recognize the BACKTAPE data type?
2) What does Survex do if both TAPE and BACKTAPE are given? Does it average the two readings? Does Therion do the same thing?
I understand that Therion will use Survex to reduce the centerline data - if Survex is installed. In case Survex is NOT installed, then Therion reduces the centerline itself. As a result they might handle this situation differently.
As I write this, another related question occurred to me. When either Therion or Survex averages a forward and backward reading, how many significant digits does it carry in the calculation? Can the significant digits be changed?
And the same question applies to loop closure calculations. How many significant digits are carried through the calculations?
Thanks! ======== Bill Gee
_______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
when I do I always do a full reading
A splay is useful only for immediate error checking on the data gathering device (magnetic or calibration), but cannot be used for automatic error correction unless the app can do it for you (like SexyTopo can, but only for opposing splays, not for opposing legs). So there is a benefit to having a full leg there. However, for Disto surveying, it feels largely unnecessary, as the accuracy is already a lot more than most surveys need when just using forward sightings.
instead Toporobot just generates a new station for the backsite.
This must be a nightmare for mistakes, always having to tell the app which station the next readings are going to be taken from, before you continue surveying.
I then manually “equate” these in the .th file, creating a loop
While nice for error handling, this must make your cave lengths completely meaningless; typically twice the correct length (if you consistently do it throughout a survey). Or do you manually set the "flags duplicate" for every backsight, then back to "flags not duplicate" for the forward sighting? Do Survex and Therion know to use only a single length if there is a forward and backsight leg between the same two stations? I sense an investigation coming soon unless Wookey appears with some knowledge gem... Cheers, Tarquin
26. 12. 2021 v 19:32, Tarquin Wilton-Jones via Therion <therion@speleo.sk>:
However, for Disto surveying, it feels largely unnecessary, as the accuracy is already a lot more than most surveys need when just using forward sightings.
But because DistoX is an electronic device it is only way to be sure it works properly. The same as check, if three (four) readings for surveying leg with rotation around the longitudinal axe of DistoX are in tolerance interval. Martin S.
Ben's comment about backshots being a form of loop is interesting. I had not though of it that way, but he is right! I do not use electronic sketching in a cave. All the data from the DistoX2 goes on paper just like Suunto/tape data. I have looked at Qave and TopoDroid, and for now I think they are not quite ready for prime time. There are people out there who swear by electronic sketching. I am just an old curmudgeon! But not quite old enough to go back to Bruntons. I am also not convinced that a DistoX is more accurate than Suunto/tape methods. It is certainly easier to use, especially for newbies. It is much less bulky since the tape reel can be left behind, and it really helps with LRUD measurements. But -- Do not assume that "electronic" is the same as "accurate". In a recent update to one of my project caves, I noticed the loop closure errors from Survex were impressive. Most were under 2%, and several under 1%. Even better - those loops used readings taken with both Suunto/tape and DistoX2 by several teams over a period of three years. I think the critical factor is getting good back shots to verify the data in the field, and then having the computer average the forward and back shots. I remember seeing a presentation at a recent NSS Convention (2020, maybe?) where someone took an uncalibrated DistoX2 and ran it around a survey course on the surface. By averaging the forward and backward shots he got loop closures around 1% over 400 feet and a dozen stations. The data using Suunto/tape and a calibrated DistoX2 was not significantly different in the loop closure. ======== Bill Gee On Monday, December 27, 2021 1:15:27 PM CST Martin Sluka via Therion wrote:
26. 12. 2021 v 19:32, Tarquin Wilton-Jones via Therion <therion@speleo.sk>:
However, for Disto surveying, it feels largely unnecessary, as the accuracy is already a lot more than most surveys need when just using forward sightings.
But because DistoX is an electronic device it is only way to be sure it works properly. The same as check, if three (four) readings for surveying leg with rotation around the longitudinal axe of DistoX are in tolerance interval.
Martin S.
On 27/12/2021 19:15, Martin Sluka via Therion wrote:
26. 12. 2021 v 19:32, Tarquin Wilton-Jones via Therion <therion@speleo.sk <mailto:therion@speleo.sk>>:
However, for Disto surveying, it feels largely unnecessary, as the accuracy is already a lot more than most surveys need when just using forward sightings.
But because DistoX is an electronic device it is only way to be sure it works properly. The same as check, if three (four) readings for surveying leg with rotation around the longitudinal axe of DistoX are in tolerance interval.
Oh very true. This is one of the checks I recommend performing occasionally to test calibration (or every leg when working around scaffolding or magnetic rocks). Create legs with one reading flat, one rotated 90 degrees longitudinally, another reading flat or in any other orientation. Then validate with a splay back to the previous station. But personally I consider the leg forward, then duplicate backsighted *leg* for every single leg, to be largely overkill. There are already 3 readings for every leg, and a rotational test (with optional backsight splay) already serves the purpose of testing calibration and magnetic effects. And standard leapfrogging can already counteract clinometer calibration faults. But I am sure someone will have their reason to make each leg from 6 readings. :)
On 27/12/2021 19:41, Tarquin Wilton-Jones via Therion wrote:
But because DistoX is an electronic device it is only way to be sure it works properly. The same as check, if three (four) readings for surveying leg with rotation around the longitudinal axe of DistoX are in tolerance interval. Oh very true. This is one of the checks I recommend performing occasionally to test calibration (or every leg when working around scaffolding or magnetic rocks). Create legs with one reading flat, one rotated 90 degrees longitudinally, another reading flat or in any other orientation. Then validate with a splay back to the previous station.
I think, but have no proof, that if you rotate the distoX it is probably better than a manual backsight. pockettopo records the rotation, and I have this in the therion file. People surveying with me know I check that they rotate at least every 5 stations and i relity they rotate most station. All the times I've been with people doing backsights (mainly US cavers) all it seems to stop is blunders, if they don't get a reading that is within their personal tolerance they do it all again. The real question is how much are they doing expectation bias? I cannot really comment further as I have too little experience of this. On 27/12/2021 19:36, Bill Gee wrote:
I remember seeing a presentation at a recent NSS Convention (2020, maybe?) where someone took an uncalibrated DistoX2 and ran it around a survey course on the surface. By averaging the forward and backward shots he got loop closures around 1% over 400 feet and a dozen stations. The data using Suunto/tape and a calibrated DistoX2 was not significantly different in the loop closure.
A compass that is 10° (well any degrees off will give a 0% loop error, the longer the loop the lower the percentage error, however bad the survey. (bit of over simplification, but relatively true) High percent loop closure is a poor survey, the opposite is not true. This could turn into a long interesting debate, please feel free to argue with me, let's see how robust our ideas are :-) Andrew
instead Toporobot just generates a new station for the backsite.
PocketTopo automatically detects backsights if they are taken immediately after a foresight. So the software handles the station naming, no user intervention required. While I processed the only survey I ever did this way with Therion, it never made it into the main cave dataset and so I didn't get as far as checking the survey length was correct (or doubled). I'm pretty sure it would be correct though. It was only done to find a reasonably precise gap in an 'almost looped' passage. The gap was too big to dig so we lost interest and I haven't drawn the enhanced accuracy map yet. It is at the bottom of my priority list. The exercise was also helpful to gauge the frequency and size of foresight/backsight mismatch with 'three-shot' disto readings. My conclusion: a few instances that may be of interest occur, but not enough to ever bother actually taking all those backsights. Have never done a backsight since. Our disto driven loop closures are typically 0.5% (I like to think) but a bad one will be 1.0%. Compared to our pre-disto loops over a decade ago which were typically 2% and a good one 1%. Although I prefer to think in Survex's standard deviation approach to loop closure, which tells a different story about which loop is good and which not... Bruce
Thanks Bill for pointing that out. BACKTAPE/BACKLENGTH reading support was added to therion in the latest commit. S. On Thu, 30 Dec 2021 at 09:44, Bruce Mutton <bruce@tomo.co.nz> wrote:
instead Toporobot just generates a new station for the backsite.
PocketTopo automatically detects backsights if they are taken immediately after a foresight. So the software handles the station naming, no user intervention required. While I processed the only survey I ever did this way with Therion, it never made it into the main cave dataset and so I didn't get as far as checking the survey length was correct (or doubled). I'm pretty sure it would be correct though. It was only done to find a reasonably precise gap in an 'almost looped' passage. The gap was too big to dig so we lost interest and I haven't drawn the enhanced accuracy map yet. It is at the bottom of my priority list. The exercise was also helpful to gauge the frequency and size of foresight/backsight mismatch with 'three-shot' disto readings. My conclusion: a few instances that may be of interest occur, but not enough to ever bother actually taking all those backsights. Have never done a backsight since. Our disto driven loop closures are typically 0.5% (I like to think) but a bad one will be 1.0%. Compared to our pre-disto loops over a decade ago which were typically 2% and a good one 1%. Although I prefer to think in Survex's standard deviation approach to loop closure, which tells a different story about which loop is good and which not... Bruce
_______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
I just checked on my little experiment of some years ago, and I misrepresented it.
PocketTopo automatically detects backsights if they are taken immediately after a foresight. So the software handles the station naming, no user intervention required. That much is true. You end up with something like: 1.4 splays 1.4 1.5 shot 1.4 1.5 shot 1.4 1.5 shot
1.5 1.4 shot # for back shots it will recognise 1 or more shots within tolerance as 'back legs', so you can do as many as you want, I think 1.5 1.4 shot 1.5 1.4 shot 1.5 splays 1.5 1.6 shot etc
While I processed the only survey I ever did this way with Therion... Not true. I did not bother parsing the data from PocketTopo, as the passage ends were too far apart to consider digging, and so no point to mapping with Therion (we already have an older map).
Never the less, I am almost curious to see what TopParser and Therion does with it. Lots of loops and doubled survey length I expect. Maybe I will process it one day. Tarquin You mention that SexyTopo deals with this by treating the backsight as a splay. Does it detect and label this splay as a backleg if it is within tolerance for the previous leg? I think it should (similarly to PocketTopo). While I don't tend to use backsights as a rule, there is that 1 in 100 occasion where I do consider it, and I expect that the field software should automatically detect this and number the shot/leg accordingly. In a generic sense, rigorous foresight/<move to next station>/backsight/splay is a valid survey protocol (along with variations of that). Bruce
On Sun, Dec 26, 2021 at 08:00:49AM +0000, Ben Cooper wrote:
What does Survex actually do for all these loops?
It deal with any backsights up front and just adds a single link to the network for each survey leg. It's harder to properly account for partial backsights if we created a separate link for the backsight (e.g. backcompass and backclino but no backtape is common). But even just from a network solving perspective, if we generated a trivial loop for each backsight the network optimisations would later rediscover each trivial loop and collapse it to a single leg, but that would be less efficient than collapsing it when we have the knowledge it's a trivial loop as we read in that leg. Cheers, Olly
On Sat, Dec 25, 2021 at 08:50:26AM -0600, Bill Gee wrote:
1) Does Therion also recognize the BACKTAPE data type?
It seems not - there's no match in the source code repo for: git grep -i BACKTAPE
2) What does Survex do if both TAPE and BACKTAPE are given? Does it average the two readings? Does Therion do the same thing?
Survex warns if they differ by more than 3 standard deviations, and then takes the mean of the readings.
I understand that Therion will use Survex to reduce the centerline data - if Survex is installed. In case Survex is NOT installed, then Therion reduces the centerline itself. As a result they might handle this situation differently.
Also, Therion generates a .svx file behind the scenes, but not necessarily the .svx file you'd write yourself given the same data. For example, I think Therion collapses backsights to a single reading itself before processing with Survex.
As I write this, another related question occurred to me. When either Therion or Survex averages a forward and backward reading, how many significant digits does it carry in the calculation? Can the significant digits be changed?
Survex uses double precision for this: https://en.wikipedia.org/wiki/Double-precision_floating-point_format That has 15-17 significant decimal digits - i.e. many more than matter here. Using more precision seems overkill. If you really want to use fewer it should be possible to build cavern to use single precision, but I don't think anyone actually does this. We made it an option as decades ago it seemed you might want to do this to reduce memory usage (4 bytes per floating point number instead of 8) but cavern's memory use is very modest by modern standards, and single precision only gives 6-9 significant decimal digits, which isn't even enough to reliably store a UTM Northing to the nearest metre so you'd be limited in what coordinate systems you could use in such a build. I don't see why you'd want to be able to dynamically force rounding to fewer significant figures - what's the benefit? Downsides of rounding are it would slow things down a bit, and it risks introducing systematic biases - a dumb example to make the issue clearer: if all readings are to the nearest inch and you round the average to the nearest inch and always round 0.5 up, then then assuming an even distribution there's an average bias of +0.25 inches on each tape reading. The problem with systematic biases is they accumulate rather than tending to cancel. There are other rounding schemes which try to address this sort of issue (like rounding 0.5 to the nearest even number) but it's hard to be certain they might not introduce a more subtle bias (some cave systems have dominant direction passage develops along so the distribution is may not uniform across odd and even) and there doesn't seem a compelling reason to be rounding in the first place.
And the same question applies to loop closure calculations. How many significant digits are carried through the calculations?
The same. The final results are then stored in the .3d file to the nearest cm (that's 0.000049710 furlongs for our US readers). That precision was chosen so that coordinate values will fit in a 32 bit integer, and seems adequate for the final results. Cheers, Olly
Hi Olly - Many thanks for the answers. I think I will give BACKTAPE a try the next time I do some survey, if only to see what happens. My question about significant digits is really related to how much precision can and should be carried through a calculation. If compass readings are taken to four significant digits, then the mean of two such numbers is really only good to about three and a half significant digits. Displaying more significant digits implies more precision that actually exists. A compass reading even from a DistoX2 is probably accurate to only +/- 2 least significant digits. https://en.wikipedia.org/wiki/Significant_figures[1] However, you have an excellent point about easting and northing data. Those number can be measure accurately to enough significant digits that a double-precision data type is needed. Speed is not an issue. Survex and Therion both compile my biggest maps in seconds. Shaving a few seconds is of little benefit. And to your last comment about furlongs ... LOL! I remember a huge argument some years ago about what the speed of light is in furlongs per fortnight. It was a hoot! Thanks! ======== Bill Gee On Monday, December 27, 2021 3:56:04 PM CST Olly Betts wrote:
On Sat, Dec 25, 2021 at 08:50:26AM -0600, Bill Gee wrote:
1) Does Therion also recognize the BACKTAPE data type?
It seems not - there's no match in the source code repo for:
git grep -i BACKTAPE
2) What does Survex do if both TAPE and BACKTAPE are given? Does it average the two readings? Does Therion do the same thing?
Survex warns if they differ by more than 3 standard deviations, and then takes the mean of the readings.
I understand that Therion will use Survex to reduce the centerline data - if Survex is installed. In case Survex is NOT installed, then Therion reduces the centerline itself. As a result they might handle this situation differently.
Also, Therion generates a .svx file behind the scenes, but not necessarily the .svx file you'd write yourself given the same data. For example, I think Therion collapses backsights to a single reading itself before processing with Survex.
As I write this, another related question occurred to me. When either Therion or Survex averages a forward and backward reading, how many significant digits does it carry in the calculation? Can the significant digits be changed?
Survex uses double precision for this:
https://en.wikipedia.org/wiki/Double-precision_floating-point_format
That has 15-17 significant decimal digits - i.e. many more than matter here.
Using more precision seems overkill.
If you really want to use fewer it should be possible to build cavern to use single precision, but I don't think anyone actually does this. We made it an option as decades ago it seemed you might want to do this to reduce memory usage (4 bytes per floating point number instead of 8) but cavern's memory use is very modest by modern standards, and single precision only gives 6-9 significant decimal digits, which isn't even enough to reliably store a UTM Northing to the nearest metre so you'd be limited in what coordinate systems you could use in such a build.
I don't see why you'd want to be able to dynamically force rounding to fewer significant figures - what's the benefit?
Downsides of rounding are it would slow things down a bit, and it risks introducing systematic biases - a dumb example to make the issue clearer: if all readings are to the nearest inch and you round the average to the nearest inch and always round 0.5 up, then then assuming an even distribution there's an average bias of +0.25 inches on each tape reading. The problem with systematic biases is they accumulate rather than tending to cancel.
There are other rounding schemes which try to address this sort of issue (like rounding 0.5 to the nearest even number) but it's hard to be certain they might not introduce a more subtle bias (some cave systems have dominant direction passage develops along so the distribution is may not uniform across odd and even) and there doesn't seem a compelling reason to be rounding in the first place.
And the same question applies to loop closure calculations. How many significant digits are carried through the calculations?
The same.
The final results are then stored in the .3d file to the nearest cm (that's 0.000049710 furlongs for our US readers). That precision was chosen so that coordinate values will fit in a 32 bit integer, and seems adequate for the final results.
Cheers, Olly
-------- [1] https://en.wikipedia.org/wiki/Significant_figures
On Mon, Dec 27, 2021 at 05:04:20PM -0600, Bill Gee wrote:
My question about significant digits is really related to how much precision can and should be carried through a calculation. If compass readings are taken to four significant digits, then the mean of two such numbers is really only good to about three and a half significant digits.
That seems wrong to me - we expect the mean to have *better* accuracy than the individual readings (otherwise one would want to avoid averaging readings!) Assuming independence (which may not an entirely valid assumption here admittedly) and that both readings have the same standard deviation x, then the s.d. of the mean of 2 readings is x/sqrt(2) - or about 0.707*x. I think that means you actually would expect to gain log(1/sqrt(2))/log(1/10) or about 0.15 decimal significant digits from taking the mean of two readings. (And it's 1/sqrt(n) for n readings, so if you average 100 independent readings then you'd expect to gain a whole decimal significant digit.)
Displaying more significant digits implies more precision that actually exists.
Right, but that argues for suitably presenting reported values, not for rounding intermediate values during the calculations.
A compass reading even from a DistoX2 is probably accurate to only +/- 2 least significant digits.
I'm not quite sure what "2 least significant digits" means, and that wikipedia page doesn't seem to help. But I think I get the gist, and indeed any instrument will have a limit to its accuracy, especially so for anything you'd be likely to consider taking underground. Cheers, Olly
participants (8)
-
Andrew Atkinson -
Ben Cooper -
Bill Gee -
Bruce Mutton -
Martin Sluka -
Olly Betts -
Stacho Mudrak -
Tarquin Wilton-Jones