Following on from this thread <https://www.mail-archive.com/therion@speleo.sk/msg04871.html> 2.5 yrs ago where I was asking about Therion support of Aven features. Therion files exported to .3d format now display 'colour by' depth, survey date, gradient and leg length when viewed with Aven. But not colour by loop error. Is 'colour by loop error' something that could readily be added to the Therion export? Bruce
22. 2. 2017 v 9:05, Bruce Mutton via Therion <therion@speleo.sk>:
Is ‘colour by loop error’ something that could readily be added to the Therion export?
Do you really need it? There is „debug on“ option in thconfig, which will produce very clear visualization of the biggest error in exported map. m.s.
I really need it! It seems the interface is and data there, just need the two to be introduced to each other. Therion debug does not show the loop errors as far as I can tell, only the single largest error, and scrap distortions. Therion.log lists all the errors, but what I need is this information in graphical form. Bruce From: Therion [mailto:therion-bounces@speleo.sk] On Behalf Of Martin Sluka via Therion Sent: Wednesday, 22 February 2017 9:45 PM To: List for Therion users <therion@speleo.sk> Cc: Martin Sluka <martinsluka@mac.com> Subject: Re: [Therion] Aven colour by loop error 22. 2. 2017 v 9:05, Bruce Mutton via Therion <therion@speleo.sk <mailto:therion@speleo.sk> >: Is ‘colour by loop error’ something that could readily be added to the Therion export? Do you really need it? There is „debug on“ option in thconfig, which will produce very clear visualization of the biggest error in exported map. m.s.
Therion strips most of the data apart from staions and legs out of 3D files it generates. Colour by error and by date are not possible because that information is not in the 3D files generated from a Therion project. Entrances are lost too. One of the reasons I plan to add Therion to Survex conversion into my cave converter program (when time permits). Footleg On Wed, Feb 22, 2017 at 6:46 PM Bruce Mutton via Therion <therion@speleo.sk> wrote:
I really need it! It seems the interface is and data there, just need the two to be introduced to each other.
Therion debug does not show the loop errors as far as I can tell, only the single largest error, and scrap distortions.
Therion.log lists all the errors, but what I need is this information in graphical form.
Bruce
*From:* Therion [mailto:therion-bounces@speleo.sk] *On Behalf Of *Martin Sluka via Therion *Sent:* Wednesday, 22 February 2017 9:45 PM *To:* List for Therion users <therion@speleo.sk> *Cc:* Martin Sluka <martinsluka@mac.com> *Subject:* Re: [Therion] Aven colour by loop error
22. 2. 2017 v 9:05, Bruce Mutton via Therion <therion@speleo.sk>:
Is ‘colour by loop error’ something that could readily be added to the Therion export?
Do you really need it? There is „debug on“ option in thconfig, which will produce very clear visualization of the biggest error in exported map.
m.s. _______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
I am able to export fixed stations and entrances with therion 5.3.15. They are nicely visible once the button is activated in the aven viewer. Am 2017-02-22 23:34, schrieb Footleg via Therion:
Therion strips most of the data apart from staions and legs out of 3D files it generates. Colour by error and by date are not possible because that information is not in the 3D files generated from a Therion project. Entrances are lost too. One of the reasons I plan to add Therion to Survex conversion into my cave converter program (when time permits).
Footleg
On Wed, Feb 22, 2017 at 6:46 PM Bruce Mutton via Therion <therion@speleo.sk [7]> wrote:
I really need it! It seems the interface is and data there, just need the two to be introduced to each other.
Therion debug does not show the loop errors as far as I can tell, only the single largest error, and scrap distortions.
Therion.log lists all the errors, but what I need is this information in graphical form.
Bruce
FROM: Therion [mailto:therion-bounces@speleo.sk [2]] ON BEHALF OF Martin Sluka via Therion SENT: Wednesday, 22 February 2017 9:45 PM TO: List for Therion users <therion@speleo.sk [3]> CC: Martin Sluka <martinsluka@mac.com [4]> SUBJECT: Re: [Therion] Aven colour by loop error
22. 2. 2017 v 9:05, Bruce Mutton via Therion <therion@speleo.sk [1]>:
Is ‘colour by loop error’ something that could readily be added to the Therion export?
Do you really need it? There is „debug on“ option in thconfig, which will produce very clear visualization of the biggest error in exported map.
m.s._______________________________________________ Therion mailing list Therion@speleo.sk [5] https://mailman.speleo.sk/listinfo/therion [6]
Links: ------ [1] mailto:therion@speleo.sk [2] mailto:therion-bounces@speleo.sk [3] mailto:therion@speleo.sk [4] mailto:martinsluka@mac.com [5] mailto:Therion@speleo.sk [6] https://mailman.speleo.sk/listinfo/therion [7] mailto:therion@speleo.sk
On Wed, Feb 22, 2017 at 09:34:03PM +0000, Footleg via Therion wrote:
Therion strips most of the data apart from staions and legs out of 3D files it generates. Colour by error and by date are not possible because that information is not in the 3D files generated from a Therion project. Entrances are lost too. One of the reasons I plan to add Therion to Survex conversion into my cave converter program (when time permits).
Therion-generated .3d files now[*] contain date and coordinate system information, thanks to a patch from Vlad: https://github.com/therion/therion/commit/43e6630e3109196d2c251f05e52c266349... [*] This isn't in a released version of therion yet, but I'm guessing there's likely to be a new release soon as it's been years since 5.3.16 and a lot of useful stuff has been merged recently. Cheers, Olly
Hi, I agree with Bruce, that it is really a very useful feature. I have just pushed it on GitHub (see commit 2ebc7ed). The only problem is, that survex calculates error which "is the ratio of the observed misclosure to the theoretical one". I have implemented more simple export (it was much easier), currently therion .3d export contains error equal to 2 * relative error of a loop. (because of 0-12 aven fixed error scale). It should be possible to extract survex relative error from either thTMPDIR/data.err or data.3d file, but it will take some time. But when I compared 3D files with relative loop error and error with survex ratio, I received two quite different pictures. Are these two measures not comparable? Is a relative error of a loop useless number? Has anybody tried to go to a cave and resurvey "red shots", whether they are really so bad? S. On 22 February 2017 at 23:20, Olly Betts via Therion <therion@speleo.sk> wrote:
On Wed, Feb 22, 2017 at 09:34:03PM +0000, Footleg via Therion wrote:
Therion strips most of the data apart from staions and legs out of 3D files it generates. Colour by error and by date are not possible because that information is not in the 3D files generated from a Therion project. Entrances are lost too. One of the reasons I plan to add Therion to Survex conversion into my cave converter program (when time permits).
Therion-generated .3d files now[*] contain date and coordinate system information, thanks to a patch from Vlad:
https://github.com/therion/therion/commit/43e6630e3109196d2c 251f05e52c2663496419d9
[*] This isn't in a released version of therion yet, but I'm guessing there's likely to be a new release soon as it's been years since 5.3.16 and a lot of useful stuff has been merged recently.
Cheers, Olly _______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
On Thu, Feb 23, 2017 at 12:08:09AM +0100, Stacho Mudrak via Therion wrote:
Hi, I agree with Bruce, that it is really a very useful feature. I have just pushed it on GitHub (see commit 2ebc7ed).
The only problem is, that survex calculates error which "is the ratio of the observed misclosure to the theoretical one". I have implemented more simple export (it was much easier), currently therion .3d export contains error equal to 2 * relative error of a loop. (because of 0-12 aven fixed error scale). It should be possible to extract survex relative error from either thTMPDIR/data.err or data.3d file, but it will take some time.
I'd definitely use the .3d file (no rounding, and a file format that's intended for machine parsing).
But when I compared 3D files with relative loop error and error with survex ratio, I received two quite different pictures. Are these two measures not comparable? Is a relative error of a loop useless number?
tl;dr: No and yes! By "relative error of a loop" I'm assuming you mean "misclosure / length of traverse * 100%". If so that's not really a useful measure, though at least historically it's been a fairly popular one, I suspect because it's an obvious one to calculate. The problem with it is that what's a "good" value depends on the length of the traverse. For random errors, the squares of the errors add, so a traverse which is 100 times as long would be expected to only have 10 times the error, which means the relative error would be expected to be 10/100 = 1/10 as much. As well as not providing a reliable measure of the random error, a blunder in a long traverse will tend to be smoothed out in the relative error, as it gets divided by the length of the traverse. Observed error divided by theoretical error should be agnostic to length of traverse. It's simply telling you which traverses don't seem to be surveyed to the accuracy that was claimed.
Has anybody tried to go to a cave and resurvey "red shots", whether they are really so bad?
I've done the armchair equivalent and used Aven's error colouring to successfully find mis-ties, reversed legs, etc. Cheers, Olly
23. 2. 2017 v 3:08, Olly Betts via Therion <therion@speleo.sk>:
I've done the armchair equivalent and used Aven's error colouring to successfully find mis-ties, reversed legs, etc.
You will find them by drawing scraps in xtherion with "debug on“ option in thconfig immediately. Typos too - like 1.23/12.3 m length. Idea of scraps and maps is the most important part of Therion. m.s.
I made one simple example file how errors in data are visualized: Sketch: .th file: survey error_visualization input nacrt.th2 centerline data normal from to tape compass clino ignoreall 1 2 11 45 -10 2 3 10.5 95 +10 no error #2 3 10.5 275 +10 backsight #2 3 1.05 95 +10 typo #2 3 10.5 65 +10 misreading 3 4 12 100 0 endcenterline endsurvey outputs without walls and with walls: On Feb 23, 2017, at 10:01 AM, Martin Sluka via Therion <therion@speleo.sk> wrote: 23. 2. 2017 v 3:08, Olly Betts via Therion <therion@speleo.sk>: I've done the armchair equivalent and used Aven's error colouring to successfully find mis-ties, reversed legs, etc. You will find them by drawing scraps in xtherion with "debug on“ option in thconfig immediately. Typos too - like 1.23/12.3 m length. Idea of scraps and maps is the most important part of Therion. m.s. _______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Martin The Therion debug feature is useful, but I would suggest that it is for micro interrogation of at most a few scraps at a time, and not for quickly identifying the macro loop error health of a project. Typos like the one you describe would not be evident from scrap distortions, at least in my case, as (in the old days of manual surveying with compass and book) the scraps would be drawn using the erroneous data, so no distortion mismatch between centreline and scrap would be flagged by Therion. Would only be found by the person drawing noticing that something looked out of proportion. I have attached a small part of a 30km project, which has at least three loops shown, and for which I had to make a number of changes (turn off passage fill colour, experiment with debug settings) especially to produce a debug pdf. The relative loop closure error does not exactly stand out. I could perhaps go to more effort and turn off drawings to leave only centreline, but my point is; With Avens ‘colour by error’ the loop information would be interactively available in an instant, for every single compile (provided you export a model.3d by default, which I do, they are fast and small, so cannot be considered a significant overhead). I’m looking forward to the next release :) By the way, is it possible to create a Therion debug with the black, yellow and orange debug features, without the ‘first’, ‘second’ and ‘scrap-names’? If so, that might ease the confusion in the image attached. Bruce -----Original Message----- From: Therion [mailto:therion-bounces@speleo.sk] On Behalf Of Martin Sluka via Therion Sent: Thursday, 23 February 2017 10:01 PM To: List for Therion users <therion@speleo.sk> Cc: Martin Sluka <martinsluka@mac.com> Subject: Re: [Therion] Aven colour by loop error
23. 2. 2017 v 3:08, Olly Betts via Therion < <mailto:therion@speleo.sk> therion@speleo.sk>:
I've done the armchair equivalent and used Aven's error colouring to
successfully find mis-ties, reversed legs, etc.
You will find them by drawing scraps in xtherion with "debug on“ option in thconfig immediately. Typos too - like 1.23/12.3 m length. Idea of scraps and maps is the most important part of Therion. m.s. _______________________________________________ Therion mailing list <mailto:Therion@speleo.sk> Therion@speleo.sk <https://mailman.speleo.sk/listinfo/therion> https://mailman.speleo.sk/listinfo/therion
On Feb 23, 2017, at 08:06 PM, Bruce Mutton via Therion <therion@speleo.sk> wrote: Martin The Therion debug feature is useful, but I would suggest that it is for micro interrogation of at most a few scraps at a time, and not for quickly identifying the macro loop error health of a project. Typos like the one you describe would not be evident from scrap distortions, at least in my case, as (in the old days of manual surveying with compass and book) the scraps would be drawn using the erroneous data, so no distortion mismatch between centreline and scrap would be flagged by Therion. Would only be found by the person drawing noticing that something looked out of proportion. It is the question of your workflow. I check each scrap individually, I check each map, each bigger map etc. I check it after each equate command (new loop closure), etc. I'm not sure if the best way is to start from huge system down to individual scraps. Martin
Martin You are right, and I do use the workflow that you describe. However if one needs a very quick and very clear macro representation of how good the loop closures are, or if you want to compare projects. Or if you inherit or need to debug someone else's project, then the methods available in Therion are tedious. Aven provides a very nice way to make these macro comparisons (once Therion can export the data). Bruce PS: I am thinking mainly of centrelines here, scraps are not so relevant. From: Therion [mailto:therion-bounces@speleo.sk] On Behalf Of Martin Sluka via Therion Sent: Friday, 24 February 2017 9:34 AM To: List for Therion users <therion@speleo.sk> Cc: Martin Sluka <martinsluka@mac.com> Subject: Re: [Therion] Aven colour by loop error On Feb 23, 2017, at 08:06 PM, Bruce Mutton via Therion <therion@speleo.sk <mailto:therion@speleo.sk> > wrote: Martin The Therion debug feature is useful, but I would suggest that it is for micro interrogation of at most a few scraps at a time, and not for quickly identifying the macro loop error health of a project. Typos like the one you describe would not be evident from scrap distortions, at least in my case, as (in the old days of manual surveying with compass and book) the scraps would be drawn using the erroneous data, so no distortion mismatch between centreline and scrap would be flagged by Therion. Would only be found by the person drawing noticing that something looked out of proportion. It is the question of your workflow. I check each scrap individually, I check each map, each bigger map etc. I check it after each equate command (new loop closure), etc. I'm not sure if the best way is to start from huge system down to individual scraps. Martin
On Thu, Feb 23, 2017 at 10:00:35AM +0100, Martin Sluka via Therion wrote:
23. 2. 2017 v 3:08, Olly Betts via Therion <therion@speleo.sk>:
I've done the armchair equivalent and used Aven's error colouring to successfully find mis-ties, reversed legs, etc.
You will find them by drawing scraps in xtherion with "debug on“ option in thconfig immediately. Typos too - like 1.23/12.3 m length. Idea of scraps and maps is the most important part of Therion.
OK, but Stacho's question was about the error measures in aven... Cheers, Olly
The new capability of exporting loop closure information to 3d format is helping to identify the bad loops on our surveys. Very nice. However the scalebar at the top right seems to be locked into a range of 0.0 to 12.0 (%) error. This is OK for our really nasty loop closures; the maps look suitably colourful, but some of our more recent cave surveys don't have error much more than 1.5%, so the picture is kind of shades of blue, as below. It doesn't help with visualising the relative quality of the loops. Would it be possible to change the default scalebar to something like 0.0 to 2.0, with the upper limit stepping upwards to suit the data, if there are larger loop errors? Bruce
On Sat, Apr 01, 2017 at 09:10:02PM +1300, Bruce Mutton via Therion wrote:
The new capability of exporting loop closure information to 3d format is helping to identify the bad loops on our surveys.
Very nice.
However the scalebar at the top right seems to be locked into a range of 0.0 to 12.0 (%) error.
Aven's error scale isn't in %, but rather in "sigmas" - i.e. it's how many times the expected error the observed error is. Assuming normal distribution of errors (which sums of random errors will tend towards) anything more than 3 is only 0.3% likely by random errors, so highly suspect: https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule If what Stacho said back in February is still true, therion outputs the wrong data in this field in the 3d file, so you actually see "2 * relative error of a loop" there, by which I think he means percentage misclosure. But as I said at the time, percentage misclosure isn't really a useful measure of loop quality because the threshold of what is reasonable varies with the length of the loop: https://mailman.speleo.sk/pipermail/therion/2017-February/006300.html
This is OK for our really nasty loop closures; the maps look suitably colourful, but some of our more recent cave surveys don't have error much more than 1.5%, so the picture is kind of shades of blue, as below. It doesn't help with visualising the relative quality of the loops.
Unless Stacho has since fixed this, you're actually seeing 0-6% there currently.
Would it be possible to change the default scalebar to something like 0.0 to 2.0, with the upper limit stepping upwards to suit the data, if there are larger loop errors?
I'd much rather we fixed therion to export the correct error information. Trying to colour based on percentage error just seems to be fundamentally a less helpful approach. Also, not varying the colours for a particular number of sigmas depending on how bad the data is was a deliberate choice. 2 sigmas is no better or worse just because someone blundered a survey elsewhere. And if data is surveyed to a lower quality the grades should be set appropriately to reflect that. Cheers, Olly
Hello, it is already fixed in the latest release and therion is parsing survex .err file for loop errors and outputs sigmas to .3d file. You may compare data.3d generated by survex (in thTMPDIR when -d is used) and therion generated .3d file, that they are equal when colored by error. S. On 3 April 2017 at 03:49, Olly Betts via Therion <therion@speleo.sk> wrote:
On Sat, Apr 01, 2017 at 09:10:02PM +1300, Bruce Mutton via Therion wrote:
The new capability of exporting loop closure information to 3d format is helping to identify the bad loops on our surveys.
Very nice.
However the scalebar at the top right seems to be locked into a range of 0.0 to 12.0 (%) error.
Aven's error scale isn't in %, but rather in "sigmas" - i.e. it's how many times the expected error the observed error is. Assuming normal distribution of errors (which sums of random errors will tend towards) anything more than 3 is only 0.3% likely by random errors, so highly suspect:
https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule
If what Stacho said back in February is still true, therion outputs the wrong data in this field in the 3d file, so you actually see "2 * relative error of a loop" there, by which I think he means percentage misclosure.
But as I said at the time, percentage misclosure isn't really a useful measure of loop quality because the threshold of what is reasonable varies with the length of the loop:
https://mailman.speleo.sk/pipermail/therion/2017-February/006300.html
This is OK for our really nasty loop closures; the maps look suitably colourful, but some of our more recent cave surveys don't have error much more than 1.5%, so the picture is kind of shades of blue, as below. It doesn't help with visualising the relative quality of the loops.
Unless Stacho has since fixed this, you're actually seeing 0-6% there currently.
Would it be possible to change the default scalebar to something like 0.0 to 2.0, with the upper limit stepping upwards to suit the data, if there are larger loop errors?
I'd much rather we fixed therion to export the correct error information. Trying to colour based on percentage error just seems to be fundamentally a less helpful approach.
Also, not varying the colours for a particular number of sigmas depending on how bad the data is was a deliberate choice. 2 sigmas is no better or worse just because someone blundered a survey elsewhere. And if data is surveyed to a lower quality the grades should be set appropriately to reflect that.
Cheers, Olly _______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
Nice, thank you all for all the hard work that has gone into the new 5.4.0 It works! Sadly many of my projects still have bright red 12 standard deviation loop errors! Perhaps time to lower the survey grade claims made in the .th files! Bruce From: Therion [mailto:therion-bounces@speleo.sk] On Behalf Of Stacho Mudrak via Therion Sent: Wednesday, 5 April 2017 11:39 PM To: List for Therion users <therion@speleo.sk> Cc: Stacho Mudrak <lists@group-s.sk> Subject: Re: [Therion] Aven colour by loop error Hello, it is already fixed in the latest release and therion is parsing survex .err file for loop errors and outputs sigmas to .3d file. You may compare data.3d generated by survex (in thTMPDIR when -d is used) and therion generated .3d file, that they are equal when colored by error. S.
participants (6)
-
Benedikt Hallinger -
Bruce Mutton -
Footleg -
Martin Sluka -
Olly Betts -
Stacho Mudrak