Wrong average calculation from compass/backcompass data
Hi, I have data from a survey with fore/back sight compass and clino. I have detected a Therion bug calculating the average data, comparing with the error loops with Auriga, Visualtopo, Compass and Therion. See the next compare. Example from the data in .th file:
data normal from to length compass backcompass clino backclino left right up down 279.0 - 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 279.0 279.0A 2.33 236.8 56.8 -11.4 11.4 0.00 0.00 0.00 0.00 # .superficie 279.0A 279.1 4.51 163.2 343.2 -32.2 32.2 0.00 0.00 0.00 0.00 # .superficie hasta anclaje desviador 279.1 279.2 12.41 1.5 181.5 -77.7 77.7 0.00 0.00 0.00 0.00 279.2 279.3 1.95 212.2 33.1 -17.7 16.6 0.00 0.00 0.00 0.00
Data transferred to Survex by the "data.svx" file
*data normal from to tape compass clino 1 2 0.000000 268.348831 0.000000 1 3 2.330000 235.148831 -11.400000 3 4 4.510000 161.548831 -32.200000 4 5 12.410000 359.848831 -77.700000 5 6 1.950000 210.998831 -17.150000
You can see Therion do a wrong fore/back sight compass average calculation. The good data average are: 1 3 236.8 -11.4 3 4 163.2 -32 4 5 1.5 -77.7 5 6 212.65 -17.15 The clino average seems OK. Searching in the mail-list I have found the bug was reported before. I think is not corrected since then. https://www.mail-archive.com/therion@speleo.sk/msg04660.html Regards, Evaristo.
Hi, I found that the error had been fixed in version 5.4.1
Therion 5.4 (5.4.1 now 2017-04-18)
* bugs and issues fixed: 27. invalid compass/backcompass averaging formula
But I am using the "Therion 5.4.1+6884381 (2018-04-12)", were I have found the actual bug. Evaristo. El 16/05/2018 a las 14:17, Evaristo Quiroga via Therion escribió:
Hi,
I have data from a survey with fore/back sight compass and clino. I have detected a Therion bug calculating the average data, comparing with the error loops with Auriga, Visualtopo, Compass and Therion.
See the next compare.
Example from the data in .th file:
data normal from to length compass backcompass clino backclino left right up down 279.0 - 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 279.0 279.0A 2.33 236.8 56.8 -11.4 11.4 0.00 0.00 0.00 0.00 # .superficie 279.0A 279.1 4.51 163.2 343.2 -32.2 32.2 0.00 0.00 0.00 0.00 # .superficie hasta anclaje desviador 279.1 279.2 12.41 1.5 181.5 -77.7 77.7 0.00 0.00 0.00 0.00 279.2 279.3 1.95 212.2 33.1 -17.7 16.6 0.00 0.00 0.00 0.00
Data transferred to Survex by the "data.svx" file
*data normal from to tape compass clino 1 2 0.000000 268.348831 0.000000 1 3 2.330000 235.148831 -11.400000 3 4 4.510000 161.548831 -32.200000 4 5 12.410000 359.848831 -77.700000 5 6 1.950000 210.998831 -17.150000
You can see Therion do a wrong fore/back sight compass average calculation. The good data average are:
1 3 236.8 -11.4
3 4 163.2 -32
4 5 1.5 -77.7
5 6 212.65 -17.15
The clino average seems OK.
Searching in the mail-list I have found the bug was reported before. I think is not corrected since then.
https://www.mail-archive.com/therion@speleo.sk/msg04660.html
Regards,
Evaristo.
_______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
I think Therion subtract the magnetic declination before to transfer data Survex. Now to compare better I have put magnetic declination to 0.
declination 0.00 degrees
Data transferred to Survex by the "data.svx" file
*data normal from to tape compass clino 1 2 0.000000 269.423853 0.000000 1 3 2.330000 236.223853 -11.400000 3 4 4.510000 162.623853 -32.200000 4 5 12.410000 0.923853 -77.700000 5 6 1.950000 212.073853 -17.150000
You can see Therion do a wrong fore/back sight compass average calculation. The good data average are:
1 3 236.8 -11.4
3 4 163.2 -32
4 5 1.5 -77.7
5 6 212.65 -17.15
I think the problem is the average calculation formula in "thdb1d.cxx"
// check backwards compass reading if ((lei->data_type == TT_DATATYPE_NORMAL) || (lei->data_type == TT_DATATYPE_DIVING) || (lei->data_type == TT_DATATYPE_CYLPOLAR)) { if (!thisnan(lei->backbearing)) { if (thisnan(lei->bearing)) { lei->backbearing -= 180.0; if (lei->backbearing < 0) lei->backbearing += 360.0; lei->bearing = lei->backbearing; } else { lei->backbearing -= 180.0; if (lei->backbearing < 0) lei->backbearing += 360.0; // calculate average of two angles //lei->bearing += lei->backbearing; //lei->bearing = lei->bearing / 2.0; double sumx, sumy; sumx = cos((90.0 - lei->bearing)/180.0*THPI) + cos((90.0 - lei->backbearing)/180.0*THPI); sumy = sin((90.0 - lei->bearing)/180.0*THPI) + sin((90.0 - lei->backbearing)/180.0*THPI); lei->bearing = 90.0 - (atan2(sumy, sumx) / THPI * 180.0); if (lei->bearing < 0.0) lei->bearing += 360.0; } } }
I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2 Regards, Evaristo.
On Wed, May 16, 2018 at 04:45:54PM +0200, Evaristo Quiroga via Therion wrote:
I think the problem is the average calculation formula in "thdb1d.cxx"
// check backwards compass reading if ((lei->data_type == TT_DATATYPE_NORMAL) || (lei->data_type == TT_DATATYPE_DIVING) || (lei->data_type == TT_DATATYPE_CYLPOLAR)) { if (!thisnan(lei->backbearing)) { if (thisnan(lei->bearing)) { lei->backbearing -= 180.0; if (lei->backbearing < 0) lei->backbearing += 360.0; lei->bearing = lei->backbearing; } else { lei->backbearing -= 180.0; if (lei->backbearing < 0) lei->backbearing += 360.0; // calculate average of two angles //lei->bearing += lei->backbearing; //lei->bearing = lei->bearing / 2.0; double sumx, sumy; sumx = cos((90.0 - lei->bearing)/180.0*THPI) + cos((90.0 - lei->backbearing)/180.0*THPI); sumy = sin((90.0 - lei->bearing)/180.0*THPI) + sin((90.0 - lei->backbearing)/180.0*THPI); lei->bearing = 90.0 - (atan2(sumy, sumx) / THPI * 180.0); if (lei->bearing < 0.0) lei->bearing += 360.0; } } }
I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2
Your proposed formula gives wrong answers in some cases - consider: bearing = 80, backbearing = 0 These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180). The therion formula is attempting to average the angles by trigonometry, which seems a reasonable approach (though probably slower than trying to average more directly like you're suggesting). It looks essentially correct to me, though it's a bit oddly written since cos(90 - x) is sin(x), sin(90 - x) is -cos(x), 90 - atan2(y,x) is atan2(x,y) (possibly +/- a multiple of 360). But if I work it out with a calculator for 236.8 and 56.8 then I do get 236.8. If it's still not working with the declination set to zero, perhaps you should show us a complete small example we can process to see what is going on? Cheers, Olly
Hi Olly, El 17/05/2018 a las 0:36, Olly Betts escribió:
I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2 Your proposed formula gives wrong answers in some cases - consider:
bearing = 80, backbearing = 0
These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180).
In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning.
The therion formula is attempting to average the angles by trigonometry, which seems a reasonable approach (though probably slower than trying to average more directly like you're suggesting). It looks essentially correct to me, though it's a bit oddly written since cos(90 - x) is sin(x), sin(90 - x) is -cos(x), 90 - atan2(y,x) is atan2(x,y) (possibly +/- a multiple of 360). But if I work it out with a calculator for 236.8 and 56.8 then I do get 236.8.
In the afternoon I have detected a strong anomaly problem in one station and corrected manually the data. Now the loop error is similar to Auriga and Compass (2.9m for a 1967m loop). But I have to check if Therion do the correct average calculation.
If it's still not working with the declination set to zero, perhaps you should show us a complete small example we can process to see what is going on?
I attach the original file with the fore/back sight (SCAGUA.th) and a manually averaged (SCAGUA_avg.th) to compare if Survex and Therion work well. The error loop is similar between the two files (2,9m) but I have to check if the station are positioned at the same coordinates. Regards, Evaristo.
17. 5. 2018 v 9:25, Evaristo Quiroga via Therion <therion@speleo.sk>:
In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning.
Evaristo, Therion is a program to interprete your data, not to solve problems with your data. It is your responsibility what the data you import into Therion. Martin S.
On 2018-05-17 09:25 +0200, Evaristo Quiroga via Therion wrote:
El 17/05/2018 a las 0:36, Olly Betts escribió:
I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2
Your proposed formula gives wrong answers in some cases - consider:
bearing = 80, backbearing = 0
These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180).
In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning.
Yes a warning should probably be issued about poor data, but it _is_ a problem with your formula. It's just an example showing that all cases have to be dealt with correctly, including the wrap-around at 0/360 (or 0/400 for grads) and your simplified formula doesn't. An example with a much smaller difference between back and foresight could still be constructed to show the issue: Fore: 175, Back: 0 AverageBearing= (175+0-180)/2 = -2.5. That's wrong. It should be 177.5 in this case. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/
Hi, Ok OIly and Wookey, you are right, my formula has not been very lucky. It was an attempt to understand what error he was experiencing when compiling data with fore/back bearings. I found a large difference in loop closure error between Auriga (2.85 m in 1967m loop), Compass (2.9m) and Therion (14m). Debugging the compilation I have found that the data transferred to Survex (data.svx), don't have sense to me. I have searched in the code source and found the complicated formula. I made my own formula in a Excell, and calculate the average. With this I have detected a strong magnetic anomaly in one leg (Auriga has this bearing manually discarded, and the translation to Compass was OK, but to Therion the fore-bearing was translated with a 0, a 56 degrees difference with the inverse back-bearing). After to correct the magnetic anomaly and to calculate the fore-bearing with the inverse back-bearing, the loop error it's OK too in Therion (2.9m). Only that it was necessary to understand what numbers Therion was going through to Survex, and that some degrees were deviating from those calculated manually. With the two files attached in the previous message: -a file with magnetic declination to 0 and fore/back bearings -a file with DM=0 and my average bearings. I have compared the results, that are the same. After looking carefully at the files and the compilation log, I found the difference. The data transferred to Survex are: bearing - magnetic declination - meridian convergence. These two components are what made me see different data between those calculated by Therion and those calculated manually. After some more checks, I have verified that Therion is correctly calculating the average between fore/back bearings. MY MISTAKE. The issue is closed. Best regards and thanksfor the patience. Evaristo. El 17/05/2018 a las 15:02, Wookey via Therion escribió:
On 2018-05-17 09:25 +0200, Evaristo Quiroga via Therion wrote:
El 17/05/2018 a las 0:36, Olly Betts escribió: I think the formula is too complicated. I purpose a simpler formula, like: If bearing <=180 AverageBearing = (bearing + (backbearing -180))/2 else AverageBearing = (bearing + (backbearing +180))/2
Your proposed formula gives wrong answers in some cases - consider:
bearing = 80, backbearing = 0
These give AverageBearing = (80 + 0 - 180) / 2 = -50 (equivalent to 310), but this should be 130 (average of 80 and 180).
In this case is not a problem with my formula, is a serious magnetic anomaly (100 degrees difference) and the program should to stop and to send a warning. Yes a warning should probably be issued about poor data, but it _is_ a problem with your formula. It's just an example showing that all cases have to be dealt with correctly, including the wrap-around at 0/360 (or 0/400 for grads) and your simplified formula doesn't.
An example with a much smaller difference between back and foresight could still be constructed to show the issue:
Fore: 175, Back: 0 AverageBearing= (175+0-180)/2 = -2.5. That's wrong. It should be 177.5 in this case.
Wookey
_______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion
On Thu, May 17, 2018 at 09:25:23AM +0200, Evaristo Quiroga wrote:
I attach the original file with the fore/back sight (SCAGUA.th) and a manually averaged (SCAGUA_avg.th) to compare if Survex and Therion work well. The error loop is similar between the two files (2,9m) but I have to check if the station are positioned at the same coordinates.
I feel I should point out that Survex's handling of backsights isn't coming in to play here at all - Therion flattens away the backsights so they don't appear in what it asks Survex to process. Out of interest I did a crude conversion of the two attached files to Survex format and processed them natively, and with scagua.279.0 fixed in each case the maximum discrepancy between the two results is: Moved by (-0.11,0.13,0.03): scagua.a.10 Which seems entirely plausible - I wouldn't expect exact agreement in this case as some of the data has backsights and some doesn't. In the data from SCAGUA.th, Survex will treat the data with backsights as slightly more accurate (because the average of two essentially independent readings is expected to be more accurate than a single reading) and that affects the distribution of loop misclosures. Cheers, Olly
participants (4)
-
Evaristo Quiroga -
Martin Sluka -
Olly Betts -
Wookey