Falling hard with an appropriate error message would be good: it’s obvious then that the available data is invalid, and why. And the message should include the hint to the declination command to fix this.

Am 05.10.2020 um 20:40 schrieb Bruce Mutton <bruce@tomo.co.nz>:

 Hi Matt I think these issues have been raised previously. The error message for dates later than those included in the Therion model was inaccurate, as you point out. I ‘think’ that in recent versions this may have been replaced with a more meaningful message.

And then the question of whether it is better for Therion to extrapolate future declinations, or set them to zero? Shades of failing gracefully or abruptly, I guess. I could take either side, but I think my preference would be for Therion to set declinations beyond those modelled to zero. This way it is more likely that users will detect the problem (as you have), and either upgrade to a more recent version of Therion, or manually set the declination for each affected survey, using the declination statement.

Bruce

From: Therion <therion-bounces@speleo.sk> On Behalf Of Matthias Keller Sent: Monday, 5 October 2020 21:40 To: therion@speleo.sk Subject: [Therion] Therion uses wrong declination for surveys 'in the future'

Hi

I just had the following problem: A current survey (with survey dates 2020.06.27 and 2020.09.19) was compiled using Therion 5.4.4 (from 2019). I got the following warning during compilation:

warning -- unable to determine magnetic declination for undated surveys

However, firstly: this warning is wrong, as all surveys are dated correctly.

However I was stunned by some large loop errors in the newest surveys (>60cm in only 10 legs). Then, just by some other impulse I updated therion to 5.5.1 and suddenly, those errors went down to <10cm and the generated models looked differently. Also, the above error message was not present anymore.

So i experimented and just added "declination 3 deg" to the two new surveys (from June and September) and suddenly, the output generated by 5.4.4 looked pretty much as the one from 5.5.1

It appears that, when therion cannot determine the magnetic declination of a survey (with valid date!), it finally just assumes 0. The problem seems to be, that the declination data stored in 5.4.4 didn't let therion determine a possible declination for 2020.06.27 and 2020.09.19 so it just assumed 0. I would have expected therion to at least consider the latest known declination which would result to around 3° (in Switzerland).

Is this a known issue? It just means that whenever you open a survey in some version being 'too old', it will kill your plans...

BTW, Therion 5.4.4 seems to have *some* data for 2020 at least, as it seems:

geomag declinations (deg): 2019.1.1 2.9201 2020.1.1 3.0503

but the only surveys from 2020 are the ones from June 2020 and September 2020, and both were assigned a declination of 0° instead of about 3°. I can verify that by adding the declination parameter with "0 deg" to those surveys and the output is identical to the one without this parameter.

Thanks

Matt

_______________________________________________ Therion mailing list Therion@speleo.sk https://mailman.speleo.sk/listinfo/therion

On 2020-10-05 23:54 +0200, Matthias Keller wrote:

Exactly this.

Unfortunately, even the current Therion 5.5.1 doesn't have a more meaningful (and correct) message than "unable to determine magnetic declination for undated surveys"

I thought this might be a proj confusing error message, but it is therion's.

I would propose:

1. If a survey is dated but is newer than available correction data, build should fail with a message like: "Error determining magnetic declination for survey <name> with date <date>. Please specify the declination explicitly using for example 'declination 3 deg'" instead of the (completely wrong) "unable to determine magnetic declination for undated surveys" 2. If at least one of the imported (and thus joined) surveys has magnetic data but at least one does not (or cannot be determined), an error shall be thrown as well, because mixing corrected and uncorrected surveys is just plain wrong and causes a lot of confusion (as it has happened to me). The same error message shall be displayed and the build fails.

This does seem reasonable behaviour. And I agree the error message is wrongly worded or issued here. I just had a look at the code that produces this error but it's rather confusing. A bitmap is used to record what declination data was used and if bits 1 and 3 are set you get the above error. I think bit 2 is 'used igrf model to get delination' and bit 3 is 'used defined declination', and bit 3 is 'didn't use any', but I'm not sure and I don't see why two bits would be set. I'll leave it to Martin/Stacho to work out how the logic should run here.

Is this the right place to request that or shall I open an issue on github?

I think this list is the canonical place to discuss such things.

I really find this issue pressing, because I just cannot imagine, that this doesn't happen to everyone here from time to time - keeping therion up to date is not the same cycle as adding new surveys, and suddenly you fall out of the prediction data.

This is really an issue of the geomag model. It does work some way into the future, but there is a cutoff. Looks like it does 10 years into the future as the latest data in the therion source (igrf12) ends in 2015 and the previos igrf11 (used in therion 5.4.4) ended in 2010. igrf13 (with data to 2020) was released in December 2019 https://www.ngdc.noaa.gov/IAGA/vmod/coeffs/igrf13coeffs.txt So clearly this will arise anytime you have survey data 10 years newer than the model in the therion you are running, which as they are only released every 5 years and therion may take a year or two to update and the installed one may be a few years old, is quite likely to happen from time to time, especially in years ending with 0 or 5.

btw, is there any way to update the data in an existing therion installation? Or is the only way the update to a (hopefully existing) new version?

There could be - the files are available online so therion could use the currently-installed version of the file. But currently the igrf coefficients file is compiled-in as thgeomagdata.h This means therion can always find the data, but it's not updateable without recompiling. I could fix this for linux relatively easily, but it's harder on Windows. (and it was making things work on windows that caused therion to compile-in all the stuff it needed from subsystems, many years ago). Ideally it would compile one in but have the ability to read a newer file if provided. But then someone has to write a parser. We could also try a little harder not to get 10 years behind again. (5.4.4 was released in May 2019 with a model that expired in mid 2020, when the 2025 model had been available since 2015 (and was included with the next therion release in May 2020). Not sure why the update was not to 2030 as that was available from Dec 2019.) Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/

Hi, On Tue, Oct 6, 2020 at 4:10 AM Wookey <wookey@wookware.org> wrote:

On 2020-10-05 23:54 +0200, Matthias Keller wrote:

...

Unfortunately, even the current Therion 5.5.1 doesn't have a more meaningful (and correct) message than "unable to determine magnetic declination for undated surveys"

This has already been fixed in the latest commit.

...

I would propose:

1. If a survey is dated but is newer than available correction data, build should fail with a message like: "Error determining magnetic declination for survey <name> with date <date>. Please specify the declination explicitly using for example 'declination 3 deg'" instead of the (completely wrong) "unable to determine magnetic declination for undated surveys"

We have discussed it with Stacho and the following seems to be the best approach: IGRF models are designed to predict over the 5 year period (IGRF 13 released at the end of 2019 is intended to be used to predict the declination up to the end of 2024, when a new version, IGRF 14 will be released). In Therion, we will allow to use the model for additional 5 years (using linear extrapolation) and produce a warning (so IGRF 13 could be used until the end of 2029). After 2029, more firm warnings will be produced if Therion still uses IGRF 13, but we don't expect anybody to use such an outdated Therion. On the other hand: is there any need to use the data older than 1900? If yes, we could implement the GUFM1 model covering the period 1590–1890.

...

2. If at least one of the imported (and thus joined) surveys has

magnetic

...

data but at least one does not (or cannot be determined), an error shall be thrown as well, because mixing corrected and uncorrected surveys is just plain wrong and causes a lot of confusion (as it

has

...

happened to me). The same error message shall be displayed and the build fails.

You can't always avoid mixing dated and undated surveys (sometimes the older survey data might be undated), so using the min(survey_dates) as a proxy for undated surveys (and producing a warning as well) should be a reasonable approach.

...

btw, is there any way to update the data in an existing therion installation? Or is the only way the update to a (hopefully existing) new version?

There could be - the files are available online so therion could use the currently-installed version of the file. But currently the igrf coefficients file is compiled-in as thgeomagdata.h This means therion can always find the data, but it's not updateable without recompiling. I could fix this for linux relatively easily, but it's harder on Windows. (and it was making things work on windows that caused therion to compile-in all the stuff it needed from subsystems, many years ago). Ideally it would compile one in but have the ability to read a newer file if provided. But then someone has to write a parser.

Not sure if writing the parser is worth the effort, as AFAIK all the real issues with the declination come from the incorrect handling of undated/out-of-range data, not from an outdated model. Even a model outdated 10 or 15 years should provide very reasonable results. E.g. for our cave area: The declination for 2015-01-01 calculated as a * 10-year prediction from IGRF 10 = 4.6206 * 5-year prediction from IGRF 11 = 4.6593 * model result from IGRF 12 = 4.6836 * definitive model result from IGRF 13 = 4.6834 The difference is absolutely negligible even between the models 3 generations apart. The declination for 2020-01-01 calculated as a * 15-year prediction from IGRF 10 = 5.1626 * 10-year prediction from IGRF 11 = 5.2685 * 5-year prediction from IGRF 12 = 5.3231 * model result from IGRF 13 = 5.4036 The difference is still less than a typical compass reading error. The example is not really representative and there might be areas (definitely near the magnetic poles) where the differences are larger, but it should illustrate the point. Also consider that the model is approximate anyway (even in its intended time range) and no local/regional deviations in the geomagnetic field are taken into account at all.

We could also try a little harder not to get 10 years behind again. (5.4.4 was released in May 2019 with a model that expired in mid 2020, when the 2025 model had been available since 2015 (and was included with the next therion release in May 2020). Not sure why the update was not to 2030 as that was available from Dec 2019.)

There was actually no 2025 model available since 2015. The models are always for 5-year predictions and there is zero overlap between them. Just to illustrate, here is an overview of the history of publication of IGRF models and their inclusion into Therion (see https://github.com/therion/therion/commits/master/thgeomagdata.h): * IGRF 11 released in December 2009, included into Therion on February 17, 2010, released as 5.3 on the same day * IGRF 12 released in December 2014, included into Therion on December 28, 2014, released as 5.3.16 on the same day * IGRF 13 released in December 2019, included into Therion on January 19, 2020, released as 5.5.0 on May 1, 2020 Cheers Martin

On 2020-10-06 21:10 +0200, Martin Budaj wrote:

On the other hand: is there any need to use the data older than 1900? If yes, we could implement the GUFM1 model covering the period 1590–1890.

We've had to use data back to about 1910 in Ireland but I'm not aware of anyone needing to go back before 1890. I'm not sure if there is/was any conventional cave survey data that old. I reckon we are safe to carry on ignoring it until someone asks.

You can't always avoid mixing dated and undated surveys (sometimes the older survey data might be undated), so using the min(survey_dates) as a proxy for undated surveys (and producing a warning as well) should be a reasonable approach.

Very good point. We have a lot of data imported from other programs which ends up with no date. And fake data generated from existing survey plots also ends up dateless. This is definitely a thing.

Ideally it would compile one in but have the ability to read a newer file if provided. But then someone has to write a parser.

Not sure if writing the parser is worth the effort, as AFAIK all the real issues with the declination come from the incorrect handling of undated/out-of-range data, not from an outdated model.  Even a model outdated 10 or 15 years should provide very reasonable results.

Agreed.

We could also try a little harder not to get 10 years behind again. (5.4.4 was released in May 2019 with a model that expired in mid 2020, when the 2025 model had been available since 2015 (and was included with the next therion release in May 2020). Not sure why the update was not to 2030 as that was available from Dec 2019.)

There was actually no 2025 model available since 2015. The models are always for 5-year predictions and there is zero overlap between them. Just to illustrate, here is an overview of the history of publication of IGRF models and their inclusion into Therion

Cheers for the corrections. (and for the quick fixes). Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/