OK - I'm trying to use the new import feature in 0.3.6. I'm not sure how the namespace hierarchy works. I have a survex dataset where everything is under terikan so we have major surveys terikan.goon terikan.eagle terikan.eagle2 terikan.elevator terikan.evening etc The .th file looks like: ------ survey tmp import terikan.3d survey terikan -title "Terikan River Caves" -declination [0.511 degrees] map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 break goon_s1 etc... input goon.th2 input soundriver.th2 etc... -------- This fails with: reading source files ... done preprocessing database ... done therion: error -- terikan/terikan.th [3] -- station doesn't exist -- eagle2.15 This is odd because there is no mention of eagle2 in any of the files in this dir. I'm not sure why it gets mentioned. The error is referencing the "import" line. If I comment out everything apart from the import line then the processing completes without error. Adding an empty file that continas nothing more than survey terikan endsurvey terikan will make it break again (same error). Changing the survey name to survey goon endsurvey goon Processes fine. Changing it to survey goon input goon.th2 endsurvey goon Now gives the error: therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon (see below) Now going back to the original file and moving the input command inside the 'survey terikan' context: ------ survey terikan -title "Terikan River Caves" -declination [0.511 degrees] import terikan.3d map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 ... ------ I get a different error, having got rather further: reading source files ... done preprocessing database ... done scanning centreline tree ... done searching for centerline loops ... done calculating station coordinates ... done calculating basic statistics ... done processing references ... therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon This suggests that the correct level to import things is inside a survey of the same name as the top level of the .3d file - correct? terikan.goon.69 is a valid station. It is the fiurst station mentioned in the first .th2 file read in. This suggests that it hasn't been read at the right level? Changing to put another .th2 file first changes the error to refer to the first station in that file instead. I'm confused - how should this feature be used? Is there a bug, or am I just doing it wrong? Wookey -- Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679 work: http://www.aleph1.co.uk/ play: http://www.chaos.org.uk/~wookey/
At 03:17 +0000 7.2.2005, Wookey wrote: *******************************************
The .th file looks like: ------ survey tmp import terikan.3d
survey terikan -title "Terikan River Caves" -declination [0.511 degrees]
map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 break goon_s1 etc... input goon.th2 input soundriver.th2 etc... --------
This fails with: reading source files ... done preprocessing database ... done therion: error -- terikan/terikan.th [3] -- station doesn't exist -- eagle2.15
Stacho and MartinB will explain this better, but if I try to check the part of cave I should use something as: survey test map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 break goon_s1 etc... endmap survey tmp import terikan.3d survey terikan -title "Terikan River Caves" -declination [0.511 degrees] # map elevatorplan -title "Terikan River Cave: Elevator entrance" # goon_s3 # break # goon_s1 # etc... input goon.th2 input soundriver.th2 etc... -------- endsurvey #tmp endsurvey #test The reference to the stations in map-endmap should be one level DOWN in survey hierarchy - as I understood this. Martin --
Wookey wrote:
OK - I'm trying to use the new import feature in 0.3.6. I'm not sure how the namespace hierarchy works.
I have a survex dataset where everything is under terikan so we have major surveys terikan.goon terikan.eagle terikan.eagle2 terikan.elevator terikan.evening etc
The .th file looks like: ------ survey tmp import terikan.3d
survey terikan -title "Terikan River Caves" -declination [0.511 degrees]
map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 break goon_s1 etc... input goon.th2 input soundriver.th2 etc... --------
So far everything looks OK.
This fails with: reading source files ... done preprocessing database ... done therion: error -- terikan/terikan.th [3] -- station doesn't exist -- eagle2.15
... this is a bug. Import command (terikan/terikan.th [3]) should never throw such an exception. Are you able to send me some files, that generate this sort of error?
This is odd because there is no mention of eagle2 in any of the files in this dir. I'm not sure why it gets mentioned. The error is referencing the "import" line. If I comment out everything apart from the import line then the processing completes without error. Adding an empty file that continas nothing more than survey terikan endsurvey terikan will make it break again (same error). Changing the survey name to survey goon endsurvey goon Processes fine.
OK - now I am guessing where the error is - I will try to find and fix it.
Changing it to survey goon input goon.th2 endsurvey goon Now gives the error: therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon (see below)
This is OK - you are trying to reference goon survey in goon context (so in upper context it is a survey goon.goon).
Now going back to the original file and moving the input command inside the 'survey terikan' context: ------ survey terikan -title "Terikan River Caves" -declination [0.511 degrees] import terikan.3d
map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 ... ------
I get a different error, having got rather further: reading source files ... done preprocessing database ... done scanning centreline tree ... done searching for centerline loops ... done calculating station coordinates ... done calculating basic statistics ... done processing references ... therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon
Again the same error.
This suggests that the correct level to import things is inside a survey of the same name as the top level of the .3d file - correct?
The correct is your first try. If you have in your .3d file all stations named terikan.*, then import terikan.3d must be outside terikan survey.
I'm confused - how should this feature be used?
Is there a bug, or am I just doing it wrong?
Yes, there is probably some bug. I will chech it. Regards, S.
I have tried to reproduce your error - but without success :( On my machine import works fine (see attached minimal sample). To fix this problem - I need some data where it fails. Thanks, S. Wookey wrote:
OK - I'm trying to use the new import feature in 0.3.6. I'm not sure how the namespace hierarchy works.
I have a survex dataset where everything is under terikan so we have major surveys terikan.goon terikan.eagle terikan.eagle2 terikan.elevator terikan.evening etc
The .th file looks like: ------ survey tmp import terikan.3d
survey terikan -title "Terikan River Caves" -declination [0.511 degrees]
map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 break goon_s1 etc... input goon.th2 input soundriver.th2 etc... --------
This fails with: reading source files ... done preprocessing database ... done therion: error -- terikan/terikan.th [3] -- station doesn't exist -- eagle2.15
This is odd because there is no mention of eagle2 in any of the files in this dir. I'm not sure why it gets mentioned. The error is referencing the "import" line. If I comment out everything apart from the import line then the processing completes without error. Adding an empty file that continas nothing more than survey terikan endsurvey terikan will make it break again (same error). Changing the survey name to survey goon endsurvey goon Processes fine.
Changing it to survey goon input goon.th2 endsurvey goon Now gives the error: therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon (see below)
Now going back to the original file and moving the input command inside the 'survey terikan' context: ------ survey terikan -title "Terikan River Caves" -declination [0.511 degrees] import terikan.3d
map elevatorplan -title "Terikan River Cave: Elevator entrance" goon_s3 ... ------
I get a different error, having got rather further: reading source files ... done preprocessing database ... done scanning centreline tree ... done searching for centerline loops ... done calculating station coordinates ... done calculating basic statistics ... done processing references ... therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon
This suggests that the correct level to import things is inside a survey of the same name as the top level of the .3d file - correct?
terikan.goon.69 is a valid station. It is the fiurst station mentioned in the first .th2 file read in. This suggests that it hasn't been read at the right level?
Changing to put another .th2 file first changes the error to refer to the first station in that file instead.
I'm confused - how should this feature be used?
Is there a bug, or am I just doing it wrong?
Wookey
+++ Stacho Mudrak [05-02-07 10:29 +0100]:
I have tried to reproduce your error - but without success :(
On my machine import works fine (see attached minimal sample). To fix this problem - I need some data where it fails.
http://wookware.org/terikan20050207.tgz (I'm using the recently-released 0.3.6 BTW) Wookey -- Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679 work: http://www.aleph1.co.uk/ play: http://www.chaos.org.uk/~wookey/
It was dirty bug, but it should be fixed now. I have put new developement snapshot on the downloads page. Your sample seems to works now. Regards, S. Wookey wrote:
+++ Stacho Mudrak [05-02-07 10:29 +0100]:
I have tried to reproduce your error - but without success :(
On my machine import works fine (see attached minimal sample). To fix this problem - I need some data where it fails.
http://wookware.org/terikan20050207.tgz
(I'm using the recently-released 0.3.6 BTW)
Wookey
+++ Stacho Mudrak [05-02-07 16:19 +0100]:
It was dirty bug, but it should be fixed now. I have put new developement snapshot on the downloads page. Your sample seems to works now.
Thanx again for the speedy fixes. Downloaded, built, and tested. Something very odd happenned the first time I ran it. It seemed to run through lots of data, looking like everything was fixed andthen it came up with a 'duplicate survey error - dummy' error. It turned out that this was due to the thconfig file being 'doubled' - it had had a second copy of itself added to the end (thus including index.th twice, hence the duplicate survey error). However having fixed that it now seems to process OK, so I'm not sure if this is a bug or finger trouble on my part, but I'm fairly sure I didn't mess it up. However - whilst it now processes a 'null' set of data as in the example, as soon as I put back any data it goes wrong again: input goon.th2 (inside the terikan survey) and I get: therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon Now it doesn't seem to matter whether the terikan.3d file is included inside or outside the terikan survey context, or even not included at all - I get the same erro in each case. The .3d file is being read because I get a cave.3d file out that looks fine (i.e. looks just the same as the terikan.3d file). So - any clues? You can test this by just uncommenting line 47 of the 20050207 dataset you have. Wookey -- Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679 work: http://www.aleph1.co.uk/ play: http://www.chaos.org.uk/~wookey/
Wookey wrote:
However - whilst it now processes a 'null' set of data as in the example, as soon as I put back any data it goes wrong again: input goon.th2 (inside the terikan survey) and I get: therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon
Well - I expected that you will report this error :) OK - the problem is - when stations are imported, they are inserted into subsurveys _if and only if this subsurvey exists_. In your example - you have no survey goon defined, therefore terikan.goon.69 is interpreted like goon.69@terikan. We came to conclusion, that this arrangement is better, like if subsurveys will be created - because we usually use "." in station names within single survey. If therion would have import command before - you would probably reference stations like goon.69 and you would not create any survey structure. So this problem is probably only in the case - when somebody converted his surveys first and then he tries to use import command instead. You can solve your problem three ways: 1. Define empty surveys at the beginning of the file import terikan.3d survey terikan survey goon endsurvey ... the same for each used survey endsurvey terikan ... and it should work. This is easiest solution. 2. Convert everything XX@YY into YY.XX using regexp replace. A little bit more work - but very clean solution. 3. Persuade us to modify therion in a way - that subsurveys will be created (may be into some - "user specified" - level). It is a litle bit more work on our side, but if you will have good arguments for it - no problem. Regards, S.
+++ Stacho Mudrak [05-02-08 08:53 +0100]:
Wookey wrote:
However - whilst it now processes a 'null' set of data as in the example, as soon as I put back any data it goes wrong again: input goon.th2 (inside the terikan survey) and I get: therion: error -- terikan/goon.th2 [10] -- survey does not exist -- goon -- invalid station reference -- 69@goon
Well - I expected that you will report this error :)
You could have mentionned it so I didn't spend two hours trying to work out if I was just being dumb... :-/
OK - the problem is - when stations are imported, they are inserted into subsurveys _if and only if this subsurvey exists_. In your example - you have no survey goon defined, therefore terikan.goon.69 is interpreted like goon.69@terikan.
If therion would have import command before - you would probably reference stations like goon.69 and you would not create any survey structure. So this problem is probably only in the case - when somebody converted his surveys first and then he tries to use import command instead.
True. Although it might be useful if it was possible to use the same dataset with either an import command or a set of converted svx->therion data. In this case the station references in the scrpas need to be consistent, and probably need to be 69@goon, not goon.69@ This makes the scraps modular so they can be put together in different ways without too much knowledge of the surrounding structure. Modularity in data is something we should strive for as much as possible. This discussion brings up another point I wanted to raise. The fact that I have to add a fake top-level survey in order to include a .3d file seems to me to be very naff. There must be a better solution. If I have a therion dataset inside survey terikan endsurvey terikan And a survex dataset which has terikan as the top level then it ought to be possible to put them together without having to makew the structure be survey dummy import terikan.3d survey terikan endsurvey terikan endsurvey dummy Maybe we need a concept like the -p1 in patch to tell it how many levels to strip off the input dataset in order to import it correctly? This is particularly an issue if I have a big dataset for the area containing several caves (Which I do:) We actually have top level: mulu (whole area) 2nd level: benarat - mountain api - another mountain 3rd level: terikan - cave bluemoon - cave cobweb - cave etc. So ideally I would want to import the top-level .3d file that included all the GPS fixes, surface surveys, surface loop closures etc - i.e the best possible relative fit of all the cave positions. Then use this at various places in the therion dataset for therion to draw separate surveys with. I can do this with the current version of therion by adding lots of empty surveys to match this structure, although still with the extra top-level 'dummy' which I don't like. It would definately be better to be able to say 'import ignoring the top 2 levels'. Ideal would be for Therion to look at the imported structure and try to match a level to the current survey level (this won't work if there are two surveys with the same name at different levels, but I don't think that happens often in real datasets?). It is obvious to a person that if a dataset contains terikan which contains goon, and the therion dataset has survey terikan and load of references to goon below then it matches up. It would be good if the computer could do that too, although I realise this is not trivial in practice.
You can solve your problem three ways:
1. Define empty surveys at the beginning of the file
import terikan.3d survey terikan survey goon endsurvey ... the same for each used survey endsurvey terikan
... and it should work. This is easiest solution.
but it is naff :-)
2. Convert everything XX@YY into YY.XX using regexp replace. A little bit more work - but very clean solution.
Except that it means I can't go back to using a full-therion dataset without changing all the structure round. I shouldn't have to decide before starting all the drawing whether I am going to get the station positions from a therion dataset or a survex dataset - it should be possible to do both with the same set of scraps.
3. Persuade us to modify therion in a way - that subsurveys will be created (may be into some - "user specified" - level). It is a litle bit more work on our side, but if you will have good arguments for it - no problem.
OK- right, I am understanding the problem now. With a full therion dataset we have each survey explicitly created. With an imported survex dataset this does not happen. I think the importing of the dataset should implicitly create the hierarchy, at least so that references in .th2 files can 'just work' without us having to explicitly re-specify it in .th files. I think there is enough data in the .3d file to do this? At the very least we need a detailed explanation of this for the time being as it's not at all obvious :-) Wookey -- Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679 work: http://www.aleph1.co.uk/ play: http://www.chaos.org.uk/~wookey/
Wookey wrote:
You could have mentionned it so I didn't spend two hours trying to work out if I was just being dumb... :-/
I am sorry... But as far as I remember, I sent you working sample of your data with import 3D file week or two ago... There was a solution with empty surveys - it could give you at least a small hint. OK - next time I will write some warnings to new stuff...
This discussion brings up another point I wanted to raise. The fact that I have to add a fake top-level survey in order to include a .3d file seems to me to be very naff. There must be a better solution.
This I have already in TODO. I agree with you - I do not like it neither. I will try to arrange things, that no top level survey - endsurvey commands will be needed (default context will be survey - even no survey will be created).
Maybe we need a concept like the -p1 in patch to tell it how many levels to strip off the input dataset in order to import it correctly?
This is very good idea - very simple to implement. I will do it ASAP.
Ideal would be for Therion to look at the imported structure and try to match a level to the current survey level (this won't work if there are two surveys with the same name at different levels, but I don't think that happens often in real datasets?).
May be second step.
Except that it means I can't go back to using a full-therion dataset without changing all the structure round. I shouldn't have to decide before starting all the drawing whether I am going to get the station positions from a therion dataset or a survex dataset - it should be possible to do both with the same set of scraps.
OK - you are probably right.
OK- right, I am understanding the problem now. With a full therion dataset we have each survey explicitly created. With an imported survex dataset this does not happen. I think the importing of the dataset should implicitly create the hierarchy, at least so that references in .th2 files can 'just work' without us having to explicitly re-specify it in .th files. I think there is enough data in the .3d file to do this?
The reason why I did not do this was, that it was again quite deep change in the complicated part of the code. But I can have a look at it once more - may be it is more simple than it looks at first sight (like a lot of other things were :-)). So until it will not be finished - you will have to create empty surveys... Regards, S.
+++ Stacho Mudrak [05-02-08 17:05 +0100]:
Wookey wrote:
You could have mentionned it so I didn't spend two hours trying to work out if I was just being dumb... :-/
I am sorry... But as far as I remember, I sent you working sample of your data with import 3D file week or two ago... There was a solution with empty surveys - it could give you at least a small hint.
Did you? I seem to have failed to look at that/appreciate it. I didn't mean to complain too much - you are doing a brilliant job overall. It was just that I spend a couple of hours trying it again and find it doesn't work and you say 'ah yes I thought youd say that' :-)
OK - next time I will write some warnings to new stuff...
That would be good - I have far toomany things going on at once and tend to forget mails from more than a couple of days ago. (my inbox is 5800 at the moment, growing at about 50/day).
So until it will not be finished - you will have to create empty surveys...
no problem. Not that I understand what is going on - thanx for the prompt explanation. (the survey I was supposed to finish in mid-jan is now going to be rather late...) Wookey -- Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679 work: http://www.aleph1.co.uk/ play: http://www.chaos.org.uk/~wookey/
At 15:39 +0000 8.2.2005, Wookey wrote: *******************************************
We actually have top level: mulu (whole area) 2nd level: benarat - mountain api - another mountain 3rd level: terikan - cave bluemoon - cave cobweb - cave etc.
It looks you waited for therion with this project, isn't it. :) Me too - 25 years old survey project of Cachtice cave. Good luck! Martin S. --
+++ Martin Sluka [05-02-08 18:07 +0100]:
At 15:39 +0000 8.2.2005, Wookey wrote: *******************************************
We actually have top level: mulu (whole area) 2nd level: benarat - mountain api - another mountain 3rd level: terikan - cave bluemoon - cave cobweb - cave etc.
It looks you waited for therion with this project, isn't it. :) Me too - 25 years old survey project of Cachtice cave.
I have indeed been waiting since about 1994 for something like Therion, before I decided to commit a lot of time to digitising cave survey drawings. I have avoided using things like coreldraw/Xara/freehand in the meantime as I didn't want to get data stuck in a proprietary format. The mulu data for the system I am currently drawing comes from 1984-2003. (and the early stuff is drawings-only - hence some of my earlier questions about that). I need to get it drawn up before the next expo in spring 2005.... Wookey -- Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679 work: http://www.aleph1.co.uk/ play: http://www.chaos.org.uk/~wookey/
At 18:29 +0000 8.2.2005, Wookey wrote: *******************************************
The mulu data for the system I am currently drawing comes from 1984-2003. (and the early stuff is drawings-only - hence some of my earlier questions about that).
I need to get it drawn up before the next expo in spring 2005....
Exact (only in scale 1:100) - the "my" cave was digged and partly explored in half past of 50th. The first map wass only 750 m and about 0,5 km nonsurveyed passages. The second - very professional surveying - with theodolite in 0.5 m high crawlings was made in 72-74 - but only 330 m of "main passages". After our Roumanian experience with real caving we started in 1979 to resurvey the cave from the "end" to entrance - now we have more than 4 km of survey shots. But all this is in very small volume - 100 x 150 x 350 m of limestone. So you may imagine, to draw it by hand or in iIlustrator. But therion works well, I have now 1,5 km in it. :) Good luck with your project, our meeting is in middle of April :( Martin --
But therion works well, I have now 1,5 km in it. :) And thanks to survex - without the analysis of surveying of caves included there the Stacho with Martin had much more complicated work. Martin --
OK, I have added two new options for import command and changed the survey structure policy. Now non-existing surveys are created. + import -filter <string> -- only stations with given prefix and shots between them will be imported - prefix will be removed from station names + import -surveys (create)|use|ignore -- specifies how to import survey structure. create = split stations into subsurveys, if subsurveys do not exists, create them use = split stations into existing subsurveys ignore = do not split stations into sub-surveys This option works only for .3d import. So now you can do following survey terikan import terikan.3d -filter terikan ... scraps & maps ... endsurvey You can download developer's snapshot to test them. S.
participants (3)
-
Martin Sluka -
Stacho Mudrak -
Wookey