Lost Catalog File - Is this a Bug?

Roger EdrinnRoger Edrinn Global Mapper UserPosts: 721Trusted User
edited April 2011 in Bug Report
I just spent all morning building a giant catalog file of GridFloat DEMs in *.flt format. Opened the file an hour later, all gone!!! Did a search on both HDD for *.gmc, not around.

Running GM 11.02 July 29, 2010

Because a FLOAT file is not recognized as a typical GM file type this catalog build was extra painful and time consuming. There were 112 folders I had to open and load a file from each to build the catalog. I tested multiple times and when I selected a small area, each time the data was there indicated by the default color and cursor elevation tests.

About half way through the catalog build I loaded a .gmw file of the state border as a reference. Then I finished loading the remainder of the 112 files.

Since there is no Save option in a catalog, only Save As, I was under the impression that a catalog saves with each new added file.

Did I do something wrong or is this a BUG?

Thanks

Comments

  • redbeardredbeard Global Mapper User Posts: 279Trusted User
    edited April 2011
    I think you can do a load all files in a directory and specify *.flt as the type.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited April 2011
    I'm not sure what happened to the catalog file. I would make sure to record where you specified to save the map catalog file when you create the catalog. Also make sure not to hit Cancel on the map catalog dialog.

    Loading that many .flt files should be easy using the method that Redbeard suggested. Another option would be to write a small script that simply imports all of the Float/Grid files in a folder tree. You can specify the type in the IMPORT command if it isn't automatically detected based on the file extension.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    BUG!!!

    Your answer was very unsatisfying. This is a reproducible BUG.
    1. Open and save a new catalog file (write down the location)
    2. Open a couple of *.flt files in the catalog (may work with others, did not test)
    3. Open a GMW file
    4. Open a couple more *.flt files
    5. Close the Catalog dialog by selecting OK
    6. Do a file Save As on the GMW
    7. Your catalog file will have no contents
    8. You GMW will have an empty Catalog file

    Some humble words are now appropriate.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited April 2011
    I am unable to reproduce what you are seeing with the latest v11 build. I never get an empty catalog file when adding .flt files. It is possible there is an issue in the building you are using which has since been fixed. I have placed a new build at http://www.globalmapper.com/global_mapper11.zip with the latest changes for you to try. Simply download that file and extract the contents into your existing v11.xx installation folder to give it a try. If you are using the 64-bit v11 version there is a new build at http://www.globalmapper.com/global_mapper11_64bit.zip .

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    Mike,

    Somewhere between July `10 and April `11 this guy was exorcised. I had no problem reproducing the problem on my July build. Works like it should today. I only wish it was able to recover the work from earlier, didn't happen . . . gone.

    My turn for humble pie. I guess the moral, if you laid off GM for months, start with a new build, may/not save some headaches.

    Thanks
  • tjhbtjhb Global Mapper User Posts: 454Trusted User
    edited April 2011
    Roger,

    It is idiotic to launch any post in giant red letters and exclamation marks the way you did.

    Even if the exclamation marks had been question marks, and even if you had been right, this would have been beneath you.

    You usually post good sense.

    Tim
  • redbeardredbeard Global Mapper User Posts: 279Trusted User
    edited April 2011
    Question -
    is there any confusion about gmw and gmc? I'm sort of losing track. Did you have a workspace file saved with a catalog in it, and added the file to the catalog and saved the workspace but not the catalog?
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    redbeard wrote: »
    I think you can do a load all files in a directory and specify *.flt as the type.
    Unless I'm missing something, typing in *.flt is the same as selecting the FLOAT type in the file open dialog. There are still 122 folders to open and select the single .flt file in each.

    Thanks
  • redbeardredbeard Global Mapper User Posts: 279Trusted User
    edited April 2011
    When you open global mapper - hit create new catalog -
    then in that dialog you can add a directory - select the folder that contains all the folders - and specify file type as *.flt

    Then it will ask if you wish to look inside folders inside the folder - and hit yes.

    I dont have it open so I wont have the exact dialogs - but I think that's how it works.
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    Loading that many .flt files should be easy using the method that Redbeard suggested.
    See my comment to Redbeard.
    Another option would be to write a small script that simply imports all of the Float/Grid files in a folder tree. You can specify the type in the IMPORT command if it isn't automatically detected based on the file extension.
    I need to add files to a Catalog, not to the workspace. I'm dealing with 50GB of DEM files and 3GB of useful memory.

    In looking over the script reference, you linked in another thread, their were only two Catalog references in the entire document. Both dealing with importing the catalog, not adding files to the catalog.

    I'm currently left with duplicating the 3-hour marathon that the code bug flushed, unless I'm missing a key point.

    Thanks
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    redbeard wrote: »
    When you open global mapper - hit create new catalog -
    then in that dialog you can add a directory - select the folder that contains all the folders - and specify file type as *.flt

    Then it will ask if you wish to look inside folders inside the folder - and hit yes.

    I dont have it open so I wont have the exact dialogs - but I think that's how it works.
    Old Dog, New Tricks!!!

    Currently chewing the data. :)

    Thanks
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited April 2011
    When on the map catalog dialog you need to press the 'Add Directory' rather than 'Add Files' button. Then you can add all files matching *.flt in the directory tree.

    Also note that loading 50GB of .flt files directly wouldn't be a problem either, they are read from disk on the fly and won't take anywhere near 50GB of memory. It would still be better to manage them from a map catalog though.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    redbeard wrote: »
    Question -
    is there any confusion about gmw and gmc? I'm sort of losing track. Did you have a workspace file saved with a catalog in it, and added the file to the catalog and saved the workspace but not the catalog?
    Not exactly sure of your question, but in summary. Adding the gmw to the New Catalog caused the catalog to be flushed, it wasn't apparent at any time during the file add sequence, nor when clicking the catalog dialog OK button. Even doing a Save As of the subsequent workspace.

    Either opening the catalog or the workspace resulted in an empty catalog in my July 2010 version of GM. All the loaded data in the catalog was lost.
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    When on the map catalog dialog you need to press the 'Add Directory' rather than 'Add Files' button. Then you can add all files matching *.flt in the directory tree.

    Also note that loading 50GB of .flt files directly wouldn't be a problem either, they are read from disk on the fly and won't take anywhere near 50GB of memory. It would still be better to manage them from a map catalog though.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
    Thanks Mike, we must have been writing simultaneously.
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    This bug has not entirely disappeared.

    I dropped the just created catalog file into GM and nothing. Opened the catalog, nothing. As a test I dropped another known good catalog unto GM, instantly, all the red boxes appeared. Yet the suspect .gmc file shows 73KB, so it's not empty.

    I attempted to attach the file to this forum "invalid file"?!?

    I loaded the file to my website, here's a link.

    Please examine why/what is going on.

    Thanks
  • redbeardredbeard Global Mapper User Posts: 279Trusted User
    edited April 2011
    Roger - you know you can open the file with a text editor right? Might help you troubleshoot - I've not had the problems you describe - but the file names looks good, though the ampersand is %amp% and I'm guessing that's correct. What are the display settings for your catalog?
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    When I open the link from the forum, it appears as text. Yes it appears correct, but I'm no expert. The bottom line I want GM to open the catalog. GM made the catalog. GM opens another catalog I made. I'll test the %amp%.

    Thanks
  • redbeardredbeard Global Mapper User Posts: 279Trusted User
    edited April 2011
    Roger, that catalog shows up fine for me.

    I guess check the following - when you open the overlay control center - select the catalog - and hit options

    You should see all the files on the left, and on the right it has display options. One is show bounding boxes - is there anything you can do there?
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    redbeard wrote: »
    Roger - you know you can open the file with a text editor right? Might help you troubleshoot - I've not had the problems you describe - but the file names looks good, though the ampersand is %amp% and I'm guessing that's correct. What are the display settings for your catalog?
    redbeard,

    You have good instincts, tested without %amp% and it worked as it should. No doubt this was the problem all along. Too bad GM didn't choke on the load, instead of when I tried to access the file.

    Any suggestion on the best contour file type? I'll be loading those 100's -1000's of contour files to another catalog using the new trick I learned a couple post ago.

    I only ask because you're on a roll.

    Thank you for the suggestion.
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    redbeard wrote: »
    Roger, that catalog shows up fine for me.

    I guess check the following - when you open the overlay control center - select the catalog - and hit options

    You should see all the files on the left, and on the right it has display options. One is show bounding boxes - is there anything you can do there?
    Tried again, nothing, not a single file in the dialog. If I click OK, I get a warning, must have a least one file.

    The good news, just tested sans %amp%, worked fine.

    Perhaps Yoda can explain all this?
  • redbeardredbeard Global Mapper User Posts: 279Trusted User
    edited April 2011
    That's an oddity. Not sure.

    As far as contour file type - I'm not sure what would be best. Shapefiles have some limitations, especially when they get big. Perhaps the globalmapper package file? There is an option (I'm on v12) to generate directly to a global mapper package and use with gridding for very large areas.
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    redbeard wrote: »
    Perhaps the globalmapper package file? There is an option (I'm on v12) to generate directly to a global mapper package and use with gridding for very large areas.
    That's not an option, at least yet. GMP will hold both vector and raster, it's like the GDB format. Hence it'll grab the DEM file too.

    Mike has offered to write either an ON/OFF switch or a Vector only switch for the Export_package command. All still pending. May need v12 to do this.

    I don't want to grid because of the niggles I've seen with auto gridding.

    Picky, picky.

    Thanks
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited April 2011
    The ampersands were indeed the problem. There was a bug dealing with those in map catalog filenames in v11 (since fixed in v12). I went ahead and fixed it in v11 as well. I have placed a new build at http://www.globalmapper.com/global_mapper11.zip with the change for you to try. Simply download that file and extract the contents into your existing v11.xx installation folder to give it a try. If you are using the 64-bit v11 version there is a new build at http://www.globalmapper.com/global_mapper11_64bit.zip .

    I also added support for adding a VECTOR_ONLY=YES parameter to the EXPORT_PACKAGE command in the latest v12 build if you decide to move to v12. I have placed a new build at http://www.globalmapper.com/global_mapper12.zip with the change for you to try. Simply download that file and extract the contents into your existing v12.xx installation folder to give it a try. If you are using the 64-bit v12 version there is a new build at http://www.globalmapper.com/global_mapper12_64bit.zip .

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • Roger EdrinnRoger Edrinn Global Mapper User Posts: 721Trusted User
    edited April 2011
    Thanks Mike,

    Turns out I misspoke, while the gridded niggles can be seen, the zoom or scale is so ridiculous that it is a total non-issue for my application. I spent a half-hour doing side-by-side comparison of GM and MapSource. The max scale of MS is 1:700, no where near enough to see any "defect".

    A surprising revelation of that comparison was the vastly smoother contours in GM vs the GPS maps. That tells me the the MP export reduces the vector point count to keep the file size reasonable. Still huge files.

    I've used the maps in the field numerous times and while the contours aren't as pretty as a USGS topo, you just don't need that detail. If its that steep, you're not looking at your GPS.

    I can live without the "&", but in case I forget later, I'll download the new file.

    Appreciate all your help.
Sign In or Register to comment.