USGS implementation of "ODDF"?

rjackrjack Global Mapper UserPosts: 8
edited February 2011 in Technical Support
Does global mapper support the USGS "ODDF" binary file format?

Webring, M. W., 1998, ODDF: a file I/O subroutine package implementing
NASA PDS data description and USGS map projections, version 1.6,
USGS Open-File Report 98-765, 132 pg.

If not, what is the best way to convert it? I can get the data as ascii XYZ but the size ballons 3.5x which requires splitting the files into chunks to be able to computationally handle it. But the chunks aren't continuous, so its just not working well. I thought I would ask about using the original native format if possible.

Comments

  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited February 2011
    The ODDF format is not directly supported. Can Global Mapper handle the larger XYZ files directly? If so you could use Global Mapper to convert the large XYZ into something easier to work with, like a Global Mapper Grid file (if working in Global Mapper), or maybe a BIL file with a header or something like that.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    Yeah I asked about gridding larger 1.2 to 1.7Gb XYZ files earlier. http://www.globalmapperforum.com/forums/elevation-data/1787-huge-xyz-file-how-open.html
    I have a 2.1Ghz processor and 4Gb of RAM running 64-bit GM in Windows7 and its not really working.

    I am currently trying to load one file as XYZ directly into a grid as you suggested. It did load the points faster (~30mins) since it didn't have to draw them. But has been struggling with the gridding for almost 2 hours already. Seems to be writing data to disk cache. Windows thinks Global Mapper is "not responding" although it seems to still be working on it. I will have to change the sleep settings to allow it to run overnight.

    Thanks for any tips
    Richard
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited February 2011
    Richard,

    I would definitely let it run overnight. It sounds like you might have exhausted your physical memory so virtual memory from disk is being used, which is definitely a lot slower than if all of it fit into memory.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    I attempted to produce a 2m grid from a 1.7Gb XYZ file and it returned the following error after about 4 hours:

    Unable to grid 3d points from <file name.txt>
    Access is denied
    MemoryMappedFile.cpp - 403
    Version: v12.01
    Build time Jan 20 2011 22:43:41

    I guess I am stuck looking for a machine with more processors and more memory. I would edit the text file into contiguous chunks but I don't have a text editor capable of manipulating the whole file at once. So far I can manipulate roughly 1/4th of the points at a time in MSaccess but it is limiting out on database size (2Gb max) as well. The entire dataset (which is currently split into 4 text files) is roughly 244 million points.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited February 2011
    It does seem like you are running out of memory. What you could try is loading the XYZ file as point-only, then right-click on it in the Control Center and select the option to generate an elevation grid. On the dialog that appears, use the Gridding tab to break the operation into smaller pieces, say a 4x4 grid, and see if that completes a lot faster.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    Turns out some of the settings I was using were causing problems gridding these data:
    I tried to specify a 2m grid (data are in UTM meters) - bad idea.
    Also turned on "grid to boundaries not just convex hull" (can't remember the exact wording of this setting).

    Allowing GM to create its own optimum grid size and gridding the entire area seems to have fixed the problems I was having with these XYZ files. I suspect its because some of these files have discontinuous "patches" of points and GM was struggling to fill in the gaps. I still need to try to grid the entire dataset which will probably take a long time but not crashing is good.
Sign In or Register to comment.