› ›

Huge XYZ-File - How to open?

pnxpnx Global Mapper UserPosts: 2
edited June 2013 in Elevation Data
Hi there,

I have three xyz-files (two of them with 1 GB and one with 700 MB) I want to open in Global Mapper, but every time I tried there was an Error "not enough memory". The files cotain the result of an Laserscan (resolution 1 meter), projection is Gauss Krueger Zone 3.

My goal is, to cut a area within the data and export it to a smaller file.

So far I tried "Open ASCII Text File Import" with
- Elevation Grid from ED Point Data
- X First
- No Coord-Prefix
- Delimiter: Auto-Detect (or Space or tab, or tab)
- Treat 3rd coordinate value as elevation

Maybe there is option to reduce the data befor importing ...
I tried to open the files with common texteditors to split them up,
but the data is to huge for them.

Some ideas?

regards,

pnx

Comments

  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited May 2008
    Try opening the data file with the "Point Only" option rather than the "Elevation Grid" option as that will require much less memory. You can then export a subset of the loaded point data to a new file using the File->Export Vector->Export Simple ASCII menu command.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • pnxpnx Global Mapper User Posts: 2
    edited May 2008
    Thx for that solution .. I will try soon.

    At the moment I use Ultraedit to split the files.
    The original Files have about 30.000.000 lines and
    I made 3 files with 10.000.000 lines each.
    (Select rows and Save selected rows as ...)

    With this smaller files (about 400 MB) Global Mapper works fine

    pnx
  • FranckiFrancki Global Mapper User Posts: 67Trusted User
    edited October 2010
    Hi,
    i have the same problem with 4 Gb file. I'm working with global mapper V12.
    Is GM could open a subset area instead of open the entire file ?

    Thank you for your answer
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited October 2010
    Do you have 64-bit Windows? If so you could run the 64-bit Global Mapper v12 and likely pull the whole thing in without any problems. There is not currently any support for subsetting the data on import to Global Mapper, you can only subset after you get the entire data set into Global Mapper.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • FranckiFrancki Global Mapper User Posts: 67Trusted User
    edited October 2010
    I run GM on Wxp 32 bits.
    Thank you anyway. I found an another solution.
    Anyway, congratulations for your software.
    Regards:)
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    I am running the 64bit version in Windows7 with 4Gb of RAM and while its seems like I can open a 1.7Gb XYZ point file I can't do much with it at all. Just attempting to save the workspace takes >1hr. Ultimately I would like to convert the points into a grid but its slower than slow attempting this. Is this typical performance or is there some setting which I can use to speed things up? I don't need to see the point data graphically for instance, but I'm sure drawing it is taking up vast quantities of resources. Just turning the point layer off after loading takes ~20mins.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited February 2011
    I would suggest gridding the data on import rather than pulling it in as point data. Once the gridding process is complete, you could export out to a gridded format, like a Global Mapper Grid file, that you can then load back in. That will take a lot less memory and be very fast to work with after the slow initial conversion process.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    If I have 4 txt files is there a setting which allows me to load them as 1 grid instead of 4 separate grids? (I think this is what's happening).

    I have upgraded to 8Gb of RAM and I can barely load all 4 files as points. If I could skip the points this might just work. But I don't have a text editor big enough to make 1 text file.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited February 2011
    The method to grid them all at once is to load the 4 separate files as "point only" then select those layers in the Control Center and grid from that. The points will have to be read into memory to grid them.

    What you can do after loading all 4 and starting the Gridding process is to split the gridding operation into multiple pieces using the Gridding tab when doing the grid operation. You could do say a 4x4 grid of smaller gridding operations to break things up and generate results much faster. You could also check the option to export the results directly to Global Mapper Grid files, then load those when the operation completes to reduce the required memory utilization.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    Thanks sounds like I have to actually draw all points in order to have them loaded.

    I am looking into making 1 XYZ file using split databases in MSAccess. That would avoid having to draw the points and I could then grid direct from the file. Unfortunately Access has a 2Gb database limit and these 4 files collectively add up to 6.5Gb or so. Otherwise, if I have to, I will load the 4 files and grid sections. I think I can do that, its just right on the edge of capability.

    FYI, There's supposedly 244,000,000 ASCII points. I'm running 64bit Win7, 2.3Ghz processor and 8Gb Ram. When USGS originally processed these data 10yrs ago, even in the native binary format, I'm pretty sure it wasn't on a laptop ;)
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited February 2011
    You do have to load them all, but if you don't want to watch them all draw you can hit the Escape key to abort the draw.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • rjackrjack Global Mapper User Posts: 8
    edited February 2011
    Excellent "escape" will help considerably!
  • lairadejaqkelairadejaqke Global Mapper User Posts: 2
    edited June 2013
    Hi, I'm new to the forum. I would like to know how I can make a map gives solar radiation. I have radiation data (z), length (y) and latitude (x). How I can load that data from excel?

    The excel data are presented as follows:
    Lat Long Radiation

    -56 -75 5,13
    -56 -74 5,03
    -56 -73 5,03
    -56 -72 4,7

    I hope you can help me
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited June 2013
    You should be able to load that using the File->Open Generic ASCII Text File menu command. Specify that the Y/latitude column is first and that the 3rd value is elevation. Also be sure to specify the delimiter as 'Space or Tab' so that the comma in the Radiation value is treated as a decimal and not a field delimiter. You can load as 'Point Only' or 'Lidar' to get 3D points, or just directly create an "elevation" grid from the Radiation values.

    Thanks,

    Mike
    Global Mapper Guru
    geohelp@bluemarblegeo.com
    Blue Marble Geographics for Coordinate Conversion, Image Reprojection and Vector Translation
  • lairadejaqkelairadejaqke Global Mapper User Posts: 2
    edited June 2013
    Thanks!! for your help! I could do it!

    global_mapper wrote: »
    You should be able to load that using the File->Open Generic ASCII Text File menu command. Specify that the Y/latitude column is first and that the 3rd value is elevation. Also be sure to specify the delimiter as 'Space or Tab' so that the comma in the Radiation value is treated as a decimal and not a field delimiter. You can load as 'Point Only' or 'Lidar' to get 3D points, or just directly create an "elevation" grid from the Radiation values.

    Thanks,

    Mike
    Global Mapper Guru
    geohelp@bluemarblegeo.com
    Blue Marble Geographics for Coordinate Conversion, Image Reprojection and Vector Translation
Sign In or Register to comment.