Global Mapper v25.0

Export error during script run

the4thworld
the4thworld Global Mapper UserTrusted User
edited September 2013 in Technical Support
I have a script that creates gridded LAS files out of point features and attaches the RGB values from underlying imagery. These are fairly large data sets (15-20 million points) and for the most part the script works fine. Every so often during the export process my script log captures this error:
WARNING: Error writing Lidar LAS file
<output path>
Timeout waiting for data from
<path to an ECW image>
ECWSDKBridgePlugin.h - 171
Version: v14.2.2 (64-bit)
Build Time: Jul 7 2013 17:55:41

What's strange is that it doesn't export anything for that grid, but it's not a corruption issue with the ECW as the previous and next grids usually export without issue. Any ideas on what I can do to fix the issue?

Thank you,
Matt

Comments

  • global_mapper
    global_mapper Administrator
    edited September 2013
    Matt,

    Actually that does sound like there may be a piece of the ECW that's corrupt and when that piece is accessed the error occurs. However Global Mapper can actually use 2 different versions of the ECW read library as they each have their own issues. To try the old version you can go to the Global Mapper install folder and rename the ECWSDKBridge.dll file to something else, then try the export again and see if you get an error.

    Alternatively to narrow down the bad piece of the image you could export the ECW to another format, like JPEG2000 or perhaps GeoTIFF. If there is corruption in the ECW I would expect an error in the export when it reaches the part of the file that the ECW library can't handle. Typically it is just one bad tile at the highest zoom level in the ECW file so you wouldn't see a problem with draw unless you zoomed in to that particular spot.

    Thanks,

    Mike
    Global Mapper Guru
    geohelp@bluemarblegeo.com
    Blue Marble Geographics for Coordinate Conversion, Image Reprojection and Vector Translation
  • the4thworld
    the4thworld Global Mapper User Trusted User
    edited September 2013
    Mike,
    Thank you for the response; I will export the data to a different format and see if it has the error, but I'm still not thinking it's a corruption error.

    We had this error 11 or so different images during processing. We had no errors with smaller point datasets in the same area and the same images using the same script (the only difference was a 4 character length difference in the file path). The smaller point datasets were about 100MB-200MB versus the one that is having issues at about 1.4GB. Is it possible that a out of memory error is manifesting as this ECW error?

    Thank you,
    Matt
  • global_mapper
    global_mapper Administrator
    edited September 2013
    Matt,

    I suppose it's possible you are out of memory, but 15-20 million Lidar points really should take that much memory in v14.2. Probably between 60MB and 200MB , depending on how well it compresses, so you shouldn't be anywhere close to out of memory. Unfortunately it's hard to tell what the problem with ECW is since it's a 3rd-party library that fails and not something we have visibility into.

    Let me know if it fails or doesn't with the other imagery format.

    Thanks,

    Mike
    Global Mapper Guru
    geohelp@bluemarblegeo.com
    Blue Marble Geographics for Coordinate Conversion, Image Reprojection and Vector Translation
  • the4thworld
    the4thworld Global Mapper User Trusted User
    edited September 2013
    Mike,
    Just to clarify; the data that's being converted to LAS is the 15-20 million points from PLS-XYZ format which seems to take up more memory. We bring in the XYZ points, convert the feature coded values to a LAS classification and then export to LAS. This seems to be more memory intensive, especially when you have to loop over all 15-20 million points several times to address multiple feature codes. The points are then exported in grids to the LAS files with the RGB option enabled.

    Thank you,
    Matt
  • global_mapper
    global_mapper Administrator
    edited September 2013
    Matt,

    Oh yes that would take a ton more memory as they look just like points rather than a Lidar point cloud. If you export to LAS first without colors, then load that back in and export again, now with colors (after unloading the XYZ too of course) that would dramatically reduce your memory usage from the point cloud (by a factor of about 10).

    Thanks,

    Mike
    Global Mapper Guru
    geohelp@bluemarblegeo.com
    Blue Marble Geographics for Coordinate Conversion, Image Reprojection and Vector Translation
  • the4thworld
    the4thworld Global Mapper User Trusted User
    edited September 2013
    Mike,
    That might be the issue then; I had to let the export of the ECWs to GeoTIFF run over the weekend but it compelted without any errors, so I don't think it's an image issue. However, I think coverting to LAS then unloading then reloading LAS with imagery for the RGB export may be the easiest solution based on the data and the nature of the data I have. I will try that.

    Thank you,
    Matt