Global Mapper Pro Coming Soon

JPEG2000 images not processing

partjohnnerpartjohnner Global Mapper UserPosts: 17Trusted User
edited October 2010 in Bug Report
I get the following error when trying to create JPG files from certain JPEG2000 images:
Unable to generate JPG file.
Timeout waiting for data from D:\_Workspace\AirPhotos\...\ortho_imagery\ortho_1-1_1m_j_tn115_2008_4.jp2
ECWOverlay.cpp - 3648
Version: v11.02
Build Time: Jun 5 2010 19:21:17
I've been told that this has to do with the JPEG2000 library that Global Mapper uses, and that it was being addressed as of August 10, 2010.

Comments

  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited August 2010
    Yes, we are still waiting for an updated JPEG2000 library from our vendor. They were supposed to get it to us last week, but they haven't provided anything yet. Hopefully it will be available this week.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited August 2010
    My concern on this front come from the fact that we have found a large data resource of JPEG2000 images from the USDA, the Geospatial Data Gateway (http://datagateway.nrcs.usda.gov/GDGOrder.aspx). This resource seems to have recent 1-meter coverage for nearly the entire U.S., and we have become to rely upon this source for free imagery to our viewer. "Free" for us, but billable work for processing time to our clients. Unfortunately I am 3 projects behind because I have acquired this JPEG2000 data and cannot process it into our JPEG tiled grid.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited August 2010
    Our library vendor indicated yesterday that they should have the read/write version of their SDK (which is what we need) available on September 7th. Hopefully we can integrate that week and this will fix the JPEG2000 issues.

    Note that Global Mapper has built-in access to NAIP imagery via the File->Download Online Imagery/Data menu command.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited September 2010
    Is this problem fixed in GM version 12?
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited September 2010
    Unfortunately we finally got the new SDK from our vendor to try early this week and it still has the same issues with some JPEG2000 files. They have been provided the files again so hopefully they will get them working soon, since they don't get our money for an upgrade until they get the problems fixed.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited September 2010
    Mike:
    Thanks for the lightning fast reply!

    Will you be fixing this in version 11? Or does that mean that you want to address the issue in version 12? I know I will eventually buy version 12, but I don't really need it yet (except for this issue if that's where the fix will live).

    Thanks!
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited September 2010
    This will likely be a v12 fix as it would be cumbersome to back-port the new ECW/JPEG2000 library to v11 once we finally get a working one.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited October 2010
    Checking in on this bug, still very relevant for me. How is the fix coming?
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited October 2010
    Supposedly our library vendor is working on it and has the problem data that they need, so hopefully soon!

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited October 2010
    As a test for a work-around to this problem, I was able to use the LizardTech GeoViewer to convert a few of my JPEG2000 files to JPG files. I could not use GeoViewer to convert them to TIF files due to the 4 GB TIF size limit. I don't think GeoViewer knows about "bigTiff" yet ...

    Anyway, I saved the JPG files to full resolution, and 85 percent JPG quality. A test view in GM looked satisfactory; but, after loading a second JPG file into GM, I ran into the error "Not enough disk space for temporary file in folder: C:\Users\<me>\AppData\Local\Temp\GlobalMapper\".

    Is there any way I can configure GM to change the path of this temporary folder so that it uses my D:\ drive (that has much more hard drive space)?
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited October 2010
    Yes, if you create a string value in your registry (using regedit) named 'HKEY_CURRENT_USER\Software\Global Mapper\TempDir' with the value being the desired temp file path then you can change the temporary folder location used by Global Mapper.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited October 2010
    I was successfully able to open the 400 MB JPG file in GM -- it took about 20 minutes for the image to load. Adding a second JPG of this size really seems to do a number on the software. I am noticing that the tempDir/ is keeping several files (e.g., "MAP12AB.tmp", 4.2 GB), and I am wondering if these files are supposed to go away after a session has loaded or been closed. They do not appear to go away in such case.

    My work-around test (converting JPEG2000 to JPEG) seems to not be successful. Has anyone seen a better converter for a JPEG2000 other than GeoViewer or MrSIDDecode, both developed by LizardTech? Neither seems able to create a MrSID file or the BigTiff image format of over 4 GB; I have no problems with either format in Global Mapper. I really do need a workable solution to this problem, so that I can bring several counties-worth of imagery (originally JPEG2000) into a single session to create the needed image tiling.
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited October 2010
    The JPG format is bad for large images in that JPG images have to be completely decompressed before accessing. For large JPG images the data is decompressed to a temporary file on disk and read from there. Those temporary files should go away when the layer is closed or you shut down Global Mapper though, or worst case the next time you start up Global Mapper. Are they sticking around beyond that for you?

    What you might do is batch-convert those large JPG images (I would export from LIzardTech with very high quality) to GeoTIFF files using JPG-in-TIFF compression using Global Mapper, then use those BigTIFF files generated by Global Mapper as they can be accessed directly without decompressing the whole thing.

    And hopefully soon there will be a fix for the JPEG2000 problems in the library we use. I don't know what is taking so long, we try and always fix bugs within 1-2 days at worst, and it's been months since I initially reported these issues.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • partjohnnerpartjohnner Global Mapper User Posts: 17Trusted User
    edited October 2010
    I am exporting GeoTIFF files with JPG-in-TIFF compression from JPEG2000 (one at a time) directly in Global Mapper, as per your suggestion. I think this duct tape will hold for a little while!
  • global_mapperglobal_mapper Administrator Posts: 17,238
    edited October 2010
    We finally received a response from our ECW/JPEG2000 library vendor regarding these JPEG2000 files that cause problems. Basically the problem is corrupt sections in the JPEG2000 files that the USDA is distributing, but they can update their library to do something a little less drastic for those problem areas. Here is their response:

    "We have tested this images in multiple application software and SDKs. ERDAS always test using multiple packages. The image fails in a number of packages based which are based on the ECW SDK, and non-ECW SDKs. ArcGIS 9.3.1 and ERDAS IMAGINE 9.2 were two of the products we tested. Both displayed the complete non-standard JPEG2000 image. Both have some display artifacts in the corrupted blocks which start at about ¼ down on the image, but the image does complete the display process. Both products use older Kakadu versions, the SDK credited with creating the image.

    As ERDAS researched this issue we decided to add an enhancement to the SDK to plow through the corrupted blocks on non-standard images whenever possible and display something that makes since visually. It may be a lower resolution, or something like that, but display something. Almost as important, report to the user the JPEG2000 image is non-standard so they know if the vendor is delivering what they paid them to deliver.

    Once we complete the ERDAS 2010 release (of which the ECW SDK v4.1 is a part) , ERDAS will estimate this enhancement to determine when we can deliver this capability to our customers. The JIRA database tracking number is EMS-351.

    Getting the US government to re-create the data will be problematic. While they have large numbers of ERDAS IMAGINE software licenses in their image production shops and can batch out many images very fast with those our multi-core and distributed processing capabilities, the issue is more problematic than simply remaking the images. My guess is, USDA long ago deleted the raw images, thus they only have the corrupted data. USDA can create new images without this problem, but the affected data blocks have some level of compromised quality, no matter what they do."

    So basically it sounds like an upcoming ECW/JPEG2000 SDK release will be able to handle these problem images better, but the real problem is that the JPEG2000 images created by the USDA are corrupt in some way. They are notifying the USDA of the problem, but as they mentioned the USDA likely no longer has the raw data available even if they were willing to recreate the JPEG2000 images without the corruption.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
Sign In or Register to comment.