Global Mapper v25.0

Problems changing ECW

STH
STH Global Mapper UserTrusted User
edited May 2009 in Technical Support
I have done a batch/reproject of a TIFF file (no coordinate system or datum loaded) to ECW- file.

Now I have got the TAB- file associated with it. When I load the original TIFF and the TAB file it displays correctly. However when I load the ECW and the TAB file it displays in the wrong place (near 0,0).

If I export the TIFF file with the TAB file loaded the exported ECW is correctly located and rotated. However I do not want the rotated image. I want the ECW to be without any black areas around, and not rotated - but I want the accompanying ERS/EWW/PRJ/TAB - whatever to describe the rotation so I do not have to regenerate the ECW- file itself.

How can I do this?

When I check the header of the ECW it says_
Datum: WGS84
Projection: GEODETIC
Units: Degrees
Y Coordinates Increase: Upward
Top left X: 0
Top Left Y: 3.14166666667
Cell width: 0.000277777778
Cell height: 0.000277777778

Since there is no information on the Datum and Projection on export I guess GlobalMapper is using the "Fake coordinates option?"
«1

Comments

  • global_mapper
    global_mapper Administrator
    edited April 2009
    It looks like your original TIFF file perhaps didn't have any coordinate information, so the default faked information is used, which is basically one arc second per pixel at the location (0,0), which corresponds to what your file shows. Does your TIFF file have any information with it that should correct place it? The TAB file should be used unless there is something else, like perhaps embedded coordinates, that are overridding the TAB file.

    For rotation, there is not any way to generate an un-rotated ECW that has the rotation information in metadata. You can export an ECW file with the background pixels marked as transparent though, so you basically get to have your cake (a correctly positioned file) and eat it to (background pixels automatically marked as transparent).

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    The TIFF file is clean, IE I do not want any coordinates, datums etc. The ECW should be the same. The reason is I need to do the conversion now - but the coordinates and datum etc I get later from another software and then the idea was just to add a TAB/EWW/TFW/ERS file to the TIFF/ECW and it would display correctly. The TIFF- file does display correctly with the TAB- file, however it does not work on the ECW.

    Changing
    Datum: WGS84 to RAW
    Projection: GEODETIC to RAW
    Top left X: 0
    Top Left Y: 3.14166666667 to 0
    Cell width: 0.000277777778 to 1
    Cell height: 0.000277777778 to 1

    Gives the ECW almost at the correct postition, however it is rotated 90 degrees wrong and the rotation is not taken into account.
  • global_mapper
    global_mapper Administrator
    edited April 2009
    Since the automatic fake coordinate system assigned really doesn't work for you since you don't want those values, try creating a TFW world file for your TIFF that looks like the following:

    1.0
    0.0
    0.0
    -1.0
    0.0
    0.0

    This should cause the TIFF file to load at 0,0 with pixel size of 1 and assuming you don't reproject or anything should export the same way.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    I tried this. Then I selected UTM32 WGS84 when asked. The output is the following:

    Datum: WGS84
    Projection: NUTM32
    Units: Meters
    Y Coordinates Increase: Upward
    Top left X: -0.5
    Top Left Y: 0.5
    Cell width: 1
    Cell height: 1

    However when I load it in GlobalMapper it does not get positioned according to the TAB- file - but according to the header of the ECW

    Can I send you an example file on the FTP to have a look at it?
  • global_mapper
    global_mapper Administrator
    edited April 2009
    You can FTP the files to ftp.globalmapper.com with a username of 'upload' and a password of 'upload'. Note that if your ECW file contains embedded position/projection information that will be used rather than any information from external files like .TAB files. External files are only used if sufficient information could not be extracted directly from the ECW file.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    Finished uploading the files in 5 minutes.

    I have also tried to remove the Datum and Projection. However when GlobalMapper then tries to load the ECW it goes very slow and after a while (5 minutes) it gives timeout.

    Also there is no way to remove the top x / y coordinates from an ECW- file. And you can not set any rotation- parameters in the ECW- header itself.

    For your information (if you do not already know about it) you can look at ECW Spy (free) to have a look at an ECW- file: Digital Earth Weblog: Downloads

    Also you can download the ECW Header Editor from ermappers homepage - its also free (you just need to register and verify by e-mail) -but can only update the header of one ECW at a time - no batch- mode.
  • global_mapper
    global_mapper Administrator
    edited April 2009
    Sorry, I told you wrong on the TFW file coordinates as I forgot they were cell-center coordinates. Your TFW file should look like:

    1
    0
    0
    -1
    0.5
    0.5

    This should then generate a "pixel-space" ECW file that will then look for external files for positioning.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    I tried also this - but it will still position the ECW file with the top left corner at the coordinates 0,0.

    Now the information written in the ECW is:
    Datum: WGS84
    Projection: NUTM32
    Units: Metres
    Y Coordinates Increase: Upward
    Top left X: 0
    Top Left Y: 1
    Cell width: 1
    Cell height: 1

    If you look at the 3_NOT_desired_output in GlobalMapper you can see the correct location of the image - this is the place where I want the ECW-file to end up also
  • global_mapper
    global_mapper Administrator
    edited April 2009
    I've done some more tests and found that the TFW file still wasn't correct. Use the following:

    1
    0
    0
    -1
    0.5
    -0.5

    In addition, when you select the projection you will have to select one that the ECW format doesn't know about, otherwise the external file still won't be used. I would choose some strange projection like Aitoff-Wagner with a oddball datum like an Interplanetary one or a custom one, that way the ECW file won't be able to directly store them. I think this will finally get you what you are after.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    I have now done as suggested in the previous post. When trying to open the new file and selecting the proper coordinate system I get the error message as before: "
    Unknown error while rendering overlays, you should probably save and quit. Timeout waiting for data.ECWOverlay.cpp - 3220. Build time Feb 16 2009 19:38:02"

    I got hold of an example file that actually works in GlobalMapper in the right position. However I can not see the difference between this file and the ones created with GlobalMapper - perhaps you can see what the difference is?

    Anyway - for the file that can be opened (the one I sent you on ftp now) it takes roughly 20 seconds to open one file on a new 64-bit computer. On my older computer it goes into timeout before it can open the file. (The same Unknown error while rendering overlays...") Perhaps it is the size of the file that is the problem? Or the DLL that can not read and translate the file suitably?

    Now the information written in the ECW is:
    Datum:
    Projection: LOCAL
    Units: Meters
    Y Coordinates Increase: Upward
    Top left X: 0
    Top Left Y: 0
    Cell width: 1
    Cell height: 1

    Opening the file in ECW-spy gives the difference between the two files:

    Working file - GlobalMapperfile
    Datum: RAW - *empty*
    Projection: RAW - LOCAL
    BRX: positive - positive
    BRY: positive - negative

    ECW file size: 9.69 MB - 31.98 MB
    Original file size: 303.75 MB - 560,12 MB
    Compression details target ratio: 10:1 - 10:1
    Compression details actual ratio: 31.34:1 - 17.52:1

    UPDATE: I have created a file which is 1/2 the size of the original file and I managed to open this ONCE in GlobalMapper. However trying to open it the next time failed with the same error. I do not have any other software for the moment to test if these ECW- files are being properly rotated in other software so I do not know if they are valid or if this is a GlobalMapper-issue.

    UPDATE2: After roughly 15-20 minutes the file is opened in GlobalMapper - so I guess it is a "bug" or something in GlobalMapper due to the large filesize? The only thing I have changed is that I now have 3 points instead of 4 points in the TAB-file. I have uploaded the working file with the correct coordinates for you to try on your ftp. See if you can open the file also and see if you have the same behaviour as me. Coordsyst: UTM32 WGS84. I notice GlobalMapper using 100% of one CPU and roughly 1000KB/s Disk I/O. I am guessing that GlobalMapper first reads the whole ECW- translates all pixels into rotated system and then displays the data. Isn`t ECW using layers so you can only read the "suitable" layer instead of doing this on all the pixels? Or am I wrong about the ECW- format? GlobalMapper does not have any problems opening ECW- files which are NOT rotated (have opened several GB large ECWs without any problem)
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    Quick update again.

    I have now tried to open the ECW file in MapInfo and it seems to load after adjusting some parameters. It also loads very quickly independent of the zoom of the ECW - so it manages to load the different layers of the ECW correct. I suspect that GlobalMapper is not capable of loading the different ECW- layers and does all the computations on the high-res layer - and therefore it takes 20-30 minutes to load this particular ECW file. (In comparison when turning on transparent background on tiff files then GlobalMapper did not support to do this operation on overviews - however when you implemented this it loaded very quickly - see post http://www.globalmapperforum.com/forums/suggestion-box/382-transparent-background-overview-not-real-image.html)
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    Update again:

    I have now got the files to load in MapInfo (they load fast there, but very slow in GlobalMapper - will tell you about that later)

    Now I have another issue:

    The images I have are orthophotos. 2 lines - one line is flown in the direction NORTH, while the other is flown in the direction SOUTH. So when using the dummy.tfw-file they should be different to compensate for this.

    I have tried to open one of the files with the dummy.tfw - then rerectified this so it is flipped 180 degrees and then saved the tfw-file however it still does not get flipped. I suspect that the given coordinates in the TFW-file are for the top left corner. Solution:

    1. Batch convert all my TIF-files in some software to flip them 180degrees
    2. GlobalMapper can learn to read the TAG Orientation within the TIFF-file. This way I can just change the orientation parameter in the TIFF-file on the images I want and use the same dummy.tfw on all the images - is this possible to do quickly? Here is a print of the header of the TIF-file:

    Output from Display Header
    File Name: X
    File Information:
    Standard : : TIFF File
    Format : : JPEG Compressed 24 bit RGB data, Q= 3
    Pixels per Line : 17310
    Number of Lines : 11310
    Samples per pixel : 3
    File bits per sample : 8
    Actual bits per sample : 8
    Tiled file, tile dimension : 256
    Number of tiles along a row : 68
    Number of tile rows : 45
    Number of overviews : 10
    Overview sampling method: Averaged
    Scanning device resolution : 0 : None Specified
    Orientation : 4 : Row major order, origin at top left
    NO scan line headers : non-scannable file
    Packet size (16-bit words) : 0
    Free vlt space (16-bit words) : 2000000000
    Free packet space (16-bit words) : 2000000000
    Raster to UOR matrix:
    Unspecified or All Zero Matrix
    Raster to World Matrix:
    Units: Unknown or Unspecified
    amx[ 0]= 1, amx[ 1]= 0, amx[ 2]= 0
    amx[ 3]= 0, amx[ 4]= -1, amx[ 5]= 0
    0 , 0
    17310 , 0
    17310 , -11310
    0 , -11310
    No GeoTIFF info found

    The other flight-direction will have the following header-TIF:
    Output from Display Header
    File Name: X
    File Information:
    Standard : : TIFF File
    Format : : JPEG Compressed 24 bit RGB data, Q= 3
    Pixels per Line : 17310
    Number of Lines : 11310
    Samples per pixel : 3
    File bits per sample : 8
    Actual bits per sample : 8
    Tiled file, tile dimension : 256
    Number of tiles along a row : 68
    Number of tile rows : 45
    Number of overviews : 10
    Overview sampling method: Averaged
    Scanning device resolution : 0 : None Specified
    Orientation : 7 : Row major order, origin at bottom right
    NO scan line headers : non-scannable file
    Packet size (16-bit words) : 0
    Free vlt space (16-bit words) : 2000000000
    Free packet space (16-bit words) : 2000000000
    Raster to UOR matrix:
    Unspecified or All Zero Matrix
    Raster to World Matrix:
    Units: Unknown or Unspecified
    amx[ 0]=0.997969052224148, amx[ 1]= 0, amx[ 2]=-17.5778529999989
    amx[ 3]= 0, amx[ 4]=-1.00179263143236, amx[ 5]=62.8708897500001
    -17.5778529999989 , 62.8708897500001
    17257.266441 , 62.8708897500001
    17257.266441 , -11267.40377175
    -17.5778529999989 , -11267.40377175
    No GeoTIFF info found

    Other software is capable of rotating the image according to this tag so it is not needed to flip the actual image 180 degrees, I hope GlobalMapper can support this also? Thanks for the good support.
  • global_mapper
    global_mapper Administrator
    edited April 2009
    I have never seen that orientation tag used. Is that something in the GeoTIFF file header? If so I can probably support it.

    For the slow ECW draws, Global Mapper will slow down tremendously on large ECW files that have rotation or some other rectification applied. This is because that Global Mapper currently switches to accessing data at the most detailed layer when a non-linear transformation is applied, and also that significant rotation can cause the internal block caching to become inefficient, causing the same blocks to be read repeatedly as a non-standard orientation is used.

    I am planning at some point to update the ECW importer to be able to use the less detailed layers when appropriate even when displaying a rotated or rectified image, which should speed things up dramatically for large ECW files.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    Thanks for the reply.

    The Orientation tag is not from the GeoTIFF but from the TIFF specification (TIFF 6.0 Specification Coverage)

    From the PDF of the specification:
    The orientation of the image with respect to the rows and columns.
    Tag = 274 (112.H)
    Type = SHORT
    N = 1
    1 = The 0th row represents the visual top of the image, and the 0th column represents
    the visual left-hand side.
    2 = The 0th row represents the visual top of the image, and the 0th column represents
    the visual right-hand side.
    3 = The 0th row represents the visual bottom of the image, and the 0th column represents
    the visual right-hand side.
    4 = The 0th row represents the visual bottom of the image, and the 0th column represents
    the visual left-hand side.
    5 = The 0th row represents the visual left-hand side of the image, and the 0th column
    represents the visual top.
    6 = The 0th row represents the visual right-hand side of the image, and the 0th column
    represents the visual top.
    7 = The 0th row represents the visual right-hand side of the image, and the 0th column
    represents the visual bottom.
    TIFF 6.0 Specification Final—June 3, 1992
    37
    8 = The 0th row represents the visual left-hand side of the image, and the 0th column
    represents the visual bottom.
    Default is 1.
    Support for orientations other than 1 is not a Baseline TIFF requirement.

    Here is a post about the tag itself and problems in different viewers:
    Nabble - Tiff / LibTiff - Image Orientation tag and rotating images

    So this is not a new problem - and ofcourse for our use (aerial photography) we have seen this problem long. I have not used GlobalMapper for processing on original images before, thats why I haven`t seen this.

    For our use depending on what is easiest for you to implement the following solutions exist:
    1. Update GM to support the Orientation tag correctly. Then we can just edit this tag easily with our own software to the correct one.
    2. Be able to batch reproject/convert and have an option FLIP image horizontaly AND an option FLIP image vertically
    3. Generate some kind of header file (ERS, TFW, EWW??) That takes this rotation into account and create a dummy-file (like the dummy.tfw I have used) that rotates the image. Do you have any suggestions on a file like this like you had with the dummy.tfw?
    For the slow ECW draws, Global Mapper will slow down tremendously on large ECW files that have rotation or some other rectification applied. This is because that Global Mapper currently switches to accessing data at the most detailed layer when a non-linear transformation is applied, and also that significant rotation can cause the internal block caching to become inefficient, causing the same blocks to be read repeatedly as a non-standard orientation is used.
    This is what i suspected - thanks for verifying it.
    I am planning at some point to update the ECW importer to be able to use the less detailed layers when appropriate even when displaying a rotated or rectified image, which should speed things up dramatically for large ECW files.
    After fixing on some settings on MapInfo I can load the ECW-files quickly - I turned of the reproject option and now they load amazingly fast - I will give you more input on this when I have time. So it will be possible to do this fast as your write.
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    By the way I know this is a "bizarre" issue - however for our industry it is quite a normal problem - and as I advertise for GlobalMapper for the people I meet I am happy to point out all features that makes this software better than other competing software - although I already have 100 things in favour of GM I hope to add even more.
  • global_mapper
    global_mapper Administrator
    edited April 2009
    I'll just add support for the orientation on import, it shouldn't be too difficult.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited April 2009
    Great, I hope it will be possible to implement as fast as always. I will test the new implementation and give you feedback as soon as the new version is available. I have software that can alter this header in a file - so I will be able to test and verify each of these settings. To implement this setting is for me very important. The ECW using overviews are not so important and should be considered future plans - perhaps for the next June- release of GlobalMapper if time permits.

    Good luck with the programming.
  • global_mapper
    global_mapper Administrator
    edited April 2009
    I was able to speed up the ECW display for your file just by increasing the cache size ECWs that will access the most detailed layer. Now display is still sluggish when zoomed all the way, but much better than before and quite snappy when zoomed in at all. I have placed a new build at http://www.globalmapper.com/global_mapper10.zip with the change for you to try. Simply download that file and extract the contents into your existing v10.xx installation folder to give it a try.

    I'll work on the orientation flag now.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • global_mapper
    global_mapper Administrator
    edited April 2009
    Can you provide me with a TIFF file with the orientation flag set so that I can give it a try? The one that you provided earlier does not appear to have that tag.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited May 2009
    Great work, I can atleast now load one of the ECW-images! (Loading one image it uses approx 500MB - so thats pretty accurat for the uncompressed image).

    I have uploaded 5 files to your ftp with different orientation flags. I have also uploaded the software where you can look/edit the TIFF-header (vras and edheader). Not sure if they work as standalone applications or if you need to install the whole package (also supplied - the exe-file)
  • global_mapper
    global_mapper Administrator
    edited May 2009
    I think I have this working now for the orientations that you sent (I didn't implement the column-major order ones yet, but I doubt they are used much). I have placed a new build at http://www.globalmapper.com/global_mapper10.zip with the change for you to try. Simply download that file and extract the contents into your existing v10.xx installation folder to give it a try.

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited May 2009
    Hi!

    I have now tried the new release with the files sent to you. However I can not manage to see them rotated in GlobalMapper - am I doing something wrong? Do you seem them correct?
  • global_mapper
    global_mapper Administrator
    edited May 2009
    Yes, they flip fine for me. The top-right file flips horizontally, the bottom-right file flips verticallyl and horizontally, and the bottom-left file flips vertically. You don't see any of that? Can you check the build date on the Help->About dialog and make sure you didn't get a cached copy or something?

    Thanks,

    Mike Childs
    Global Mapper Software LLC
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited May 2009
    Build time: May 2 2009 20:35:55

    Do I need to add a dummy world-file or something?
  • global_mapper
    global_mapper Administrator
    edited May 2009
    Yeah, I was using a dummy world file for my tests.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited May 2009
    I have tried the files I sent you - orientation 0,4,5,6,7 - and created a dummy.tfw for each (with the respecting filename) containing the following:
    1
    0
    0
    -1
    0.5
    -0.5

    When opening I have tried the default suggestion GeograLatLon/WGS84 and UTM32/WGS84. However I still can not see the difference.
  • global_mapper
    global_mapper Administrator
    edited May 2009
    It looks like I messed up the build on the web site. I have placed a new build of Global Mapper 10 that should work at http://www.globalmapper.com/global_mapper10.zip . Simply download that file and extract the contents over your existing v10.xx installation to give it a try.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com
  • STH
    STH Global Mapper User Trusted User
    edited May 2009
    Now it works like a charm! Thanks a lot once again for the good support (well, actually it is not support - it is hard development for a user of your software!)

    UPDATE: Of course I found some files with orientation 1 and 2 also - and these does not work. I guess I have to ask you to kindly add support for all of the orientation- parameters - I hope it is not too much of a hassle this last update.
  • STH
    STH Global Mapper User Trusted User
    edited May 2009
    UPDATE: Of course I found some files with orientation 1 and 2 also - and these does not work. I guess I have to ask you to kindly add support for all of the orientation- parameters - I hope it is not too much of a hassle this last update.
  • global_mapper
    global_mapper Administrator
    edited May 2009
    Can you provide samples of the other orientations? They are a bit more complex, but shouldn't be hard to add with some samples.

    Thanks,

    Mike
    Global Mapper Support
    support@globalmapper.com