Mosaicking Failure

[jmwheatley]

Ah Mike... I'm sure your tired of seeing my name pop up with my ECW mosaicking woes. Here I am again.

I'm having some major issues getting these to run. Just to give you background on what I'm trying to accomplish. I'm am trying to create an ECW mosaic for each of the 28 counties in Maryland, USA. They were delivered to us in SID format and we are trying to "simplify" the tiles and make these mosaics available to us internally, and potentially the state. We have had great successes in the past converting TIFF and SID tiles into mosaics with Global Mapper, but I'm running into many issues with the new imagery.

This imagery is 6"pixel and the tile count per county ranges from 1,500 - to 5,000 plus. I have attempted to run 4 counties now (smaller ones) and it keeps locking up at a particular percentage (different for each one). The machine I'm running them on is not the most powerful fella (2.4Ghz pentium, 2GB RAM) so I'm not sure if that is the problem or what.

For example I have attempted to run a county containing 2,336 tiles 3 tiles and each time it will freeze at 19%. I've let it run for 2 days and the status never changes. When I look at my process tree, Global Mapper is grinding along the whole time. But I'm assuming nothing is happening and its locked up since no progress is being made. I kill the process and try again.

My next attempt may be to try and break the mosaic up into sections. Run each section and then mosaic the sections together at the end. Thoughts on this? Would you recommend adding the compression to the sections (20:1) or run them with no compression and then add compression to the final section mosaic.

Thanks for your help Mike. We love the product. I think were putting it and our hardware to the test with these mosaicking tasks.

Jason

[STH]

Thought I would share my advice, but I`m sure you might be aware most of them:

1. Use a Map Catalogue containing all the images
2. When exporting to ECW it will create an image as large as the bounding box of all the images - so try to see if the bounding box is very large. Remeber ECW will also try to "compress" all lines outside your actual images inside the bounding box. You can get a very large bounding box in atleast two cases:
2A: for instance if you have two areas far away from each other.
2B: If one or more images are incorrectly georeferenced (meaning one image will end up in coordinatespace 0,0 while the rest will be at the correct place - thus creating a bounding box from 0,0 to 500000, 500000 or whatever your coordinates are)
3. GlobalMapper for the moment only use one core for processing, ER Mapper which costs more money will use all available processors decreasing the time
4. Networks are unstable - so my advice is to have the files available locally or on a USB/firewire disk so they are always available - I have experienced conversion stopping after 2 days just because the network was down for 1 sec - NEVER again !
5. ECW files generate temp-files in the same catalogue as the output - so I suggest the output file to be on a local or USB/firewire (not the same as the input files) for fastest times
6. Multiple instances running once. I have experienced problems running two instances of GlobalMapper on the SAME dataset but I guess you are running each instance of GlobalMapper on completely different files? (ie no overlapping etc..).
7. Turn off anti-aliasing/fill small gaps to speed up processing.
8. Perhaps you can try a test with smaller resolution first, for instance 50m, just to see if it manages to create "something" and that there is nothing wrong with the input files?
9. Running multiple instances eat RAM - remember the limitation on Win 32 bit and your 2GB of RAM. Try to start only one instance of GlobalMapper and do one county and if it "hangs" for a long time , check the amount of used RAM and the Windows SWAP file to see if it is swapping to disk all the time or not.

Sorry for the long reply, however I did not have time to shorten it Let us know how it goes.

GlobalMapper: Will the new v11 be able to use more than 4GB ram and more CPUs when creating ECWs or is the number of CPUs a limitation on the ECW-dll itself?

[global_mapper]

Thanks for providing the support and saving me some time!

The upcoming v11 will have a native 64-bit version. In fact, I would suggest maybe downloading that from GIS Software Download - Topographic Mapping Software - Mapping Software With GPS and giving it a try. Just email me at support@globalmapper.com for a demo license key.

Thanks,

Mike
Global Mapper Support
support@globalmapper.com

[STH]

Always trying to help as best I can as I know you are busy with programming new features and updates of GlobalMapper. Just worried that I might give "wrong" answers - however I watch the forum closely so I try to give as good help as possible.

Will try the 64bit version tomorrow at work and experience with it trying to find some bugs before the release.

[jmwheatley]

I'm just writing within your post below...

Originally Posted by STH

Thought I would share my advice, but I`m sure you might be aware most of them:

1. Use a Map Catalogue containing all the images
I've tried that per Mike's suggestion in a previous post. That did save some, but it still is not enough to kick out these large datasets.
2. When exporting to ECW it will create an image as large as the bounding box of all the images - so try to see if the bounding box is very large. Remeber ECW will also try to "compress" all lines outside your actual images inside the bounding box. You can get a very large bounding box in atleast two cases:
2A: for instance if you have two areas far away from each other.
2B: If one or more images are incorrectly georeferenced (meaning one image will end up in coordinatespace 0,0 while the rest will be at the correct place - thus creating a bounding box from 0,0 to 500000, 500000 or whatever your coordinates are)
Bounding Box is not an issue. Most of the counties are fairly "square" so there is not a lot of "empty" space in the bounding coordinates. All files are in proper coordinate systems.
3. GlobalMapper for the moment only use one core for processing, ER Mapper which costs more money will use all available processors decreasing the time
4. Networks are unstable - so my advice is to have the files available locally or on a USB/firewire disk so they are always available - I have experienced conversion stopping after 2 days just because the network was down for 1 sec - NEVER again !
LOL! I feel ya there. I learned very quickly to work locally. I've been running these locally.
5. ECW files generate temp-files in the same catalogue as the output - so I suggest the output file to be on a local or USB/firewire (not the same as the input files) for fastest times
6. Multiple instances running once. I have experienced problems running two instances of GlobalMapper on the SAME dataset but I guess you are running each instance of GlobalMapper on completely different files? (ie no overlapping etc..).
I've only run one instance of GM at a time. I fear this tower of mine would really choke if I tried to run any more than one.
7. Turn off anti-aliasing/fill small gaps to speed up processing.
I'll try that!, Thanks.
8. Perhaps you can try a test with smaller resolution first, for instance 50m, just to see if it manages to create "something" and that there is nothing wrong with the input files?
That is almost what I'm doing now. See below. I've also been running older imagery that we have with no problems. Its only 10" pixel resolution and has half to maybe even a quarter of the tiles. Those go well.
9. Running multiple instances eat RAM - remember the limitation on Win 32 bit and your 2GB of RAM. Try to start only one instance of GlobalMapper and do one county and if it "hangs" for a long time , check the amount of used RAM and the Windows SWAP file to see if it is swapping to disk all the time or not.

Sorry for the long reply, however I did not have time to shorten it Let us know how it goes.

GlobalMapper: Will the new v11 be able to use more than 4GB ram and more CPUs when creating ECWs or is the number of CPUs a limitation on the ECW-dll itself?

Thanks guys for the reply. I think the method to get these done at this time is to run multiple smaller mosaics for each county and then to do a global mosaic of the smaller mosaics. It seems to be working so far. I believe it's just an issue of my hardware. Its just not enough to make it happen.

As always, thanks for the help! One of these days I'll be done with this mosaicking stuff and I can explore all the other things possible with GM.