jpeg tile validity checker

I encountered an unexpected issue during my mbtiles output. The tiles are in jpeg format. When I browse through the tiles, I noticed that some tiles are invalid jpeg files. I have checked and confirmed that the drive is without any error. The tiles are legitimate files, only that they cannot be read as a jpeg picture. The percentage is not high, around 0.1% - 0.01%. I can of course pick the files up and delete them, and run the output once more to fill the gaps, but I have 3 million tiles in one layer. It will take hours to finish the eyeball checking. Is there any automatic way to do this?
Best Answer
-
bmg_bob Global Mapper Programmer Posts: 2,069
I suggest writing a Global Mapper script to iterate through the folders where the JPEG files are stored, and attempt to load each one. You should get error messages in the output log when the import fails. Be sure to log the name of each file before you attempt to load it, and use UNLOAD_ALL after each import command.
If you are not familiar with scripting, we have a webinar you can look at to get an introduction.
When you ran your MBTiles export, did you choose the "Skip Empty Tiles" option? I wonder if these invalid JPEGs are the result of exporting a tile with no data.
Answers
Not a JPEG file: Starts with 0x00 0x00
JPG Overlay.cpp -67
followed by version info etc.
The percentage of such bad files is not high, in about 0.1-0.01%. But I have 3 million files to examine. Can you suggest any automatic way to pick these bad files?
Your suggested way will work, I am sure. But please keep this in mind for future GM updates, either improve the reliability of the output, or provide a build in checking mechanism. Thank you very much.