Temp file housekeeping...

Fuzzman

I download quite a lot of online imagery. I find that every few days my 3 Global Mapper installations start to become unstable with crashes, apparent hangs and being unable to load workspace files as well as other issues. Each time this starts to happen I know what to do and how to fix the issue and that is  to go into the temp directory and clear the 65,000+ OSM????.tmp files that have accumulated in that directory. When a directory starts to get that many files, NTFS begins to have a hard time working efficiently and will cause apparent hangs (you just need to wait for the program to respond) and other such inconsistent behaviors all caused by NTFS having to deal with so many files in one subdirectory.

Global Mapper should do some housekeeping at startup and delete these files. They are not removed by "delete cached files" in the online section of the program.

When I am downloading I can go into this folder and watch these files accumulate at a rate of about 10 to 20  per minute. Global Mapper seems to do nothing to delete or remove these files when done. I believe this can be a cause of many problems for most users who aren't computer savvy enough to know to look in and clear temporary files if their installations start to become unstable.

Thanks

bmg_bob

Hello,

I opened bug #18609 to address this issue.  When I download online data (MapQuest OpenStreetMap data), I see tile files (.png) in the temporary folder, but I don't see any .tmp files.  Which data source are you using?

Cheers,

Bob

bmg_mike

I took a look and found that OSM????.tmp files could accumulate if accessing a tiled source where you failed to download a tile. Each failed download would leave a .tmp file around. The .tmp files do get cleaned up when GM starts up once they are at least 3 days old, but if you are doing a lot of access to problem sources these could build up. I've fixed the issue that caused them to stick around so they should be immediately deleted.

I have placed new builds with the latest changes at http://data.bluemarblegeo.com/downloads/global-mapper/dailybuilds/ for you to try.

Thanks,

Mike
Global Mapper Guru
geohelp@bluemarblegeo.com
http://www.bluemarblegeo.com/

Fuzzman

Thanks for your quick response and modification to the program. I can confirm, at least after initial testing, that it is working fine and I am watching those files flash up for a moment in the temp directory and then being immediately deleted. Before that I had accumulated 3600 OSM????.tmp files with the previous installation in just 2 hours.

In case you need to revisit this issue, I want to confirm that it was not happening as a result of failed downloads because they were accumulating as files were successfully being downloaded. I was downloading from a very reliable source - NAIP & USGS 1 foot.

Thanks

Fuzzman

LOL. You just made a liar out of me! The problem is still there. They were being deleted initially but in then in the few minutes I was responding to your message, I went back to check and 1200 had built up in 5 minutes. There were no failed downloads, they were proceeding normally.

So you might need to take another look at the problem.

I can confirm it is the latest daily build b062916.

Thanks

bmg_mike

That is very odd. Can you check the 'Enable Detailed Logging' option at the bottom of the Advanced section of the General tab of the Configuration dialog, then send me the errlog.txt file from your user settings folder (listed on Help->About) when this behavior is happening? It should contain more information about why the files can't be deleted.

I have also made some further updates to improve that logging to help track down what is happening on your machine. I have placed new builds with the latest changes at http://data.bluemarblegeo.com/downloads/global-mapper/dailybuilds/ for you to try.

Thanks,

Mike
Global Mapper Guru
geohelp@bluemarblegeo.com
http://www.bluemarblegeo.com/

Fuzzman

Ok. I will do this next time I am downloading which should be within the next 48 hours.

Fuzzman

Ok. I have installed the latest build and attached is the log of a download that is currently taking place. I also took a snapshot of my temp directory and the rate at which these TMP files are being created. I had just deleted 25,000 of them that had accumulated since the download began around 18 hours ago.

BUT.... please don't mess with things too much!!! Why I say that is that in that latest build you also fixed a problem that was going to be the next problem I was going to post about. The problem that was occurring was that randomly, downloads would just stop but looked like they were still progressing. So nothing would appear as crashed but the small download windows showing that a download was in progress would still be up but data was not being downloaded. The only way to stop the program and exit the download process would be to kill the process in task manager. This would happen randomly on 3 different machines with either NAIP, EROS or even the world imagery server. This frustrating problem has GONE in this build and the CO download that is shown in the log went for around 18 hours without any crashes or hangs as described. It was so stable that I was even able to load multiple instances of GM and have 3 instances all downloading at once - all multi-threaded too. This is the most stable I have had the download experience be, so PLEASE don't break what you inadvertently fixed while trying to solve this problem. :-) In the meantime I can just write a small batch file to run on a schedule to delete the files so they don't build up to the 65,000 mark.

Files attached.

bmg_mike

I discovered another place that OSM files were being generated and could be left behind. I have fixed it as well. Neither of these should affect the recent stability and threading improvements, so now you should have everything working automatically. I have placed new builds with the latest changes at http://data.bluemarblegeo.com/downloads/global-mapper/dailybuilds/ for you to try.

Thanks,

Mike
Global Mapper Guru
geohelp@bluemarblegeo.com
http://www.bluemarblegeo.com/