Building a shapefile as a raster tileindex in GMS (gdaltileindex from GDAL)
RobertR
GlobalMapper Fan!Trusted User
I would like to use the GM Scripting Language to build a shapefile as a raster tileindex;
Currently, I am using gdaltileindex to do this, and it runs like this in Linux:
synopsis: gdaltindex [-tileindex field_name] [-write_absolute_path] [-skip_different_projection] [-t_srs target_srs] index_file [gdal_file]*
example: gdaltindex /me/maps/data/tiles.shp /me/maps/data/final/*.tif;
Is it possible to do this using GMS? I would like to transfer the entire project in the GM script..I managed to do all except this one. I am kinda new with GM Scripting so bear with me a bit.
Many thanks for the support!
Currently, I am using gdaltileindex to do this, and it runs like this in Linux:
synopsis: gdaltindex [-tileindex field_name] [-write_absolute_path] [-skip_different_projection] [-t_srs target_srs] index_file [gdal_file]*
example: gdaltindex /me/maps/data/tiles.shp /me/maps/data/final/*.tif;
Is it possible to do this using GMS? I would like to transfer the entire project in the GM script..I managed to do all except this one. I am kinda new with GM Scripting so bear with me a bit.
Many thanks for the support!
Comments
-
If you use the GENERATE_LAYER_BOUNDS (http://www.bluemarblegeo.com/knowledgebase/global-mapper/ScriptReference.html#cmd_generate_layer_bounds) script command you can create bounding areas for any loaded raster layers, then use EXPORT_VECTOR to export that out to a file.
Thanks,
Mike
Global Mapper Guru
gmsupport@bluemarblegeo.com
http://www.bluemarblegeo.com/ -
Hello Mike, that's almost good, but what if I have 30.000 raster files and I want to create the tile index for all? In batch, by using gdal, I do not need to load all of them, it would surely cause problems to load such an amount of raster files.
-
Update. I think that loading so many tiff files will be an issue and I encounter the same problem. I would like to keep only one or two columns in the resulted shapefile, the attribute that contains the full path adn maybe the EPSG code.
ATTR_TO_DELETE is not working for me, it does not delete the column(s). I was thinking of something like EDIT_VECTOR FILENAME="xxxx" KEEP_COLUMN="LOCATION" would be awesome but I dunno if we have this option somewhere, under another name that I am missing. I skimmed every post in the Scripting thread and did not find something similar.
Here is my code with a few comments with questions on enhancements and/or fixes:
GLOBAL_MAPPER_SCRIPT VERSION=1.00
/ We import the TIFF files from default directory
/ I want to check if all the tif files have the correct EPSG code, if not, reproject it.
/(here I have two folders "tiff_one" and "tiff_two" but there can be up to 100 folders eventually)
/would it be ok to unload the folders at some point? it takes a while for these two, imagine for all 100
IMPORT_DIR_TREE DIRECTORY="C:\GMFiles" FILENAME_MASKS="*.tif"
/Generate shapefile with path towards location
GENERATE_LAYER_BOUNDS
EXPORT_VECTOR FILENAME="C:\GMFiles\layer_bounds.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES"
UNLOAD_ALL
/Import resulted shapefile to delete attribute columns which are not needed
IMPORT FILENAME="C:\GMFiles\layer_bounds.shp"
/How could I insert a whole list of attribute columns?
/It would be easier to add the one that we need to keep in this case
EDIT_VECTOR FILENAME="C:\GMFiles\layer_bounds.shp" ATTR_TO_DELETE="ELEVATION"
/Export the result which should contain a clean shapefile
EXPORT_VECTOR FILENAME="C:\GMFiles\layer_bounds_no_elev.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES"
Result Log
Importing directory C:\GMFiles...
Generating layer bounds areas from loaded data...
Exporting vector data to file C:\GMFiles\layer_bounds.shp...
Removed all loaded overlays.
Importing file C:\GMFiles\layer_bounds.shp...
Editing vector features...
WARNING: Empty COMPARE_STR list, all features will be matched on.
Updated 0 features.
Exporting vector data to file C:\GMFiles\layer_bounds_no_elev.shp...
Script processing COMPLETED.
In the end, I still have all the columns (a number of 40) and I need to keep only one or two.
Can you recommend a way to do this? It seems to be almost what I need, except for the questions raised in the comments. Anyway, GM Scripting is a great way to enhance tasks.
Thanks for the support -
You don't actually need to export your intermediate results to a Shapefile and reload them. Just add LAYER_DESC="Bounds" or something like that to GENERATE_LAYER_BOUNDS and then you can use FILENAME="Bounds" to refer to that layer later on.
I don't think the layer bounds areas would have an ELEVATION attribute to delete so that ATTR_TO_DELETE wouldn't do anything. You can provide multiple on the same line though. So something like:
EDIT_VECTOR FILENAME="Bounds" ATTR_TO_DELETE="UPPER LEFT X" ATTR_TO_DELETE="UPPER LEFT Y" \
ATTR_TO_DELETE="LOWER RIGHT X"
Just add all of the attributes you want to get rid of. There isn't currently a way to specify just those attributes to keep.
Thanks,
Mike
Global Mapper Guru
gmsupport@bluemarblegeo.com
http://www.bluemarblegeo.com/products/global-mapper.php -
Ok, I will try again, but somehow, I have it:
attribue_table.JPG
Would it help to load all the tif s from a folder, create tiles, then unload_all and proceed to the next one? -
Ok.
I found the problem. When I click to edit a feature, I get the actual names of the columns. When I click on the info tool, I have those that you see in the attach. Different display of column names. -
Ah, when you export to a Shapefile as an intermediary the field names are cropped to 10 characters (a format limitation). So if you are exporting to Shapefile first and reloading you would need to delete the modified attribute names. I would just not export to Shapefile and then you don't have to worry about the names changing.
Thanks,
Mike
Global Mapper Guru
gmsupport@bluemarblegeo.com
http://www.bluemarblegeo.com/products/global-mapper.php -
Ok, I understand. Thanks for clarifying that.
I would have a final question regarding my script. I am repeating a command for 37 folders located in a parent folder (like in the first attach).
snip1.JPG
My code:[SIZE=2]GLOBAL_MAPPER_SCRIPT VERSION=1.00 IMPORT_DIR_TREE DIRECTORY="C:\GMScript\ALL_DATA[B][COLOR=#b22222]\week1\[/COLOR][/B]" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS[B][COLOR=#b22222]\shapefile1.shp[/COLOR][/B]" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL IMPORT_DIR_TREE DIRECTORY="C:\GMScript\ALL_DATA[B][COLOR=#008000]\week2\[/COLOR][/B]" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS[B][COLOR=#008000]\shapefile2.shp[/COLOR][/B]" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL IMPORT_DIR_TREE DIRECTORY="C:\GMScript\ALL_DATA[B][COLOR=#0000ff]\week3\[/COLOR][/B]" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS[B][COLOR=#0000ff]\shapefile3.shp[/COLOR][/B]" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL IMPORT_DIR_TREE DIRECTORY="C:\GMScript\ALL_DATA\week4\" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS\shapefile4.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL IMPORT_DIR_TREE DIRECTORY="C:\GMScript\ALL_DATA\week5\" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS\shapefile5.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL IMPORT_DIR_TREE DIRECTORY="C:\GMScript\ALL_DATA\week6\" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS\shapefile6.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL[/SIZE]
The only thing that changes from one set to another is the folder that is read/imported (week1,2,3...37) and its corresponding shapefile output (shapefile1,2,3...shp)
I used this method to avoid loading so many .tif files at once. I tried to look over the DIR_LOOP command but as I saw in the example ( http://www.globalmapper.com/helpv14/ScriptReference.html#sample_create_contours , I just cannot make it work in my case.
Is there a solution to maybe reduce the code length so I will not have to insert 37 paragraphs and another extra paragraph of code every time I add a new folder with data?
In batch I used a "for": for I in {1..37}; do gdaltindex [...]
Basically what I want to do is to "read" the folders and for each, to export a shapefile with layer_bounds. A single shapefile output would also work but how can I do that without reading/importing the .tif files all at once? (i have thousands of them); -
This is what I tried and failed. Reason: It keeps running the generate and export command for the same folder on and on (week10), it does not go to the next one (week29 for eg).
GLOBAL_MAPPER_SCRIPT VERSION=1.00 UNLOAD_ALL DIR_LOOP_START DIRECTORY="C:\GMScript\ALL_DATA\" FILENAME_MASKS="*.tif" RECURSE_DIR=YES IMPORT_DIR_TREE DIRECTORY="%DIR%" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS\%FNAME_WO_EXT%.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL DIR_LOOP_END UNLOAD_ALL
Another try which I made was by following your example in a previous topic ("DIR_LOOP_START DIRECTORY="D:\TEST\100*" FILENAME_MASK="..") but again, The result was negative, see bottom of code example;GLOBAL_MAPPER_SCRIPT VERSION=1.00 UNLOAD_ALL DIR_LOOP_START DIRECTORY="C:\GMScript\ALL_DATA\week*\" FILENAME_MASKS="*.tif" RECURSE_DIR=YES IMPORT_DIR_TREE DIRECTORY="%DIR%" FILENAME_MASKS="*.tif" GENERATE_LAYER_BOUNDS EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS\%FNAME_WO_EXT%.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES" UNLOAD_ALL DIR_LOOP_END UNLOAD_ALL -------------------------------------------------------- Removed all loaded overlays. Starting loop over all files in directory... WARNING: No files matching the specified mask were found under the specified directory. The loop will be skipped. Loop over files in directory completed. Removed all loaded overlays. Script processing COMPLETED.
-
There is a special feature allowing you to hit each folder in a directory tree just once. If you use the following DIR_LOOP_START you should get what you want:
DIR_LOOP_START DIRECTORY="C:\GMScript\ALL_DATA\" FILENAME_MASKS=".." RECURSE_DIR=YES
Thanks,
Mike
Global Mapper Guru
gmsupport@bluemarblegeo.com
http://www.bluemarblegeo.com/products/global-mapper.php -
Sorry, to ask again, but isn't that exactly what I used in the Code sample? What seems to be incorrect in my code which causes the repetition of the same folder over and over again?
DIR_LOOP_START DIRECTORY="C:\GMScript\ALL_DATA\" FILENAME_MASKS="*.tif" RECURSE_DIR=YES
IMPORT_DIR_TREE DIRECTORY="%DIR%" FILENAME_MASKS="*.tif"
GENERATE_LAYER_BOUNDS
EXPORT_VECTOR FILENAME="C:\GMScript\LAYER_BOUNDS\%FNAME_WO_EXT%.shp" TYPE="SHAPEFILE" SHAPE_TYPE="AREAS" GEN_PRJ_FILE="YES"
UNLOAD_ALL
DIR_LOOP_END
UNLOAD_ALL -
I tried with IMPORT FILENAME="%FNAME_W_DIR%" FILENAME_MASKS="*.tif" instead of IMPORT_DIR_TREE DIRECTORY="%DIR%" FILENAME_MASKS="*.tif" and it works.
The setback is that it exports a tile for each TIFF file which is too much as I have thousands of TIFFs.
I think that dir_loop_start does not work properly with IMPORT_DIR_TREE DIRECTORY="%DIR%". -
You missed the key change, which is using FILENAME_MASKS=".." . Note that the mask is ".." and not "*.tif". The ".." is special and causes each folder to be hit just once as there is just a single ".." file inside each folder.
Thanks,
Mike
Global Mapper Guru
gmsupport@bluemarblegeo.com
http://www.bluemarblegeo.com/products/global-mapper.php -
Damn me! Many thanks and sorry for that. Now it's clear.
Categories
- 12.7K All Categories
- 5.6K Features Discussion
- 342 Downloading Imagery
- 1.3K Elevation Data
- 380 Georeferencing Imagery Discussion
- 628 GM Script Language
- 53 User Scripts
- 113 GPS Features
- 414 Projection Questions
- 819 Raster Data
- 1.3K Vector Data
- 6.6K Support
- 177 Announcement and News
- 908 Bug Report
- 558 SDK
- 1.2K Suggestion Box
- 3.7K Technical Support
- 562 Other Discussion
- 129 GIS Data Sources
- 27 Global Mapper Showcase
- 233 How I use Global Mapper
- 107 Global Mapper Forum Website