How to crop a lot of .shp with a loop

Carlospernac

Hello there!

I'm trying to crop a large amount of .SHP in diferent folders using a loop. The idea is to use a base map of spain and crop lots of restrictions for each region.

Here is the script I'm trying to use:

GLOBAL_MAPPER_SCRIPT VERSION=1.00

DIR_LOOP_START DIRECTORY="MyFolder" FILENAME_MASKS=".SHP" RECURSE_DIR=YES

IMPORT_DIR_TREE DIRECTORY="%PARENT_DIR%" FILENAME_MASKS=".SHP" RECURSE_DIR=YES

POLYGON_CROP_FILE="Folder with the shp used to crop" POLYGON_CROP_USE_EACH=YES \

POLYGON_CROP_COMBINE_DUPS=YES POLYGON_CROP_NAME_ATTR="<nombre>" \ POLYGON_CROP_FILENAME_SUFFIX="_crop"

UNLOAD_ALL

DIR_LOOP_END

Can you guys help me fixing this code? The first problem I have is that Global Mapper doesnt recognice my Filename mask.

Thank you in advance

bmg_mike

You need to update your FILENAME_MASKS to be "*.SHP". With just ".SHP" you are only matching on an empty filename with a .shp extension, which likely matches nothing.

You might also want to specify the full path to the folder.

Assuming you want to crop each file and export to a new file (or over the same file), you will also need an EXPORT_VECTOR command. You also are importing every file in the folder for each file that you loop through by using IMPORT_DIR_TREE rather than IMPORT.

Something like the following would be better:

GLOBAL_MAPPER_SCRIPT VERSION=1.00

DIR_LOOP_START DIRECTORY="MyFolder" FILENAME_MASKS="*.SHP" RECURSE_DIR=YES

 // Import the current Shapefile

 IMPORT FILENAME="%FNAME_W_DIR%" 

 // Export to a new file with the crop in place

 EXPORT_VECTOR FILENAME="%FNAME_W_DIR%" TYPE=SHAPEFILE \

  POLYGON_CROP_FILE="Folder with the shp used to crop" POLYGON_CROP_USE_EACH=YES \

  POLYGON_CROP_COMBINE_DUPS=YES POLYGON_CROP_NAME_ATTR="<nombre>" \ POLYGON_CROP_FILENAME_SUFFIX="_crop"

 // Unload the file from this pass of the loop

 UNLOAD_ALL

DIR_LOOP_END

Note that your POLYGON_CROP_FILE parameter should contain the full path and filename of the file containing the crop polygons.

Carlospernac

Hi Mike,

FIrst of all thanks for your help. Let me explain better my case:

I have the map of Spain in a folder (lets call it "RegionsFolder") in shp, wich i want to use as the base map (That shp has the regions diferenced with the attribute <name>).

I also have a folder with lots of restricctions ("RestrictionsFolder"). I want to use all of the restrictions in loop to crop the Spain base map to obtain viable areas for contructions so, the final result might be the different regions cropped with only the viable areas of construction. One .shp for each region.

I've been trying to do it myself but I still have the problem of the FIlemask (image attached)

Thanks for your help Mike, you are always pretty helpful!

bmg_mike

I wonder if the accent characters inside the directory are causing the issue. Global Mapper would expect the code page of the path to be displayable in the non-Unicode code page of the system, but Windows itself supports full Unicode code page.

Can you try renaming the path with the accent character to use simple ASCII just to see if that works?

Carlospernac

That was one issue. Thanks! Now the loop runs properly, but other errors appeared:

Starting loop over files in directory <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Restricciones\2DOMINIOPUBLICO\HIDROLOGIA\DPH\> that match '*.SHP'...

Importing file <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Restricciones\2DOMINIOPUBLICO\HIDROLOGIA\DPH\DPH\DPHCARTOGRAFICO_S.shp>...

WARNING: Unknown parameter <\> ignored for EXPORT_VECTOR command.

Exporting vector data to <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Restricciones\2DOMINIOPUBLICO\HIDROLOGIA\DPH\DPH\DPHCARTOGRAFICO_S.shp>...

WARNING: Unable to load overlay Provincias

Acceso a C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Provincias denegado.

Cause: 5, IOS Error: 5

File <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Provincias> Exists, Size: 4096 bytes, Read-Only: N, Attributes: 0x00000010

Disk Free Space: 369,648,992,256

ERROR: No data could be loaded from POLYGON_CROP_FILE <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Provincias> for export.

Removed all loaded overlays.

Importing file <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Restricciones\2DOMINIOPUBLICO\HIDROLOGIA\DPH\DPH\DPHDESLINDADO_S.shp>...

WARNING: Unknown parameter <\> ignored for EXPORT_VECTOR command.

Exporting vector data to <C:\Users\c.perez\Documents\00.TAREAS DIARIAS\06. Scripting GM\Restricciones\2DOMINIOPUBLICO\HIDROLOGIA\DPH\DPH\DPHDESLINDADO_S.shp>...

And that continues for every file in the loop.

I tried with other codes to try doing what i told in the second answer, but it didnt work. Let me explain the case better. Thanks for your patience 😊.

The .shp of the image is the map of Spain. As you can see the Feature "Nombre" is the one I want to use to crop and export.

The final result of the script might be one .shp for each feature "nombre" but cropped with a lot of restrictions (As we saw, that restrictions are also SHP and are storaged in another directory with subfolders) so the result is the viable areas in each region of Spain. I think that the script you sent me (thanks 😉) doesn't have that objective.

This script is driving me crazy 😓. Thank you in advance!

bmg_mike

I think the issue is the line continuation character (\) being in the line before the POLYGON_CROP_FILENAME_SUFFIX parameter. The forum is not ideal for posting formatted text. So the POLYGON_CROP_FILENAME_SUFFIX was being chopped off and ignored, so '_crop' was not appended to the filename and you were trying to overwrite a file that was open.

 EXPORT_VECTOR FILENAME="%FNAME_W_DIR%" TYPE=SHAPEFILE \

  POLYGON_CROP_FILE="Folder with the shp used to crop" POLYGON_CROP_USE_EACH=YES \

  POLYGON_CROP_COMBINE_DUPS=YES POLYGON_CROP_NAME_ATTR="nombre" \

POLYGON_CROP_FILENAME_SUFFIX="_crop"

In addition, you would want to use POLYGON_CROP_NAME_ATTR="nombre" (i.e. no <>) as that is the name of the attribute in the file.

Carlospernac

Hi Mike,

This time the script run for some regions, but an error ocurred in others. The problem is that the result of one region overwritte my base resticction layer, so now i've lost it 😥.

Anyway, do you have something in mind to change the script code to make only one .shp as a result with the areas that are not cropped after all the loop?

I tried to write it but i feel I'm pretty far away to get a good result 😅. With the spatial operations tool I can do it but it's too slow.

So, ill tryto clarify more my case:

• I have one folder with subfolders witch contain lots of .shp with restrictions.
• In another folder, I have my Spain map with the regions differenced with the attribute "Nombre"
• I only want one result per region. Each region should have only the areas without the restrictions.

For example, if my restrictions are rivers and cities, the result might be all the areas of the region that doesnt have a river or a city, and everything included in the same shp.

Thanks for your patience.

bmg_mike

I guess it depends on how exactly the restriction layers are set up, is there an attribute or something that specifies whether or not there are rivers or cities?

If you can set up some kind of attribute query to identify what you want to keep out of the restrictions, you could have the script load all of the data from the restrictions folder up front, then use an EDIT_VECTOR script command (https://www.bluemarblegeo.com/knowledgebase/global-mapper-23/cmd/EDIT_VECTOR.htm#EDIT_VECTOR) to identify all of the regions that you want to ignore and mark them as deleted, then export cropped to your Spain map polygons.

Make sure the output FILENAME in your EXPORT_VECTOR command isn't overwriting something that you want to keep (maybe build a more complex output filename in a different folder using different filename variables).

JeffH@BMG

Hi Carlospernac,

Pardon me for butting in with a side question. You said, "With the spatial operations tool I can do it but it's too slow." I'm a colleague of Mike's and have been adding the spatial operations facilities to Global Mapper, bit by bit, for the last couple of years. I'd be interested in what your approach was in using spatial operations, and what was too slow, and anything else related to making them more useful. Any feedback would be helpful.

Best regards,

~Jeff

Carlospernac

Hi Jeff!

Nice to meet you. Let me explain better.

With the spatial operations tool I can do it very quickly but I have to cut one by one all the restrictions (almost 50) and after that export the uncropped areas for each region of Spain . All of the restrictions are for all Spain, so sometimes if I load one of the heavy ones my computer crash.

That's why I was asking for scripts, because I can do that crops without loading the files.

If you know any other posibility I would be very thankful!

JeffH@BMG

Hi Carlos,

Thanks for getting back to me. If I read your reply correctly, it sounds like it might have helped to be able to integrate scripting the spatial operations stuff into your process (and if I have that wrong, don't hesitate to clarify). So you may not know that we do have a spatial scripting language that can allow you to do spatial operations in a Global Mapper script, or (as of 23.1) in its own separate script, in a .gmss file. At this point, though, in the first case, it's not as well integrated into the GM scripting language as I'd like, and in the second case, the spatial operations scripting language isn't as powerful as it could be. I have several ideas / improvements on my radar for these types of additions. If that would help you with the types of processes that you do, or have other ideas, I'd be happy to hear more from you.

Thanks again. Cheers.

~Jeff