Apply Formula Errors for simple layer joining

burt46

Hi,

I am trying to apply a basic raster calculation formula to some loaded layers in my current workspace and can’t get this to work with the scripting (I can do it manually in the UI). Each layer is a single band only. I have tried to approach these two ways as follows and can’t get either to work.

////1. Direct Apply Formula Method////

DEFINE_VAR NAME="a" VALUE="%FULL_PATH%*B02.jp2"

DEFINE_VAR NAME="b" VALUE="%FULL_PATH%*B03.jp2"

DEFINE_VAR NAME="c" VALUE="%FULL_PATH%*B04.jp2"

DEFINE_VAR NAME="d" VALUE="%FULL_PATH%*B08.jp2"

APPLY_FORMULA FILENAME="%a%" FILENAME="%b%" FILENAME="%c%" FILENAME="%d%" BAND_EXPORT_SETUP="2?1? %a%" BAND_EXPORT_SETUP="3?1? %b%" BAND_EXPORT_SETUP="4?1? %c%" BAND_EXPORT_SETUP="8?1? %d%" FORMULA="(B2+B3+B4+B8)/(B8+B4)"

Error output is (note the files do exist and I even hardcoded the route and get the same error):

ERROR: No layers matching <C:\Users\burt\Desktop\TEST\*B02.jp2<-~|~-> C:\Users\burt\Desktop\TEST\*B03.jp2<-~|~-> C:\Users\burt\Desktop\TEST\*B04.jp2<-~|~-> C:\Users\burt\Desktop\TEST\*B08.jp2> for APPLY_FORMULA

Be sure to provide the full path and filename or the description for the loaded file to use, or * for all files

ERROR: No exportable raster or terrain layers with the specified filename

////2. Prestep to generate a multiband geotiff////

EXPORT_RASTER TYPE="GEOTIFF" FILENAME="%OUTPUT_FOLDER%_TSF SG_NDWI TEMP_WGS84_res 10m.tif" EXPORT_LAYER="*B02*" EXPORT_LAYER="*B03*" EXPORT_LAYER="*B04*" EXPORT_LAYER="*B08*" NUM_BANDS=4 PALETTE=MULTIBAND BAND_EXPORT_SETUP="1?1?*B02*" BAND_EXPORT_SETUP="2?1?*B03*" BAND_EXPORT_SETUP="3?1?*B04*" BAND_EXPORT_SETUP="4?1?*B08*" FORCE_SQUARE_PIXELS="YES" BG_TRANSPARENT=YES COMPRESSION=LZW

Error output is:

WARNING: PALETTE="MULTIBAND" is incompatible with INC_VECTOR_DATA="YES". Vector data will not be exported.

ERROR: Data must be loaded for EXPORT_RASTER command.

bmg_mike

You need to load the data that you are working with so the script can see it. The FILENAME parameter for APPLY_FORMULA refers to already loaded files, it won't go find the files and load them if not already loaded. Just add IMPORT commands for each band file that you need to use.

Thanks,

Mike

Global Mapper Guru

geohelp@bluemarblegeo.com

burt46

Hi Mike,

Thanks for the help.

Yes, i have this in my code prior to what was copied previsouly (i did not copy all as its quite long). The layers are all present prior to execution of the two versions of code as above that i have been trying to use. In fact it cant even do a simple a + b, its very strange behaviour. The manual needs more examples of APPLY_FORMULA for rasta calculations. What would be good is to see the code for a simple layer combination.

If i run manually and see the code in the workspace i just see all the gobbly text so i cant use it with a different layer setup, i have to generate the original execution code.

Any other ideas?

Thanks again,

Cheers

bmg_mike

I was able to setup a test on my end and get it working with similar data. I did have to fix an issue with forcing the 16-bit JP2 files to load as imagery rather than terrain. The RASTER_TYPE="IMAGE" parameter should have worked for that, but it wasn't until I fixed it in code. If you are having the same issue, adding LOAD_FLAGS="1~0~0" to your IMPORT commands should do the same thing, just in a much less readable manner.

I've attached my script that gets through the APPLY_FORMULA part with recognized filenames. I did have to add the NUM_BANDS parameter to what you had to get it to work. Normally if you are specifying 8 bands of output you probably want to have 8 bands of input, otherwise you will just get 0s filled in for the rest.

ApplyFormulaTest.gms.txt

If you still have trouble, can you let me know what version of Global Mapper you are using? There was an issue parsing wildcards in some cases during some v23.0.x versions and perhaps that is hitting you.

Thanks,

Mike

Global Mapper Guru

geohelp@bluemarblegeo.com

bmg_mike

Ignore that comment about needing 8 bands of input data as that doesn't apply. You do need the NUM_BANDS parameter as it would default to 1 in this case.

burt46

Thanks Mike,

Very grateful for the help.

I tried your script and it is importing and processing, but it asks for "Select elevation units to use", so its not importing as a raster image file but a DEM even with the RASTER_TYPE="IMAGE". The output is therfore not the same, but it is processing. From your code I just changed NUM_BANDS to 4 as i only have 4 layers each with a single band.

Going back to my original code, i added RASTER_TYPE="IMAGE" (i already had the LOAD_FLAGS="1~0~0" in my import) but get the same "ERROR: No layers matching" problem as before. So there are two things, its my import file loop or my GM version.

Attached is my import file process in your original code. It promts for a folder with the .jp2 files to load first, then should run the APPLY_FORMULA comment. You should see the error i see, If you dont, then its my GM version which is V23.0.1 (b112321).

Thanks again

burt46

It wont let me upload, so here is a copy:

//DEFINE_VAR NAME="FULL_PATH" VALUE="E:\Data\Imagery\Multispectral\Tile Sentinel 2\"

// Define filenames to load

DEFINE_VAR NAME="a" VALUE="%FULL_PATH%*B02.jp2"

DEFINE_VAR NAME="b" VALUE="%FULL_PATH%*B03.jp2"

DEFINE_VAR NAME="c" VALUE="%FULL_PATH%*B04.jp2"

DEFINE_VAR NAME="d" VALUE="%FULL_PATH%*B08.jp2"

DEFINE_VAR NAME="FULL_PATH" PROMPT=DIR ABORT_ON_CANCEL=YES FILE_MUST_EXIST=YES

DIR_LOOP_START DIRECTORY="%FULL_PATH%" FILENAME_MASKS="*.jp2"

IMPORT FILENAME="%FNAME_W_DIR%" \

TYPE="JPEG2000" RASTER_TYPE="IMAGE" LABEL_FIELD_FORCE_OVERWRITE="YES" LABEL_FORMAT_NUMBERS="YES" LABEL_PRECISION="-1" \

LABEL_REMOVE_TRAILING_ZEROS="YES" LABEL_USE_SCIENTIFIC_NOTATION="NO" HIDDEN="YES" \

LOAD_FLAGS="1~0~0" CLIP_COLLAR="POLY" CLIP_COLLAR_POLY="CUSTOM_CROP_POLY" SAMPLING_METHOD="NEAREST_NEIGHBOR" \

RASTER_STATS="1882.5791,495.26169,1882.5791,495.26169,1882.5791,495.26169,0,17431,0,17431,0,17431,7535025,4,<sep>0,1882.5791,495.26169,0,17431" \

AUTO_CONTRAST="YES" CONTRAST_SHARED="YES" CONTRAST_STRETCH_SIZE="0.000" CONTRAST_MODE="PERCENTAGE" \

TEXTURE_MAP="NO"

DIR_LOOP_END

// Load files

//IMPORT FILENAME="%a%" RASTER_TYPE="IMAGE"

//IMPORT FILENAME="%b%" RASTER_TYPE="IMAGE"

//IMPORT FILENAME="%c%" RASTER_TYPE="IMAGE"

//IMPORT FILENAME="%d%" RASTER_TYPE="IMAGE"

////1. Direct Apply Formula Method////

APPLY_FORMULA FILENAME="%a%" FILENAME="%b%" FILENAME="%c%" FILENAME="%d%" NUM_BANDS="4" \

  BAND_EXPORT_SETUP="2?1? %a%" BAND_EXPORT_SETUP="3?1? %b%" BAND_EXPORT_SETUP="4?1? %c%" BAND_EXPORT_SETUP="8?1? %d%" \

FORMULA="(B2+B3+B4+B8)/(B8+B4)"

bmg_mike

In your sample you need to move the DEFINE_VAR commands for a, b, c, and d to after the DEFINE_VAR that gets the %FULL_PATH% variable, otherwise the path isn't known to be inserted.

You will also want to remove the RASTER_STATS parameter as that is for the calculated band statistics for the layer being imported. You would normally just find that in a workspace file with a separate IMPORT for each layer. If you do it in a loop like this, the band statistics for each layer will be set to the same which is almost certainly incorrect. It might not affect the APPLY_FORMULA since that would use the raw 16-bit values, but the draw of each layer would be wrong.

The RASTER_TYPE="IMAGE" parameter wasn't supported for JPEG2000 until I fixed it for the upcoming v23.1 release yesterday, so you'd need to stick with the LOAD_FLAGS to have the data loaded as imagery rather than terrain.

Your NUM_BANDS value needs to be as high as the largest band you reference in the formula (B8 is the largest you have assigned). So change NUM_BANDS=4 to NUM_BANDS=8 as well.

The final issue is the HIDDEN="YES" parameter. The APPLY_FORMULA command filters out any hidden layers, so nothing ends up matching. I will update this for the next version so that hidden layers will be checked if no visible layers match the specified filename, but for now, just remove that parameter and it should work for you.

Here is a script that I made that works:

// Prompt user for path to data

DEFINE_VAR NAME="FULL_PATH" PROMPT=DIR ABORT_ON_CANCEL=YES FILE_MUST_EXIST=YES

// Define filenames to load

DEFINE_VAR NAME="a" VALUE="%FULL_PATH%*B02.jp2"

DEFINE_VAR NAME="b" VALUE="%FULL_PATH%*B03.jp2"

DEFINE_VAR NAME="c" VALUE="%FULL_PATH%*B04.jp2"

DEFINE_VAR NAME="d" VALUE="%FULL_PATH%*B08.jp2"

// Loop over files

DIR_LOOP_START DIRECTORY="%FULL_PATH%" FILENAME_MASKS="*.jp2"

 IMPORT FILENAME="%FNAME_W_DIR%" \

  TYPE="JPEG2000" LABEL_FIELD_FORCE_OVERWRITE="YES" LABEL_FORMAT_NUMBERS="YES" LABEL_PRECISION="-1" \

  LABEL_REMOVE_TRAILING_ZEROS="YES" LABEL_USE_SCIENTIFIC_NOTATION="NO" HIDDEN="NO" \

  LOAD_FLAGS="1~0~0" SAMPLING_METHOD="NEAREST_NEIGHBOR" \

  AUTO_CONTRAST="YES" CONTRAST_SHARED="YES" CONTRAST_STRETCH_SIZE="0.000" CONTRAST_MODE="PERCENTAGE" \

  TEXTURE_MAP="NO"

DIR_LOOP_END

// Create a new layer by applying the formula

APPLY_FORMULA FILENAME="%a%" FILENAME="%b%" FILENAME="%c%" FILENAME="%d%" NUM_BANDS="8" \

 BAND_EXPORT_SETUP="2?1? %a%" BAND_EXPORT_SETUP="3?1? %b%" BAND_EXPORT_SETUP="4?1? %c%" BAND_EXPORT_SETUP="8?1? %d%" \

 FORMULA="(B2+B3+B4+B8)/(B8+B4)"

burt46

Hi Mike,

Firstly, thanks for the patience with this. I learnt a few things from your last reply and can confirm it now works.

Just to assist others, the main error that did not allow me to debug this on my own was that i had DEFINE_VAR values after the loop call that did not allow the full path to load.

We have a few GM licences and i think the oldest is a v19 so i wont update the code to V23.1 just yet.

Thanks again. 👍️

bmg_mike

For future versions, you will also be able to run APPLY_FORMULA on loaded but hidden layers so it's more obvious what is happening. It will only look for hidden layers matching the specified filenames if no visible layers match. This will be in v23.1.1 and later.

Davesdewas

Very grateful for the help.

https://teatv.ltd/dl/ https://hellodear.in