Global Mapper v25.0

compare Grid elevations

sjf
sjf Global Mapper User

Hi Mike,

I run into several problems comparing (subtracking) gridded elevations data:

I have 2 similar 10 cm grid files: When I use combine - compare terrain layers the resulting file is not correct:

  1. the grid cells are different (base grid is clamped to meters grid and are both OK)
  2. the result is different ( when I select one cell and \compare by hand: the result cell is different
  3. then when I export the result file to a new grid: the legend values are different ( I think it is something to do with rounding related to accuracy settings (meters , centimeters): the values of the result grid and the export (GM grid) are the same but the colors are picked different from the Shader (customized). that means: 2cm (gray) on the result grid is displayed as 2cm gray but on the gridded (exported GM grid) the color is changed to 1cm (or something). This is only noticable on the very small values (1 cm differences). but that is essential because I compare gridded data from to runs of lidar data and want to detect and visualize very menute changes in height (and zero change defined as values between -1 and +1 cm).

  4. what am I doing wrong - or is this a bug? What i did not expect is a shifted result grid: it should clamp to the grid like the other GM grids (derived from LAS files). I also played around with the elevation settings since sometimes I wil have to be meters and sometimes cm to have elevation data displayed as 0.01 meters (and raw data being 1 cm (reported by GM).

  5. I have to have it 100% reliable and consistent - because I a looking for minute changes and can not have 'GM interfere' - the workflow wil be described and repeated and shaders will be matched etc etc.

long story short... gridding is a powerfull tool but should be 100% reliable.

cheers

Stefan

Answers

  • sjf
    sjf Global Mapper User

    addendum:

    BUG (seems like a bug): If I shift layers a fixed position (0.125 and -0.125 for a 25 cm gridded result file) when I click < apply the shift is performed BUT ON EXIT AGAIN (2 times shift);

    then when I export the result file to grid (25 cm etc etc): the result is NOT SHIFTED.

    PLUS: the colours are off in relation to the result grid (due to points above related to rounding and or precision settings (meters, CM mm ...).

    stefan

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    Hi Stefan,

    If you are exporting to the GMG format, the elevation units are the maximum precision, so if you choose 'Meters', all values are rounded to the nearest whole meter. If you are comparing things at the centimeter level, I would choose Millimeters as the precision. This fixed precision allows GMG files to be compressed very well. Also make sure to check the option to 'Only Use Lossless Compression' so you don't get any fuzziness near the specified precision.

    For the Shifting (and really any dialog), an OK button means 'Apply, then Close', so when you press Apply, then OK, you are doing the shift twice. We actually just updated this for the upcoming v23.1 release, so now that dialog just has Apply and Close buttons so people don't accidentally double shift.

    When exporting to grid formats, you might also check the options on the General -> Export tab of the Configuration dialog as there are lots of different snapping options that you can enable which might mess with the default export bounds.

    Thanks,

    Mike

    Global Mapper Guru

  • sjf
    sjf Global Mapper User

    Addendum 2: example: where does this 'precision' come from (suspect a bug)


    Notice: height being: 0.0099999905 (but actually 0.01 meters) and all base precision is cm in meters so 0.01 meters.


  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    That type of precision would just come from the inherent limitations of a single-precision floating point number. The GMG would store the elevation as exactly 1 centimeter. When loaded, that would be saved as 0.01 to a single-precision floating point number. Floating point numbers can't exactly store many fractions, so 0.01 ends up as 0.009999999905. Unless you have extreme precision reporting turned on, that would typically be displayed as 0.01 in Global Mapper.

  • sjf
    sjf Global Mapper User

    addendum 3 - forgive me digging in the deep -

    I see blocking in blocks of processed change-map data (can not attach an image for some reason) - so when I compute the difference between two grid-set (GM grids) of exactly similar composition only different time I see blocks of different changes (al at 2-3-4 cm level). will try to attach an image later of separately,

    I suspect something with GM processing... That would be something ...

  • sjf
    sjf Global Mapper User

    But Mike: it is all very random: the precision is at one cell 0.01 meters and in others goes to 0.009999999123

    And all base cells are in 0.01 meter precision.

    This is confusing and I just need a specific precision. (cm).

    And then the blocks ( I get status code 403 when I upload or attach)

    I looks like the precision varies in areas of in computing blocks... Could that be?

    PS where can i enable extreme precision reporting (is this similar to extreme debug reporting?

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    I just checked in the debugger and a full precision value of '0.1' becomes '0.00999999978' in single-precision floating point.

    The random-ness is a bit odd, I wonder if some resampling was done somewhere and the standard loss of precision due to floating point math is causing very slightly different numbers. There does happen to be a cut-off in the GM code where an elevation value is formatted for display where anything 0.1 or larger uses a maximum of 3 decimal digits. Smaller than that and you will end up with 10 digits. It's kind of an arbitrary threshold that should probably extend further down to at least 1 millimeter.

    If you have some sample data, you could send it directly to me at mikec@bluemarblegeo.com .

  • sjf
    sjf Global Mapper User

    I will set the combine terrain options to mm

    Do I understand correctly that the mm setting is more a precision setting (round values to a precision of 3 units) than a vertical elevation unit setting? At most other transforms you don't get that option: so if I transform from LAS (points) to GM grid you just have to option of meters (or feet). But then if you want to compare two grids you have to set the precision to mm...

    Anyway: I managed to squeeze a good result out of the compare. Except the shift... How do I make that permanent? I guess if I save the grid again I will need to do the shift again.

    Sorry to bother you: but in between I also got some results that I thought just was the gridding but actually is quite spectacular so i want to be absolutely sure it is not an error or oversight in the computation, gridding etc et.

    greetings

    Stefan

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    For precision the units that matter are the units that you exported with to the GMG. Elsewhere, like in Combine Terrain, the units are just for specifying the native units to store the results in. They will be stored as full single precision floating point numbers, so there isn't any rounding until you hit the limits of a 'float' (about 10 digits of precision).

    I would expect the shift to stick even on export. If you save a workspace and then reload is the shift maintained there? For a terrain layer, the shift just creates an image rectifier for the layer. The bounds in metadata should update as well.

  • sjf
    sjf Global Mapper User

    Keep running into 'problems'- One is like this:

    I compare two grids and subtract them. But both grids also have no data cells (because there is water and- this is not in the lidar data). Then I get the results and the cell are empty where one of the grids had no data: but when I export this to GMG some cells with empty values are calculated! They have the value of one of the grids that had data, so the result file and the exported GMG file are not similar. I don't understand where this data is coming from.

    ( Plus strange rounding errors that pup up somewhere from deep in the floating point pool (like suddenly reading: 0.0100000979 ) )

    A simple substraction of two identically produced grids takes a lot of time to set up a reliable workflow.

    It simply is not working reliable.

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    When you do the export to GMG, do you have the box checked to 'Interpolate to Fill Small Gaps in Data'? If so, any small no-data areas will be filled in.

    Also, before you export, make sure you have hidden the original grids. If they are still there, any holes in the combined result will end up filled in with any visible data that is behind it, like the original data. You can also right-click on a layer in the Control Center and select the Layer submenu option to just export that layer.

    You may also want to disable resampling for the layers before combining and export. Otherwise, if the input data cells do not perfectly align with the sampling grid for the subtract operation, your input data will be interpolated to match the sample location. If you set the Sampling Method for each layer to Nearest Neighbor, then no resampling will occur and just the raw values should be worked with.

  • sjf
    sjf Global Mapper User

    interpolate to fill small gaps is unchecked: This is because I need to determine point density vs grid size. I want to see NO data when there is no data to fill a cell (like open water but also our way to fly with a drone with laser and optimize operation and setting).

    I just export the layer: but from now on I will prepare the gridding in a separate project and make the grids, unload the LAs point files and make the change map comparison between the two grids (with no point there).

    When I go from las (point) to grid I check the make GMG tick box so the grid file is made.

    It is a surprise to me that the original points are still used when I make comparisons between to resulting GMG files. But anyway: I remove the Las files to avoid any interference.

    Resampling and all is off and on Nearest Neighbour (which has a disturbing by-line actually it reads: No resampling (nearest Neighbour) - it should just read < No Resampling > because I want a cell for cell copy and no relation with any neighboring cells.

    workflow:

    1- from LAS to Grid (no precision setting) - 2 grid files

    2 - remove the LAs files

    3 - combine/compare the grids: will give intermediate result - setting mm (the results read height 0.1 m raw value 99,999908 mm !)

    4 - export to GMG with setting on mm precision: results read: 0.1m raw value 100 mm

    Now the Legenda of the change map can use the same precision: 0.01cm 0.02 cm etc and I get no rounding confusion (99.999908 mm is not 0.10m but will fall in a class lower... And this is fixed now too.

    Thanks for your help. Still feel this is a complicated route and you need to know a lot of details for it to work consistently. just unticking the LAS files was something I thought should have no influence - but now I know better.

    thanks

    Stefan