Ghost precision

sjf

Hi Mike,

mentioned this earlier: when I do elevation calculations on lidar data - las input and gridded to a certain GMG I notice what I call: ghost precision. This is a precision (way) beyond the input precision.

In this case the LAS input is mm precision - as is standard in lidar input data, so 1.123 meters.

When doing calculations I frequently see numbers appearing far beyond the input precision. Including rounding to insignificant precision like 19.00000999 mm (see example)

Typical workflow allows LAS analysis GMG settings on the input to be set as meters of feet (I work in meters) to convert las data to GMG.

I typically use an average (binning) within a gridcell - so basically the average is the total elevation / divided by the number of points. The resulting number can never be more precise than the input precision:

1.123 meters + 1.122 meters = 1.1225 so should be 1.123m (and I will chose to use 1.12)

For further analysis and calculations (combine/compare, visualisation) I make sure to set the precision to cm (2 significant numbers). Still I notice that the RAW number displayed can have (not always, so: can have) a ghost precision, way beyond the cm level.

I don't know where the extra precision comes from. I guess from somewhere in the code.

Please note that more complex calculations might in theory lead to more figures behind the comma (like when doing a triangulation) but the result should never get beyond the input precision and should be rounded to the input precision. In an export the user should be able to round even further (p.e. to cm precision if wanted). You can do that to set the level in the export box.

I now manage the process because I am very careful in my procedure. I will remove the original LAS file and use the save GMG and load separately to avoid cross-over from the original files.

Please have a look at this!

Are there more users who noticed this behaviour?

greetings

Stefan

bmg_mike

Hi Stefan,

I have made an update in the code in the last week to prevent the huge number of digits of precision from being displayed for elevation values <= 0.01 (a centimeter of less typically).

The issue is that all elevation values are converted to floating point for display, and floating point doesn't exact represent many decimal values (power of 2 vs. power of 10 thing). So long as you aren't showing a huge number of digits of precision, you won't notice as the rounding will take care of everything. There was just code that made any very small elevation values display with a huge amount of precision, so anything <= 0.1 would explode out to 10 digits of precision, which exposes the floating point rounding. Once v23.1 is out next week, the daily build updates for that will have that extra precision only shown for values of 1 millimeter or less.

For generated elevation grids using an average operation, you certainly could end up with non-zero digits beyond the precision of the original data as the elevation grid itself will be a floating point grid. The averaging is just adding up the floating point representation of the input values in each cell and averaging them. So if you had millimeter precision input Lidar and one value in a cell was 1 mm and one was 2 mm, you would end up with 1.5 mm as the value. It is more precision than the original data and not meaningful in the real world, but I think most users would expect that as the value. Then can always export to a rounded format at the desired precision if they want.

Choosing elevation units like centimeters during the grid process doesn't affect this, that just means that the floating point values will be 100X what they would be stored as if you had instead chosen meters. Where the units does matter is when exporting to a Global Mapper Grid (or another terrain format with fixed precision). In that case, values are stored as integers at the specified units, so if you choose Centimeters for units, the values will be rounded to the nearest centimeter. Most terrain formats using floating point, so you will get the exact values that are in GM regardless of the units selected (the values are just converted to the output units, but not rounded in any way).

Thanks,

Mike

sjf

Good to have that clarification.

And indeed: I export to GM grid with cm precision (as a last step) - so values will be stored as integers representing cm values.

I will check my workflow again after version v23.1 is released.

Thanks for your explanation and effort: this is an important and tricky part in lidar calculations.

greetings Stefan

bmg_mike

Stefan,

Note that my update to how much precision is displayed for elevations will not be in the initial v23.1.0 release, but will be in the daily builds as soon as that is out. It also only affects values very near zero (was <= 0.01, now <= 0.002), so it would mostly show up on difference grids like you were doing vs. normal data.