Global Mapper v25.0

sorting GPStime LAS/LAZ file

sjf
sjf Global Mapper User

Hi Mike and team,

I am working with Drone-LiDAR data and I need to extract single sweep lidar datasets.

This means I need to sort the LAS file to GPStime (ascending) and then extract or select single sweeps based on time and scan angle.

All points are perfectly in a time series. From the set I need sweeps from scan angle to scan angle of a bout 850 points and process them per individual sweep set.

I can do this by hand but it takes a very long time to sort a large Lidar set (not that large actually but still 17 to 20 million points).

I tried exporting the LAS file to LAS/LAZ and using the sort on GPStime (ascending) setting and this only works for a first part of the set: after that the GPS time sorting is broken.

I think there is some sorting limitation that is not there when I sort the dataset in the attributes table (click on the GPStime - it will start sorting). When I do that I can see the attributes - go to a GPStime and select a time series and export this: but after that the attributes table will reset and will start sorting again... Anyway - this only works on smaller sets.

(For 17 million poinst thsi takes a few hours...)

I have also set up a querry - this works OK on smaller sets but is also limited to a max number. I can not sort based on the view because still too large (there is sweep overlap so in a specific spot there are more sweeps and thus time series.

what can I do?

Cheers - Stefan

Answers

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    Hi Stefan,

    It sounds like there might be a bug sorting large point clouds by GPS time or something. I've created a bug (GM-13145) for this and will add some validation in the sorting code to see if I see the same issue.

    The sorting in the attribute table is an entirely different code path. I would actually expect that path to be much slower, but perhaps there is some performance issue in the direct Lidar sorting that I can identify as well. I would not expect it to take anywhere near that long.

    I'll let you know what I find.

    Thanks,

    Mike

    Global Mapper Guru

    geohelp@bluemarblegeo.com

  • sjf
    sjf Global Mapper User

    Thanks,

    One addition: the test file Lidar set is 77 million points (not 17 mentioned) still relatively small.

    Smaller sets are easier to handle: they sort relatively quick and (more importantly) I can select using a querry. Larger datasets are not easily querried (I get a too larges dataset: use the selction on screen - messge - but even that set is too large or can not be querried).

    Anyway: I think the essence of using LiDAR data is: Large file sets. (for sorting and querrying you can not tile the data). LAS/LAZ is an eay way to handle. Since LiDAR data is always time related (one pint at a time) a basic sorting nd selections method is time: would be helpfull to be able to sort and index using the GPStime code.

    Looking forward to the results.

    Stefan

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    Hi Stefan,

    I added validation to the sorting of Lidar on export, but so far haven't seen any issues. It is also quite fast (just a few seconds to sort 45 million points on my machine).

    The next v23.0.x daily build will do the sort validation if your Log Level is set to 'Debug (Extreme)' on the Configuration -> General -> Advanced dialog tab. You can try that and see if you are able to reproduce any issues with the sorting being incorrect on export to LAS / LAZ. If you can find a reproducible test case I would like to see that.

    There is another existing bug (GM-12149) about the slow performance of the Attribute Table with large Lidar data sets. I will try and look at that in the near future.

    Thanks,

    Mike

    Global Mapper Guru

    mikec@bluemarblegeo.com

  • sjf
    sjf Global Mapper User

    Will check.

    Did you check the sorting result until the last point. Easy to check because the new las file will have a new index number.

    I noticed that the first part was sorted but the rest was not (total 77 million points). Will check again and do a new test with the update.

    If it sorts well on export - that will do most of the work: sorting the attribute table is not convenient because sorting is done again after a selection of a sub-set and unselecting the selection.

    Stefan

  • bmg_mike
    bmg_mike Global Mapper Guru Moderator, Trusted User

    Stefan,

    Yes, the new validation steps through every point in the newly sorted point cloud and confirms that every point is <= the previous one based on the sort criteria.

    Thanks,

    Mike