Global Mapper v25.0

Grib2 support

Ice Age Mark
Ice Age Mark Global Mapper UserTrusted User
edited October 2012 in Technical Support
Howdy there Mike,

The WMO and NWS are now putting out all of their gridded forecasts, models, and climate data as a GRIB2 file format (.grb2) instead of the previous GRIB1(?) which GM supports. (Don't know why they do such things.) Any chance of getting the support updated to the new kind, at your convenience of course? This would save the long step of using their cumbersome software decoding program first to get such data into GM for some adjusting by script.

Thanks much,

Mark

Comments

  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    There should already be some support for GRIB edition 2 data. Do you have any files in GRIB2? Do they load?

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • Ice Age Mark
    Ice Age Mark Global Mapper User Trusted User
    edited March 2012
    Hey Mike,


    The analysis and forecast model GRIB2 files I'm trying to use aren't working. The examples are too large to attach (35 & 17mb), so I used the links to where I get them instead. The first kind at

    http://soostrc.comet.ucar.edu/data/grib/ruc/20120308/hybrid/

    should contain a dozen or so model weather parameters, with some numeric value for that parameter for each 13km grid cell. It doesn't open at all.


    The second kind

    ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/rtma/prod/rtma.20120308/

    opens, but it opens like a raster image with RGB/palette values for each cell, instead of the numeric value that should be there. On loading I have to choose the GRIB file type, so it seems like GM is not recognizing the newer .grb2 file type. Then, the individual parameter names don't show up in the sub-dialogue choice that should allow a single parameter and it's grid cell values to be selected for loading.


    The raw .grb2 files export out of the "NOAA Weather and Climate Toolkit" program as a "GeoTIFF (32-bit Real Values)" file; which works well in GM (except for the units of the real values, which I'll tackle next). An example is also too large to attach (7mb), but it seems to produce an "elevation" Geotiff with the model numeric values replacing elevation values for each grid cell.


    These kinds of files are updated regularly, like satellite or radar data, and download on an FTP type server, so if I can skip the NOAA software, I can download and overlay this data with other layers and use a customized background; like I do with other weather charts, sat pics, and radar using simple scripts. It appears that all U.S. and global weather and climate data is now in this newer format, and none is in the older type already supported by GM. There are tons of this data available, and loading them directly into GM would be very convenient.


    Thanks again,


    Mark


    V13.10 (B030512)


    p.s. - If you're into the severe storms, the free NOAA software mentioned above can decode the new Dual-Polarization radar (.bin) files you looked at for me recently that GM cannot open. (Presumably, because they have a "radial" coordinate system commensurate with the way raw NEXRAD data is collected.) The exported "GeoTIFF 32-bit real-value" image loads right into GM (in my case over a nice customized background), and gets rid of the annoying distorted Geographic Projection of all Nexrad displays. It can also create a really nice 3D KMZ radar image that looks/works great in Google Earth viewing. It even has either a split screen or alternate screen GE viewer built in that automatically loads the decoded file. Check it out.
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    Thanks for the info, I will take a look and see if I can get our GRIB importer to pull these files in. GRIB is one of those generic formats like HDF/HDF5 and netCDF that is generic so each type has to have a separate importer written.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • Ice Age Mark
    Ice Age Mark Global Mapper User Trusted User
    edited March 2012
    Hey there Mike,

    Any word on getting those GRIB2 files to import directly into GM without pre-processing them with the NOAA software?

    Thanks,

    Mark
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    Sorry it got pushed to the back-burner with the v13.1 release, but I'll try and take a look at it today or tomorrow.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    Do you have a new link for the 2nd type of file that does open? The link in your original message doesn't work.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    For the first set of GRIB2 files, the issue is that the data is encoded using JPEG2000 encoding inside the GRIB2 files and the GRIB2 libraries only work with the JASPER JPEG2000 decoding library, which we don't currently use. I have added this to the todo list, but unfortunately it is a larger task than I originally imagined.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • Ice Age Mark
    Ice Age Mark Global Mapper User Trusted User
    edited March 2012
    Hi Mike,


    Thanks for taking a look at this (again). Sorry about the bad timing. I was hoping it was only necessary to make some small changes to the existing GRIB support. I would only reiterate that, rather than being yet another pesky "unique" file type, huge amounts of public Climate, Weather (observed and forecasted), and Geophysical data (both archived and real-time) use this GRIB2 format. Maybe that can influence where it falls on the ToDo list.


    For your own personal interest (if you don't already have it), here is a link to some less well known weather satellite products. Colorado is unique in being visible to both the GOES East and the GOES West geostationary satellites, and gets it's own special 1km resolution visible images from each during daylight hours, as you can see here (top two images):


    http://rammb.cira.colostate.edu/ramsdis/online/goes-west_goes-east.asp


    The "loops" are great for watching convective activity develop. Also, check out the "GOES-R..." link (upper left) for other interesting products. (GOES West imagery is now from the newer and better GOES 15 satellite.)


    Thanks again,


    Mark
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    I am definitely a weather junkie, although here in the Denver area there hasn't been much to get excited about this year other than one big snow storm last month. Not that I'm complaining about warm sunny weather!

    I will bump this to a higher priority and maybe even be able to pull in that Jasper JPEG2000 library just for the GRIB library to use and then maybe support for these can be added. It would sure have been nice if they had just used one of the multitude of already standard formats rather than inventing yet another format to distribute their data in.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    I was able to pull in the JPEG2000 library that the GRIB2 stuff wants and get those files decoding. I have placed a new build at http://www.globalmapper.com/global_mapper13.zip with the latest changes for you to try. Simply download that file and extract the contents into your existing v13.xx installation folder to give it a try. If you are using the 64-bit v13 version there is a new build at http://www.globalmapper.com/global_mapper13_64bit.zip .

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • Ice Age Mark
    Ice Age Mark Global Mapper User Trusted User
    edited March 2012
    Hey Mike,


    Well, we're off to a running start with this, I hope it's not a "hairball". (I hate it when someone does you a favor, and then you need more still from them.) It seems like GM is rendering the maps as raster imagery that is not geo-located or scaled; rather than as "psuedo"-terrain data like a "32 bit elevation GeoTIFF", where it is rectified/scaled but the elevation value for each grid cell pixel represents some single atmospheric parameter. (I was a little worried when you mentioned JPG2000, which I think I know to just be imagery, but I figured I didn't understand about GRIB2.) Some clues: In the metadata, a pixel is showing up as about 5070 arc degrees square, and the whole Earth's global grid is a match-head sized area at the lower left of the image. The "Z" value for pixels is the raster image palette value instead of a numeric value. The 2 files I tested open quickly, the sub-menu selection works right, and I don't get asked for the file type now, but I do get asked for a projection. It seems like the raw GRIB2 files must contain their location information because they have it for GM coming out of the NOAA software (Geographic Projection). Somehow GM has to think of it like a DEM and not raster imagery, I guess. Here is the link to a small and simple kind of data file I would be using very often, as it is almost real-time:


    ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/rtma/prod/rtma.20120325/rtma.t18z.2dvarges_ndfd.grb2


    Using one such pre-processed file in GM I produced the attached map by using "Combine terrain layers" to subtract the "dew point temperature" layer from the "temperature" layer to get a "dew point depression" map, which directly reflects the "dryness" of the air. This one was for mid-day Friday. With the HSV shading, red is very dry (Death Valley, southern High Plains) and purple is very humid (Delaware).


    Thanks for your time with this (once again) unconventional use of GM. Maybe someday there will be a Weather and Climate "version".


    Mark
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    The GRIB2 files that wouldn't load previously use JPEG2000 to encode the data for each layer, so each one seems to be just a RGB image already rendered as JPEG2000 and not a grid of raw values (like an elevation grid).

    The scale problem is just a matter of what projection is selected. These GRIB2 files appear to use a Lambert Conformal Conic projection of some kind, I just haven't quite worked out how to decode it yet. If you select the right projection it should be placed and scaled correctly, but I will work on getting that decoded automatically. I'll also take a look at your sample file and see what I can find there.

    Thanks,

    Mike
  • global_mapper
    global_mapper Administrator
    edited March 2012
    Mark,

    I was able to get these working now as grids of data rather than just a rendered image so whatever shader you want will work. I also got the positioning of the Lambert grids correct automatically so they should come in properly placed. I have placed a new build at http://www.globalmapper.com/global_mapper13.zip with the latest changes for you to try. Simply download that file and extract the contents into your existing v13.xx installation folder to give it a try. If you are using the 64-bit v13 version there is a new build at http://www.globalmapper.com/global_mapper13_64bit.zip .

    Let me know if I can be of further assistance.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com
  • Ice Age Mark
    Ice Age Mark Global Mapper User Trusted User
    edited April 2012
    Hello Mike,


    I just got to fooling around with the GRIB2 stuff using the support you added. Everything seems to be working perfectly, the only caveat being that I have to set my display units to meters so that the numeric grid values match the "elevation" value so I can contour, combine layers, etc. Not a problem for me, I just mention it in case anybody else uses this stuff. (I guess it's because meters are inherited from the projection as the vertical unit.) Thanks very much for this, Mike. It was taking 10 minutes to run each grid layer through the other software before even using GM, and some of it gets updated every hour I think you can gain some new users as well from the direct load GRIB2 support.


    I attached a map made from RTMA GRIB2 data from late afternoon on the day the wildfires started there last week. The hatched area is sustained wind of over 30 mph, and inside the purple line the dew point depression was over 55 F degrees! Definitely what I used to call "nose-bleed weather" down in Southern California, where the Santa Ana Winds produce similar red flag fire weather conditions.


    Thanks once again,


    Mark
  • Marcel Wijnberger
    Marcel Wijnberger Global Mapper User
    edited October 2012
    hello, i'm also interested in importing grib-files into global mapper v13. i life in the netherlands. how would i start. Perhaps also import metar data. regards, Marcel
  • global_mapper
    global_mapper Administrator
    edited October 2012
    Marcel,

    Are you able to just load the files using the File->Open Data File menu command. The GRIB files should load in the latest release.

    Thanks,

    Mike
    Global Mapper Guru
    gmsupport@bluemarblegeo.com
    http://www.globalmapper.com