Ancillary Land and Ocean data

Two data sets for Ocean and Land have been released through the GES DISC, derived from MERRA, but not included in the original data collection. The ocean data has 1 hourly fluxes, meteorology and stresses over both open water and ice. the MERRA-Land data is an offline reprocessing using MERRA atmospheric forcing with bias corrected precipitation to produce a new set of surface fluxes and land states (Reichle et al. 2011).

See the release page for more information.

April 2011 Precipitation Extremes

During last April, record or near-record precipitation occurred across a large section of the United States (from the Midwest through the Ohio Valley). This rain, and likely snow melt from the Northern Great Plains, are contributing to the current prolonged and severe flooding along the Mississippi River and its delta. In addition, a prolonged drought of varying degrees persists across the Gulf Coast states.

MERRA precipitation (color shaded) with CPC gauge observations (black contour) time averaged for April 2011. (units: mm/day)

This week, MERRA data for April 2011 was released at the MDISC, roughly two weeks behind real time. Preliminary comparison with the CPC gauge data shows that MERRA precipitation is generally weaker than observed especially in southern Missouri, though the maximum in Pennsylvania is an overestimate. Because the reanalysis system is strongly constrained by observations, the weather systems that produce the rain, and hence the occurrence of rain events, are faithfully reproduced. The physical process of producing the precipitating water then leads to the error in the data product (assuming that the rain gauges capture the extent of the precipitating mass). It is worthwhile to note that MERRA does not assimilate precipitation observations over land, as in NARR.

Giovanni for Quick look and data download

Recently a question about some regional climate data came in, it needed to be independently checked. It seemed like a good opportunity to make use of the Giovanni portal into the MERRA data. Many of the MERRA data collections are available to Giovanni, even 2D one hourly data. MERRA data is directly linked through the Giovanni page, but also from the MERRA data holdings page, which may be more desirable since there is more documentation about what is in each MERRA collection.

Once at the MERRA/Giovanni page, there are options to choose variable(s), region, level and time. There are several hardwired calculation/visualization options in which the data can be presented. Depending on the size of the request, there may be a few minutes before the results are available. However, once they are there is options to download the visualizations, or the data. Both source data and processed data can be retrieved. At this time, HDF, NetCDF and ASCII formats are supported for download.

The FTP Subset is the best way for large requests, and OpenDAP/GDS still serves a purpose for some on the fly live calculation jobs needing flexibility. However, Giovanni fills the middle ground in data visualization and retrieval. It certainly is worth a test drive, and may be a useful option in many cases. Given the range of what can be done with MERRA data, the wide array of ways to access the data should fit most needs.

Latest Data being reprocessed

Earlier this year, a compiler was upgraded on our supercomputing platform. This upgrade has apparently introduced a problem into the post processing program that compresses and prepares data files for transfer to the DISC where the data are publicly accessible. So that: all MERRA data after data date June 1, 2010 will be replaced and is currently no longer accessible at the DISC. This includes Monthly means from May 2010 on, but not the May 2010 data.

For those that may have downloaded data for the period June 1 - August 31, 2010, consider very carefully whether to continue using the data. So far, the only variables we have confirmed are corrupted are the roughness lengths for momentum and heat. We have no reason to think that other variables are not affected, hence the recall. The GMAO does not recommend the use of this data if you downloaded it before this recall.

The MERRA system and archive data are not affected by this issue. The system continues to run in near real time. Once the post processing code is fixed, the archive data will be reprocessed and posted to the DISC.

Late Oct 29 Update: The code has been fixed and tested. Corrected files are being reprocessed and sent to the DISC.

Converting MERRA data to Grib2

Forwarded from Wesely Ebisuzaki:

Hi,

   I wanted the MERRA dataset (HDF/GDS) in grib2 format so g2grb.gs
was written.
g2grb allows you to write grib2 files from grads.

Testers wanted:
 requirements:  latest version of wgrib2 installed
                know grib2
                send results to wesley.ebisuzaki at noaa.gov

instructions: http://www.cpc.noaa.gov/products/wesley/g2grb.html


          Wesley Ebisuzaki

MERRA real time production schedule

Stream three has finished March 2010, signifying a transition to real time production schedule. Production will lag real time by approximately 3 weeks, allowing collection of all the observations used in the retrospective periods. We are planning one week to review and quality assure each new month's data files, before being sent to the GES DISC. So, Each new month of MERRA data will be available online roughly 1 month behind present time, and will be released in 1 month segments.

Jan-Mar 2010 are currently being reviewed and should be available on the DISC within the next couple of days.

Update to Data Streams

We are updating MERRA data to permit longer spinup of streams 2 and 3. The announcement and how it affects data downloads follows. Data files formally available in the MERRA data streams are being renamed "SPINUP_*" and will also be available for download. Most research and applications for the data will only require use of the "Mainstream" MERRA data.

Updating of the MERRA data products site to contain only Mainstream data (http://disc.sci.gsfc.nasa.gov/mdisc/data-holdings/merra-mainstream-and-spinup-data) will begin Thursday, March 11, 2010. A two step procedure will be followed to achieve this. In step one, all current  products for the years 1989 through 1992 and 1998 through 2000 will be deleted. It is strongly advised that the MERRA data for the two aforementioned periods not be accessed until this data update is complete.  In step two,  
 products will be used to repopulate these data years. We anticipate that it will take about two weeks to complete the data update. The remaining year
s of the data and all the MERRA data services will still be available during this time period. Once the update activities are completed, a new notification will be sent out and posted on the web. We thank you for your patience during this period.  If you have any questions or concerns please send an email to the GES DISC User Services (help-disc@listserv.gsfc.nasa.gov).


So when the update is complete, Stream 1 will contain 1979 through 1992, Stream 2 will have 5 years spinup (4 available for download) containing 1993 through 2000, and stream 3 will have 4 years spinup (3 available for download) containing 2001 through present.

2009 data should become available during the week of March 15.

MERRA Turns 30

Data availability at the DISC now extends for thirty years 1979 through 2008. This data is all online, and open to access. Production is presently in May 2009, and so should catch up to real time around the end of February. MERRA will continue into the future, as long as it is feasible to do so. At this point, we do not have a set production terminus.

MERRA Turns 30

31DEC2008 has just been produced by the reanalysis system. This extends the MERRA Mainstream to 30 Years. 2008 is still being checked for quality, but should be posted online at the MDISC within a week or so.

At approximately 1 year per six weeks, 2009 should be produced by the end of February. MERRA production will make a transition then, where we expect data to be released monthly. The production is delayed from real time to allow ingest of the most complete observations, but also to quality check the data files.

Accessing MERRA: Data Subsetter

This week, I needed a daily average surface flux subset of MERRA data. Locally, we have the 1 hourly data on a mass store system, but that is primarily archive, and not best used for routine analysis. The source data files are 269Mb each, and I needed one for each day from July 1987 through Dec 2007, which would have been a 2TB request. So, I used the MDISC to create data files specifically for the comparison from the Data Subsetter.

With the subsetter, I selected the 4 variables needed for the experiment, the time range (Jul1987-Dec2007, or 7489 days), the region could have been trimmed, but was left at the default (global). Daily averages, not the 1 hourly averages were preferable, so the the daily mean box was checked. HDF was suitable, so it was left at the default (as opposed to NetCDF, more formats may be added later). The subsetter provided a text file with the http links to the reduced data request. The links activate a program that does the subsetting and streams the requested data back. This text file is used as input to a Linux call to wget, which does the work of opening the http.

It took only about 16 hours (mostly over night), but the result was only 23Gb of disc space. The daily mean check box also saved me the time to process those daily means. Lastly, I did use the Mirador search to access and download one of the unaltered source data files, just to verify the variables and the daily averaging, and there was no difference between the Subset processed averages and daily averages computed manually.

The subsetter has evolved a lot over this past year, and additional functionality is planned. However, in it's present form, it should be a very useful tool in accessing MERRA data.