Main stream and spinup data

The original plan for MERRA production was to run in three streams, optimizing the computers processes. The original three streams were 1) 1979-1988, 2) 1989-1997 and 3) 1998-present. These were initialized with 2 years of coarse resolution analysis, followed by 1 year at the native (1/2 degree) resolution. Recently, streams 1 and 2 caught up to the beginning of the respective subsequent streams, connecting the time series. Streams 1 and 2 were continued providing overlapping data with the beginning of the next streams, in order to test the variance of the system and the viability of the initialization. At present, each of the overlapping periods are now 3 years duration (1989-1991, and 1998-2000).

We have been evaluating the initial conditions and the continued spinup of streams 2 and 3. In general, there are few differences in the meteorology between the overlapping data. In fluxes (such as precipitation), we do not a small difference at the beginning which gets smaller in time. The differences would likely not affect any scientific results. Slightly larger differences can be seen in slowly varying states, such as the root zone wetness.

At present we are documenting these differences and will provide a report on the overlapping period. In the mean time, this letter is provided to users to alert them that we will be changing the transition times of the streams to utilize the additional data produced at the ends of stream 1 and 2. This does not invalidate the data currently available, but is considered only a minor scientific revision. Once the data is provided to the DAAC and prepared for user access, the new streams and transitions will be as follows:

• Stream 1 1979-1991
• Stream 2 1992-2000
• Stream 3 2001-present

Connecting these streams will be presented to users as the primary MERRA data, or the “main stream” data. Access to Stream 2 1989-1991 and Stream 3 1998-2000 data will still be provided, but the data will be considered secondary and called “spinup” data. This will effectively increase the spinup period for Streams 2 and 3 to 4 years of native resolution analysis.

We anticipate the transition from the original streams to this extended spinup configuration to occur in February 2010, nearly coincident with Stream 3 catching up to real time. Regular updates on this will be made after the holidays.

Have a wonderful Holiday Break!!

Accessing MERRA: Data Subsetter

This week, I needed a daily average surface flux subset of MERRA data. Locally, we have the 1 hourly data on a mass store system, but that is primarily archive, and not best used for routine analysis. The source data files are 269Mb each, and I needed one for each day from July 1987 through Dec 2007, which would have been a 2TB request. So, I used the MDISC to create data files specifically for the comparison from the Data Subsetter.

With the subsetter, I selected the 4 variables needed for the experiment, the time range (Jul1987-Dec2007, or 7489 days), the region could have been trimmed, but was left at the default (global). Daily averages, not the 1 hourly averages were preferable, so the the daily mean box was checked. HDF was suitable, so it was left at the default (as opposed to NetCDF, more formats may be added later). The subsetter provided a text file with the http links to the reduced data request. The links activate a program that does the subsetting and streams the requested data back. This text file is used as input to a Linux call to wget, which does the work of opening the http.

It took only about 16 hours (mostly over night), but the result was only 23Gb of disc space. The daily mean check box also saved me the time to process those daily means. Lastly, I did use the Mirador search to access and download one of the unaltered source data files, just to verify the variables and the daily averaging, and there was no difference between the Subset processed averages and daily averages computed manually.

The subsetter has evolved a lot over this past year, and additional functionality is planned. However, in it's present form, it should be a very useful tool in accessing MERRA data.

Water vapor feedback

An early decision in producing MERRA was to release data to the science community before the completion of the full time series, in the hopes that early analysis would provide insight to the data in a timely manner. In this months Journal of Climate, Dessler and Wong evaluate the water vapor feedback in the climate system using AR5 models, ERA40 and MERRA. The models and reanalyses all show consistent positive feedback (see the excerpt figure below). MERRA (point L in the figure) does show a bit more variability than the climate models (points A-J). A contributing factor to that variability may be a smaller number of years considered for MERRA, as the data was analyzed early in the production. Even so, the inclusion of MERRA in this research does help characterize the system and contribute to the understanding of its capabilities for climate research. At present, 1979 through Feb 2008 are approved for release at the MDISC data site.