Latest Data being reprocessed

Earlier this year, a compiler was upgraded on our supercomputing platform. This upgrade has apparently introduced a problem into the post processing program that compresses and prepares data files for transfer to the DISC where the data are publicly accessible. So that: all MERRA data after data date June 1, 2010 will be replaced and is currently no longer accessible at the DISC. This includes Monthly means from May 2010 on, but not the May 2010 data.

For those that may have downloaded data for the period June 1 - August 31, 2010, consider very carefully whether to continue using the data. So far, the only variables we have confirmed are corrupted are the roughness lengths for momentum and heat. We have no reason to think that other variables are not affected, hence the recall. The GMAO does not recommend the use of this data if you downloaded it before this recall.

The MERRA system and archive data are not affected by this issue. The system continues to run in near real time. Once the post processing code is fixed, the archive data will be reprocessed and posted to the DISC.

Late Oct 29 Update: The code has been fixed and tested. Corrected files are being reprocessed and sent to the DISC.

MERRA Climate Atlas

The MERRA Climate Atlas http://gmao.gsfc.nasa.gov/ref/merra/atlas/ has been opened and linked from the homepage. Numerous figures compare MERRA with previous and current reanalyses as well as global observation data when available. As is mentioned in the Introduction, new figures and comparisons are still planned. Comments and suggestions on the Atlas content are welcome.

MERRA Status and some images

Most of the MERRA analysis and results that have come along are intended for publication but not yet submitted. So, we are holding off posting them here until peer reviews are in. Quite a few are going to a special collection at the Journal of Climate (Deadline Oct 30). In addition, between the upcoming IESA workshop in Baltimore (Nov 1-3), AGU Fall Meeting and AMS Annual Meeting, there are a lot of interesting abstract and work to write on. So, expect more posts on the science being done with MERRA in the coming months.

The system continues at about 2 months behind present time available for download.

Here are some nice trajectory images and animations produced at the Scientific Visualization Studio, and written up for the Earth Observatory Image of the day.

Last Call for IESA workshop (more info in previous post)

Evaluation of Reanalyses - Developing an Integrated Earth System Analysis (IESA) Capability November 1-3, 2010, Sheraton Hotel, Baltimore, Maryland (Inner Harbor)

NOTE: The government perdiem rate in Baltimore has dropped to $144 for FY2011. If you have already made a reservation ,your room rate will be adjusted accordingly. If you have any questions or problems reserving at the new rate, you can contact Jill Reisdorf at 303-497-8636 or reisdorf at ucar.edu

Online registration available - abstract deadline 30 September 2010

Agenda: http://www.usclivar.org/Reanalysis2010.php

(Update) USCLIVAR IESA Workshop

Note that Travel Support and deadlines have been included:

Evaluation of Reanalyses - Developing an Integrated Earth System Analysis (IESA) Capability
November 1-3, 2010,
Sheraton Hotel, Baltimore, Maryland (Inner Harbor)
Online registration available - abstract deadline 10 September 2010
Sponsors: NOAA, NASA, NSF, and U.S. CLIVAR

Workshop objectives:
*Assess strengths and limitations of the new recent U.S. reanalyses and suggest where improvements of reanalysis products can be made; identify additional studies necessary to further elucidate the fidelity and usefulness of recent U.S. reanalyses;
*Develop definitions and identify goals of U.S. efforts leading to the forthcoming generation of integrated Earth system analyses (IESA);
*Develop diagnostics to quantitatively assess needed improvements in IESA products; and
*Demonstrate applications of reanalyses in climate and weather that would further highlight needed improvements in reanalysis products

The workshop welcomes contributions in the following areas:
*Assessment of the scientific strengths and limitations of the latest reanalyses, including for example: NASA MERRA, NOAA CFS, NOAA-CIRES 20th Century Reanalysis, NOAA GFDL coupled, SODA, ECCO.
*Applications of and requirements for reanalyses to support climate and weather research
*Efforts to move forward toward integrating (either directly through coupled assimilation or through co-varying products) multiple components of the Earth system (Atmosphere, Ocean, Land, Cryosphere, Chemistry, Carbon, etc.) in reanalyses.

The workshop is intended to bring together reanalysis developers and reanalysis users, and also to bring together various discipline specific analysis systems (including atmosphere, ocean, sea ice, land, and carbon). The anticipated result from the workshop would be improving two-way communications among the reanalysis developers and the climate and weather community, and increasing coordination among developers of the reanalyses for different components.

TRAVEL SUPPORT FOR THE REANALYSIS/IESA WORKSHOP, Nov 1-3, 2010
Please note there is a limited amount of funding available to offset travel to support students, postdocs, and/or early career scientists (receiving a PhD after Jan 1, 2008) from US institutions. If you are interested in support, please provide your name, institution and the type/amount of support you are requesting along with a vita and abstract for presentation at the workshop. A short letter confirming your affiliation should also be provided if you are a student. Please be aware federal employees are NOT eligible to receive funding.
Please submit this information by September 30, 2010.

AGU Session on Reanalyses

Please consider presenting at or attending session A20 "Progress and Uncertainty in Reanalysis Datasets" at the AGU 2010 Fall Meeting , 13-17 December 2010, San Francisco, California, USA.

By synthesizing observations from various conventional and satellite observing systems, reanalysis datasets provide comprehensive long-term descriptions of the climate system that are used in numerous researches and applications. Recently five new global reanalyses (ECMWF ERA-interim, Japanese 25-year ReAnalysis (JRA-25), NASA Modern Era Retrospective-analysis for Research and Applications (MERRA), NOAA-CIRES 20th Century Reanalysis (20CR), NOAA-NCEP Climate Forecast System Reanalysis (CFSR)) based on advanced assimilation systems from different agencies became available to the research community. The goal of this session is to advance the awareness of the progress and uncertainty in these new reanalyses, so as to benefit the research based on reanalysis datasets.

Presentations on the introduction, evaluation, inter-comparison, and application of reanalysis datasets are all welcome.

You may submit your abstract online at:
http://www.agu.org/meetings/fm10/program/abstract_submissions.php
The deadline to submit an abstract is 02 September 2010 — 23:59 Eastern Daylight Time (EDT).

Session A20 convener
Junye Chen, University of Maryland, jchen at essic.umd.edu
Michael Bosilovich, NASA/GMAO, Michael.G.Bosilovich at nasa.gov
Phillip Arkin, University of Maryland, parkin at essic.umd.edu
Wesley Ebisuzaki, NOAA/NCEP, wesley.ebisuzaki at noaa.gov

New results

Sincere apologies for the lack of postings on new results. We are pushing forward with quite a few papers on MERRA, and they are nearly ready for peer review submission. Most of what has been published here before was either accepted, or some straight forward analysis that would likely not be submitted. As the peer reviews come in and these manuscripts get accepted, more results will be shared here.

By the same token, the GMAO is very interested in research using MERRA and plans to host a listing of journal articles using MERRA data. While we scan the journals regularly, we appreciate notification of submitted and accepted papers. There are several contacts through the GMAO web page to do that.

20CR Status

For long term climate applications, 20CR should be a good option. Hopefully we will compare and contrast with MERRA soon. Here is the latest status from 20CR:

14 July 2010

Dear Colleagues,

Global four-times-daily atmospheric and surface fields spanning 1871 to 2008 from the Twentieth Century Reanalysis Project version 2 dataset are now available. They can be obtained in GRIB format via our partners at the National Center for Atmospheric Research http://dss.ucar.edu/datasets/ds131.1, and are also available at http://www.esrl.noaa.gov/psd/data/gridded/data.20thC_ReanV2.html in netCDF format, courtesy of our partners at the NOAA Earth System Research Laboratory, Physical Sciences Division (PSD), and University of Colorado CIRES/Climate Diagnostics Center (CDC). They will also become available via our partners at NOAA’s National Climatic Data Center (NCDC) through the NOAA Operational Model Archive and Distribution System (NOMADS) http://nomads.ncdc.noaa.gov in the near future. Beta composite plotting tools for 4-times-daily, daily and monthly means are now available courtesy of Cathy Smith of NOAA/ESRL/PSD and CU/CIRES/CDC at http://www.esrl.noaa.gov/psd/data/20thC_Rean/.

Additionally, pre-generated synoptic maps showing the ensemble mean analysis and analysis uncertainty of Sea Level Pressure and 500 hPa height over the Northern Hemisphere and Southern Hemisphere and for Sea Level Pressure and 850 hPa zonal wind in the tropics are available at http://www.esrl.noaa.gov/psd/data/20thC_Rean/hem_images.html.

Example animations of the Galveston hurricane of 1900 are available courtesy of NCAR at http://dss.ucar.edu/datasets/ds131.1/docs/galveston/Galveston1900.avi and for the Science on a Sphere visualization platform courtesy of Cathy Smith and Beth Russell at http://sos.noaa.gov/datasets/Atmosphere/reanalysis.html

A brief description of the project is given below. Please also visit the project home page at http://www.esrl.noaa.gov/psd/data/20thC_Rean/. A journal article describing the dataset has been submitted to the Quarterly Journal of the Royal Meteorological Society at their invitation:
Compo, G.P., J.S. Whitaker, P.D. Sardeshmukh, N. Matsui, R.J. Allan, X. Yin, B.E. Gleason, R.S. Vose, G. Rutledge, P. Bessemoulin, S. Brönnimann, M. Brunet, R.I. Crouthamel, A.N. Grant, P.Y. Groisman, P.D. Jones, M.C. Kruk, A.C. Kruger, G.J. Marshall, M. Maugeri, H.Y. Mok, Ø. Nordli, T.F. Ross, R.M. Trigo, X.L. Wang, S.D. Woodruff, S.J. Worley, 2009: The Twentieth Century Reanalysis Project. Quart. J. Roy. Meteor. Soc., submitted. Available online at http://www.esrl.noaa.gov/psd/people/gilbert.p.compo/20CRv2_Compoetal2010.pdf

Please feel free to contact the Project Leads, Gil Compo compo at colorado.edu and Jeff Whitaker Jeffrey.S.Whitaker at noaa.gov, with questions.

USCLIVAR Reanalysis Workshop Nov 1-3

Within the last year or so, and in addition to the availability of MERRA data, ECMWF Interim, NCEP CFSR and NOAA 20CR have made data available. Each system has its own strengths and weaknesses. USCLIVAR has initiated a workshop this year. One of the foci is to evaluate these new reanalyses in a climate sense, but also to determine what is needed for the integration of Earth system components (IESA, Integrated Earth System Analysis). In addition to the atmospheric reanalyses, there will be sessions for various disciplinary reanalyses (ocean, land, cryosphere, chemistry and aerosols), and how to couple them. This should be an excellent opportunity for the research community to interact with the reanalysis developers and begin to define the needs for subsequent reanalyses.

Evaluation of Reanalyses - Developing an Integrated Earth System Analysis (IESA) Capability
November 1-3, 2010,
Sheraton Hotel, Baltimore, Maryland (Inner Harbor)
For registration:
http://www.regonline.com/Checkin.asp?EventId=872029
Additional Information:
http://www.usclivar.org/Reanalysis2010.php

MERRA Downloads

The GES DISC has been tracking MERRA downloads, and there are a few interesting occurrences. In the first 5 months of 2010, the average download is 24Tb per month, and the total for 2010 is nearly equal to all of 2009. Since the purpose of the Data Subsetter is to help reduce the volume for faster downloads (and is the most popular method of access), we also look at number of files. On average, the number of files per month being downloaded in 2010 is 2x that in 2009. This is to be expected. The long time series became connected in August of 2009, and MERRA just caught present in April. Mostly, this is a sign that MERRA is getting out there. It's also not surprising that most downloads are in the US, but the access from the international community is growing as well, and Brazil has downloaded the most MERRA files outside the US to date.

Data through March 2010 is available on line, and MERRA processing has nearly reached the end of May 2010.


Figure 1 MERRA Monthly Downloads in Tb per month.

Figure 2 MERRA downloads in total number of files (in Millions) by country (Top 10).

Converting MERRA data to Grib2

Forwarded from Wesely Ebisuzaki:

Hi,

   I wanted the MERRA dataset (HDF/GDS) in grib2 format so g2grb.gs
was written.
g2grb allows you to write grib2 files from grads.

Testers wanted:
 requirements:  latest version of wgrib2 installed
                know grib2
                send results to wesley.ebisuzaki at noaa.gov

instructions: http://www.cpc.noaa.gov/products/wesley/g2grb.html


          Wesley Ebisuzaki

GEO on Continuing Reanalyses

GEO has published an article supporting the reanalyses development and international collaboration and coordination. Reanalyses represent a significant confrontation between the models and observations, and can be used to improve both, and produce a research quality value-added data product. While CFSR, Interim, 20CR and MERRA are all being released about the same time, they were not planned that way. So, there is a unique opportunity to have a community-wide check on our technological development in the models, assimilation, observations quality and computational resources (which affect the resolution and numerics). Generally, there will be advantages and disadvantages to each of these, which will take some time and effort to sort out. While they will be different, the important point will be is if they are making improvements on the representation of the Earth system.

In order to make progress, continuity and traceability in the system development will be crucial, but can only be attained if the respective centers have regular plans to produce the next round. If too many years go by, the next systems will be so different isolating and understanding the reasons for differences will not be possible. To little time, and the next reanalysis will not represent much difference, and the community will begin to feel the workload updating to the latest. Ideally, the next reanalyses should utilize all the information gained on the quality of the observations from the previous reanalyses, while advancing the representation of physical processes and data assimilation.

Effect of the Eyjafjallajokull eruption

Aircraft observations are a substantial part of the reanalysis system, especially over the oceans and regions where other high altitude conventional observations are infrequent. So that, the eruption of Eyjafjallajokull volcano will have many effects on reanalyses in general. Increased concentrations of aerosols can effect the remote sensing of atmospheric properties (Mount Pinatubo is the typical historical case). The aerosols will affect the radiance observations, and we will be monitoring MERRA's variational bias correction and assimilation of the radiance observations. It's also worthwhile to note that the MERRA system does not include interactive aerosols or aerosol assimialtion (currently under development), but rather incorporates only prescribed mean seasonal cycle variations.

The commercial aircraft restrictions that occurred also have affected the aircraft observing data set.The figures below show the number and locations of aircraft observations in a 6 hour window from ASDAR data system, before and after the major eruption that disrupted air travel.

It appears that the numbers of flights are beginning to recover, and the altitude of the eruption is on the order of 10km, so that the effects should be short lived, assuming the activity subsides.

Before: 12425 obs in 6 hours (green are used in the assimilation, red indicates qc rejection)
After: 6038 obs in 6 hours

MERRA real time production schedule

Stream three has finished March 2010, signifying a transition to real time production schedule. Production will lag real time by approximately 3 weeks, allowing collection of all the observations used in the retrospective periods. We are planning one week to review and quality assure each new month's data files, before being sent to the GES DISC. So, Each new month of MERRA data will be available online roughly 1 month behind present time, and will be released in 1 month segments.

Jan-Mar 2010 are currently being reviewed and should be available on the DISC within the next couple of days.

New Results: High Latitude Fluxes

There are a number of presentations that include MERRA data at the Workshop: Surface Fluxes: Challenges for High Latitudes in Boulder CO this week. Given the scarcity of observations at high latitudes and uncertainty in satellite observations, reanalyses may be useful data sets.

Michael Brunke, Xubin Zeng,( Uncertainties in global surface flux datasets in high latitudes.) compare various reanalyses and remote sensing products with tower and ship data from various field experiments (CATCH, FASTEX and SHEBA). The results show MERRA is much more comparable to the station data than the previous generations reanalyses, and in range of the remotely sensed products (e.g. Figure 1). The poster is available online.


Figure 1 Comparison of 6-hourly mean wind stress from MERRA, ERA-40, NCEP-R1, and NCEP-R2. The solid lines are the one-to-one slope and the dashed lines are the straight-line regressions with the regression slopes indicated.

Richard Cullather. Evaluation of Arctic Energy and Moisture Budgets in the MERRA Reanalysis. has also compared MERRA Arctic data with long term station measurements and some field experiment sites. For example, Figure 2 shows the time series of MERRA precipitation compared to observations made at a drift camp site. While there are some discrepancies in the magnitude of a few events, the comparison (and correlations of 0.74) seems quite reasonable given the uncertainties at high latitudes.

Figure 2 Drift Camp observed precipitation compared to MERRA. The points are 7 day running means over the period late 1987 through 1990 (a total of 1200 days).

In addition, there is:
J. Brent Roberts, Franklin R. Robertson, Carol Anne Clayson, Analysis of atmosphere-ocean surface flux feedbacks in recent satellite and model reanalysis products.

More on that later.

Full presentations will be posted online at the workshop site shortly after the conclusion.

Update to Data Streams

We are updating MERRA data to permit longer spinup of streams 2 and 3. The announcement and how it affects data downloads follows. Data files formally available in the MERRA data streams are being renamed "SPINUP_*" and will also be available for download. Most research and applications for the data will only require use of the "Mainstream" MERRA data.

Updating of the MERRA data products site to contain only Mainstream data (http://disc.sci.gsfc.nasa.gov/mdisc/data-holdings/merra-mainstream-and-spinup-data) will begin Thursday, March 11, 2010. A two step procedure will be followed to achieve this. In step one, all current  products for the years 1989 through 1992 and 1998 through 2000 will be deleted. It is strongly advised that the MERRA data for the two aforementioned periods not be accessed until this data update is complete.  In step two,  
 products will be used to repopulate these data years. We anticipate that it will take about two weeks to complete the data update. The remaining year
s of the data and all the MERRA data services will still be available during this time period. Once the update activities are completed, a new notification will be sent out and posted on the web. We thank you for your patience during this period.  If you have any questions or concerns please send an email to the GES DISC User Services (help-disc@listserv.gsfc.nasa.gov).


So when the update is complete, Stream 1 will contain 1979 through 1992, Stream 2 will have 5 years spinup (4 available for download) containing 1993 through 2000, and stream 3 will have 4 years spinup (3 available for download) containing 2001 through present.

2009 data should become available during the week of March 15.

Update to Precipitation Validation

Precipitation is a reasonable metric for evaluating reanalyses, because it is coupled to both the water and energy cycles and will be affected by the heating and water vapor increments, as well as variations in the dynamic response of the system to the data assimilation. Bosilovich et al (2008) evaluated precipitation among reanalyses, and also included results from the MERRA validation experiments. Below is an update of some of the Taylor diagrams presented there, now including both MERRA and ERA Interim reanalyses. See Bosilovich et al. (2008) for more explanaiton and the original citations for Taylor diagrams. These are showing that both MERRA and Interim precipitation are close in skill, and ahead of the previous generations of reanalyses. NCEP CFSR will be included in the next version of this evaluation. This was presented in part at the last AMS Annual Meeting (presentation at AMS).

MERRA Turns 30

Data availability at the DISC now extends for thirty years 1979 through 2008. This data is all online, and open to access. Production is presently in May 2009, and so should catch up to real time around the end of February. MERRA will continue into the future, as long as it is feasible to do so. At this point, we do not have a set production terminus.

Arctic System Reanalysis

Recognizing the importance of the Arctic region, as well as the tremendous amount of field work and supporting data sets, the Arctic System Reanalysis will perform a high resolution limited domain reanalysis for the Arctic. Recently, there was a Call for Community Involvement (unfortunately AGU member can only download this), printed in EOS (v 91, #2, 12 Jan 2010) looking for supplemental data and science interest in the project.

There has been quite a bit of recent evaluations of MERRA polar energy and water cycles. Some will be posted here shortly.

Pressure levels greater than surface pressure

Some questions have come in regarding differences between MERRA and other reanalysis at pressure 1000mb and 850mb pressure levels. It is very important to note that MERRA does not extrapolate pressure level data vertically greater than the surface pressure. The result is that there is undefined data points in much of the 1000mb fields. This will affect the representativeness of both time and area averages of MERRA data compared with reanalyses or other data that extrapolates gridpoints to pressure levels greater than the surface pressure. The GMAO provides a summary of the impact that this has on averaging. In addition, the FAQ will be updated to call out this difference with other data sets. The main GMAO www page also has other pages with useful practical information about the data and assimilation system.

CFSRR

Some News which may interest those who read these pages:

On 1/14/2010 6:16 PM, Suranjana.Saha@noaa.gov wrote:
> Dear Colleagues
>
> I am happy to inform you that the CFS Reanalysis is now complete for 31
> years (1979-2009) and the paper has just been submitted to BAMS.
>
> You can download it at the CFS website at:
>
> http://cfs.ncep.noaa.gov
>
> The first set of CFSR data will be made available to the public at NCDC
> and NCAR on Feb 1, 2010. The complete high resolution datasets will be
> distributed in the next 6 months or so.
>
> Thanks
> Suru Saha

MERRA Turns 30

31DEC2008 has just been produced by the reanalysis system. This extends the MERRA Mainstream to 30 Years. 2008 is still being checked for quality, but should be posted online at the MDISC within a week or so.

At approximately 1 year per six weeks, 2009 should be produced by the end of February. MERRA production will make a transition then, where we expect data to be released monthly. The production is delayed from real time to allow ingest of the most complete observations, but also to quality check the data files.

1993 Flood

Hydrologic anomalies can be persistent leading to extreme regional conditions. Much has been written on the 1993 flood in the central United States. The soil was saturated at the start of the summer becasue of above normal spring precipitation. During June, several strong synoptic scale events brought continued wet conditions. During July at the center of the flooding rains, convective events occurred almost every day. This post is just a quick look at the MERRA anomalies. It is interesting to note that the magnitude of the heavy precipitation anomaly is weaker than what was observed by gage measurement (figure below). Likewise, the precipitation anomaly for 1988 (severe persistent drought) is also a smaller magnitude than observed (not shown).


The following figure shows the time series of MERRA and gage July-Aug (JJA) precipitation area averaged for the sub-basins of the Mississippi. The curious feature in this comparison is that the various sub-basins in MERRA tend to track similar to each other, or perhaps, more than the gage observations. This is apparent from the mid 1990s through the early 2000s. It seems that the spatial scale of precipitation anomalies in MERRA is larger than observed.

These features still need to be more thoroughly studied.