MERRA turns 11, Global Energy Budget

Last week, MERRA had produced 11 years of data across all 3 streams (79-82, 89-92 and 98-00). Here is a quick evaluation of the global energy budget so far. The reference values are a recent update from Trenberth et al. (2009, BAMS accepted DOI: 10.1175/2008BAMS2634.1). There is a difference between MERRA and this reference, where MERRA is averaging all 11 years, while the reference data was developed from theory as well as observations and reanalyses from March 2000 - May 2004. MERRA is presently in 2001, so it will be a couple months before we can reproduce the global energy budget for this exact time.

(In the figure, MERRA data includes 130 months written in red)

At the surface, too much shortwave, and not enough longwave reach the surface, while at the top of the atmosphere, too little shortwave is reflected and too much longwave is emitted. This suggests that the cloud fraction is too low or cloud optical properties too thin (or both). It is interesting to note that the values tend toward improving in time toward the 2000s. A discussion on the trends in the global energy budget is being prepared.

Some other interesting features occur. The overestimate of the shortwave energy (primarily over the ocean) into the surface is not translating into increased sensible, latent or upward longwave fluxes. Instead, it contributes to the imbalance at the surface, and since the SSTs are prescribed the surface is not warming. Also, the difference of precipitation and evaporation reflect the influence of hte analysis increments on the water budget (globally taking water out of the system).

Trenberth et al (linked above) provide more discussion on the global budget, as well as comparisons to the NCEP and JRA reanalyses.

Severe Freeze in South America

While looking over the area average time series of the Amazon River basin produced by MERRA, a very cold event was apparent in July 1981. Figure 1 shows the time series of 850 mb temperature for the basin, and the event is several K colder than earlier in the reanalysis. The cold event starts around July 16 1981, and lasts a few days. Looking at the observation reports from this period, did not show any major problems with the input obs, yet the analysis increments of water vapor show a strong shift, starting with this event (Figure 2). So, the shift in the increments along with the temperature spike suggests, at first glance, a problem. However, the lack of rejected observations or increased error in the forecast says the observation and assimilation are OK.

While considering the problem, Austin Conaty used Google Scholar to search for cold events in the Amazon. It turns out that there was a strong system pushed northward across the continent bring cold air and a freeze to Brazil (Fig 3 from Fortune and Kousky, 1983). By looking at the Amazon Basin area average, we see the cold anomaly push into the basin. While it looks out of place, it is a real feature. We'll file this under, you learn something new every day.

Figure 1 Time series of Amazon basin average 1 hourly 850 mb temperature (K) for 1979 through 1981. The July 1981 cold anomaly stands out in this relatively short time series.

Figure 2 Time series of area averaged analysis increment of water vapor daily averages (dqvdt_ana, mm/day) for the Amazon basin from May 1981 through Sept 1981. As the cold air enters the region, the analysis increments shift (becoming smaller negative values).

Figure 3 Synoptic maps of the cold air event, brining a sever freeze to Brazil, copied from Fortune and Kousky (1983, MWR).

Known Issue #1, east Atlantic anomaly

Recently, an evaluation of the global energetics identified an anomalous feature, a bullseye, off the coast of Portugal during most of 1979 and 1980. The anomaly affects many tendencies and variables in this localized region. Figure 1 shows the JJA 1979 Analysis Increments of temperature at 500 mb, the center of the strong negative increment is a persistent radiosonde that appears to be flawed, compared to other nearby sondes. However, it was not rejected by the data assimilation's quality control. The sonde itself is being researched as to whether it is mislocated or just poor quality. Regardless, 1979 and 1980 data have a localized problem. The data will still likely be made available, as other regions around the globe, and certain global parameters are not as dramatically affected. However, we will try to alert users to this issue, as it will affect any investigation of the weather in the eastern Atlantic Ocean or possible western Europe (Figure 2).

The current plan is to re-run the affected period at a later date, however, that re-run will not replace the existing data, but be placed alongside the existing data. We will continue to use and evaluate the current 1979 and 1980 data, and if other issues arise, fixes could also be incorporated into the re-run. Also, with this problem, a new page will be created for the MERRA WWW site to catalog issues with the data as they arise, and include any actions to those issues.



Figure 1 JJA 1979 analysis increments of temperature (DTDT_ANA). The questionable sonde is causing the negative increments circled.


Figure 2 Temperature differences (MERRA-JRA25) at 850 mb for JJA 1979. The low temperatures resulting from the questionable sonde are apparent west of Portugal. The featuer is also apparent in the surface pressure fields (not shown)

Integrating Earth Observations

One of the advantages of reanalyses is that many different sources of observations are combined in a global gridded consistent system. We have a brief overview of MERRA and the challenges facing reanalyses published in the online IEEE magazine Earthzine www.earthzine.org. There are a number of interesting interviews and articles at the site as well.

MERRA Short Course at AMS

We will have a short course at the AMS Annual meeting January 11 in Phoenix AZ. It will cover an overview of the system and physics, the validation that we have already done, but the most important and interesting component will include several hands on tutorials working with data online. Given that the volume of data will be quite large, the subsetting and previewing tools should be useful to researchers to target the data they require. The audience will likely be graduate students or researchers who work directly with the data. Several tools that are used regualrly at the GMAO to manipulate the data, formats and grids will be demonstrated.

The announcment is on this page: http://www.ametsoc.org/MEET/annual/call.html and the registration is currently open. The AMS will shortly publish an agenda, and we are currently working on adding more details to that regarding the tutorials.

While MERRA won't be completed by then, it will be nearly halfway complete. Also, by then data will be accessible online.

Status and File Spec

Production of MERRA data has been progressing at 10 days per day on three streams, which is what is expected. Data monitoring is going on regularly, and the system seems to be moving along at the expected data quality. We have some time series routines being run, and will report on those when we achieve 8 full annual cycles. Currently, Streams 1 and 2 have more than 2.5 years each, and stream 3 has 22 months.

The MERRA File specification document has also been updated and reflects exactly the data files being produced. The document can be found at the bottom of the MERRA home page, and a link is provided here.

MERRA mail list

A mail list has been established to pass on news and alerts regarding MERRA especially the data. Send an email to majordomo@listserv.gsfc.nasa.gov with the body of the message "subscribe merra-news"

You will receive a message with instructions to confirm the subscription.

AMS Annual Meeting - Abstract deadline Aug 1

There is a planned session on reanalyses at the AMS Annual meeting, co sponsored by the Committee on Hydrology and Climate Variability and Change Committee. The deadline is August 1. Please see the call for papers:
http://www.ametsoc.org/MEET/annual/index.html

Status July 17

Late last week, the computing platform running MERRA was brought down for scheduled maintenance including an upgrade to the operating system. When it was brought back online, numerous scripts and control jobs for MERRA were broken. The science and executables for MERRA remain, but the control of jobs and data flow needed to be patched for the new OS. At this point, the three MERRA streams are processing forward. However, some post processing scripts are still being patched, such as the information on our Production Progress and Quick View pages. Those should be back on line in a few days.

Brief Comparison with Interim Reanalysis

Recently the ECMWF Interim reanalysis has been released (see http://data.ecmwf.int/data/). We have a quick overlap and look at a comparison with MERRA at monthly time scales for January 1998.

The figure below shows two features of the MERRA system we have been tracking since the system was under development, 1) negative zonal wind bias in the tropics and extratropics and 2) dry bias in the lower troposphere, especially the tropics. These are apparent against each of the existing long reanalyses (see the quick look page for more comparisons). However, in comparing ERA40 and the new Interim reanalyses with MERRA, the magnitude of the differences is smaller compared to Interim.

At this time monthly files are not available on the ECMWF site. When those become available for download, we will integrate the Interim data into the quick look pages for comparisons.



(Gary Partyka, GMAO, downloaded the Interim data and performed the comparison and evaluation.)

Status June 15

Just a brief update. The updated sonde fixes seem to be going well. The restarted Stream 3 is into March 1998, and we have evaluated the early data. Stream 1 has completed 1979, and Stream 2 will complete 1989 in a day or so. While their has been scheduled downtime for the computing system (Discover), recent upgrades have paid off with 11 days/day of throughput per stream.

Status June 8

Streams 1 and 2 have been progressing, and full years should be available by the end of this week.

Stream 3 was held for the revision to the sonde data (station identifiers and launch times, see the May 23 post). However, it was decided to restart from January. Stream 3 is presently restarted and at Jan 11 1998.

MERRA Quick View

We've just opened a new link on the MERRA WWW page. It is a version of what our monitoring team is using to look quickly at various aspects of the circulation and water cycle. We'll be adding some other figures, like time series of the monthly data as well.

http://gmao.gsfc.nasa.gov/research/merra/prequel/view.php

Status May 23

All three runs streams are progressing forward, 5 months in, approximately. However, as some folks here were making corrections a few weeks ago there were some unusual reports in the sonde data. Further investigation has found a problem that is fairly easily correctable. Actually there are two problems, and they change in time.

Starting around 1988, the actual launch time isn't being considered, just the synoptic time of the launch. From 1988-1992, this occurs in less than 1% of the launches at any given synoptic time. After 1992, the rate increases to around 10-15%. The consensus is that the impact to the 1989 MERRA stream is small, and we can have this fixed by the time we cross 1992. However, in 1992, a more serious problem occurs. Then, some sondes are not labeled correctly as to their manufacturer. This then affects whether the radiation correction for certain sondes is called, and the radiation correction is important to handle correctly. This occurs at a rate of about 15% at any given synoptic time, also a fairly large number. Again it can be corrected for the 1989 stream, before it gets serious.

The 1998 stream is another matter. Both of these problems are occurring at a rate of about 15% per synoptic time in January 1998 and on. So, the data we have processed for the 1998 stream has a flaw than needs to be corrected. In order to stay timely with the MERRA processing, we will continue the 1998 stream to 1 July 1998, at which time the fixed sonde data will be implemented. The Jan - Jun 1998 data processed so far will not be released. Instead, we will continue the 1989 stream to recover this period, and there will be overlap through 1998 to ensure that the transition of streams is as smooth as possible.

A word about the source of this error. We are using recently reprocessed sonde data from NCEP, and working with NCEP closely on monitoring and quality controlling this data. This is the data set that would also go into the NCEP CFSRR, but is not the same as has been used in previous reanalyses.

So in summary, there is no impact on the 1979 data stream. The extremely small number of launch time errors early in the 1989 stream are considered to be nominal, and the sonde data will be fixed as soon as possible. The 1998 data from Jan - Jun must be redone. The fix to the data will be implemented on Jul 1998.

Status May 19

All three streams have progressed through March and into April of their respective first years. Evaluation of the primary diagnostics and data usage seems as expected, and these first few months have passed monitoring tests. The data has been approved for transfer to the GES DISC, where the data will be available for download.

We are still testing some of the software for accessing the data files. However, it won't be held for any significant length of time. When we get a schedule for opening the data, it will be posted on this site.

The runs seem to be going smoothly, and with throughput back near the 10 days per day level. (a good sign).

Status May 14

All three MERRA streams have been running, and past their respective first month. We have started evaluating the first months, so far no surprises. The evaluation is not yet completed. The vertically integrated budget diagnostics are fully incorporated in this version, and the online file specification document should be updated soon. The budgets are complete and deep into the terms of the equations, owing to the thoroughness of Larry Takacs and Max Suarez. We'll try to post some more information on this soon.

We still are only getting 8 days per day throughput. A planned upgrade to the computing system should improve I/O, and is expected to help regain the throughput.

Status May 2

The fixed tag of the data assimilation system has been handed off to operations. They are preparing to restart the streams. However, we found that the updated PAOBS data had a problem in 1988. A different data format has caused problems there, but not in any other
data files. The 1988 PAOBS are being reprocessed. Since this affects the start of the 1989 stream (stream 2), it will continue to hold for the data fix. Otherwise, we should see Streams 1 and 3 restart today.

Reanalysis Information and Precipitation work

The previous post on reanalysis precipitation climatology was some extra information from a paper recently accepted at the Journal of Applied Meteorology and Climatology, the pre-press version is available online at AMS. Evaluations of the GEOS5 precipitation are discussed in the last section.

More generally, the extended abstracts from the 3rd International Reanalysis Conference has been published online. See the email copied below.

begin message -----

Dear Participants

We would like to announce that the extended abstracts of oral and poster presentations at the third WCRP International Conference on Reanalysis that took place in Tokyo, Japan, 28 January- February 2008, are now available at following URL.

[World Climate Reanalysis Programme]
http://wcrp.ipsl.jussieu.fr/Workshops/Reanalysis2008/abstract.html

Thank you for your cooperation,

Best regards,

Eriko Moriai


Secretariat for WCRP International Conference on Reanalysis

Status April 24

The three streams are holding at the end of their spin up periods. Some refinements are being made to the budget output diagnostics for better closure. We have also been reviewing our input data holdings. A gap in the PAOB data set was found from 1997 through the end of 2000. The PAOB data starts back up in 2001 through our CDAS input data stream.

After checking this issue with Jack Woolen at NCEP, we found that he had just finished preparing revised PAOB data set (including about 4x more data) from 1985-2000 using JRA25 and ERA40's data holdings. In short, we are aiming to implement these data (many thanks to Jack, and counterparts at JMA and ECMWF). So, the restarts will be delays a few days to test this implementation.

For further information on PAOB, see http://www.bom.gov.au/bmrc/basic/wksp16/papers/Seaman.pdf

A consistent positive impact from the PAOB data suggests that this is worth a couple days to put into the system.

Status April 18 - Rewind

The data drop reported in the last status posting prompted some deeper digging in the input data streams. One issue led to another, then another. Briefly, a bug fix that should have been zero difference actually caused double counting in some of the input data streams, affecting the 1979 and 1989 streams. Further, a resource file had incorrect settings in the 1998 stream, leading to several observation systems not being assimilated.

The net result is that the data cannot be used as expected, and almost all the production processing needs to be re-run. The estimate for getting production restarted is Wednesday Apr 23. In addition, these errors have exposed some blind spots in our monitoring routines and adjustments are being made, and more personnel time is being devoted to monitoring.