Atmosphere

20th Century Reanalysis References

Compo,G.P., J.S. Whitaker, and P.D. Sardeshmukh, 2006: Feasibility of a 100 year reanalysis using only surface pressure data. Bull. Amer. Met. Soc., 87, 175-190.
Compo, G.P., J.S. Whitaker, P.D. Sardeshmukh, N. Matsui, R.J. Allan, X. Yin, B.E. Gleason, R.S. Vose, G. Rutledge, P. Bessemoulin, S. Brönnimann, M. Brunet, R.I. Crouthamel, A.N. Grant, P.Y. Groisman, P.D. Jones, M. Kruk, A.C. Kruger, G.J. Marshall, M. Maugeri, H.Y. Mok, Ø. Nordli, T.F. Ross, R.M. Trigo, X.L. Wang, S.D. Woodruff, and S.J. Worley, 2011: The Twentieth Century Reanalysis Project. Quarterly J. Roy. Met. Soc., in press.
Kanamitsu, M, and Coauthors 1991: Recent changes implemented into the global forecast system at NMC. Wea. Forecasting, 6, 425-435. Moorthi, S., H.-L. Pan, and P. Caplan, 2001: Changes to the 2001 NCEP operational MRF/AVN global analysis/forecast system. NWS Tech. Procedures Bulletin 484, 14 pp. [Available online at http://www.nws.noaa.gov/om/tpb/484.htm.]
Rayner, N.A., D.E. Parker, E.B. Horton, C.K. Folland, L.V. Alexander, D.P. Rowell, E.C Kent, and A. Kaplan, 2003: Global analyses of SST, sea ice and night marine air temperature since the late nineteenth century. J. Geophys. Res., 108, 4407, doi:10.1029/2002JD002670.
Saha, S. and Coauthors, 2006: The NCEP Climate Forecast System. J. Climate, 19, 3483-3517. Whitaker, J.S., G.P.Compo, X. Wei, and T.M. Hamill 2004: Reanalysis without radiosondes using ensemble data assimilation. Mon. Wea. Rev., 132, 1190-1200.

Add new comment

How to obtain/plot/analyze data

Data Extraction

NCEI NOMADS and NCMP: reanalysis (CFSR,NARR,R1,R2); NWP (NAM, GFS, RUC); GENS ensembles, SST
NOAA PSL Search and Plot. (R1,R2,20CR, NARR)
NOAA IRI (CFSR,20CR,R1,R2)
ECMWF:
1. ERA5: access via the Climate Data Store https://cds.climate.copernicus.eu/#!/home and https://cds-beta.climate.copernicus.eu/
2. Alternative: https://www.ecmwf.int/en/forecasts/datasets/search
NASA
1. MERRA: GES DISC Data Collections: MERRA
2. MERRA-2: GES DISC Data Collections: MERRA-2
3. MERRA and MERRA2 Data Subsetter (variable list for MERRA-2)
NCAR, Highest-resolution files for all reanalyses, except MERRA. GRIB parameter field extraction using cURL, and some conversion to netCDF as noted.
1. JRA-25: Data Access > Web File Listing > Create Your Own File List (e.g. anl_p), use cURL or wget (for files).
2. JRA-55
3. ERA5, ERA-Interim
4. CFSR - (also subset with net CDF format conversion)
5. 20CR
6. 20CRv2c
7. 20CRv3
OpenDAP servers:
1. NOAA PSL (20CR,R1,R2,NARR), NCEP, MERRA2D, MERRA3D
2. MERRA Gridded Innovations and Observations (GIO)
OpenGrADS.org: GrADS software with additional user functionality, including GUI for reanalyses including NCEP and MERRA
Earth System Grid Federation (CFSR, MERRA, 20CR, JRA-25, ERA-Interim) Obs4MIPS: Easy access to NetCDF reanalyses data of selected variables corresponding to the CMIP5 climate model output
GOAT: Geophysical Observations Analysis Tool for MATLAB

Step-by-Step Guide to obtaining data files

Step by step descriptions of how to get the data at the reanalyses centers are available.

Post Processing Routines and Algorithms

Extrapolate MERRA pressure-level data below the surface

Webtools to plot/analyze data by Function

Basic Maps

NOAA IRI (CFSR,20CR,R1,R2)
NOAA PSL Search and Plot (20CR,R1,R2,NARR)
NASA MERRA: Uses Giovanni to produce maps or animations of some monthly fields. Can average over successive times.
NOAA NOMADS (CFSR,NARR,R1,R2)
ECMWF ERA-40, ERA-Interim (Plot maps)
NASA MERRA Atlas
NOAA/PSL Web-based Reanalysis Intercomparison Tool for maps makes user-selected reanalysis fields for monthly data. It can also difference two reanalyses or selected observational datasets with user-selected climatologies.
The Climate Reanalyzer makes user-selected reanalysis fields and differences for monthly data.
MeteoCentre provides pre-generated synoptic maps of SLP, 1000-500 thickness, and 500 hPa height (20CR and R1).
GOAT: Geophysical Observations Analysis Tool for MATLAB

Other Basic Geographic Plots

NOAA PSL:Crossections
NOAA PSL: Search and Plot
MERRA: Uses Giovanni to produce hovmollers of some monthly fields.
The Royal Netherlands Meteorological Institute (KNMI) Climate Explorer
GOAT: Management and Analysis of Geophysical-Data Made Simple for Matlab
Met Data Explorer

Hovmollers

NOAA/PSL: Hovmollers (R1)
IRI
GOAT: Geophysical Observations Analysis Tool:Management and Analysis of Geophysical-Data Made Simple for Matlab

Advanced Plots

NOAA PSL: Can plot monthly, daily and sub-daily of crossections from composite plotting pages (R1). Anomalies are available.
NOAA/PSL: Hovmollers of means, anomalies of daily data. Anomalies are available (R1).
GOAT: Temporal and spatial subsettting is supported via a GUI or built in function. Built-in calculation of anomalies and climatology. Superimpose or show the difference between two fields. Display land-cover or topography.

Composite Maps (Average different, possibly non-contiguous dates together)

NOAA PSL: Can plot composite maps and vertical crossectons from composite plotting pages on monthly, daily and sub-daily time scales for R1. Anomalies are available.
NOAA PSL: Can plot composite maps from plotting pages on monthly, daily and sub-daily timescales for 20CR. Anomalies are available. For monthly, plot composite maps of the 20CR ensemble spread (uncertainty).
GCOS/WGSP: Can plot composite maps of sea level pressure from plotting pages on monthly timescales. Anomalies are available.
WRIT maps: Can plot composite maps of a reanalysis or the difference of composites from two reanalyses.
GOAT: Can plot composites of non-contiguous dates.

Correlation Maps

NOAA PSL WRIT Correlations
NOAA PSL: From monthly NCEP/NCAR R1
KNMI
The Climate Reanalyzer (ERA-Interim, NCEP/NCAR R1, NCEP/DOE R2, 20CR, observational datasets: PRISM precipitation, ERSSTv3b)

Timeseries Plots

NOAA PSL Plot simple timeseries of NCEP/NCAR R1 and 20thC Reanalysis monthly variables
IRI Data Library
Met Data Explorer: Unidata
Web-based Reanalysis Intercomparison Tool for timeseries (WRIT) makes user-selected reanalysis timeseries, scatter plots, cross-correlation functions, and probability density functions for monthly data. It can also difference two reanalyses or selected observational datasets.
The Climate Reanalyzer makes user-selected reanalysis time series with land/ocean masking.

Timeseries Analysis

KNMI: Plot, compute annual cycle, filter and other tools available for time series analysis. Provides many climate/ocean time-series.
NOAA/PSL: Correlate and do some other simple analysis on pregenerated or user supplied monthly time-series
NOAA/PSL: Extract daily timeseries from datasets. Can supply user criteria (e.g. top 10 temperatures in January for a point). R1,20CR
NOAA/PSL: Extract monthly timeseries from datasets. R1,20CR
IRI Data Library
NOAA PSL: Web-based Reanalysis Intercomparison Tool for timeseries (WRIT) makes user-selected reanalysis timeseries, scatter plots, cross-correlation functions, wavelets, and probability density functions for monthly data. It can also difference two reanalyses or selected observational datasets.

Google Earth

NOAA PSL: Create in KML plots (20CR, R1)

Miscellaneous

NOAA/PSL Lead/Lag for Composites
NOAA\/PSL Lead/Lag for Correlations (maps)
KNMI Smoothed fields, EOF, SVD and other analysis
NOAA/PSL WRIT Trajectory calculator (20CR, CFSR)

Applications that read/plot/analyze netCDF and/or grib data (non-web)

A complete list is at Unidata

NCL: NCAR Command Language (no longer updated but has full functionality)
GrADS: The Grid Analysis and Display System (GrADS)
IDV: Integrated Data Viewer
Ferret: Data Visualization and Analysis: From NOAA/PMEL
NCO: NetCDF Operators. The NCO toolkit manipulates and analyzes data stored in netCDF-accessible formats, including DAP, HDF4, andHDF5. It exploits the geophysical expressivity of many CF (Climate & Forecast) metadata conventions, the flexible description of physical dimensions translated by UDUnits, the network transparency of OPeNDAP, the storage features (e.g., compression, chunking, groups) of HDF (the Hierarchical Data Format), and many powerful mathematical and statistical algorithms of GSL (the GNU Scientific Library). NCO is fast, powerful, and free.
CDO: Climate Data operators. A collection of command-line operators to manipulate and analyze climate and numerical weather prediction data; includes support for netCDF-3, netCDF-4 and GRIB1, GRIB2, and other formats.
MATLAB
IDL: Interactive Data Language
CDAT Community Data Analysis Tools: CDAT: Note this will be replaced soon by xCDAT
xCDAT: xCDAT is an extension of xarray for climate data analysis on structured grids. It serves as a spiritual successor to the Community Data Analysis Tools (CDAT) library.
GOAT: Geophysical Observations Analysis Tool: A MATLAB based tool that integrates with NetCDF files and OPeNDAP sources.

open and read ERA5 grid data inside IDL

Dear Sir,

I would like to open ERA5 grid data that was downloaded from ECMWF.

Could you please send me some code to open and read with a visualize such kind of data.

I wait your reply as soon as possible.

Best wishes

Khaled

I don't have ERA code…

I don't have ERA code explicitly but there is this page

https://psl.noaa.gov/data/gridded_help/tools.html#idl

you can also check the MERRA2 webpages and email them with questions. I believe they have a helpdesk.

Cathy

Help

Please can anyone help me? I failed install NCL on my window platform, i fallowed the instruction but could not start to work.

How to download hourly data for radiation balance components

I really need help.

I want to download data for

-Incoming shortwave and outgoing shortwave radiation

-Incoming longwave and outgoing longwave radiation

-Albedo and net radiation.

Location: Lagos Nigeria

These data must be hourly with good resolution.

Please any help?

Thanks

What is the time period of interest?

Dear Moses,

There are very few datasets that are hourly and extend over a long time period. Is there a specific time period that you are interested in?

best wishes,

Reply: How to download hourly data for radiation balance compone

Moses,

A couple of points. MERRA, MERRA-2 and CSFR have 1 hourly data, others may too, you should review the characteristics of these and others to see which best suits your needs. When you say resolution must be good, do you mean spatial resolution? And if so, what is good resolution?

Keep in mind that when using reanalysis data, the radiation parameters are strongly dependent on *modeled* cloud fields. This can lead to significant uncertainty in the values.

Once you figure out which you want, then you can start looking to see if they have tools that permit the download of point data (MERRA/MERRA-2 both do). Their documentation is listed here.

hourly data

For there ERA5

https://www.ecmwf.int/en/newsletter/147/news/era5-reanalysis-production

For CFSR

https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/climate-forecast-system-version2-cfsv2#CFS%20Reanalysis%20(CFSR)

The 20CRV2c has 3 hourly data for surface fluxes. The MERRA and ERA have higher spatial resolution.

As the others asked, what time period?

Hi, …

Hi,

Need to plot streamlines using ECMWF winds (U and V) over Asian region ...Can I have matlab code?

Suggest a matlab users group

Dear Andy,

I suggest also trying a matlab user group for plotting tools specific to matlab.

best wishes,

streamlines and matlab

While I don't think matlab has an email list, they do have extensive help pages. I searched and see

https://plot.ly/matlab/streamline-plots/

https://www.mathworks.com/help/matlab/ref/streamline.html

https://www.mathworks.com/help/matlab/ref/stream2.html

They also have libraries to read netCDF files.

Cathy S.

Re: Need to plot zonal means of meridinal winds-stream functions

Hi, I am in need of plotting zonal means of meridinal winds and its streamlines in order to know a Hadley Cell circulation over Indonesian region. Can you please guide me by providing Grads software codes. thanks,

Re: Need to plot zonal means of meridinal winds-stream functions

For GrADS, this page has an example http://www.jamstec.go.jp/frsgc/research/iprc/nona/GrADS/plot-strem-line.html You can use this plotting page http://www.esrl.noaa.gov/psd/cgi-bin/data/testdap/plot.comp.pl to plot zonal means of meridional winds (and omega) to look at the Hadley cell. NCL will also plot streamlines http://www.ncl.ucar.edu/Applications/stream.shtml Cathy Smith

20CR - pr_wtr vs prate variables

I'd like to access from 20CR (ver 2c) daily rainfall for a specific lat/lon reference point. Is there a relationship between the daily pr_wtr output variable and the monthly mean prate variable? Is it valid to compare actual daily rainfall with data derived from pr_wtr?

Re: 20CR - pr_wtr vs prate variables

Mac, the pr_wtr is the precipitable water: the vertical integral of water vapor. prate is the rate of precipitation at the given averaging period. They should co-vary but not perfectly.

Problems with incoming shortwave from ERA-I

Hi, I trying to extract daily incoming shortwave from ERA-I, the variable is ssrd (Surface solar radiation downwards), which I downloaded from ECMWF website (http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/). The problem is that the daily values that I obtain from ERA-I are too low. ssrd comes in J/m2 and in a daily resolution (I cannot get higher temporal resolutions), and I get values up to 7 MJ/m2/d, in which values around 20+ MJ/m2/d are expected for mid-summer in this location (lat = 68N Lon 134W). Has anyone run into these problems with radiation (not able to download subdaily radiation and getting too small values). Any answer is appreciated, thanks Sebastian

Re: Problems with incoming shortwave from ERA-I

These are not daily values. If you look above "Select parameter" you will see "Select step" and "Select time". Time (and date) are the start time of the forecasts (twice daily) and step is the number of hours into the forecast. Ssrd is an accumulated field, from the beginning of the forecast to the particular step. Steps are 3 hourly, out to 12 hours. However, note that further steps, out to 240 hours are available with batch access, see "Access Public Datasets" in the left hand menu. To convert Jm-2 to Wm-2, simply divide by the number of seconds in step hours ie step*60*60.

Re: Problems with incoming shortwave from ERA-I

In addition to my previous reply, ssrd is a surface quantity. You might be interested in "TOA incident solar radiation" which is for the top of atmosphere.

Re: Problems with incoming shortwave from ERA-I

Please ignore this question since I found my error. The error had to do with the way I was downloading the data, which was not including the entire day. Thanks anyways,
Sebastian

Re: How to read the SAGE II V7 file with Matlab

Hello i want to open the SAGE II V7 aerosol data by using Matlab software !!!

Re: How to read the SAGE II V7 file with Matlab

Dear Toihir, I'm not sure what a SAGE II V7 file has to do with reanalysis, but perhaps these suggestions will help. I suggest you consult the lead author of any reference you are using for the SAGE II data, or consult with the data center from which you procured the data. From a google search on SAGE II, I see that the SAGE II home page is http://sage.nasa.gov/SAGE2/ . Read software is provided at https://eosweb.larc.nasa.gov/project/sage2/sage2_v7_table . Both Fortran and IDL code are provided there, so some modification will be necessary for matlab. I suggest you consult with a local matlab expert about how to interpret either the Fortran or IDL. best wishes

Re: How to read the SAGE II V7 file with Matlab

Dear Toihir, You may also want to post your question with considerably more details about the file type and structure to a matlab forum. Perhaps http://www.mathworks.com/matlabcentral/newsreader/ is such a venue. best wishes

add_offset and scale factor in ERA-I

I have been puzzled about odd looking scales in some downloaded ERA-I netCDF files. However, I have found out about scale factors and add-offset. However, when I apply them to some datasets, e.g. temperature and mean SLP, the new "unpacked" values are quite obviously not right, and the original values were. I tried to check by downloading an equivalent grib file, converting to net CDF with cod then examining the new output values. They were the same as the packed version. This is confusing. Heat flux values on the other hand seem wrong in both packed and unpacked, as ocean values in the Arctic (Barents Sea) appear more negative than do those over the ice.

Re: add_offset and scale factor in ERA-I

To assist you, anyone will probably need significantly more information. I suggest you start a page at reanalyses.org under Help (visible once authenticated) and describe precisely what you steps you followed and what values you are seeing. Including screen shots of the data access request and then the output of ncdump will be helpful.

Re: add_offset and scale factor in ERA-I

I can offer some general advice, but as Gilbert Compo said, a precise answer would require more information. Firstly, beware that some software automatically unpacks netCDF data. Also note, that the scale factors and add-offset vary from variable to variable and file to file. ERA-Interim fluxes, defined to be positive downwards, are accumulated from the beginning of the forecast for +step hours, so you need to divide by the number of seconds in step to obtain units in "per second".

Re: add_offset and scale factor in ERA-I

Hi,I came across this problem recently, I also found that the temperature after operated is not correct,I want to know how this problem solved?should I operate the formula unpack_value= scale factors*value_original+add-offset?Can you help me?Thanx

Re: add_offset and scale factor in ERA-I

Hi- We need more information. Where did you get your files and how were you trying to read them? The post below about different scale/add offsets for different variables also might be apropos. Cathy Smith NOAA/ESRL PSD and CU CIRES

Re: How to obtain/plot/analyze data

open GrADS is not processing SST files of CMIP5 data. But other files like surface temperature it is processing. Please give me a solution

Re: How to obtain/plot/analyze data

Dear Subramanyam, I'll need access and/or location of the 'problem' SST files to understand why opengrads is not working. Mike (via Gil Compo)

Re: How to obtain/plot/analyze data

Hi subramanyam , In order to help, anyone looking at this would need to have a link to the data you are trying to open. Grads and opengrads are similar in their capacity to open files. You may prefer to search the grads forum, then ask this question if it hasn't already been discussed. I did a quick google search and found:

http://gradsusr.org/pipermail/gradsusr/2012-December/033787.html

Good Luck

Re: How to obtain/plot/analyze data

Dear Subramanyam, CMIP5 data are not reanalysis data, so this question is probably better directed to the generator or provider of the CMIP5 data, or to the provider of the opengrads software at http://opengrads.org/ . best wishes,

Re: How to obtain/plot/analyze data

Dear Subramanyam, It looks like bug reports for opengrads are entered at http://sourceforge.net/p/opengrads/bugs/ I suggest submitting a detailed bug report, including the particular CMIP5 file that is causing the problem, and any error messages. Additionally, try downloading the CMIP5 files that are causing problems, or demonstrate that some other software opens it correctly in your bug report. best wishes,

Any hourly mean sea level pressure data

I need hourly mean sea level pressure data for the recent years. Do you know where can I find such reanalysis data?

Re: Any hourly mean sea level pressure data

Dear Mostafa, Hourly data are available from NOAA NCEP's Climate Forecast System Reanalysis (CFSR, http://reanalyses.org/atmosphere/overview-current-reanalyses#CFSR) See the Data Access links. Please post back with any additional questions.

Problem with longitude in 20CR data

I think I have found a problem with the longitude coordinate of the data files of the 20CR under http://portal.nersc.gov/archive/home/projects/incite11/www/20C_Reanalysis/everymember_grib_indi_fg_variables ncl_convert2nc fails to convert these files. CDO does convert them to NetCDF but after this the longitude coordinate values range from about -1.8 to 0 (from West to East). The latitude coordinates are correct. The CDO operator setgrid,t62 fixes this problem.

Re: Problem with longitude in 20CR data

Camiel, Does your cdo returned netCDF file show grid_type = "gaussian" before you use setgrid,t62 ? When I use cdo on these files without the setgrid,t62 I see that metadata. The longitude coordinate goes from 0 to 358.125. But, we use "wgrib" to separate out each ensemble member as a separate grib file before passing to cdo. Perhaps the ensemble dimension is confusing cdo? Looks like by specifying the setgrid,t62 you have found a great workaround to a cdo problem! thanks for sharing this, best wishes, gil

Re: How to obtain/plot/analyze data

Yeah, I agree that it is better to read every single ensemble member out from TMP.2m.1871.grb type files and convert them to be nc files. It works for me in this way. But I use "cdo -f nc copy filename.grb filename.nc" to convert grb files to be nc files. Thanks.

How to read GFS SFC data files

To force an ensemble of ocean model experiments I would like to use (near) surface data from individual members of the 20CR. I downloaded a file 187501_sfcanl_mem01.tar which I assume contains the data I am looking for. However, the files in this tar-file are not in GRIB format. The first few bytes contain the string 'GFS SFC'. Can anyone tell me how this files are formatted?

Re: How to read GFS SFC data files

Dear Camiel, For the GRIB formatted data from the NERSC Science Tape Gateway at http://portal.nersc.gov/archive/home/projects/incite11/www/20C_Reanalysis/everymember_full_analysis_fields you want the surface flux grib files "sflx", rather than the binary "sfcanl" files. E.g. for the first member of the 0 to 3 hour forecast 187501_sflxgrbens_fhr03_mem01.tar and the first member of the 3 to 6 hour forecast 187501_sflxgrbens_fhr06_mem01.tar These are the file types that most other groups have used. Alternatively, you may want to obtain only your variables of interest. If a variable that you need is not at http://portal.nersc.gov/archive/home/projects/incite11/www/20C_Reanalysis/everymember_grib_indi_fg_variables we can generate it if it is in the "sflx" files. Obtaining the individual variables you need, rather than the complete "sflxgrbens" file may save transfer time. Please reply if you one or the other of these solutions does not work for your purposes. best wishes, gil compo

Re: How to read GFS SFC data files

Dear Gill, I have found the data files and tried to convert the grib files to netcdf using ncl_convert2nc (version 6.1.2). This tool stops with the following warnings (there are lots more of those preceeding) and two fatal errors: warning:./TMP.2m.1871.ens.grb->TMP_98_HTGL is missing ens: 54 it: 12/31/1871 (18:00) ft: 6 warning:./TMP.2m.1871.ens.grb->TMP_98_HTGL is missing ens: 55 it: 12/31/1870 (18:00) ft: 3 warning:./TMP.2m.1871.ens.grb->TMP_98_HTGL is missing ens: 55 it: 12/31/1871 (18:00) ft: 6 fatal:NclGRIB: Couldn't handle dimension information returned by grid decoding fatal:NclGRIB: Deleting reference to parameter because of decoding error Classic model NetCDF does not support string types, converting initial_time1 to a character array Dimension 'ncl_strlen_0' will be added Classic model NetCDF does not support string types, converting ensemble0_info to a character array Dimension 'ncl_strlen_1' will be added Do you know what might be the problem here? Thanks, Camiel

Re: How to read GFS SFC data files

Dear Camiel, Which member file are you testing? Would you send the URL you used to access it? thanks, gil

Re: How to read GFS SFC data files

Camiel, I was able to use ncl_convert2nc 6.0.0 to convert the file sflxgrbens_fhr03_1871010100_mem01 (add .grib suffix) to netcdf. This file is contained in the tarfile accessed from http://portal.nersc.gov/archive/home/projects/incite11/www/20C_Reanalysis/everymember_full_analysis_fields/1871/187101_sflxgrbens_fhr03_mem01.tar Conversely, when I tried the file TMP.2m.1871.ens.grb accessed from http://portal.nersc.gov/archive/home/projects/incite11/www/20C_Reanalysis/everymember_grib_indi_fg_variables/TMP/TMP.2m.1871.ens.grb.tar I get ncl_convert2nc error messages very similar to yours. I access the TMP.2m.1871.ens.grb in python. I suspect that there is a bug in the ncl_convert2nc for very large files. You may want to use wgrib http://www.cpc.ncep.noaa.gov/products/wesley/wgrib.html to slice up the file into smaller pieces and see if that works. Alternatively, since the sflxgrb file does work with ncl_convert2nc, perhaps using those will be better? best wishes, gil

Re: How to read GFS SFC data files

Camiel, You may want to see if you can enable the "large file support" in ncl_convert2nc. compo/test_ncl_convert2nc> ncl_convert2nc -h ncl_convert2nc inputFile(s) OPTIONS inputFile(s) name(s) of data file(s) [required] [valid types: GRIB1 GRIB2 HDF HDF-EOS netCDF shapefile] [-i input_directory] location of input file(s) [default: current directory] [-o output_directory] location of output file(s) [default: current directory] [-e extension] file type, defined by extension, to convert [example: grb] [-u time_name] name of the NCL-named time dimension to be UNLIMITED [-U new_time_name] if -u is specified: new name of UNLIMITED variable and dimension [-sed sed1[,...]] GRIB files only; set single element dimensions [default: none] choices are initial_time, forecast_time, level, ensemble, probability, all, none [-itime] GRIB files only; set initial time as a single element dimension (same as -sed initial_time) [-ftime] GRIB files only; set forecast time as a single element dimension (same as -sed forecast_time) [-tps] GRIB files only; remove suffix representing a time period (e.g. 2h) from statistically processed variables, leaving only type of processing as a suffix (e.g. _acc, _avg) [-v var1[,...]] user specified subset of variables [default: all variables] ncl_filedump can be used to determine desired variable names [-L] support for writing large (>2Gb) netCDF files [default: no largefile support] Note, though, from the ncl_convert2nc help page http://www.ncl.ucar.edu/Document/Tools/ncl_convert2nc.shtml -L Specifies that the resultant netCDF output file may exceed 2Gb in size on platforms that have "large file support" (LFS). However, no single variable may exceed 2Gb in the current implementation. You may need to slice out individual ensemble members for ncl_convert2nc to work on the TMP.2m.1871.grb type files. I hope that this helps. best wishes, gil

Re: How to read GFS SFC data files

Hi Gil, After extracting the T2M data for one member only, ncl_convert2nc still fails with the same error (-L option makes no difference). CDO has no problems with converting the single member GRIB file to NetCDF. The variable name is wrong but this can be fixed. My guess now is that something is wrong with ncl_convert2nc. Regards, Camiel

Re: How to read GFS SFC data files

Hi Gil, Ncl_convert2nc has no problems with the GRIB files contained in 187101_sflxgrbens_fhr03_mem01.tar. Adding the -L option to the ncl_convert2nc command does not make a difference. Regards, Camiel

Re: How to read GFS SFC data files

Camiel, To answer your original question, sfcio_module.f at http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/lib/sorc/sfcio/ is what you will need to read the "sfcanl" type files. best wishes, gil

Re: How to obtain/plot/analyze data

Does GOAT support automatic downloading of ERA-Interim?

Re: How to obtain/plot/analyze data

Unfortunately, it is not straight forward to automate the download of ERA-Interim and ERA-40 fields. I do have automated routines for the conversion of ERA-40 and ERA-Interim to GOAT format but you need to download the NC files yourself. If you are interested in monthly means, some of these are available at the goat-geo.org site. I can add more upon request. GOAT does support automated download for NCEPI, NCEPII, 20CRenalysis, ORAS4, TRMM, CloudSatCalipso composite, ERSST, MERRA, and others.

Re: How to obtain/plot/analyze data

for automated download, visualization and basic analysis of reanalyses try GOAT, freely available at www.goat-geo.org

Re: How to obtain/plot/analyze data

Can GOAT generate its own anomalies or differences? If so, please add it to some of the other listings above as applicable. Thanks! best wishes, gil compo

Re: How to obtain/plot/analyze data

You may also want to add a summary of GOAT on the Software page https://reanalyses.org/atmosphere/software and on the Ocean reanalysis pages. best wishes, gil

Re: How to obtain/plot/analyze data

I think you had better look at the original satellite ozone measurements for your purpose. The Stratospheric Processes and their Role in Climate (SPARC) project has been doing ozone measurement validation and evaluation for many years. Please go to http://www.sparc-climate.org/activities/ozone-profile-ii/ and contact with the activity leaders shown there, and/or check "Website for further information" at the end of the page (i.e., http://igaco-o3.fmi.fi/VDO/index.html). Actually, there are several choices for satellite ozone measurements, but the latest version SAGE data set may be most useful for you. For ozone in the reanalyses, I think we need validation and evaluation before using it for climate studies. The SPARC Reanslysis Intercomparison Project (S-RIP, http://s-rip.ees.hokudai.ac.jp/index.html) has this component. For your information, the following is my quick survey for the 9 reanalyses. Please confirm by yourself by checking the references. NCEP/NCAR & NCEP/DOE: (Kalnay et al., 1996; Kistler et al., 2001; Kanamitsu et al., 2002): - Zonally averaged seasonal climatological ozone used in the radiation computation - (In NCEP/DOE, the latitudinal orientation was reversed north to south) ERA-40: (Uppala et al., 2005; Dethof and Holm, 2004): - TOMS and SBUV ozone retrievals (not radiance) are assimilated (1978-). Ozonesondes not assimilated. - Ozone in the ECMWF model is described by a tracer transport equation including a parametrization of photochemical sources and sinks. - The ozone climatology is used in the radiation calculations of the forecast model. ERA-Interim: (Dee et al., 2011; Dragani, 2011): - TOMS, SBUV, GOME (1996-2002), MIPAS (2003-2004), SCIAMACHY (2003-), MLS (2008-), OMI (2008-) are assimilated. SAGE, HALOE, and POAM are not assimilated. – Ozone model and radiation calculations are basically the same as ERA-40. JRA-25: (Onogi et al., 2007): – Ozone observations are not assimilated directly. – Daily ozone distribution is prepared in advance using a CTM with “nudging” to the satellite total ozone measurements and provided to the forecast model (the radiative part). JRA-55 (Ebita et al., 2011): - similar to JRA-25 for 1979-; monthly climatology for -1978 MERRA: (Rienecker et al., 2011): – SBUV2 ozone (version 8 retrievals) is assimilated for Oct 1978–present. – The MERRA AGCM uses the analyzed ozone generated by the DAS. (cf. a climatology for aerosol) NCEP-CFSR: (Saha et al., 2010) – SBUV profiles and total ozone retrievals are assimilated (but not bias-adjusted; should not be used for trend detection) – Prognostic ozone with climatological production and destruction terms computed from 2D chemistry models (for radiation parameterization) 20CR: (Compo et al., 2011): – "A prognostic ozone scheme includes parametrizations of ozone production and destruction (Saha et al., 2010)."

Re: How to obtain/plot/analyze data

Dear samudraval59, Some atmospheric reanalyses, such as NCEP/NCAR http://reanalyses.org/atmosphere/overview-current-reanalyses#NCEP1 do not provide ozone. some, such as CFSR http://reanalyses.org/atmosphere/overview-current-reanalyses#CFSR, ERA-Interim http://reanalyses.org/atmosphere/overview-current-reanalyses#ERAINT, MERRA http://reanalyses.org/atmosphere/overview-current-reanalyses#MERRA provide ozone on levels. Links to the data are provided at each overview. 20th Century Reanalysis (20CR) http://reanalyses.org/atmosphere/overview-current-reanalyses#TWENT provides only the total column ozone. Note that while ozone is prognostic in 20CR, that system assimilates only surface pressure. Please read the linked references to determine what each system is doing and what data are being assimilated, particularly related to ozone. Links to various tools are given on this page where you left this comment, i.e., http://reanalyses.org/atmosphere/how-obtainplotanalyze-data and are also http://reanalyses.org/atmosphere/tools . If those do not include ozone, you may want to leave a comment on each page or use the contact on the respective linked sites. For the Web-based Reanalysis Intercomparison Tool, you can leave comments at https://reanalyses.org/atmosphere/web-based-reanalysis-intercomparison-tools-writ best wishes, gil compo

Re: How to obtain/plot/analyze data

My present work of study of warming regimes and the trends require me acquire and capture data and analysis tools on open domain vis-à-vis ozone profiling, the ozone mixing ration and partial pressures.I would be very grateful if I could be given a peek to get the above in the most reliable free sources. thanking you. Samudrlav59

Re: ASCII timeseries of daily surface parameters from CFSR

Dear reanalyses.org I am trying to get daily weather data from CFSR to run an ecosystem model for a geographic area (say Italy) by using the NCDC OPENDAP server, e.g. http://nomads.ncdc.noaa.gov/thredds/dodsC/modeldata/cmd_flxf/2000/200005/20000504/flxf01.gdas.2000050400.grb2.html but with no luck so far. I was wondering whether there is a more direct way to get daily time series data (in ASCII) from CFSR that people uses routinely. Daily time series for surface parameters such as max/min temperature, solar radiation, precipitation, relative humidity, and wind, are standard for ecosystem models as life works on a circadian rhythm on Earth. Thanks for any hint and kind regards, Luigi

Re: How to obtain/plot/analyze data

Dear sir, I need historiacl /longterm wind data for a specific site in order to obtain correlation with actual data/nearby metmast data,so how can i get it ?and where from?.Kindly guide me with a procedure to download the data with an exact link. Regards, Easwar.

Re: How to obtain/plot/analyze data

Dear Easwar, Happy to help, but this area is for reanalysis data. See http://reanalyses.org/ for the definition of reanalysis data. This may be what you need but your question is not clear in this respect. If you want data from a station, you should post your question in the Observations area http://reanalyses.org/observations/surface . Is your site over the ocean or over land? How close do the data need to be to your site? What is your site location? While posting at http://reanalyses.org/observations/surface, you may want to make your question a bit clearer. What do you mean by "historical/longterm" wind data? Do you want a long-term average or do you want a long time series at some temporal resolution? What is the temporal resolution you need? What is the temporal resolution you can still use (e.g., monthly averages, daily averages, once-per-day)? What is the height of the data you need? Do you want data from satellites, such as scatterometers? By providing more information in the Observations area, someone may be able to help you better. best wishes, gil compo

Re: How to obtain/plot/analyze data

Stefan,

Adding panoply is a great idea, but Reanalyses.org is a wiki site that depends on users. You can login and add it where you feel it is appropriate. If you have any questions, please feel free to ask or add a question to the Help section.

best wishes,
gil compo

Re: How to obtain/plot/analyze data

I find Panoply incredibly useful to look at NetCDF and GRIB data, maybe you want to list it here too: http://www.giss.nasa.gov/tools/panoply/

Re: How to obtain/plot/analyze data

This page needs more work. What do users find useful here?

Re: How to obtaindata

Hi please guide me what gain daily data (tepreture,pressure &etc)

Add new comment

Inter-reanalysis studies

This page provides a sampling of recent research on reanalysis comparisons.^*

Antarctic

30 year comparison between Antarctic stations (temperature, heights, and surface pressure) and 2nd and 3rd generation reanalyses - Bracegirdle, Thomas J., Gareth J. Marshall, 2012: The Reliability of Antarctic Tropospheric Pressure and Temperature in the Latest Global Reanalyses. J. Climate, 25, 7138–7146. doi: http://dx.doi.org/10.1175/JCLI-D-11-00685.1

Arctic

This study shows comparisons of seven different reanalyses with independent observations of temperature, radiative fluxes, precipitation, and wind speed and intercomparisons with with each other, both for seasonal mean values and for trends. It also shows their relative ability to provide forcing for a sea ice model.

Lindsay, R., M. Wensnahan, A. Schweiger, J. Zhang, 2014: Evaluation of Seven Different Atmospheric Reanalysis Products in the Arctic*. J. Climate, 27, 2588–2606. doi: http://dx.doi.org/10.1175/JCLI-D-13-00014.1

Many additional seasonal intercomparison maps (many more than could be included in the publication) showing the median from the seven reanalyses and the deviations of each from the median are found at http://psc.apl.uw.edu/arctic_reanalyses. Both seasonal means and seasonal trends for the period 1981-2010 are shown. 34 different variables are included: surface fluxes, temperature, humidity, wind, precipitation, and pressure; layer heights and temperatures; and TOA radiative fluxes.

Arctic temperatures and trends over the 20th century in reanalyses, reconstructions, and upper-air observations are compared in Brönnimann, S., A.N. Grant, G.P. Compo,T. Ewen, T. Griesser, A.M. Fischer, M. Schraner, and A. Stickler, 2012: A multi-data set comparison of the vertical structure of temperature variability and change over the Arctic during the past 100 years. Cli. Dyn., 39, 1577-1598, doi:10.1007/s00382-012-1291-6. Discussion Page

Several reanalysis datasets are compared with satellite observations of Arctic clouds in Alexander Chernokulsky and Igor I. Mokhov, “Climatology of Total Cloudiness in the Arctic: An Intercomparison of Observations and Reanalyses,” Advances in Meteorology, vol. 2012, Article ID 542093, 15 pages, 2012. doi:10.1155/2012/542093. Discussion Page

The Arctic cloud fraction and radiative fluxes of several reanalyses are compared with Baseline Surface Radiation Network observations in Zib, B.J., X. Dong, B. Xi, A. Kennedy, 2012: Evaluation and Intercomparison of Cloud Fraction and Radiative Fluxes in Recent Reanalyses over the Arctic Using BSRN Surface Observations. J. Climate, 25, 2291-2305. doi:10.1175/JCLI-D-11-00147.1

Circulation

Nine global reanalysis datasets consistenly show an increase in deep-cyclone counts over the past half centure - Wang, X.L., Y. Feng, R. Chan, and V. Isaac, 2016: Inter-comparison of extra-tropical cyclone activity in nine reanalysis datasets. Atmospheric Research, 181 (2016), 133-153. doi:10.1016/j.atmosres.2016.06.010. Discussion Page

What's the difference between GFS and FNL? From the NCAR RDA blog: http://ncarrda.blogspot.com/2015/04/whats-difference-between-gfs-and-fnl.html

Analysis, Forecast, Reanalysis–What's the difference? From the NCAR RDA blog: http://ncarrda.blogspot.com/2015/04/analysis-forecast-reanalysis-whats.html

Interannual Variability of the Atmospheric Angular Momentum in 2nd and 3rd generation reanalyses - Paek, Houk, Huei-Ping Huang, 2012: A Comparison of interannual variability in atmospheric angular momentum and length-of-day using multiple reanalysis datasets, J. Geophys. Res., 117, D20102, doi:10.1029/2012JD018105. Discussion Page

Comparing CFSR and other reanalyses in the troposphere - Chelliah, M., W. Ebisuzaki, S. Weaver, and A. Kumar, 2011: Evaluating the tropospheric variability in National Centers for Environmental Prediction's climate forecast system reanalysis, J. Geophys. Res., 116, D17107, doi:10.1029/2011JD015707. Discussion Page

Comparing trends in sea level pressure from ERA-Interim, ERA-40, CFSR, NCEP-NCAR, MERRA, and 20CR - L'Heureux, M.L., S. Lee, and B. Lyon, 2013: Recent multidecadal strengthening of the Walker circulation across the tropical Pacific, Nature Clim Change, doi: 10.1038/nclimate1840. Discussion Page

Diabatic heating

3rd generation reanalyses compared with some observations - Jian Ling and Chidong Zhang, Diabatic Heating Profiles in Recent Global Reanalyses (doi: 10.1175/JCLI-D-12-00384.1)

Fluxes

Hydrologic cycle

Intercomparison of the diurnal cycle of upper-tropospheric humidity in satellite observations and reanalyses - Chung, E.-S., B. J. Soden, B. J. Sohn, and J. Schmetz (2013), An assessment of the diurnal variation of upper tropospheric humidity in reanalysis data sets, J. Geophys. Res. Atmos., 118, doi:10.1002/jgrd.50345. Discussion Page

Consideration of the reanalysis water budget closure, along with observational uncertainties including precipitaiton station density - Lorenz, Christof, Harald Kunstmann, 2012: The Hydrological Cycle in Three State-of-the-Art Reanalyses: Intercomparison and Performance Analysis. J. Hydrometeor, 13, 1397–1420. doi: http://dx.doi.org/10.1175/JHM-D-11-088.1

Precipitation

US Regional evaluation of reanalyses summertime interannual variability of precipitation and temperature - Bosilovich, Michael G., 2013: Regional Climate and Variability of NASA MERRA and Recent Reanalyses: U.S. Summertime Precipitation and Temperature. J. Appl. Meteor. Climatol., 52, 1939–1951. doi: http://dx.doi.org/10.1175/JAMC-D-12-0291.1

Monthly intercomparison of tropical precipitation of observations and reanalyses - Uwe Pfeifroth, Richard Mueller, and Bodo Ahrens, Evaluation of Satellite-based and Reanalysis Precipitation in the Tropical Pacific (doi: 10.1175/JAMC-D-12-049.1) In Press (12/18/12)

Sub-monthly intercomparison of tropical precipitation in reanalyses - Ji-Eun Kim and M. Joan Alexander,Tropical precipitation variability and convectively coupled equatorial waves on submonthly time-scales in reanalyses and TRMM (doi: 10.1175/JCLI-D-12-00353.1) In press (12/18/12)

Process study of precipitation in reanalysis - Arun Kumar, Li Zhang, and Wanqiu Wang, Sea Surface Temperature - Precipitation Relationship in Different Reanalyses (doi: 10.1175/MWR-D-12-00214.1) In Press (12/18/12) Discussion Page

Southern African precipitation in ERA-40, ERA-interim, JRA-25, MERRA, CFSR, NCEP-R1, NCEP-R2 and 20CRv2 are compared in Zhang, Q., H. Körnich and K. Holmgren, 2012: How well do reanalyses represent the southern African precipitation? Clim. Dyn., in press (doi: 10.1007/s00382-012-1423-z). Discussion Page

Update to the 2008 precipitation statistics below, focusing more on MERRA but includes other reanalyses, and intercompares the water and energy budgets: Bosilovich, Michael G., Franklin R. Robertson, Junye Chen, 2011: Global Energy and Water Budgets in MERRA. J. Climate, 24, 5721–5739. doi: http://dx.doi.org/10.1175/2011JCLI4175.1

Focus on global and regional precipitation biases and spatio-temporal variability at the long time scales: Bosilovich, Michael G., Junye Chen, Franklin R. Robertson, Robert F. Adler, 2008: Evaluation of Global Precipitation in Reanalyses. J. Appl. Meteor. Climatol., 47, 2279–2299. doi: http://dx.doi.org/10.1175/2008JAMC1921.1

Temperature

Temperature trends over the continental United States for the period 1979 to 2008 compared to observations - Vose, R. S., S. Applequist, M. J. Menne, C. N. Williams Jr., and P. Thorne. 2012: An intercomparison of temperature trends in the U.S. Historical Climatology Network and recent atmospheric reanalyses. Geophys. Res. Lett., 39, L10703. (doi:10.1029/2012GL051387) Discussion Page

^*This list is not comprehensive, but the interested reader should feel free to request an account and include any relevant published research the reader thinks may be useful to communicate. Please use the existing categories, only adding new as needed. Add a brief description and use reverse chronological order under each topic. Users are encouraged to link a Discussion Page summarizing their paper, presentation, or other results.

Add new comment

Observation and Reanalysis Studies

Arctic Clouds : several reanalysis datasets are compared with satellite observations of Arctic clouds in Alexander Chernokulsky and Igor I. Mokhov, “Climatology of Total Cloudiness in the Arctic: An Intercomparison of Observations and Reanalyses,” Advances in Meteorology, vol. 2012, Article ID 542093, 15 pages, 2012. doi:10.1155/2012/542093.

Arctic Radiation and Clouds: the Arctic cloud fraction and radiative fluxes of MERRA, CFSR, 20CR, ERA-Interim, and NCEP-DOE R2 are compared with Baseline Surface Radiation Network observations in Zib, B.J., X. Dong, B. Xi, A. Kennedy, 2012: Evaluation and Intercomparison of Cloud Fraction and Radiative Fluxes in Recent Reanalyses over the Arctic Using BSRN Surface Observations. J. Climate, 25, 2291-2305. doi:10.1175/JCLI-D-11-00147.1

Arctic Temperature Trends: Seasonal arctic temperatures and trends over the 20th century in reanalyses, reconstructions, and upper-air observations are compared in Brönnimann, S., A.N. Grant, G.P. Compo,T. Ewen, T. Griesser, A.M. Fischer, M. Schraner, and A. Stickler, 2012: A multi-data set comparison of the vertical structure of temperature variability and change over the Arctic during the past 100 years. Cli. Dyn., 39, 1577-1598, doi:10.1007/s00382-012-1291-6.

Decadal-to-Interdecadal Variability and Trend in reanalyses: Paek, Houk, Huei-Ping Huang, 2012: A Comparison of Decadal-to-Interdecadal Variability and Trend in Reanalysis Datasets Using Atmospheric Angular Momentum. J. Climate, 25, 4750-4758. doi: 10.1175/JCLI-D-11-00358.1

Extratropical Storminess in the Northern Hemisphere: Wang, X. L., Y. Feng, G.P. Compo, F.W. Zwiers, R.J. Allan, V.R. Swail,and P.D. Sardeshmukh, 2014:

Is the storminess in the Twentieth Century Reanalysis really inconsistent with observations?

A reply to the comment by Krueger et al. (2013b). Climate Dynamics, 42, 1113-1125,

doi:10.1007/s00382-013-1828-3

Hadley Circulation: Trends in the Hadley Cell intensity as diagnosed in ERA40, ERA-Interim, CFSR, JRA25, NCEP-NCAR, NCEP-DOE R2, MERRA, 20CR are compared with each other and with climate model output in Stachnik, J. P., and C. Schumacher, 2011: A comparison of the Hadley circulation in modern reanalyses, J. Geophys. Res., 116, D22102, doi:10.1029/2011JD016677.

Global land temperature trends and variations in the independent 20CR and observational station temperature datasets are shown to be very similar since 1901 in Compo, G.P., P.D. Sardeshmukh, J.S. Whitaker, P. Brohan, P.D. Jones, and C. McColl, 2013: Independent confirmation of global land warming without the use of station temperatures. Geophys. Res. Letters, in press, doi:10.1002/grl.50425. Auxiliary Material.

Global land hourly temperature dataset is developed and compared to NCEP/NCAR, ERA-40, ERA-Interim, and MERRA reanalyses in Wang, A., X. Zeng, 2013: Development of globaly hourly 0.5-degree land surface air temperature datasets. J. Climate, in press, doi:10.1175/JCLI-D-12-00682.1.

Interannual Variability in reanalyses: Paek, Houk, Huei-Ping Huang, 2012: A Comparison of interannual variability in atmospheric angular momentum and length-of-day using multiple reanalysis datasets, J. Geophys. Res., 117, D20102, doi:10.1029/2012JD018105.

Ocean Atmosphere Feedbacks in NCEP/NCAR, NCEP-DOE, ERA-40, JRA-25, CFSR, and MERRA are compared with each other and observational estimates in Kumar, A., and Z.-Z. Hu, 2012: Uncertainty in the ocean-atmosphere feedbacks associated with ENSO in the reanalysis products. Clim. Dyn., 39 (3-4), 575-588. DOI: 10.1007/s00382-011-1104-3.

Surface solar radiation in North America is compared across multiple in situ, satellite, and reanalyses datasets, incl. 20CRv2, ERA-Interim, ...Slater AG, (2016), Surface Solar Radiation in North America: A Comparison of Observations, Reanalyses, Satellite, and Derived Products. J. Hydrometeor, 17, 401–420. doi: 10.1175/JHM-D-15-0087.1.

Temperature and precipitation extremes compared across multiple in situ based and reanalyses datasets, incl. ERA-40, ERA-Interim, JRA-25, NCEP-R1, NCEP-R2: Donat, M. G., J. Sillmann, S. Wild, L. V. Alexander, T. Lippmann, F. W. Zwiers, 2014: Consistency of temperature and precipitation extremes across various global gridded in situ and reanalysis data sets, Journal of Climate, 27, 5019–5035, doi:10.1175/JCLI-D-13-00405.1

Tropospheric Variability in CFSR and other reanalyses: Chelliah, M., W. Ebisuzaki, S. Weaver, and A. Kumar, 2011: Evaluating the tropospheric variability in National Centers for Environmental Prediction's climate forecast system reanalysis, J. Geophys. Res., 116, D17107, doi:10.1029/2011JD015707.

Upper Tropospheric Humidity from reanalyses and satellite data are compared in Chung, E.-S., B. J. Soden, B. J. Sohn, and J. Schmetz (2013), An assessment of the diurnal variation of upper tropospheric humidity in reanalysis data sets, J. Geophys. Res. Atmos., 118, doi:10.1002/jgrd.50345.

U.S. Temperature Trends in USHCN and reanalyses (1979-2008): Temperature trends over the continental United States for the period 1979 to 2008 are diagnosed in the observations from the U.S. Historical Climate Network and compared to 20CR, ERA-Interim, CFSR, JRA25, MERRA, and NARR in Vose, R. S., S. Applequist, M. J. Menne, C. N. Williams Jr., and P. Thorne. 2012: An intercomparison of temperature trends in the U.S. Historical Climatology Network and recent atmospheric reanalyses. Geophys. Res. Lett., 39, L10703, doi:10.1029/2012GL051387.

Walker Circulation: Trends in sea level pressure are compared in ERA-Interim, ERA-40, CFSR, NCEP-NCAR, MERRA, 20CR in L'Heureux, M.L., S. Lee, and B. Lyon, 2013: Recent multidecadal strengthening of the Walker circulation across the tropical Pacific, Nature Clim Change, doi: 10.1038/nclimate1840.

Walker Circulation is not a proxy for the global convective mass flux found using 20CR, models, and SST reconstructions in Sandeep, S., F. Stordal, P.D. Sardeshmukh, and G.P. Compo, 2014: Pacific Walker Circulation variability in coupled and uncoupled climate models. Cli. Dyn., in press, doi:10.1007/s00382-014-2135-3.

Above alphabetical list by topic of papers and reanalyses.org pages that compare reanalysis and observational datasets. Use form

{Topic with link to reanalyses.org Discussion Page (or paper when page doesn't exist)}: {One sentence summary}, {citation}, digital object identifier linked to journal article or presentation.

Re: Observational Studies

Dear Gil,

I hope your are doing well.

About your e-mail requesting 20CR results, we are currently working in reconstructing observational series back to the past using present observations, statistical downscaling techniques and the 20th century reanalysis.

This work is still very preliminary and before to submit any result there are many tests and checking that we need to make. For this reason we have obtained result that are raising some questions to us about the 20CR.

The encloses figure shows an example for the Mean Temperature at "Berlin-Dahlem", which is part of the ECA dataset (http://eca.knmi.nl/utils/stationdetail.php?stationid=41). For the downscaling process we have used the training period 1981-2010, and as test period from the 1881-1980. Only the test period is shown in the figure. In the upper part the means for the DJF period for each year (1880-1980) is plot. In the lower part the same for the JJA period.

In blue line corresponds to the observed values and three different downscaling techniques (green, red and light blue). These downscaling techniques are made day-by-day and using 2m temperature from the 20CR as predictor. The Pearson Correlation between the downscaled and observed monthly means are indicated in the legend.

As you can see the results turn out to be suspiciously good (especially in case of DJF) , therefore the questions that we are making is:

degree the reanalysis 2m temperatures are constrained to the observations used in the assimilation process?.
The observations from "Berlin-Dahlem" has been assimilated?
If yes, which variables have been used for this purpose?
We suppose that only MSLP has been assimilated in the 20CR this it's correct?
there is a list of observations assimilated by the 20CR specially over Europe?

These results are very promising, but before we want avoid to make wrong assumptions.

Thank you very much in advance and "Saludos desde Santander"

Antonio

Re: Observational Studies

Dear Antonio,
Great to hear from you!

The 2m air temperatures are generated by the integration of the ensemble from the analyzed states. The analyzed states are only constrained by the assimilated surface and sea level pressure observations and the boundary conditions of monthly sea surface temperatures and sea ice concentration (note the comments and issues about the sea ice in Compo et al. 2011). CO2, solar variability, and volcanic aerosols are also imposed.

Your results are consistent with others, see, e.g., http://www.esrl.noaa.gov/psd/data/20thC_Rean/pubs/
Parker, D. E., 2011: Recent land surface air temperature trends assessed using the 20th Century Reanalysis., J. Geophys. Res., doi:10.1029/2011JD016438, in press.

I'll need more information about this station than the name to find out if it was assimilated and for what period - lat, lon, elevation, WMO id. You can query the International Surface Pressure Databank version 2 at http://rda.ucar.edu/datasets/ds132.0/ to generate the list you are interested in.
You can also look at maps from https://reanalyses.org/observations/international-surface-pressure-data… (you want version 2).

Mean Sea Level Pressure and surface pressure observations are the only subdaily data assimilated.

You can also examine pre-generated maps of the SLP field and its uncertainty and the 500 hPa geopotential height field and its uncertainty, along with the station locations that were assimilated at http://www.esrl.noaa.gov/psd/data/20thC_Rean/ and go to Obtain plots of the ensemble means and spreads.

If you would, please consider posting your results to reanalyses.org (Even just put up this email and this dialogue can then preserved and expanded upon there).

Please let me know if you have additional questions. I'm happy to help.

best wishes,

gil

Add new comment

Inter-reanalysis studies

Add new comment

Tools

List of web-based tools for comparing reanalysis datasets:

GOAT (Geophysical Observation Analysis Tool) is a free MATLAB based tool for the analysis and visualization of various types of geophysical data.

GOAT archives all geophysical data in a consistent format thereby alleviating many of the annoyances related to the inconsistency of spatial and temporal resolutions, naming, units and file formats across the ever growing number of observed and simulated geophysical datasets.

model diagnostics packages

Does anyone know of model diagnostics packages one can get. NCAR has a few. NASA has UV-CDAT. Tx.

MERRA-2 precipitation rate in WRIT time series plotter

Nice tool. However, the MERRA-2 global average precipitation rate is different from that shown in figure 6 Bosilovich et al, 2017, Atmospheric Water Balance and Variability in the MERRA-2 Reanalysis, J. Clim, 30.

I can send a screen shot and the plot if you need, just email me.

The plotting options I used are:

Dataset: MERRA-2

Variable: Precipitation Rate (note selecting this variable gives an option to select a level, I assume this is irrelevant but selected 1000 hPa)

Time averaging: None

Variable statistic: mean

Grid Point: -90 to 90 N, 0 to 360 E, no mask (All)

Output type: time series

Smoothing: 12 month running mean

Cheers

Andy Barrett

NSIDC, CIRES, University of Colorado at Boulder

Plots

Hi- I'll look at this. Other single variables look fine as a global average for the MERRA-2. I think it might be a data issue for the SH.

RE: MERRA-2 precipitation rate in WRIT time series plotter

There are two global precipitation fields in the MERRA-2 flux data collection. PRECTOT is determined from the model physics and represents the condensation sink of water vapor out of the atmosphere. However, this field is not used for water source/forcing in the land surface. PRECTOTCORR was developed by Reichle et al. to provide observation corrected precipitation to the land surface. Fig 6 is PRECTOT, the modeled precipitation. Make sure that PRECTOTCORR isnt being presented without explaining its purpose.

There is a substantial difference between these two fields over land. These need consideration in any water budget study.

Add new comment

Comparison

Climate Comparisons in the MERRA Atlas

The MERRA Atlas provides monthly, annual, seasonal and climate comparisons among several of the latest atmospheric reanalyses (including ERA-Interim, CFSR, and JRA25) and various global observation datasets.

Reanalysis Comparisons on the WRIT webpages

The NOAA-CIRES Web-based Reanalysis Intercomparison Tools (WRIT) has a prototype plotting/comparison page that is available for testing and commenting by reanalyses.org users and others. To obtain a login to reanalyses.org go to Create New Account.

Observational Studies that examine reanalysis datasets and compare them and observations are described here.

Inter-reanalysis studies that intercompare reanalyses are described here.

WCRP Task Team for the Intercomparison of ReAnalyses (TIRA) - A group of reanalysis developers and users charged to develop a broad reanalysis intercomparison project.

SPARC Reanalysis Intercomparison Project

A coordinated activity under the WCRP/SPARC focusing on reanalysis output in the stratosphere, upper troposphere, and lower mesosphere.

Tables:

Reanalyses.org Overview Comparison

ClimateDataGuide Summary and Comparison

Ana4MIPS Data Description

Comparison of ERA-Interim and NCEP/NCAR

Hi, Is NCEP/NCAR reanalysis an outdated model? . Can there be any need to compare surface variables from ERA-Interim and NCEP/NCAR reanalysis over Africa

Re: Comparison of ERA-Interim and NCEP/NCAR

While NCEP/NCAR is using a much older model for the first guess, and an older data assimilation system, it is still widely used around the world. The main paper on the NCEP/NCAR reanalysis, Kalnay et al. (1996) http://dx.doi.org/10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2. , specifically lists the expected quality of surface-related variables. For the case of southern africa, the recent paper Zhang et al (2012) http://dx.doi.org/10.1007/s00382-012-1423-z examines precipitation. You can make quick comparisons of several surface variables using the Web-based reanalysis intercomparison tool or other tools listed at https://reanalyses.org/atmosphere/tools. The other question you need to ask is what observations are included. The 2m air temperature from stations is indirectly assimilated in ERA-Interim, while it is not in NCEP/NCAR. Please let us know of other questions, or if anything can be clarified. best wishes,

Re: Comparison of ERA-Interim and NCEP/NCAR

Is there any comparison of 2m daily mean temperature between ERA-Interim and observations (maybe gridded)?

Re: Comparison of ERA-Interim and NCEP/NCAR

ERA-Interim (like ERA-40 and the JRA reanalyses, but unlike the NCEP/NCAR reanalysis) used the 2m temperature observations to produce its 2m temperature values. Any comparisons thus have to be carried out particularly carefully. We did this recently for the Arctic (Simmons & Poli, doi: 10.1002/qj.2422), though for instantaneous temperatures not daily means. This paper and a couple of Simmons et al. papers it refers to provide details on how we analyse screen-level observations. The ERA-Interim system (which is more than nine years old) is rather limited in the feedback information it saves from its 2m temperature analysis. This will be remedied in the replacement for ERA-Interim that is currently being prepared for production.

North American atm temp: 20cr vs ncep/ncar

I am no expert on reanalyses, so I thought I would ask this community this question.

I have been comparing 20cr to ncep/ncar reanalysis. I have computed DJF mslp anomalies wrt 1981-2010 over part of the North Atlantic (40W-5E, 45-70N). Not only do the two agree very well in the overlap period but there is very little spread in the 20cr ensemble in values going back to 1900. Perhaps to be expected, but I am very pleased by this!

For 850hPa T over part of North America (70-110W, 40-65N), however, the range in the early part of the series is larger and while the comparison with the ncep/ncar shows reasonable agreement in terms of the year-to-year variations, there are instances where the ncep/ncar data are outside the range of the 20cr ensemble, even in the most recent years.

Presumably ncep/ncar is a better estimate in recent times as it includes radiosonde/satellite observations, but would the range of the 20cr be expected to include the other reanalysis? Or is that not expected based on the biases in the underlying model?

Any guidance would be very gratefully received.

Re: North American atm temp: 20cr vs ncep/ncar

Ideally you would expect the 20CR range to cover the other reanalyses in most instances if both are unbiased and NCEP/NCAR is error free. Temperatures in 20CR have problems over the Arctic, so there might be biases in the northernmost part of your domain (and they have a seasonal cycle), and generally that region is not very well represented prior to around 1940. But other than that, I am not aware of problems so if you could name the instances that might help further. Stefan Brönnimann

Re: Comparison

I'm new to this site and have just prepared a page in my own 'area'. After some fiddling around, I seemed to get something going, though will need some fine-tuning as I re-learn Wiki. It deals with an issue regarding some trends in the US Reanalysis data sets and I've obtained some EC data for comparison. Hence it will have some focus on comparison, so I suppose the heading of this thread is apt.

My question is: If I have done the right thing by doing it in my area, how do I get it linked into the main site for others to have a read?

Sorry for troubling you but I sincerely think is an important issue for the folks here to consider.

I warmly appreciate what the site is about and look forward to helping the cause.

Dave

Re: Comparison

Dave, Happy to help. Welcome! Please be very specific when you refer to something. What is "EC data"? This could refer to Environment Canada or sometimes to the European Centre for Medium Range Weather Forecasts. When you are authenticated, look at the Help area. Directions for how to link and how to include images should be there, e.g., https://reanalyses.org/help/how-edit-site Please put Help related comments under the Help area if at all possible. Then the Help pages can themselves be updated as your questions are answered. best wishes, gil compo

Re: Comparison

To answer the question about the link, it should probably be linked from here https://reanalyses.org/atmosphere/inter-reanalysis-studies-0 On there top bar, there is a help link. If you click it, it tells you how to a) create the page and b) link to it. If you have problems, comment and we will try to help/improve directions. Cathy

Wind Velocity

I am using wind velocity and I want to calculate it at a height of 80mtrs from the ground . So could you tell me that at what height from the ground these data have been taken????

Re: Wind Velocity

Typically, there is meteorology produced at 2m and 10m above the surface. MERRA produces winds at 50m, and also the lowest model level. the lowest model level does not have a fixed height above the surface, but the geopotential height of the lowest model level is also provided. pressure level wind profile data is also provided, but again, not at a fixed height, geopotential height is provided. 80m is an atypical height for most reanalysis output. In the case of MERRA, you would need the 2 lowest model levels to interpolate to that height. These are available in 3D data files, but, there would be a substantial amount of work to get the interpolated winds.

Another general answer to your question is to read the documentation associated with the reanalysis system that you are using to determine which height the data are at, in case it is not obvious (such as u2m and v2m).

I hope this helps,

Add new comment

Pregenerated Images

MERRA Atlas: Climatological comparisons with JRA, CFSR, NCEP II, Interim, ERA 40, GPCP, SRB and more

JRA-25 Atlas: climatologies (annual, seasonal) of most variables. Long timeseries of some

IRI Maproom: recent climate on weeks, months and seasonal timescale including animations (R1).

ESRL/PSD maproom: recent climate on weeks, months and seasonal timescale including animations (R1).

MERRA-2 Weather Maps: Hourly and 3 hourly fields from MERRA-2, produced on the fly, for the purpose of efficient synoptic evaluations of historical weather events (1980-present).

Add new comment

Software

List of Plotting/Analysis Software useful for Reanalysis Datasets

CDAT: Climate Data Analysis Tools (CDAT) is a software system designed to provide access to and management of gridded climate data. It uses the Python scripting language which provides a general purpose and full-featured scripting language with a variety of user interfaces including command-line interaction, stand-alone scripts (applications) and graphical user interfaces (GUI). The modular CDAT subsystems provide access to the data, to large-array numerical operations (via Numerical Python), and visualization.
Ferret*: Ferret is an interactive computer visualization and analysis environment designed to meet the needs of oceanographers and meteorologists analyzing large and complex gridded data sets. It runs on most Unix systems, and on Windows XP/NT/9x using X windows for display.
GrADS*: The Grid Analysis and Display System (GrADS) is an interactive desktop tool that is used for easy access, manipulation, and visualization of earth science data. GrADS has two data models for handling gridded and station data. GrADS supports many data file formats, including binary (stream or sequential), GRIB (version 1 and 2), NetCDF, HDF (version 4 and 5), and BUFR (for station data). GrADS has been implemented worldwide on a variety of commonly used operating systems and is freely distributed over the Internet.
IDL*: IDL is a commercial cross platform data analysis/mapping/plotting code that is extremely customizable.
IDV*: is a Java(TM)-based software framework for analyzing and visualizing geoscience data. The IDV brings together the ability to display and work with satellite imagery, gridded data, surface observations, balloon soundings, NWS WSR-88D Level II and Level III radar data, and NOAA National Profiler Network data, all within a unified interface.
Matlab: MATLAB is a high-level language and interactive environment w/extensive plotting and numerical processing available.
ncBrowse: ncBrowse is a Java application that provides flexible, interactive graphical displays of data and attributes from a wide range of netCDF data file conventions.
NCL*: Is an interpreted language designed specifically for data analysis and visualization.
Panoply: Panoply is a cross-platform application which plots geo-gridded arrays from netCDF, HDF and GRIB datasets. You can plot data as well as do some simple data analysis. Outputs are images, postscript and kml

*'d are OPeNDAP enabled

Dataset Analysis Tools

Excel: A desktop analysis tool. It can read netCDF files.
CDO: CDO is a collection of command line Operators to manipulate and analyse Climate and forecast model Data.

Supported data formats are GRIB, netCDF, SERVICE, EXTRA and IEG. There are more than 400 operators available.
NCO: A (free) suite of programs that operate on netCDF files. Each operator is a standalone, command line program which is executed at the UNIX (or NT) shell-level like, e.g., ls or mkdir. The operators take netCDF or HDF4 files as input, then perform a set of operations (e.g., deriving new data, averaging, hyperslabbing, or metadata manipulation) and produce a netCDF file as output. The operators are primarily designed to aid manipulation and analysis of gridded scientific data. These tools are a powerful and easy way to perform simple manipulations on netCDF files without a major programming effort.

Read Files Remotely

GDS: The GrADS Data Server (GDS, formerly known as GrADS-DODS Server) is a stable, secure data server that provides subsetting and analysis services across the internet.
TDS: The THREDDS Data Server (TDS) is a web server that provides metadata and data access for scientific datasets, using OPeNDAP, OGC WMS and WCS, HTTP, and other remote data access protocols.

Add new comment

In the news

20th Century Reanalysis: In the news, new articles

NASA Center for Climate Simulation which provided computing for MERRA and GISS Climate simulations: June 2010

Japanese 55-year Reanalysis :JRA-55

JRA-55 (1958-2012) had been completed in 2013 and it is coninuously operated in real time basis after 2013 to present. A full Report of JRA-55 is available from JMSJ.

Add new comment

Subscribe to Atmosphere

Recent Updates

Overview of current atmospheric reanalyses
4 months 3 weeks ago
Atmospheric Reanalyses Comparison Table
4 months 3 weeks ago
ocada references
4 months 3 weeks ago
ocada notes questions and discussion
4 months 3 weeks ago
core-references
4 months 3 weeks ago

Search

Add new comment

Data Extraction

Step-by-Step Guide to obtaining data files

Post Processing Routines and Algorithms

Webtools to plot/analyze data by Function

Basic Maps

Other Basic Geographic Plots

Hovmollers

Advanced Plots

Composite Maps (Average different, possibly non-contiguous dates together)

Correlation Maps

Timeseries Plots

Timeseries Analysis

Google Earth

Miscellaneous

Applications that read/plot/analyze netCDF and/or grib data (non-web)

Add new comment

Add new comment

Add new comment

Add new comment

List of web-based tools for comparing reanalysis datasets:

Add new comment

Climate Comparisons in the MERRA Atlas

Reanalysis Comparisons on the WRIT webpages

Observational Studies that examine reanalysis datasets and compare them and observations are described here.

Inter-reanalysis studies that intercompare reanalyses are described here.

WCRP Task Team for the Intercomparison of ReAnalyses (TIRA) - A group of reanalysis developers and users charged to develop a broad reanalysis intercomparison project.

Add new comment

Add new comment

Dataset Analysis Tools

Add new comment

Add new comment