Downloading CMIP5 data

The ESGF portal I used is this one hosted by Department of Energy and Lawrence Livermore National Laboratory. In order to download data, one needs to register an account and get an OpenID. For my use case, I filter by the following criteria

project = CMIP5, experiment = rcp45, time frequency = day, 
variable = tasman,tasmin

This gives me 98 results of about 30 models.

screen shot 2019-01-13 at 1.11.11 pm

The most convenient way to bulk download data is to use the WGET script. First, I add the ones I want to download into the “Data Cart”. Then click “My Data Cart” to go to the cart. You can filter the results by variables by entering the variable name in the top right text box.

screen shot 2019-01-13 at 1.23.42 pm

For example, before filtering, there will be 319 files in the first dataset. After filtering with “tasmin”, when you click “List Files” of the first dataset, there are only 11 files left.

After applying the filters, I click “Select All Datasets”, then click on the “WGET Script” at the very top. This prompt me a small window. I click on the “WGET Script for esgf-node.llnl.gov” link in that window. This will download the WGET script for downloading all selected items in my data cart.

Last, I open the terminal and type the following to start downloading. It will prompt you to enter the OpenID and password.

bash wget-20190113094925.sh -H

Files will be downloaded in the same folder as the wget script. Depending on time duration, some files are several hundred MBs, and some are over 1GB.

How to read and process the downloaded .nc files is described in another post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s