NCEAS Scientific Computing: Reading NetCDF Files with MATLAB

Reading and Processing NetCDF-Format Climate Model Data Using MATLAB

This case demonstrates use of a MATLAB script to read and process long-range 'reanalysis' climate data (relative humidity) that is delivered in the industry-standard Network Common Data Format (NetCDF).

This Use Case demonstrates how to:

Browse NetCDF-format file headers with NetCDF Explorer

Read NetCDF files with the MATLAB maxinc package routines

View the data with a MATLAB plot

The Network Common Data Form (NetCDF) is a machine-in dependant binary data format widely used by scientists to store and distribute gridded climate data sets, including multidimensional data sets such as time series data. As the use of the NetCDF format has grown, most standard data analysis software packages have added the capability to read and analyze data in this format. Click here for a list of organizations distributing NetCDF data.

Acquiring Coupled Atmosphere-Ocean General Circulation Model (AOGCM) Data sets

Ecologists often require long-range estimates of key climate parameters such as surface temperature, precipitation, humidity, and wind speed/direction. Coupled atmosphere-ocean General Circulation Models (AOGCM) are excellent sources of large-scale (e.g., global) climate parameter estimates. Many are in use worldwide, thus the issue arises: which model is most appropriate for a particular research project?

The mission of the Program for Climate Model Diagnosis and Intercomparison (PCMDI) established in 1989 at the Lawrence Livermore National Laboratory (LLNL), is to develop improved methods and tools for the diagnosis and intercomparison of general circulation models (GCMs) that simulate the global climate. Click here for a brief introduction to CMIP. Among the many features and services offered by this program is a climate model data set archive developed as part of the Coupled Model Intercomparison Project (CMIP3). The CMIP3 project collects and compares the output from approximately 35 General Circulation Models. This is a very good place to obtain such data; however, the site stewards require that scientists register with them (and describe the ways in which they expect to use the data) before gaining access to the data.

Step 1: Search for and acquire data set

For this use case, we require a global-scale, long-range (100 years moving forward from the year 2000) monthly Sea Surface Temperature estimate. After securing an account for the CMIP Archive, we searched the available data sets and selected as a source the SRES A2 experiment produced using the UK Metrological Office Hadley Centre Hadley CM3 (HadCM3) climate model. the SRES scenario A2, generated as part of the IPCC Special Report on Emissions Scenarios. We then downloaded the data set, in NetCDF format, via File Transfer Protocol (FTP).

Step 2: Review NetCDF metadata with HDF Explorer

We use the HDF Explorer software, which reads NetCDF files as well as the more complex HDF files to review the time series data file's metadata. HDF Explorer's 'file tree viewer' interface, similar to Microsoft Explorer, enables users to drill down into the NetCDF file's metadata. Click here to see the HDF Explorer view of the sample Relative Humidity file's metadata, which has three major levels: Dimensions, Variables, and Attributes. The Dimensions section depicts the nesting order of the parameters and the dimensions of the sea surface temperature data: 144 rows, 288 columns, and 1200 time periods: monthly averages for 100 years extending forward from the year 2000.

Step 3: Read and explore Ocean Surface Temperature time series with MATLAB
We intend to use these data as part of an analysis model developed using MATLAB, so we will augment the standard MATLAB installation with a the mexnc package, which is an implementation of the NetCDF data access library developed by the inventors of the NetCDF format.

We install the package on our Linux workstation (which has MATLAB R14 installed) by downloading the mexcdf package, placing the files in a folder accessible to the MATLAB executable, and adding that folder's path name to the MATLAB executable path variable.

Reading the NetCDF file using the mexinc API

The mexinc command API consists of the mexinc() function call in combination with specific parameters. The following script demonstrates connecting a NetCDF file to a MATLAB session and then reading the surface temperature data into a MATLAB matrix
% March 19,2007 : script to read and view part of Hadley Climate Model output 
% file in NetCDF format using Matlab and the 'mexinc' NetCDF library 
% implementation for Matlab R14
%
% Requirements: mexnc package must be integrated into the Matlab environment 
% (the .mex file and support files must be installed on the system and the 
% path to the file declared in the Matlab environment). This can be performed 
% by the System Administrator, or by the user, using the SetPath feature of the 
% Matlab GUI or the Matlab path() function.
%
% Open the file and get the handle for later use.
%
[ncid,status] = 
mexinc('open','/data/computer/reeves/mbaskett/pcmdi.ipcc4.ukmo_hadcm3.sresa2.run
1.mont
hly.tos_O1.nc',nc_nowrite_mode)
%
% Get the dimensions of the file
%
[ndims,nvars,ngatts,unlimdim,status] = mexnc('inq',ncid)
% 
% Read the ID for the variable in the file: Sea Surface Temperature, or 'tos'. 
% I learned the name of the variable by using the HDF Explorer utility, 
% which reads NetCDF file metadata.
%
[varid,status] = mexnc('inq_varid',ncid,'tos')
%
[varname,datatype,ndims,dimids,natts,status] = mexnc('inq_var',ncid,varid)
%
% Read the entire 'tos' data cube into a three-multidimensional Matlab matrix. 
% this matrix will have 288 rows, 144 columns, and 1200 'planes' 
%
[tosdata,status] = mexnc('get_var_double',ncid,varid);
%
% Extract rows 100-120, cols 20-40 of plane 100
%
tosplane = tosdata(100:120,20:40,100)   // displays a sub set of the the three-dim matrix.....
%
% test plot: extract center of this sub-cube for the first 100 months.
% squeeze function removes 'singleton' dimensions from the 1 row, 1 column,
% 100 'deep' 3-d matrix 'tosdata'.
%
tosvec = squeeze(tosdata(100:100,20:20,1:100)) 
%
% a simple vector with consecutive month numbers
%
timevec = 1:100
%
% display a simple plot: month # on X, temperature on Y
%
plot (timevec,tosvec)
Click here to download this script file.

Learning More:

Coupled Model Inter Comparison Project (CMIP): CMIP archive home page

NetCDF File Format: NetCDF (Network Common Data Form) file format home page

HDF Explorer: Space Research Software (distributor of HDF Explorer) home page

Point of Contact for this Use Case: Rick Reeves, NCEAS Scientific Programmer reeves@nceas.ucsb.edu
This Use Case compiled April, 2007

	NCEAS Scientific Computing: Solutions Center Use Case
Home Meet the Staff Reports/Tutorials Software Tools Data Resources Solutions Center	Reading and Processing NetCDF-Format Climate Model Data Using MATLAB This case demonstrates use of a MATLAB script to read and process long-range 'reanalysis' climate data (relative humidity) that is delivered in the industry-standard Network Common Data Format (NetCDF). This Use Case demonstrates how to: Browse NetCDF-format file headers with NetCDF Explorer Read NetCDF files with the MATLAB maxinc package routines View the data with a MATLAB plot The Network Common Data Form (NetCDF) is a machine-in dependant binary data format widely used by scientists to store and distribute gridded climate data sets, including multidimensional data sets such as time series data. As the use of the NetCDF format has grown, most standard data analysis software packages have added the capability to read and analyze data in this format. Click here for a list of organizations distributing NetCDF data. Acquiring Coupled Atmosphere-Ocean General Circulation Model (AOGCM) Data sets Ecologists often require long-range estimates of key climate parameters such as surface temperature, precipitation, humidity, and wind speed/direction. Coupled atmosphere-ocean General Circulation Models (AOGCM) are excellent sources of large-scale (e.g., global) climate parameter estimates. Many are in use worldwide, thus the issue arises: which model is most appropriate for a particular research project? The mission of the Program for Climate Model Diagnosis and Intercomparison (PCMDI) established in 1989 at the Lawrence Livermore National Laboratory (LLNL), is to develop improved methods and tools for the diagnosis and intercomparison of general circulation models (GCMs) that simulate the global climate. Click here for a brief introduction to CMIP. Among the many features and services offered by this program is a climate model data set archive developed as part of the Coupled Model Intercomparison Project (CMIP3). The CMIP3 project collects and compares the output from approximately 35 General Circulation Models. This is a very good place to obtain such data; however, the site stewards require that scientists register with them (and describe the ways in which they expect to use the data) before gaining access to the data. Step 1: Search for and acquire data set For this use case, we require a global-scale, long-range (100 years moving forward from the year 2000) monthly Sea Surface Temperature estimate. After securing an account for the CMIP Archive, we searched the available data sets and selected as a source the SRES A2 experiment produced using the UK Metrological Office Hadley Centre Hadley CM3 (HadCM3) climate model. the SRES scenario A2, generated as part of the IPCC Special Report on Emissions Scenarios. We then downloaded the data set, in NetCDF format, via File Transfer Protocol (FTP). Step 2: Review NetCDF metadata with HDF Explorer We use the HDF Explorer software, which reads NetCDF files as well as the more complex HDF files to review the time series data file's metadata. HDF Explorer's 'file tree viewer' interface, similar to Microsoft Explorer, enables users to drill down into the NetCDF file's metadata. Click here to see the HDF Explorer view of the sample Relative Humidity file's metadata, which has three major levels: Dimensions, Variables, and Attributes. The Dimensions section depicts the nesting order of the parameters and the dimensions of the sea surface temperature data: 144 rows, 288 columns, and 1200 time periods: monthly averages for 100 years extending forward from the year 2000. Step 3: Read and explore Ocean Surface Temperature time series with MATLAB We intend to use these data as part of an analysis model developed using MATLAB, so we will augment the standard MATLAB installation with a the mexnc package, which is an implementation of the NetCDF data access library developed by the inventors of the NetCDF format. We install the package on our Linux workstation (which has MATLAB R14 installed) by downloading the mexcdf package, placing the files in a folder accessible to the MATLAB executable, and adding that folder's path name to the MATLAB executable path variable. *Reading the NetCDF file using the mexinc* API The mexinc command API consists of the mexinc() function call in combination with specific parameters. The following script demonstrates connecting a NetCDF file to a MATLAB session and then reading the surface temperature data into a MATLAB matrix % March 19,2007 : script to read and view part of Hadley Climate Model output % file in NetCDF format using Matlab and the 'mexinc' NetCDF library % implementation for Matlab R14 % % Requirements: mexnc package must be integrated into the Matlab environment % (the .mex file and support files must be installed on the system and the % path to the file declared in the Matlab environment). This can be performed % by the System Administrator, or by the user, using the SetPath feature of the % Matlab GUI or the Matlab path() function. % % Open the file and get the handle for later use. % [ncid,status] = mexinc('open','/data/computer/reeves/mbaskett/pcmdi.ipcc4.ukmo_hadcm3.sresa2.run 1.mont hly.tos_O1.nc',nc_nowrite_mode) % % Get the dimensions of the file % [ndims,nvars,ngatts,unlimdim,status] = mexnc('inq',ncid) % % Read the ID for the variable in the file: Sea Surface Temperature, or 'tos'. % I learned the name of the variable by using the HDF Explorer utility, % which reads NetCDF file metadata. % [varid,status] = mexnc('inq_varid',ncid,'tos') % [varname,datatype,ndims,dimids,natts,status] = mexnc('inq_var',ncid,varid) % % Read the entire 'tos' data cube into a three-multidimensional Matlab matrix. % this matrix will have 288 rows, 144 columns, and 1200 'planes' % [tosdata,status] = mexnc('get_var_double',ncid,varid); % % Extract rows 100-120, cols 20-40 of plane 100 % tosplane = tosdata(100:120,20:40,100) // displays a sub set of the the three-dim matrix..... % % test plot: extract center of this sub-cube for the first 100 months. % squeeze function removes 'singleton' dimensions from the 1 row, 1 column, % 100 'deep' 3-d matrix 'tosdata'. % tosvec = squeeze(tosdata(100:100,20:20,1:100)) % % a simple vector with consecutive month numbers % timevec = 1:100 % % display a simple plot: month # on X, temperature on Y % plot (timevec,tosvec) Click here to download this script file. Learning More: Coupled Model Inter Comparison Project (CMIP): CMIP archive home page NetCDF File Format: NetCDF (Network Common Data Form) file format home page HDF Explorer: Space Research Software (distributor of HDF Explorer) home page Point of Contact for this Use Case: Rick Reeves, NCEAS Scientific Programmer reeves@nceas.ucsb.edu This Use Case compiled April, 2007** Site Home \| NCEAS Home \| KNB Home