Data Sources

Local Module

local.load_GPM_IMERG_files(file_path=None, filename_pattern=None, filelist=None, variable_name='precipitationCal', name='GPM_IMERG')

Load multiple GPM Level 3 IMEGE files containing calibrated precipitation and generate an OCW Dataset obejct.

Parameters:
  • file_path (string) – Directory to the HDF files to load.
  • filename_pattern (string) – Path to the HDF files to load.
  • filelist (string) – A list of filenames
  • variable_name (string) – The variable name to load from the HDF file.
  • name (string) – (Optional) A name for the loaded dataset.
Returns:

An OCW Dataset object with the requested variable’s data from the HDF file.
Return type:

dataset.Dataset
Raises:

ValueError –

local.load_GPM_IMERG_files_with_spatial_filter(file_path=None, filename_pattern=None, filelist=None, variable_name='precipitationCal', user_mask_file=None, mask_variable_name='mask', user_mask_values=[10], longitude_name='lon', latitude_name='lat')

Load multiple GPM Level 3 IMEGE files containing calibrated precipitation and generate a two-dimensional array for the masked grid points. :param file_path: Directory to the HDF files to load. :type file_path: string :param filename_pattern: Path to the HDF files to load. :type filename_pattern: string :param filelist: A list of filenames :type filelist: string :param variable_name: The variable name to load from the HDF file. :type variable_name: string :param name: (Optional) A name for the loaded dataset. :type name: string :user_mask_file: user’s own gridded mask file(a netCDF file name) :type name: string :mask_variable_name: mask variables in user_mask_file :type name: string :longitude_name: longitude variable in user_mask_file :type name: string :latitude_name: latitude variable in user_mask_file :type name: string :param user_mask_values: grid points where mask_variable == user_mask_value will be extracted. :type user_mask_values: list of strings :returns: A two-dimensional array with the requested variable’s MASKED data from the HDF file. :rtype: dataset.Dataset :raises ValueError:

local.load_NLDAS_forcingA_files(file_path=None, filename_pattern=None, filelist=None, variable_name='APCPsfc_110_SFC_acc1h', name='')

Load multiple NLDAS2 forcingAWRF files containing 2D fields such as precipitation and surface variables into a Dataset. The dataset can be spatially subset.

Parameters:
  • file_path (string) – Directory to the NetCDF file to load.
  • filename_pattern (string) – Path to the NetCDF file to load.
  • filelist (string) – A list of filenames
  • variable_name (string) – The variable name to load from the NetCDF file.
  • name (string) – (Optional) A name for the loaded dataset.
Returns:

An OCW Dataset object with the requested variable’s data from the NetCDF file.
Return type:

dataset.Dataset
Raises:

ValueError –

local.load_WRF_2d_files(file_path=None, filename_pattern=None, filelist=None, variable_name='T2', name='')

Load multiple WRF (or nuWRF) original output files containing 2D fields such as precipitation and surface variables into a Dataset. The dataset can be spatially subset.

Parameters:
  • file_path (string) – Directory to the NetCDF file to load.
  • filename_pattern (string) – Path to the NetCDF file to load.
  • filelist (string) – A list of filenames
  • variable_name (string) – The variable name to load from the NetCDF file.
  • name (string) – (Optional) A name for the loaded dataset.
Returns:

An OCW Dataset object with the requested variable’s data from the NetCDF file.
Return type:

dataset.Dataset
Raises:

ValueError –

local.load_WRF_2d_files_RAIN(file_path=None, filename_pattern=None, filelist=None, name='')

Load multiple WRF (or nuWRF) original output files containing 2D fields such as precipitation and surface variables into a Dataset. The dataset can be spatially subset.

Parameters:
  • file_path (string) – Directory to the NetCDF file to load.
  • filename_pattern (string) – Path to the NetCDF file to load.
  • name (string) – (Optional) A name for the loaded dataset.
Returns:

An OCW Dataset object with the requested variable’s data from the NetCDF file.
Return type:

dataset.Dataset
Raises:

ValueError –

local.load_dataset_from_multiple_netcdf_files(variable_name, variable_unit=None, lat_name=None, lon_name=None, time_name=None, name='', file_list=None, file_path=None, filename_pattern=None, mask_file=None, mask_variable=None, mask_value=0)

Load multiple netCDF files from the same source (an observation or a model) into a Dataset. The dataset can be spatially subset.

Parameters:
  • variable_name (string) – The variable name to load from the NetCDF file.
  • variable_unit (string) – The variable’s unit to load from the NetCDF file.
  • lat_name (string) – (Optional) The latitude variable name to extract from the dataset.
  • lon_name (string) – (Optional) The longitude variable name to extract from the dataset.
  • time_name (string) – (Optional) The time variable name to extract from the dataset.
  • name (string) – (Optional) A name for the loaded dataset.
  • file_list (string) – A text file including a list of filenames
  • file_path (string) – Directory to the NetCDF file to load.
  • filename_pattern (string) – Path to the NetCDF file to load.
  • mask_file – A netcdf file with two-dimensional mask indices
  • mask_variable – The variable name to load from the mask_file.
  • mask_value (int) – an index for spatial subsetting a dataset
Returns:

An OCW Dataset object with the requested variable’s data from the NetCDF file.
Return type:

dataset.Dataset
Raises:

ValueError –

local.load_file(file_path, variable_name, variable_unit=None, elevation_index=0, name='', lat_name=None, lon_name=None, time_name=None)

Load a NetCDF file into a Dataset.

Parameters:
  • file_path (string) – Path to the NetCDF file to load.
  • variable_name (string) – The variable name to load from the NetCDF file.
  • variable_unit (string) – (Optional) The variable unit to load from the NetCDF file.
  • elevation_index (int) – (Optional) The elevation index for which data should be returned. Climate data is often times 4 dimensional data. Some datasets will have readins at different height/elevation levels. OCW expects 3D data so a single layer needs to be stripped out when loading. By default, the first elevation layer is used. If desired you may specify the elevation value to use.
  • name (string) – (Optional) A name for the loaded dataset.
  • lat_name (string) – (Optional) The latitude variable name to extract from the dataset.
  • lon_name (string) – (Optional) The longitude variable name to extract from the dataset.
  • time_name (string) – (Optional) The time variable name to extract from the dataset.
Returns:

An OCW Dataset object with the requested variable’s data from the NetCDF file.
Return type:

dataset.Dataset
Raises:

ValueError – When the specified file path cannot be loaded by ndfCDF4 or when the lat/lon/time variable name cannot be determined automatically.
local.load_multiple_files(file_path, variable_name, generic_dataset_name=False, dataset_name=None, variable_unit=None, lat_name=None, lon_name=None, time_name=None)

load multiple netcdf files with common filename pattern and return an array of OCW datasets

Parameters:
  • file_path (string) – directory name and (optionally common file name patterns) where the NetCDF files to load are stored.
  • dataset_name (:mod:'list') – a list of dataset names. Data filenames must include the elements of dataset_name list.
  • generic_dataset_name (:mod:'bool') – If false, data filenames must include the elements of dataset_name list.
  • variable_name (string) – The variable name to load from the NetCDF file.
  • variable_unit (string) – (Optional) The variable unit to load from the NetCDF file.
  • lat_name (string) – (Optional) The latitude variable name to extract from the dataset.
  • lon_name (string) – (Optional) The longitude variable name to extract from the dataset.
  • time_name (string) – (Optional) The time variable name to extract from the dataset.
Returns:

An array of OCW Dataset objects
Return type:

list

RCMED Module

Classes:
RCMED - A class for retrieving data from Regional Climate Model Evalutaion Database (JPL). More information about the RCMED Query Specification can be found below: https://rcmes.jpl.nasa.gov/query-api/query.php?
rcmed.get_parameters_metadata()

Get the metadata of all parameter from RCMED.

Returns:Dictionary of information for each parameter stored in one list
Return type:list of dict
rcmed.parameter_dataset(dataset_id, parameter_id, min_lat, max_lat, min_lon, max_lon, start_time, end_time, name='')

Get data from one database(parameter).

Parameters:
  • dataset_id (int) – Dataset id.
  • parameter_id (int) – Parameter id
  • min_lat (float) – Minimum latitude
  • max_lat (float) – Maximum latitude
  • min_lon (float) – Minimum longitude
  • max_lon (float) – Maximum longitude
  • start_time (datetime.datetime) – Start time
  • end_time (datetime.datetime) – End time
  • name (string) – (Optional) A name for the loaded dataset.
Returns:

An OCW Dataset object contained the requested data from RCMED.
Return type:

dataset.Dataset

DAP Module

ESGF Module

PODAAC Module

podaac_datasource.convert_times_to_datetime(time)

Convert the time object’s values to datetime objects

The time values are stored as some unit since an epoch. These need to be converted into datetime objects for the OCW Dataset object.

Parameters:time (pydap.model.BaseType) – The time object’s values to convert
Returns:list of converted time values as datetime objects
podaac_datasource.extract_l4_granule(variable, dataset_id='', name='', path='/tmp')

Loads a Level4 gridded Dataset from PODAAC :param variable: The name of the variable to read from the dataset. :type variable: string

Parameters:
  • dataset_id (string) – dataset persistent ID. datasetId or shortName is required for a granule search. Example: PODAAC-ASOP2-25X01
  • name (string) – (Optional) A name for the loaded dataset.
  • path (string) – a path on the filesystem to store the granule.
Returns:

A dataset.Dataset containing the dataset pointed to by the OpenDAP URL.
Raises:

ServerError
podaac_datasource.list_available_extract_granule_dataset_ids()

Convenience function which returns an up-to-date list of available granule dataset id’s which can be used in the granule extraction service. :returns: a comma-seperated list of granule dataset id’s.

podaac_datasource.subset_granule(variable, dataset_id='', name='', path='/tmp', input_file_path='')

Subset Granule service allows users to Submit subset jobs. Use of this service should be preceded by a Granule Search in order to identify and generate a list of granules to be subsetted.

Parameters:
  • variable (string) – The name of the variable to read from the dataset.
  • dataset_id (string) – dataset persistent ID. datasetId or shortName is required for a granule search. Example: PODAAC-ASOP2-25X01
  • name (string) – (Optional) A name for the loaded dataset.
  • path (string) – (Optional) a path on the filesystem to store the granule.
  • input_file_path (string) – path to a json file which contains the the subset request that you want to send to PO.DAAC
Returns:

a token on successful request reception. This can be further used to check the status of the request.