Raw Data I/O#
Please have a look at the tutorial Data Input - Data Output for an introduction on how to deal with different file formats.
Xarray backends#
Reading radar data into xarray Datasets using xarray.open_dataset
and xarray.open_mfdataset
.
Former available backends have been ported to xradar-package.
Xarray BackendEntrypoint for RADOLAN data. |
- class wradlib.io.backends.RadolanBackendEntrypoint[source]#
Bases:
BackendEntrypoint
Xarray BackendEntrypoint for RADOLAN data.
- url: ClassVar[str] = 'https://docs.wradlib.org/en/stable/notebooks/fileio/backends/radolan_backend.html'#
- open_dataset(filename_or_obj, *, mask_and_scale=True, decode_times=True, concat_characters=True, decode_coords=True, drop_variables=None, use_cftime=None, decode_timedelta=None, fillmissing=False, copy=False, ancillary=False)[source]#
Backend open_dataset method used by Xarray in
open_dataset
.
Digital Elevation Model Data I/O#
Provide surface/terrain elevation information from SRTM data
Download NASA SRTM elevation data Only available with login/password |
|
Get NASA SRTM elevation data |
|
Get NASA SRTM elevation data tile names |
- wradlib.io.dem.download_srtm(filename, destination, *, resolution=3, session=None)[source]#
Download NASA SRTM elevation data Only available with login/password
- wradlib.io.dem.get_srtm(extent, *, resolution=3, merge=True, session=None)[source]#
Get NASA SRTM elevation data
- Parameters:
- Returns:
dataset (
osgeo.gdal.Dataset
) – gdal.Dataset Raster dataset containing elevation information
GDAL Raster/Vector Data I/O#
Open vector file, return gdal.Dataset and OGR.Layer |
|
Open raster file, return gdal.Dataset |
|
Read MSG SAFNWC hdf5 file into a gdal georeferenced object |
|
Creates GDAL.DataSet object. |
|
Write raster dataset to file format |
|
DataSource class for handling ogr/gdal vector data |
- wradlib.io.gdal.open_vector(filename, *, driver=None, layer=0)[source]#
Open vector file, return gdal.Dataset and OGR.Layer
Warning
dataset and layer have to live in the same context, if dataset is deleted all layer references will get lost
- Parameters:
- Returns:
dataset (
osgeo.gdal.Dataset
) – gdal.Datasetlayer (
osgeo.ogr.Layer
) – ogr.Layer
- wradlib.io.gdal.open_raster(filename, *, driver=None)[source]#
Open raster file, return gdal.Dataset
- Parameters:
- Returns:
dataset (
osgeo.gdal.Dataset
) – gdal.Dataset
- wradlib.io.gdal.read_safnwc(filename)[source]#
Read MSG SAFNWC hdf5 file into a gdal georeferenced object
- Parameters:
filename (
str
) – satellite file name- Returns:
ds (
osgeo.gdal.Dataset
) – gdal.DataSet with satellite data
- wradlib.io.gdal.gdal_create_dataset(drv, name, cols=0, rows=0, bands=0, *, gdal_type=None, remove=False)[source]#
Creates GDAL.DataSet object.
- Parameters:
drv (
str
) – GDAL driver stringname (
str
) – path to filenamecols (
int
) – number of columnsrows (
int
) – number of rowsbands (
int
) – number of raster bandsgdal_type (
gdal:osgeo.ogr.DataType
) – raster data type, e.g. gdal.GDT_Float32remove (
bool
) – if True, existing gdal.Dataset will be removed before creation
- Returns:
out (
osgeo.gdal.Dataset
) – gdal.Dataset
- wradlib.io.gdal.write_raster_dataset(fpath, dataset, *, driver='GTiff', **kwargs)[source]#
Write raster dataset to file format
- Parameters:
fpath (
str
) – A file path - should have file extension corresponding to format.dataset (
osgeo.gdal.Dataset
) – gdal.Dataset gdal raster datasetdriver (
str
, optional) – gdal raster format driver string, defaults to “GTiff”
- Keyword Arguments:
Note
For format and options refer to formats_list.
Examples
- class wradlib.io.gdal.VectorSource(data=None, trg_crs=None, name='layer', source=0, **kwargs)[source]#
Bases:
object
DataSource class for handling ogr/gdal vector data
DataSource handles creates in-memory (vector) ogr DataSource object with one layer for point or polygon geometries.
- Parameters:
data (sequence or
str
) – sequence of source points (shape Nx2) or polygons (shape NxMx2) or Vector File (GDAL/OGR) filename containing source points/polygonstrg_crs (
osgeo.osr.SpatialReference
) – GDAL OSR SRS describing target CRS the source data should be projected to
- Keyword Arguments:
name (
str
) – Layer Name, defaults to “layer”.source (
int
) – Number of layer to load, if multiple layers in source shape file.mode (
str
) – Return type of class access functions/properties. Can be either of “numpy”, “geo” and “ogr”, defaults to “numpy”.src_crs (
osgeo.osr.SpatialReference
) – GDAL OGR SRS describing projection source in which data is provided in.
Warning
Writing shapefiles with the wrong locale settings can have impact on the type of the decimal. If problem arise use
LC_NUMERIC=C
in your environment.Examples
See Vector Source.
- property mode#
- property ds#
Returns VectorSource
- property extent#
- property crs#
- property data#
Returns VectorSource geometries as numpy arrays
Note
This may be slow, because it extracts all source polygons
- property geo#
Returns VectorSource geometries as GeoPandas Dataframe
- get_data_by_att(attr=None, value=None, mode=None)[source]#
Returns DataSource geometries filtered by given attribute/value
- get_data_by_geom(geom=None, mode=None)[source]#
Returns DataSource geometries filtered by given geometry
- Keyword Arguments:
geom (
osgeo.ogr.Geometry
|geopandas.GeoDataFrame
) – OGR.Geometry object or geopandas.GeoDataFrame containing the Geometrymode (
str
) – return type (“numpy”, “geo”, “ogr”), defaults to “numpy”
- dump_vector(filename, *, driver='ESRI Shapefile', remove=True)[source]#
Output layer to OGR Vector File
- load_vector(filename, *, source=0, driver='ESRI Shapefile')[source]#
Read Layer from OGR Vector File
- dump_raster(filename, *, driver='GTiff', attr=None, pixel_size=1.0, **kwargs)[source]#
Output layer to GDAL Rasterfile
- Parameters:
- Keyword Arguments:
- set_attribute(name, values, *, reset_filter=False)[source]#
Add/Set given Attribute with given values
- Parameters:
name (
str
) – Attribute Namevalues (
numpy.ndarray
) – Values to fill in attributes.reset_filter (
bool
, optional) – reset any layer filter (spatial/attribute), defaults to False.
HDF Data I/O#
Former available xarray based code has been ported to xradar-package.
Reads GPM files version V07A. |
|
Reads hdf5 files according to their structure |
|
Reads hdf5 files according to OPERA conventions |
|
Data reader for hdf5 files produced by the commercial GAMIC Enigma V3 MURAN software |
|
Quick storage of one <data> array and a <metadata> dict in an hdf5 file |
|
Loading data from hdf5 files that was stored by |
|
Reads GPM files for matching with GR |
|
Reads TRMM files for matching with GR |
- wradlib.io.hdf.open_gpm_dataset(filename, group)[source]#
Reads GPM files version V07A.
- Parameters:
- Returns:
ds (
xarray.Dataset
) – xarray.Dataset representation of GPM file with requested group.
- wradlib.io.hdf.read_generic_hdf5(fname)[source]#
Reads hdf5 files according to their structure
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the Group/Subgroup directory branches of the original file.- Parameters:
fname (
str
or file-like) – a hdf5 file path or file-like object- Returns:
output (
dict
) – a dictionary that contains both data and metadata according to the original hdf5 file structure
Examples
- wradlib.io.hdf.read_opera_hdf5(fname)[source]#
Reads hdf5 files according to OPERA conventions
Please refer to the OPERA data model documentation [B1] in order to understand how an hdf5 file is organized that conforms to the OPERA ODIM_H5 conventions.
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the Group/Subgroup directory branches of the original file. If the end member of a branch (or path) is “data”, then the corresponding item of output dictionary is a numpy array with actual data.Any other end member (either how, where, and what) will contain the meta information applying to the corresponding level of the file hierarchy.
- wradlib.io.hdf.read_gamic_hdf5(filename, *, wanted_elevations=None, wanted_moments=None)[source]#
Data reader for hdf5 files produced by the commercial GAMIC Enigma V3 MURAN software
See GAMIC homepage for further info (https://www.gamic.com).
- Parameters:
- Returns:
Examples
See GAMIC HDF5.
- wradlib.io.hdf.to_hdf5(fpath, data, *, mode='w', metadata=None, dataset='data', compression='gzip')[source]#
Quick storage of one <data> array and a <metadata> dict in an hdf5 file
This is more efficient than pickle, cPickle or numpy.save. The data is stored in a subgroup named data (i.e. file[“data”]). See
from_hdf5
for retrieving stored data.- Parameters:
fpath (
str
or file-like) – path to the hdf5 file or file-like objectdata (
numpy.ndarray
)mode (
str
) – file open mode, defaults to “w” (create, truncate if exists)metadata (
dict
) – dictionary of data’s attributesdataset (
str
) – describing datasetcompression (
str
) – h5py-compression type {“gzip”|”szip”|”lzf”}, see h5py-documentation for details
- wradlib.io.hdf.from_hdf5(fpath, *, dataset='data')[source]#
Loading data from hdf5 files that was stored by
to_hdf5
- wradlib.io.hdf.read_gpm(filename, *, bbox=None)[source]#
Reads GPM files for matching with GR
- Parameters:
- Returns:
gpm_data (
dict
) – dictionary of gpm data
Examples
- wradlib.io.hdf.read_trmm(filename1, filename2, *, bbox=None)[source]#
Reads TRMM files for matching with GR
- Parameters:
- Returns:
trmm_data (
dict
) – dictionary of trmm data
Examples
IRIS/Sigmet Data I/O#
Reads data from Vaisala’s IRIS data formats. Former available code was ported to xradar-package and is imported from there.
IRIS (Vaisala Sigmet Interactive Radar Information System)
See M211318EN-F Programming Guide ftp://ftp.sigmet.com/outgoing/manuals/
Reading sweep data can be skipped by setting loaddata=False. By default, the data is decoded on the fly. Using rawdata=True the data will be kept undecoded.
Read Iris file and return dictionary. |
|
Class for retrieving data from Sigmet IRIS Product files. |
|
Class for retrieving data from Sigmet IRIS Cartesian Product files. |
- wradlib.io.iris.read_iris(filename, *, loaddata=True, rawdata=False, debug=False, **kwargs)[source]#
Read Iris file and return dictionary.
- Parameters:
filename (
str
or file-like) – Filename of data file or file-like object.loaddata (
bool
ordict
) – If true, retrieves whole data section from file. If false, retrievs only ingest_data_headers, but no data. If dict, retrieves according to given dict:loaddata = {'moment': ['DB_DBZ', 'DB_VEL'], 'sweep': [1, 3, 9]}
rawdata (
bool
) – If true, returns raw unconverted/undecoded data.debug (
bool
) – If true, print debug messages.
- Returns:
data (
dict
) – Ordered Dictionary with data and metadata retrieved from file.
Examples
See Vaisala Sigmet IRIS.
- class wradlib.io.iris.IrisProductFile(filename, **kwargs)[source]#
Bases:
IrisRecordFile
Class for retrieving data from Sigmet IRIS Product files.
- product_identifier = ['CATCH', 'FCAST', 'NDOP', 'SLINE', 'TDWR', 'TRACK', 'VAD', 'VVP', 'WARN', 'WIND', 'STATUS']#
- property data#
- property protect_setup#
- class wradlib.io.iris.IrisCartesianProductFile(irisfile, **kwargs)[source]#
Bases:
IrisRecordFile
Class for retrieving data from Sigmet IRIS Cartesian Product files.
- product_identifier = ['MAX', 'TOPS', 'HMAX', 'BASE', 'THICK', 'PPI', 'RHI', 'CAPPI', 'RAINN', 'RAIN1', 'CROSS', 'SHEAR', 'SRI', 'RTI', 'VIL', 'LAYER', 'BEAM', 'MLHGT']#
- property data#
- get_image(header)[source]#
Retrieve cartesian image.
- Parameters:
header (
dict
) – header dictionary- Returns:
data (
numpy.ndarray
) – 3D array of cartesian data
- decode_data(data, prod)[source]#
Decode data according given prod-dict.
- Parameters:
data (
numpy.ndarray
) – data to decodeprod (
dict
) – dictionary holding decoding information
- Returns:
data (
numpy.ndarray
) – decoded data
Miscellaneous Data I/O#
Writes Polygons to a Text file which can be interpreted by ESRI ArcGIS's "Create Features from Text File (Samples)" tool. |
|
Pickle object <obj> to file <fpath> |
|
Return pickled object from file <fpath> |
|
Download radiosonde data from internet. |
|
Reads membership function parameters from wradlib-data file. |
- wradlib.io.misc.write_polygon_to_text(fname, polygons)[source]#
Writes Polygons to a Text file which can be interpreted by ESRI ArcGIS’s “Create Features from Text File (Samples)” tool.
This is (yet) only a convenience function with limited functionality. E.g. interior rings are not yet supported.
- Parameters:
- Returns:
Note
As Polygons are closed shapes, the first and the last vertex of each polygon must be the same!
Examples
- Writes two triangle Polygons to a text file::
poly1 = [[0.,0.,0.,0.],[0.,1.,0.,1.],[1.,1.,0.,2.],[0.,0.,0.,0.]] poly2 = [[0.,0.,0.,0.],[0.,1.,0.,1.],[1.,1.,0.,2.],[0.,0.,0.,0.]] polygons = [poly1, poly2] write_polygon_to_text(‘polygons.txt’, polygons)
- The resulting text file will look like this::
Polygon 0 0 0 0.000000 0.000000 0.000000 0.000000 1 0.000000 1.000000 0.000000 1.000000 2 1.000000 1.000000 0.000000 2.000000 3 0.000000 0.000000 0.000000 0.000000 1 0 0 0.000000 0.000000 0.000000 0.000000 1 0.000000 1.000000 0.000000 1.000000 2 1.000000 1.000000 0.000000 2.000000 3 0.000000 0.000000 0.000000 0.000000 END
- wradlib.io.misc.get_radiosonde(wmoid, date, *, cols=None)[source]#
Download radiosonde data from internet.
Based on http://weather.uwyo.edu/upperair/sounding.html.
- Parameters:
wmoid (
int
) – WMO radiosonde IDdate (
datetime.datetime
) – Date and Timecols (
tuple
, optional) – tuple of int or strings describing the columns to consider, defaults to None (all columns)
- Returns:
data (
numpy.ndarray
) – Structured array of radiosonde datameta (
dict
) – radiosonde metadata
- wradlib.io.misc.get_membership_functions(filename)[source]#
Reads membership function parameters from wradlib-data file.
- Parameters:
filename (
str
) – Filename of wradlib-data file- Returns:
msf (
numpy.ndarray
) – Array of membership funcions with shape (hm-classes, observables, indep-ranges, 5)
NetCDF Data I/O#
Former available xarray based code has been ported to xradar-package.
Data reader for netCDF files exported by the EDGE radar software |
|
Reads netcdf files and returns a dictionary with corresponding structure. |
- wradlib.io.netcdf.read_edge_netcdf(filename, *, enforce_equidist=False)[source]#
Data reader for netCDF files exported by the EDGE radar software
The corresponding NetCDF files from the EDGE software typically contain only one variable (e.g. reflectivity) for one elevation angle (sweep). The elevation angle is specified in the attributes keyword “Elevation”. Please note that the radar might not return data with equidistant azimuth angles. In case you need equidistant azimuth angles, please set enforce_equidist to True.
- wradlib.io.netcdf.read_generic_netcdf(fname)[source]#
Reads netcdf files and returns a dictionary with corresponding structure.
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the Group/Subgroup directory branches of the original file. Please see the examples below on how to browse through a return object. The most important keys are the “dimensions” which define the shape of the data arrays, and the “variables” which contain the actual data and typically also the data that define the dimensions (e.g. sweeps, azimuths, ranges). These keys should be present in any netcdf file.Note
The returned dictionary could be quite big, depending on the content of the file.
- Parameters:
fname (
str
or file-like) – a netcdf file path or file-like object- Returns:
out (
dict
) – an ordered dictionary that contains both data and metadata according to the original netcdf file structure
Examples
See Reading NetCDF.
RADOLAN and DX Data I/O#
Reading DX and RADOLAN data from German Weather Service
Warning
Additionally to the binary composite formats DWD also provides data in ASCII format, which have a very limited header and need to extract product and datetime from the filename. Use on your own risk.
Open and decode a RADOLAN dataset from a file or file-like object. |
|
Open multiple RADOLAN files as a single dataset. |
|
Data reader for German Weather Service DX product raw radar data files. |
|
Read quantitative radar composite format of the German Weather Service |
|
Opens radolan file and returns file handle |
|
Reads radolan ASCII header and returns it as string |
|
Parses the ASCII header of a DWD quantitative composite file |
|
Read binary data from file given by filehandle |
|
Decodes the binary runlength coded section from DWD composite file and return decoded numpy array with correct shape |
- wradlib.io.radolan.open_radolan_dataset(filename_or_obj, **kwargs)[source]#
Open and decode a RADOLAN dataset from a file or file-like object.
- Parameters:
filename_or_obj (
str
,Path
, file-like orDataStore
) – Strings and Path objects are interpreted as a path to a local or remote file.- Keyword Arguments:
fillmissing (
bool
) – Fill truncated data, defaults to False.copy (
bool
) – Create copies instead of views into the data, defaults to False.ancillary (
bool
,tuple
ofstr
) – If True, resturns ancillary masks (“secondary”, “nodatamask”, “cluttermask”) as additional data variables. Can be specified as tuple of strings. Defaults to False.**kwargs (
dict
, optional) – Additional arguments passed on toxarray.open_dataset
.
- Returns:
dataset (
xarray.Dataset
)
- wradlib.io.radolan.open_radolan_mfdataset(paths, **kwargs)[source]#
Open multiple RADOLAN files as a single dataset.
Needs dask-package to be installed.
- Parameters:
paths (
str
or sequence) – Either a string glob in the form"path/to/my/files/*"
or an explicit list of files to open. Paths can be given as strings or as pathlib Paths.- Keyword Arguments:
fillmissing (
bool
) – Fill truncated data, defaults to False.copy (
bool
) – Create copies instead of views into the data, defaults to False.ancillary (
bool
,tuple
ofstr
) – If True, resturns ancillary masks (“secondary”, “nodatamask”, “cluttermask”) as additional data variables. Can be specified as tuple of strings. Defaults to False.**kwargs (
dict
, optional) – Additional arguments passed on toxarray.open_mfdataset
.
- Returns:
dataset (
xarray.Dataset
)
- wradlib.io.radolan.read_dx(filename)[source]#
Data reader for German Weather Service DX product raw radar data files.
This product uses a simple algorithm to compress zero values to reduce data file size.
Notes
While the format appears to be well-defined, there have been reports on DX- files that seem to produce errors. e.g. while one file usually contains a 360 degree by 128 1km range bins, there are files, that contain 361 beams. Also, while usually azimuths are stored monotonously in ascending order, this is not guaranteed by the format. This routine does not (yet) check for this and directly returns the data in the order found in the file. If you are in doubt, check the ‘azim’ attribute. Be aware that this function does no extensive checking on its output. If e.g. beams contain different numbers of range bins, the resulting data will not be a 2-D array but a 1-D array of objects, which will most probably break calling code. It was decided to leave the handling of these (hopefully) rare events to the user, who might still be able to retrieve some reasonable data, instead of raising an exception, making it impossible to get any data from a file containing errors.
- Parameters:
filename (
str
or file-like) – filename of binary file of DX raw data or file-like object- Returns:
data (
numpy.ndarray
) – Array of image data [dBZ]; shape (360,128)attributes (
dict
) – dictionary of attributes - currently implemented keys:’azim’ - azimuths array of shape (360, )
’elev’ - elevations (1 per azimuth); array of shape (360, )
’clutter’ - clutter mask; boolean array of same shape as data; corresponds to bit 15 set in each dataset.
’bytes’- the total product length (including header). Apparently, this value may be off by one byte for unknown reasons
’version’- a product version string - use unknown
’cluttermap’ - number of the (DWD internal) cluttermap used
’dopplerfilter’ - number of the dopplerfilter used (DWD internal)
’statfilter’ - number of a statistical filter used (DWD internal)
’elevprofile’ - as stated in the format description, this list indicates the elevations in the eight 45 degree sectors. These sectors need not start at 0 degrees north, so it is advised to explicitly evaluate the elev attribute, if elevation information is needed.
’message’ - additional text stored in the header.
Examples
- wradlib.io.radolan.read_radolan_composite(f, *, missing=-9999, loaddata=True, fillmissing=False)[source]#
Read quantitative radar composite format of the German Weather Service
The quantitative composite format of the DWD (German Weather Service) was established in the course of the RADOLAN project and includes several file types, e.g. RX, RO, RK, RZ, RP, RT, RC, RI, RG, PC, PG and many, many more. (see format description on the RADOLAN project homepage [DWD, 2009]). At the moment, the national RADOLAN composite is a 900 x 900 grid with 1 km resolution and in polar-stereographic projection. There are other grid resolutions for different composites (e.g. PC, PG)
Note
DWD also provides data in ASCII format, which have a very limited header and need to extract product and datetime from the filename. Use on your own risk.
Added in version 1.17.
Warning
This function already evaluates and applies the so-called PR factor which is specified in the header section of the RADOLAN files. The raw values in an RY file are in the unit 0.01 mm/5min, while read_radolan_composite returns values in mm/5min (e.g. factor 100 higher). The factor is also returned as part of attrs dictionary under keyword “precision”.
Note
You might also use
wradlib.io.radolan.open_radolan_dataset
orxarray.open_dataset
with keyword engine=’radolan’ to import into xarray Dataset.- Parameters:
f (
str
or file-like) – path to the composite file or file-like objectmissing (
int
) – value assigned to no-data cellsloaddata (
bool
) – True | False, If False function returns (None, attrs)fillmissing (
bool
) – If True fills truncated values with “missing”. Defaults to False. Does not work for run-length encoded files (“PC” and “PG).
- Returns:
output (
tuple
) –- tuple of two items (data, attrs):
data :
numpy.ndarray
of shape (number of rows, number of columns) orxarray.Dataset
attrs : dict of metadata information from the file header
Examples
See RADOLAN data formats.
- wradlib.io.radolan.get_radolan_filehandle(fname)[source]#
Opens radolan file and returns file handle
- wradlib.io.radolan.read_radolan_header(fid)[source]#
Reads radolan ASCII header and returns it as string
- wradlib.io.radolan.parse_dwd_composite_header(header)[source]#
Parses the ASCII header of a DWD quantitative composite file
- wradlib.io.radolan.read_radolan_binary_array(fid, size, *, raise_on_error=True)[source]#
Read binary data from file given by filehandle
- wradlib.io.radolan.decode_radolan_runlength_array(binarr, attrs)[source]#
Decodes the binary runlength coded section from DWD composite file and return decoded numpy array with correct shape
- Parameters:
- Returns:
arr (
numpy.ndarray
) – Array of decoded values
Rainbow Data I/O#
Reads data from Leonardo’s Rainbow5 data formats. Former available code was ported to xradar-package and is imported from there.
read_rainbow
reads all data and metadata into a dictionary.
Reading sweep data can be skipped by setting loaddata=False.
Reads Rainbow files according to their structure |
- wradlib.io.rainbow.read_rainbow(filename, *, loaddata=True)[source]#
Reads Rainbow files according to their structure
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the XML outline in the original data file. The radar data will be extracted from the data blobs, converted and added to the dict with key ‘data’ at the place where the @blobid was pointing from.- Parameters:
- Returns:
rbdict (
dict
) – a dictionary that contains both data and metadata according to the original rainbow file structure
Examples
See Leonardo Rainbow.
Xarray based Data I/O#
Note
The Xarray backend code has been moved to xradar-package.
Minimal variable wrapper. |
- class wradlib.io.IrisCartesianProductFile(irisfile, **kwargs)[source]#
Bases:
IrisRecordFile
Class for retrieving data from Sigmet IRIS Cartesian Product files.
- product_identifier = ['MAX', 'TOPS', 'HMAX', 'BASE', 'THICK', 'PPI', 'RHI', 'CAPPI', 'RAINN', 'RAIN1', 'CROSS', 'SHEAR', 'SRI', 'RTI', 'VIL', 'LAYER', 'BEAM', 'MLHGT']#
- property data#
- get_image(header)[source]#
Retrieve cartesian image.
- Parameters:
header (
dict
) – header dictionary- Returns:
data (
numpy.ndarray
) – 3D array of cartesian data
- decode_data(data, prod)[source]#
Decode data according given prod-dict.
- Parameters:
data (
numpy.ndarray
) – data to decodeprod (
dict
) – dictionary holding decoding information
- Returns:
data (
numpy.ndarray
) – decoded data
- class wradlib.io.IrisProductFile(filename, **kwargs)[source]#
Bases:
IrisRecordFile
Class for retrieving data from Sigmet IRIS Product files.
- product_identifier = ['CATCH', 'FCAST', 'NDOP', 'SLINE', 'TDWR', 'TRACK', 'VAD', 'VVP', 'WARN', 'WIND', 'STATUS']#
- property data#
- property protect_setup#
- class wradlib.io.RadolanBackendEntrypoint[source]#
Bases:
BackendEntrypoint
Xarray BackendEntrypoint for RADOLAN data.
- url: ClassVar[str] = 'https://docs.wradlib.org/en/stable/notebooks/fileio/backends/radolan_backend.html'#
- open_dataset(filename_or_obj, *, mask_and_scale=True, decode_times=True, concat_characters=True, decode_coords=True, drop_variables=None, use_cftime=None, decode_timedelta=None, fillmissing=False, copy=False, ancillary=False)[source]#
Backend open_dataset method used by Xarray in
open_dataset
.
- class wradlib.io.VectorSource(data=None, trg_crs=None, name='layer', source=0, **kwargs)[source]#
Bases:
object
DataSource class for handling ogr/gdal vector data
DataSource handles creates in-memory (vector) ogr DataSource object with one layer for point or polygon geometries.
- Parameters:
data (sequence or
str
) – sequence of source points (shape Nx2) or polygons (shape NxMx2) or Vector File (GDAL/OGR) filename containing source points/polygonstrg_crs (
osgeo.osr.SpatialReference
) – GDAL OSR SRS describing target CRS the source data should be projected to
- Keyword Arguments:
name (
str
) – Layer Name, defaults to “layer”.source (
int
) – Number of layer to load, if multiple layers in source shape file.mode (
str
) – Return type of class access functions/properties. Can be either of “numpy”, “geo” and “ogr”, defaults to “numpy”.src_crs (
osgeo.osr.SpatialReference
) – GDAL OGR SRS describing projection source in which data is provided in.
Warning
Writing shapefiles with the wrong locale settings can have impact on the type of the decimal. If problem arise use
LC_NUMERIC=C
in your environment.Examples
See Vector Source.
- property mode#
- property ds#
Returns VectorSource
- property extent#
- property crs#
- property data#
Returns VectorSource geometries as numpy arrays
Note
This may be slow, because it extracts all source polygons
- property geo#
Returns VectorSource geometries as GeoPandas Dataframe
- get_data_by_att(attr=None, value=None, mode=None)[source]#
Returns DataSource geometries filtered by given attribute/value
- get_data_by_geom(geom=None, mode=None)[source]#
Returns DataSource geometries filtered by given geometry
- Keyword Arguments:
geom (
osgeo.ogr.Geometry
|geopandas.GeoDataFrame
) – OGR.Geometry object or geopandas.GeoDataFrame containing the Geometrymode (
str
) – return type (“numpy”, “geo”, “ogr”), defaults to “numpy”
- dump_vector(filename, *, driver='ESRI Shapefile', remove=True)[source]#
Output layer to OGR Vector File
- load_vector(filename, *, source=0, driver='ESRI Shapefile')[source]#
Read Layer from OGR Vector File
- dump_raster(filename, *, driver='GTiff', attr=None, pixel_size=1.0, **kwargs)[source]#
Output layer to GDAL Rasterfile
- Parameters:
- Keyword Arguments:
- set_attribute(name, values, *, reset_filter=False)[source]#
Add/Set given Attribute with given values
- Parameters:
name (
str
) – Attribute Namevalues (
numpy.ndarray
) – Values to fill in attributes.reset_filter (
bool
, optional) – reset any layer filter (spatial/attribute), defaults to False.
- class wradlib.io.WradlibVariable(dims, data, attrs)[source]#
Bases:
object
Minimal variable wrapper.
- property dimensions#
- property data#
- property attributes#
- wradlib.io.decode_radolan_runlength_array(binarr, attrs)[source]#
Decodes the binary runlength coded section from DWD composite file and return decoded numpy array with correct shape
- Parameters:
- Returns:
arr (
numpy.ndarray
) – Array of decoded values
- wradlib.io.download_srtm(filename, destination, *, resolution=3, session=None)[source]#
Download NASA SRTM elevation data Only available with login/password
- wradlib.io.from_hdf5(fpath, *, dataset='data')[source]#
Loading data from hdf5 files that was stored by
to_hdf5
- wradlib.io.gdal_create_dataset(drv, name, cols=0, rows=0, bands=0, *, gdal_type=None, remove=False)[source]#
Creates GDAL.DataSet object.
- Parameters:
drv (
str
) – GDAL driver stringname (
str
) – path to filenamecols (
int
) – number of columnsrows (
int
) – number of rowsbands (
int
) – number of raster bandsgdal_type (
gdal:osgeo.ogr.DataType
) – raster data type, e.g. gdal.GDT_Float32remove (
bool
) – if True, existing gdal.Dataset will be removed before creation
- Returns:
out (
osgeo.gdal.Dataset
) – gdal.Dataset
- wradlib.io.get_membership_functions(filename)[source]#
Reads membership function parameters from wradlib-data file.
- Parameters:
filename (
str
) – Filename of wradlib-data file- Returns:
msf (
numpy.ndarray
) – Array of membership funcions with shape (hm-classes, observables, indep-ranges, 5)
- wradlib.io.get_radiosonde(wmoid, date, *, cols=None)[source]#
Download radiosonde data from internet.
Based on http://weather.uwyo.edu/upperair/sounding.html.
- Parameters:
wmoid (
int
) – WMO radiosonde IDdate (
datetime.datetime
) – Date and Timecols (
tuple
, optional) – tuple of int or strings describing the columns to consider, defaults to None (all columns)
- Returns:
data (
numpy.ndarray
) – Structured array of radiosonde datameta (
dict
) – radiosonde metadata
- wradlib.io.get_srtm(extent, *, resolution=3, merge=True, session=None)[source]#
Get NASA SRTM elevation data
- Parameters:
- Returns:
dataset (
osgeo.gdal.Dataset
) – gdal.Dataset Raster dataset containing elevation information
- wradlib.io.open_gpm_dataset(filename, group)[source]#
Reads GPM files version V07A.
- Parameters:
- Returns:
ds (
xarray.Dataset
) – xarray.Dataset representation of GPM file with requested group.
- wradlib.io.open_radolan_dataset(filename_or_obj, **kwargs)[source]#
Open and decode a RADOLAN dataset from a file or file-like object.
- Parameters:
filename_or_obj (
str
,Path
, file-like orDataStore
) – Strings and Path objects are interpreted as a path to a local or remote file.- Keyword Arguments:
fillmissing (
bool
) – Fill truncated data, defaults to False.copy (
bool
) – Create copies instead of views into the data, defaults to False.ancillary (
bool
,tuple
ofstr
) – If True, resturns ancillary masks (“secondary”, “nodatamask”, “cluttermask”) as additional data variables. Can be specified as tuple of strings. Defaults to False.**kwargs (
dict
, optional) – Additional arguments passed on toxarray.open_dataset
.
- Returns:
dataset (
xarray.Dataset
)
- wradlib.io.open_radolan_mfdataset(paths, **kwargs)[source]#
Open multiple RADOLAN files as a single dataset.
Needs dask-package to be installed.
- Parameters:
paths (
str
or sequence) – Either a string glob in the form"path/to/my/files/*"
or an explicit list of files to open. Paths can be given as strings or as pathlib Paths.- Keyword Arguments:
fillmissing (
bool
) – Fill truncated data, defaults to False.copy (
bool
) – Create copies instead of views into the data, defaults to False.ancillary (
bool
,tuple
ofstr
) – If True, resturns ancillary masks (“secondary”, “nodatamask”, “cluttermask”) as additional data variables. Can be specified as tuple of strings. Defaults to False.**kwargs (
dict
, optional) – Additional arguments passed on toxarray.open_mfdataset
.
- Returns:
dataset (
xarray.Dataset
)
- wradlib.io.open_raster(filename, *, driver=None)[source]#
Open raster file, return gdal.Dataset
- Parameters:
- Returns:
dataset (
osgeo.gdal.Dataset
) – gdal.Dataset
- wradlib.io.open_vector(filename, *, driver=None, layer=0)[source]#
Open vector file, return gdal.Dataset and OGR.Layer
Warning
dataset and layer have to live in the same context, if dataset is deleted all layer references will get lost
- Parameters:
- Returns:
dataset (
osgeo.gdal.Dataset
) – gdal.Datasetlayer (
osgeo.ogr.Layer
) – ogr.Layer
- wradlib.io.parse_dwd_composite_header(header)[source]#
Parses the ASCII header of a DWD quantitative composite file
- wradlib.io.read_dx(filename)[source]#
Data reader for German Weather Service DX product raw radar data files.
This product uses a simple algorithm to compress zero values to reduce data file size.
Notes
While the format appears to be well-defined, there have been reports on DX- files that seem to produce errors. e.g. while one file usually contains a 360 degree by 128 1km range bins, there are files, that contain 361 beams. Also, while usually azimuths are stored monotonously in ascending order, this is not guaranteed by the format. This routine does not (yet) check for this and directly returns the data in the order found in the file. If you are in doubt, check the ‘azim’ attribute. Be aware that this function does no extensive checking on its output. If e.g. beams contain different numbers of range bins, the resulting data will not be a 2-D array but a 1-D array of objects, which will most probably break calling code. It was decided to leave the handling of these (hopefully) rare events to the user, who might still be able to retrieve some reasonable data, instead of raising an exception, making it impossible to get any data from a file containing errors.
- Parameters:
filename (
str
or file-like) – filename of binary file of DX raw data or file-like object- Returns:
data (
numpy.ndarray
) – Array of image data [dBZ]; shape (360,128)attributes (
dict
) – dictionary of attributes - currently implemented keys:’azim’ - azimuths array of shape (360, )
’elev’ - elevations (1 per azimuth); array of shape (360, )
’clutter’ - clutter mask; boolean array of same shape as data; corresponds to bit 15 set in each dataset.
’bytes’- the total product length (including header). Apparently, this value may be off by one byte for unknown reasons
’version’- a product version string - use unknown
’cluttermap’ - number of the (DWD internal) cluttermap used
’dopplerfilter’ - number of the dopplerfilter used (DWD internal)
’statfilter’ - number of a statistical filter used (DWD internal)
’elevprofile’ - as stated in the format description, this list indicates the elevations in the eight 45 degree sectors. These sectors need not start at 0 degrees north, so it is advised to explicitly evaluate the elev attribute, if elevation information is needed.
’message’ - additional text stored in the header.
Examples
- wradlib.io.read_edge_netcdf(filename, *, enforce_equidist=False)[source]#
Data reader for netCDF files exported by the EDGE radar software
The corresponding NetCDF files from the EDGE software typically contain only one variable (e.g. reflectivity) for one elevation angle (sweep). The elevation angle is specified in the attributes keyword “Elevation”. Please note that the radar might not return data with equidistant azimuth angles. In case you need equidistant azimuth angles, please set enforce_equidist to True.
- wradlib.io.read_gamic_hdf5(filename, *, wanted_elevations=None, wanted_moments=None)[source]#
Data reader for hdf5 files produced by the commercial GAMIC Enigma V3 MURAN software
See GAMIC homepage for further info (https://www.gamic.com).
- Parameters:
- Returns:
Examples
See GAMIC HDF5.
- wradlib.io.read_generic_hdf5(fname)[source]#
Reads hdf5 files according to their structure
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the Group/Subgroup directory branches of the original file.- Parameters:
fname (
str
or file-like) – a hdf5 file path or file-like object- Returns:
output (
dict
) – a dictionary that contains both data and metadata according to the original hdf5 file structure
Examples
- wradlib.io.read_generic_netcdf(fname)[source]#
Reads netcdf files and returns a dictionary with corresponding structure.
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the Group/Subgroup directory branches of the original file. Please see the examples below on how to browse through a return object. The most important keys are the “dimensions” which define the shape of the data arrays, and the “variables” which contain the actual data and typically also the data that define the dimensions (e.g. sweeps, azimuths, ranges). These keys should be present in any netcdf file.Note
The returned dictionary could be quite big, depending on the content of the file.
- Parameters:
fname (
str
or file-like) – a netcdf file path or file-like object- Returns:
out (
dict
) – an ordered dictionary that contains both data and metadata according to the original netcdf file structure
Examples
See Reading NetCDF.
- wradlib.io.read_gpm(filename, *, bbox=None)[source]#
Reads GPM files for matching with GR
- Parameters:
- Returns:
gpm_data (
dict
) – dictionary of gpm data
Examples
- wradlib.io.read_iris(filename, *, loaddata=True, rawdata=False, debug=False, **kwargs)[source]#
Read Iris file and return dictionary.
- Parameters:
filename (
str
or file-like) – Filename of data file or file-like object.loaddata (
bool
ordict
) – If true, retrieves whole data section from file. If false, retrievs only ingest_data_headers, but no data. If dict, retrieves according to given dict:loaddata = {'moment': ['DB_DBZ', 'DB_VEL'], 'sweep': [1, 3, 9]}
rawdata (
bool
) – If true, returns raw unconverted/undecoded data.debug (
bool
) – If true, print debug messages.
- Returns:
data (
dict
) – Ordered Dictionary with data and metadata retrieved from file.
Examples
See Vaisala Sigmet IRIS.
- wradlib.io.read_opera_hdf5(fname)[source]#
Reads hdf5 files according to OPERA conventions
Please refer to the OPERA data model documentation [B1] in order to understand how an hdf5 file is organized that conforms to the OPERA ODIM_H5 conventions.
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the Group/Subgroup directory branches of the original file. If the end member of a branch (or path) is “data”, then the corresponding item of output dictionary is a numpy array with actual data.Any other end member (either how, where, and what) will contain the meta information applying to the corresponding level of the file hierarchy.
- wradlib.io.read_radolan_binary_array(fid, size, *, raise_on_error=True)[source]#
Read binary data from file given by filehandle
- wradlib.io.read_radolan_composite(f, *, missing=-9999, loaddata=True, fillmissing=False)[source]#
Read quantitative radar composite format of the German Weather Service
The quantitative composite format of the DWD (German Weather Service) was established in the course of the RADOLAN project and includes several file types, e.g. RX, RO, RK, RZ, RP, RT, RC, RI, RG, PC, PG and many, many more. (see format description on the RADOLAN project homepage [DWD, 2009]). At the moment, the national RADOLAN composite is a 900 x 900 grid with 1 km resolution and in polar-stereographic projection. There are other grid resolutions for different composites (e.g. PC, PG)
Note
DWD also provides data in ASCII format, which have a very limited header and need to extract product and datetime from the filename. Use on your own risk.
Added in version 1.17.
Warning
This function already evaluates and applies the so-called PR factor which is specified in the header section of the RADOLAN files. The raw values in an RY file are in the unit 0.01 mm/5min, while read_radolan_composite returns values in mm/5min (e.g. factor 100 higher). The factor is also returned as part of attrs dictionary under keyword “precision”.
Note
You might also use
wradlib.io.radolan.open_radolan_dataset
orxarray.open_dataset
with keyword engine=’radolan’ to import into xarray Dataset.- Parameters:
f (
str
or file-like) – path to the composite file or file-like objectmissing (
int
) – value assigned to no-data cellsloaddata (
bool
) – True | False, If False function returns (None, attrs)fillmissing (
bool
) – If True fills truncated values with “missing”. Defaults to False. Does not work for run-length encoded files (“PC” and “PG).
- Returns:
output (
tuple
) –- tuple of two items (data, attrs):
data :
numpy.ndarray
of shape (number of rows, number of columns) orxarray.Dataset
attrs : dict of metadata information from the file header
Examples
See RADOLAN data formats.
- wradlib.io.read_rainbow(filename, *, loaddata=True)[source]#
Reads Rainbow files according to their structure
In contrast to other file readers under
wradlib.io
, this function will not return a two item tuple with (data, metadata). Instead, this function returns ONE dictionary that contains all the file contents - both data and metadata. The keys of the output dictionary conform to the XML outline in the original data file. The radar data will be extracted from the data blobs, converted and added to the dict with key ‘data’ at the place where the @blobid was pointing from.- Parameters:
- Returns:
rbdict (
dict
) – a dictionary that contains both data and metadata according to the original rainbow file structure
Examples
See Leonardo Rainbow.
- wradlib.io.read_safnwc(filename)[source]#
Read MSG SAFNWC hdf5 file into a gdal georeferenced object
- Parameters:
filename (
str
) – satellite file name- Returns:
ds (
osgeo.gdal.Dataset
) – gdal.DataSet with satellite data
- wradlib.io.read_trmm(filename1, filename2, *, bbox=None)[source]#
Reads TRMM files for matching with GR
- Parameters:
- Returns:
trmm_data (
dict
) – dictionary of trmm data
Examples
- wradlib.io.to_hdf5(fpath, data, *, mode='w', metadata=None, dataset='data', compression='gzip')[source]#
Quick storage of one <data> array and a <metadata> dict in an hdf5 file
This is more efficient than pickle, cPickle or numpy.save. The data is stored in a subgroup named data (i.e. file[“data”]). See
from_hdf5
for retrieving stored data.- Parameters:
fpath (
str
or file-like) – path to the hdf5 file or file-like objectdata (
numpy.ndarray
)mode (
str
) – file open mode, defaults to “w” (create, truncate if exists)metadata (
dict
) – dictionary of data’s attributesdataset (
str
) – describing datasetcompression (
str
) – h5py-compression type {“gzip”|”szip”|”lzf”}, see h5py-documentation for details
- wradlib.io.write_polygon_to_text(fname, polygons)[source]#
Writes Polygons to a Text file which can be interpreted by ESRI ArcGIS’s “Create Features from Text File (Samples)” tool.
This is (yet) only a convenience function with limited functionality. E.g. interior rings are not yet supported.
- Parameters:
- Returns:
Note
As Polygons are closed shapes, the first and the last vertex of each polygon must be the same!
Examples
- Writes two triangle Polygons to a text file::
poly1 = [[0.,0.,0.,0.],[0.,1.,0.,1.],[1.,1.,0.,2.],[0.,0.,0.,0.]] poly2 = [[0.,0.,0.,0.],[0.,1.,0.,1.],[1.,1.,0.,2.],[0.,0.,0.,0.]] polygons = [poly1, poly2] write_polygon_to_text(‘polygons.txt’, polygons)
- The resulting text file will look like this::
Polygon 0 0 0 0.000000 0.000000 0.000000 0.000000 1 0.000000 1.000000 0.000000 1.000000 2 1.000000 1.000000 0.000000 2.000000 3 0.000000 0.000000 0.000000 0.000000 1 0 0 0.000000 0.000000 0.000000 0.000000 1 0.000000 1.000000 0.000000 1.000000 2 1.000000 1.000000 0.000000 2.000000 3 0.000000 0.000000 0.000000 0.000000 END
- wradlib.io.write_raster_dataset(fpath, dataset, *, driver='GTiff', **kwargs)[source]#
Write raster dataset to file format
- Parameters:
fpath (
str
) – A file path - should have file extension corresponding to format.dataset (
osgeo.gdal.Dataset
) – gdal.Dataset gdal raster datasetdriver (
str
, optional) – gdal raster format driver string, defaults to “GTiff”
- Keyword Arguments:
Note
For format and options refer to formats_list.
Examples