Load ODIM_H5 Volume data from German Weather Service¶
In this example, we obtain and read the latest 30 minutes of available volumetric radar data from German Weather Service available at opendata.dwd.de. Finally we do some plotting.
This retrieves 6 timesteps of the 10 sweeps (moments DBZH and VRADH) of the DWD volume scan of a distinct radar. This amounts to 120 data files which are combined into one volumetric Cf/Radial2 like xarray powered structure.
Exports to single file Odim_H5 and Cf/Radial2 format are shown at the end of this tutorial.
Note
The used open_odim_mfdataset
implementation is based on xarray. It claims multiple data files and presents them in a simple structure. See also the notebook wradlib_odim_backend for further details.
[1]:
import wradlib as wrl
import warnings
warnings.filterwarnings('ignore')
import matplotlib.pyplot as pl
import numpy as np
import xarray as xr
try:
get_ipython().magic("matplotlib inline")
except:
pl.ion()
from wradlib.io import open_odim_mfdataset
[2]:
import urllib3
import os
import io
import glob
import shutil
import datetime
Download radar volumes of latest 30 minutes from server using wetterdienst
¶
wetterdienst
is a neat package for easy retrieval of data primarily from DWD. For further information have a look at their documentation.
[3]:
from wetterdienst.provider.dwd.radar import DwdRadarDataFormat, DwdRadarDataSubset, DwdRadarParameter, DwdRadarValues
from wetterdienst.provider.dwd.radar.sites import DwdRadarSite
[4]:
elevations = range(10)
end_date = datetime.datetime.utcnow()
start_date = end_date - datetime.timedelta(minutes=30)
results_velocity = []
results_reflectivity = []
for el in elevations:
# Horizontal Doppler Velocity
request_velocity = DwdRadarValues(
parameter=DwdRadarParameter.SWEEP_VOL_VELOCITY_H,
start_date=start_date,
end_date=end_date,
site=DwdRadarSite.ESS,
elevation=el,
fmt=DwdRadarDataFormat.HDF5,
subset=DwdRadarDataSubset.POLARIMETRIC,
)
# Horizontal Reflectivity
request_reflectivity = DwdRadarValues(
parameter=DwdRadarParameter.SWEEP_VOL_REFLECTIVITY_H,
start_date=start_date,
end_date=end_date,
elevation=el,
site=DwdRadarSite.ESS,fmt=DwdRadarDataFormat.HDF5,
subset=DwdRadarDataSubset.POLARIMETRIC,
)
# Submit requests.
results_velocity.append(request_velocity.query())
results_reflectivity.append(request_reflectivity.query())
Acquire data as memory buffer¶
[5]:
%%time
volume_velocity = []
for item1 in results_velocity:
files = []
for item2 in item1:
files.append(item2.data)
volume_velocity.append(files)
3%|▎ | 7/240 [00:04<02:19, 1.67it/s]
3%|▎ | 7/240 [00:02<01:10, 3.29it/s]
3%|▎ | 7/240 [00:01<01:05, 3.56it/s]
3%|▎ | 7/240 [00:02<01:08, 3.41it/s]
3%|▎ | 7/240 [00:02<01:08, 3.39it/s]
2%|▎ | 6/240 [00:01<01:10, 3.31it/s]
2%|▎ | 6/240 [00:01<01:09, 3.39it/s]
2%|▎ | 6/240 [00:01<01:11, 3.26it/s]
2%|▎ | 6/240 [00:01<01:07, 3.47it/s]
2%|▎ | 6/240 [00:01<01:01, 3.80it/s]
CPU times: user 5.39 s, sys: 104 ms, total: 5.49 s
Wall time: 21.2 s
[6]:
%%time
volume_reflectivity = []
for item1 in results_reflectivity:
files = []
for item2 in item1:
files.append(item2.data)
volume_reflectivity.append(files)
3%|▎ | 7/240 [00:02<01:10, 3.32it/s]
3%|▎ | 7/240 [00:01<01:05, 3.53it/s]
3%|▎ | 7/240 [00:02<01:11, 3.28it/s]
3%|▎ | 7/240 [00:01<01:05, 3.56it/s]
3%|▎ | 7/240 [00:02<01:09, 3.33it/s]
2%|▎ | 6/240 [00:02<01:49, 2.13it/s]
2%|▎ | 6/240 [00:01<01:09, 3.35it/s]
2%|▎ | 6/240 [00:01<01:02, 3.75it/s]
2%|▎ | 6/240 [00:01<01:07, 3.48it/s]
2%|▎ | 6/240 [00:01<01:01, 3.80it/s]
CPU times: user 5.65 s, sys: 77.5 ms, total: 5.73 s
Wall time: 19.8 s
Read the data into xarray powered structure¶
[7]:
vol = wrl.io.RadarVolume()
for r, v in zip(volume_reflectivity, volume_velocity):
ds0 = wrl.io.open_odim_mfdataset(r, group="dataset1",
concat_dim="time",
combine="nested",
)
ds1 = wrl.io.open_odim_mfdataset(v, group="dataset1",
concat_dim="time",
combine="nested",
)
vol.append(xr.merge([ds0, ds1], combine_attrs="override"))
vol.sort(key=lambda x: x.time.min().values)
100%|██████████| 1/1 [00:00<00:00, 3.31it/s]
100%|██████████| 1/1 [00:00<00:00, 4.30it/s]
100%|██████████| 1/1 [00:00<00:00, 4.35it/s]
100%|██████████| 1/1 [00:00<00:00, 4.43it/s]
100%|██████████| 1/1 [00:00<00:00, 4.43it/s]
100%|██████████| 1/1 [00:00<00:00, 4.25it/s]
100%|██████████| 1/1 [00:00<00:00, 4.22it/s]
100%|██████████| 1/1 [00:00<00:00, 4.32it/s]
100%|██████████| 1/1 [00:00<00:00, 4.35it/s]
100%|██████████| 1/1 [00:00<00:00, 4.32it/s]
100%|██████████| 1/1 [00:00<00:00, 5.12it/s]
100%|██████████| 1/1 [00:00<00:00, 5.05it/s]
100%|██████████| 1/1 [00:00<00:00, 5.33it/s]
100%|██████████| 1/1 [00:00<00:00, 5.25it/s]
100%|██████████| 1/1 [00:00<00:00, 5.10it/s]
100%|██████████| 1/1 [00:00<00:00, 5.08it/s]
100%|██████████| 1/1 [00:00<00:00, 5.21it/s]
100%|██████████| 1/1 [00:00<00:00, 5.18it/s]
100%|██████████| 1/1 [00:00<00:00, 5.22it/s]
100%|██████████| 1/1 [00:00<00:00, 5.17it/s]
Inspect structure¶
Root Group¶
[8]:
vol.root
[8]:
<xarray.Dataset> Dimensions: (sweep: 10) Coordinates: sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 altitude float64 185.1 latitude float64 51.41 Dimensions without coordinates: sweep Data variables: volume_number int64 0 platform_type <U5 'fixed' instrument_type <U5 'radar' primary_axis <U6 'axis_z' time_coverage_start <U20 '2022-05-19T11:15:35Z' time_coverage_end <U20 '2022-05-19T11:47:30Z' sweep_group_name (sweep) <U7 'sweep_0' 'sweep_1' ... 'sweep_8' 'sweep_9' sweep_fixed_angle (sweep) float64 5.5 4.5 3.5 2.5 ... 8.0 12.0 17.0 25.0 Attributes: version: None title: None institution: None references: None source: None history: None comment: im/exported using wradlib instrument_name: None fixed_angle: 5.5
[9]:
vol.root.sweep_fixed_angle
[9]:
<xarray.DataArray 'sweep_fixed_angle' (sweep: 10)> array([ 5.5, 4.5, 3.5, 2.5, 1.5, 0.5, 8. , 12. , 17. , 25. ]) Coordinates: sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 altitude float64 185.1 latitude float64 51.41 Dimensions without coordinates: sweep
Sweep Groups¶
[10]:
vol
[10]:
<wradlib.RadarVolume>
Dimension(s): (sweep: 10)
Elevation(s): (5.5, 4.5, 3.5, 2.5, 1.5, 0.5, 8.0, 12.0, 17.0, 25.0)
[11]:
vol[0]
[11]:
<xarray.Dataset> Dimensions: (azimuth: 360, time: 7, range: 720) Coordinates: * azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5 elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray> rtime (time, azimuth) datetime64[ns] dask.array<chunksize=(1, 360), meta=np.ndarray> * range (range) float32 125.0 375.0 625.0 ... 1.796e+05 1.799e+05 * time (time) datetime64[ns] 2022-05-19T11:15:35 ... 2022-05-19T11:4... sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 latitude float64 51.41 altitude float64 185.1 Data variables: DBZH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray> VRADH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray> Attributes: fixed_angle: 5.5
plot sweeps¶
DBZH¶
[12]:
fig = pl.figure(figsize=(20, 30))
import matplotlib.gridspec as gridspec
gs = gridspec.GridSpec(4, 3, wspace=0.4, hspace=0.4)
for i, ts in enumerate(vol):
swp = ts.isel(time=0)
swp.DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot(ax=gs[i], fig=fig)
ax = pl.gca()
ax.set_title(vol.root.sweep_fixed_angle[i].values)
VRADH¶
[13]:
fig = pl.figure(figsize=(20, 30))
import matplotlib.gridspec as gridspec
gs = gridspec.GridSpec(4, 3, wspace=0.4, hspace=0.4)
for i, ts in enumerate(vol):
swp = ts.isel(time=0)
swp.VRADH.pipe(wrl.georef.georeference_dataset).wradlib.plot(ax=gs[i], fig=fig)
ax = pl.gca()
ax.set_title(vol.root.sweep_fixed_angle[i].values)
Plot single sweep using cartopy¶
[14]:
import cartopy
import cartopy.crs as ccrs
import cartopy.feature as cfeature
map_trans = ccrs.AzimuthalEquidistant(central_latitude=vol.root.latitude.values,
central_longitude=vol.root.longitude.values)
[15]:
vol[-1]
[15]:
<xarray.Dataset> Dimensions: (azimuth: 360, time: 6, range: 240) Coordinates: * azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5 elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray> rtime (time, azimuth) datetime64[ns] dask.array<chunksize=(1, 360), meta=np.ndarray> * range (range) float32 125.0 375.0 625.0 ... 5.962e+04 5.988e+04 * time (time) datetime64[ns] 2022-05-19T11:18:51 ... 2022-05-19T11:4... sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 latitude float64 51.41 altitude float64 185.1 Data variables: DBZH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray> VRADH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray> Attributes: fixed_angle: 25.0
[16]:
map_proj = ccrs.AzimuthalEquidistant(central_latitude=vol.root.latitude.values,
central_longitude=vol.root.longitude.values)
pm = vol[-1].isel(time=0).DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot_ppi(proj=map_proj)
ax = pl.gca()
ax.gridlines(crs=map_proj)
print(ax)
< GeoAxes: +proj=aeqd +ellps=WGS84 +lon_0=6.967111 +lat_0=51.405649 +x_0=0.0 +y_0=0.0 +no_defs +type=crs >
[17]:
map_proj = ccrs.Mercator(central_longitude=vol.root.longitude.values)
fig = pl.figure(figsize=(10,8))
ax = fig.add_subplot(111, projection=map_proj)
pm = vol[-1].isel(time=0).DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot_ppi(ax=ax)
ax.gridlines(draw_labels=True)
[17]:
<cartopy.mpl.gridliner.Gridliner at 0x7fa11f1a84c0>
[18]:
fig = pl.figure(figsize=(10, 8))
proj=ccrs.AzimuthalEquidistant(central_latitude=vol.root.latitude.values,
central_longitude=vol.root.longitude.values)
ax = fig.add_subplot(111, projection=proj)
pm = vol[-1].isel(time=0).DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot_ppi(ax=ax)
ax.gridlines()
[18]:
<cartopy.mpl.gridliner.Gridliner at 0x7fa11fcf8490>
Inspect radar moments¶
The DataArrays can be accessed by key or by attribute. Each DataArray inherits dimensions and coordinates of it’s parent dataset. There are attributes connected which are defined by Cf/Radial and/or ODIM_H5 standard.
[19]:
vol[-1].isel(time=0).DBZH
[19]:
<xarray.DataArray 'DBZH' (azimuth: 360, range: 240)> dask.array<getitem, shape=(360, 240), dtype=float32, chunksize=(360, 240), chunktype=numpy.ndarray> Coordinates: * azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5 elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray> rtime (azimuth) datetime64[ns] dask.array<chunksize=(360,), meta=np.ndarray> * range (range) float32 125.0 375.0 625.0 ... 5.962e+04 5.988e+04 time datetime64[ns] 2022-05-19T11:18:51 sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 latitude float64 51.41 altitude float64 185.1 Attributes: _Undetect: 0.0 long_name: Equivalent reflectivity factor H units: dBZ standard_name: radar_equivalent_reflectivity_factor_h
[20]:
vol[-1].isel(time=0).sweep_mode
[20]:
<xarray.DataArray 'sweep_mode' ()> array('azimuth_surveillance', dtype='<U20') Coordinates: time datetime64[ns] 2022-05-19T11:18:51 sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 latitude float64 51.41 altitude float64 185.1
[21]:
vol.root
[21]:
<xarray.Dataset> Dimensions: (sweep: 10) Coordinates: sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 altitude float64 185.1 latitude float64 51.41 Dimensions without coordinates: sweep Data variables: volume_number int64 0 platform_type <U5 'fixed' instrument_type <U5 'radar' primary_axis <U6 'axis_z' time_coverage_start <U20 '2022-05-19T11:15:35Z' time_coverage_end <U20 '2022-05-19T11:47:30Z' sweep_group_name (sweep) <U7 'sweep_0' 'sweep_1' ... 'sweep_8' 'sweep_9' sweep_fixed_angle (sweep) float64 5.5 4.5 3.5 2.5 ... 8.0 12.0 17.0 25.0 Attributes: version: None title: None institution: None references: None source: None history: None comment: im/exported using wradlib instrument_name: None fixed_angle: 5.5
Plot Quasi Vertical Profile¶
[22]:
vol
[22]:
<wradlib.RadarVolume>
Dimension(s): (sweep: 10)
Elevation(s): (5.5, 4.5, 3.5, 2.5, 1.5, 0.5, 8.0, 12.0, 17.0, 25.0)
[23]:
ts = vol[-1]
ts
[23]:
<xarray.Dataset> Dimensions: (azimuth: 360, time: 6, range: 240) Coordinates: * azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5 elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray> rtime (time, azimuth) datetime64[ns] dask.array<chunksize=(1, 360), meta=np.ndarray> * range (range) float32 125.0 375.0 625.0 ... 5.962e+04 5.988e+04 * time (time) datetime64[ns] 2022-05-19T11:18:51 ... 2022-05-19T11:4... sweep_mode <U20 'azimuth_surveillance' longitude float64 6.967 latitude float64 51.41 altitude float64 185.1 Data variables: DBZH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray> VRADH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray> Attributes: fixed_angle: 25.0
[24]:
fig = pl.figure(figsize=(10, 4))
ax = fig.add_subplot(111)
ts.DBZH.median('azimuth').plot(x='time', vmin=-10, vmax=30, ax=ax)
ax.set_title(f"{np.datetime_as_string(ts.time[0].values, unit='D')}")
ax.set_ylim(0, 20000)
[24]:
(0.0, 20000.0)
Export to OdimH5¶
This exports the radar volume at given timestep including all moments into one ODIM_H5 compliant data file.
[25]:
vol.to_odim('dwd_odim.h5', timestep=0)
Export to Cf/Radial2¶
This exports the radar volume at given timestep including all moments into one Cf/Radial2 compliant data file.
[26]:
vol.to_cfradial2('dwd_cfradial2.nc', timestep=0)
Import again and check equality¶
[27]:
vol1 = wrl.io.open_odim_dataset('dwd_odim.h5')
vol2 = wrl.io.open_cfradial2_dataset('dwd_cfradial2.nc')
[28]:
xr.testing.assert_equal(vol1.root, vol2.root)
for i in range(len(vol1)):
xr.testing.assert_equal(vol1[i].drop_vars("rtime"), vol2[i].drop_vars("rtime"))