Load ODIM_H5 Volume data from German Weather Service¶
In this example, we obtain and read the latest 30 minutes of available volumetric radar data from German Weather Service available at opendata.dwd.de. Finally we do some plotting.
This retrieves 6 timesteps of the 10 sweeps (moments DBZH and VRADH) of the DWD volume scan of a distinct radar. This amounts to 120 data files which are combined into one volumetric Cf/Radial2 like xarray powered structure.
Exports to single file Odim_H5 and Cf/Radial2 format are shown at the end of this tutorial.
Note
The used open_odim_mfdataset implementation is based on xarray. It claims multiple data files and presents them in a simple structure. See also the notebook wradlib_odim_backend for further details.
[1]:
import wradlib as wrl
import warnings
warnings.filterwarnings('ignore')
import matplotlib.pyplot as pl
import numpy as np
import xarray as xr
try:
get_ipython().magic("matplotlib inline")
except:
pl.ion()
from wradlib.io import open_odim_mfdataset
[2]:
import urllib3
import os
import io
import glob
import shutil
import datetime
Download radar volumes of latest 30 minutes from server using wetterdienst¶
wetterdienst is a neat package for easy retrieval of data primarily from DWD. For further information have a look at their documentation.
[3]:
from wetterdienst.provider.dwd.radar import DwdRadarDataFormat, DwdRadarDataSubset, DwdRadarParameter, DwdRadarValues
from wetterdienst.provider.dwd.radar.sites import DwdRadarSite
[4]:
elevations = range(10)
end_date = datetime.datetime.utcnow()
start_date = end_date - datetime.timedelta(minutes=30)
results_velocity = []
results_reflectivity = []
for el in elevations:
# Horizontal Doppler Velocity
request_velocity = DwdRadarValues(
parameter=DwdRadarParameter.SWEEP_VOL_VELOCITY_H,
start_date=start_date,
end_date=end_date,
site=DwdRadarSite.ESS,
elevation=el,
fmt=DwdRadarDataFormat.HDF5,
subset=DwdRadarDataSubset.POLARIMETRIC,
)
# Horizontal Reflectivity
request_reflectivity = DwdRadarValues(
parameter=DwdRadarParameter.SWEEP_VOL_REFLECTIVITY_H,
start_date=start_date,
end_date=end_date,
elevation=el,
site=DwdRadarSite.ESS,fmt=DwdRadarDataFormat.HDF5,
subset=DwdRadarDataSubset.POLARIMETRIC,
)
# Submit requests.
results_velocity.append(request_velocity.query())
results_reflectivity.append(request_reflectivity.query())
Acquire data as memory buffer¶
[5]:
%%time
volume_velocity = []
for item1 in results_velocity:
files = []
for item2 in item1:
files.append(item2.data)
volume_velocity.append(files)
3%|▎ | 7/240 [00:04<02:19, 1.67it/s]
3%|▎ | 7/240 [00:02<01:10, 3.29it/s]
3%|▎ | 7/240 [00:01<01:05, 3.56it/s]
3%|▎ | 7/240 [00:02<01:08, 3.41it/s]
3%|▎ | 7/240 [00:02<01:08, 3.39it/s]
2%|▎ | 6/240 [00:01<01:10, 3.31it/s]
2%|▎ | 6/240 [00:01<01:09, 3.39it/s]
2%|▎ | 6/240 [00:01<01:11, 3.26it/s]
2%|▎ | 6/240 [00:01<01:07, 3.47it/s]
2%|▎ | 6/240 [00:01<01:01, 3.80it/s]
CPU times: user 5.39 s, sys: 104 ms, total: 5.49 s
Wall time: 21.2 s
[6]:
%%time
volume_reflectivity = []
for item1 in results_reflectivity:
files = []
for item2 in item1:
files.append(item2.data)
volume_reflectivity.append(files)
3%|▎ | 7/240 [00:02<01:10, 3.32it/s]
3%|▎ | 7/240 [00:01<01:05, 3.53it/s]
3%|▎ | 7/240 [00:02<01:11, 3.28it/s]
3%|▎ | 7/240 [00:01<01:05, 3.56it/s]
3%|▎ | 7/240 [00:02<01:09, 3.33it/s]
2%|▎ | 6/240 [00:02<01:49, 2.13it/s]
2%|▎ | 6/240 [00:01<01:09, 3.35it/s]
2%|▎ | 6/240 [00:01<01:02, 3.75it/s]
2%|▎ | 6/240 [00:01<01:07, 3.48it/s]
2%|▎ | 6/240 [00:01<01:01, 3.80it/s]
CPU times: user 5.65 s, sys: 77.5 ms, total: 5.73 s
Wall time: 19.8 s
Read the data into xarray powered structure¶
[7]:
vol = wrl.io.RadarVolume()
for r, v in zip(volume_reflectivity, volume_velocity):
ds0 = wrl.io.open_odim_mfdataset(r, group="dataset1",
concat_dim="time",
combine="nested",
)
ds1 = wrl.io.open_odim_mfdataset(v, group="dataset1",
concat_dim="time",
combine="nested",
)
vol.append(xr.merge([ds0, ds1], combine_attrs="override"))
vol.sort(key=lambda x: x.time.min().values)
100%|██████████| 1/1 [00:00<00:00, 3.31it/s]
100%|██████████| 1/1 [00:00<00:00, 4.30it/s]
100%|██████████| 1/1 [00:00<00:00, 4.35it/s]
100%|██████████| 1/1 [00:00<00:00, 4.43it/s]
100%|██████████| 1/1 [00:00<00:00, 4.43it/s]
100%|██████████| 1/1 [00:00<00:00, 4.25it/s]
100%|██████████| 1/1 [00:00<00:00, 4.22it/s]
100%|██████████| 1/1 [00:00<00:00, 4.32it/s]
100%|██████████| 1/1 [00:00<00:00, 4.35it/s]
100%|██████████| 1/1 [00:00<00:00, 4.32it/s]
100%|██████████| 1/1 [00:00<00:00, 5.12it/s]
100%|██████████| 1/1 [00:00<00:00, 5.05it/s]
100%|██████████| 1/1 [00:00<00:00, 5.33it/s]
100%|██████████| 1/1 [00:00<00:00, 5.25it/s]
100%|██████████| 1/1 [00:00<00:00, 5.10it/s]
100%|██████████| 1/1 [00:00<00:00, 5.08it/s]
100%|██████████| 1/1 [00:00<00:00, 5.21it/s]
100%|██████████| 1/1 [00:00<00:00, 5.18it/s]
100%|██████████| 1/1 [00:00<00:00, 5.22it/s]
100%|██████████| 1/1 [00:00<00:00, 5.17it/s]
Inspect structure¶
Root Group¶
[8]:
vol.root
[8]:
<xarray.Dataset>
Dimensions: (sweep: 10)
Coordinates:
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
altitude float64 185.1
latitude float64 51.41
Dimensions without coordinates: sweep
Data variables:
volume_number int64 0
platform_type <U5 'fixed'
instrument_type <U5 'radar'
primary_axis <U6 'axis_z'
time_coverage_start <U20 '2022-05-19T11:15:35Z'
time_coverage_end <U20 '2022-05-19T11:47:30Z'
sweep_group_name (sweep) <U7 'sweep_0' 'sweep_1' ... 'sweep_8' 'sweep_9'
sweep_fixed_angle (sweep) float64 5.5 4.5 3.5 2.5 ... 8.0 12.0 17.0 25.0
Attributes:
version: None
title: None
institution: None
references: None
source: None
history: None
comment: im/exported using wradlib
instrument_name: None
fixed_angle: 5.5[9]:
vol.root.sweep_fixed_angle
[9]:
<xarray.DataArray 'sweep_fixed_angle' (sweep: 10)>
array([ 5.5, 4.5, 3.5, 2.5, 1.5, 0.5, 8. , 12. , 17. , 25. ])
Coordinates:
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
altitude float64 185.1
latitude float64 51.41
Dimensions without coordinates: sweepSweep Groups¶
[10]:
vol
[10]:
<wradlib.RadarVolume>
Dimension(s): (sweep: 10)
Elevation(s): (5.5, 4.5, 3.5, 2.5, 1.5, 0.5, 8.0, 12.0, 17.0, 25.0)
[11]:
vol[0]
[11]:
<xarray.Dataset>
Dimensions: (azimuth: 360, time: 7, range: 720)
Coordinates:
* azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5
elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray>
rtime (time, azimuth) datetime64[ns] dask.array<chunksize=(1, 360), meta=np.ndarray>
* range (range) float32 125.0 375.0 625.0 ... 1.796e+05 1.799e+05
* time (time) datetime64[ns] 2022-05-19T11:15:35 ... 2022-05-19T11:4...
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
latitude float64 51.41
altitude float64 185.1
Data variables:
DBZH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray>
VRADH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray>
Attributes:
fixed_angle: 5.5plot sweeps¶
DBZH¶
[12]:
fig = pl.figure(figsize=(20, 30))
import matplotlib.gridspec as gridspec
gs = gridspec.GridSpec(4, 3, wspace=0.4, hspace=0.4)
for i, ts in enumerate(vol):
swp = ts.isel(time=0)
swp.DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot(ax=gs[i], fig=fig)
ax = pl.gca()
ax.set_title(vol.root.sweep_fixed_angle[i].values)
VRADH¶
[13]:
fig = pl.figure(figsize=(20, 30))
import matplotlib.gridspec as gridspec
gs = gridspec.GridSpec(4, 3, wspace=0.4, hspace=0.4)
for i, ts in enumerate(vol):
swp = ts.isel(time=0)
swp.VRADH.pipe(wrl.georef.georeference_dataset).wradlib.plot(ax=gs[i], fig=fig)
ax = pl.gca()
ax.set_title(vol.root.sweep_fixed_angle[i].values)
Plot single sweep using cartopy¶
[14]:
import cartopy
import cartopy.crs as ccrs
import cartopy.feature as cfeature
map_trans = ccrs.AzimuthalEquidistant(central_latitude=vol.root.latitude.values,
central_longitude=vol.root.longitude.values)
[15]:
vol[-1]
[15]:
<xarray.Dataset>
Dimensions: (azimuth: 360, time: 6, range: 240)
Coordinates:
* azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5
elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray>
rtime (time, azimuth) datetime64[ns] dask.array<chunksize=(1, 360), meta=np.ndarray>
* range (range) float32 125.0 375.0 625.0 ... 5.962e+04 5.988e+04
* time (time) datetime64[ns] 2022-05-19T11:18:51 ... 2022-05-19T11:4...
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
latitude float64 51.41
altitude float64 185.1
Data variables:
DBZH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray>
VRADH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray>
Attributes:
fixed_angle: 25.0[16]:
map_proj = ccrs.AzimuthalEquidistant(central_latitude=vol.root.latitude.values,
central_longitude=vol.root.longitude.values)
pm = vol[-1].isel(time=0).DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot_ppi(proj=map_proj)
ax = pl.gca()
ax.gridlines(crs=map_proj)
print(ax)
< GeoAxes: +proj=aeqd +ellps=WGS84 +lon_0=6.967111 +lat_0=51.405649 +x_0=0.0 +y_0=0.0 +no_defs +type=crs >
[17]:
map_proj = ccrs.Mercator(central_longitude=vol.root.longitude.values)
fig = pl.figure(figsize=(10,8))
ax = fig.add_subplot(111, projection=map_proj)
pm = vol[-1].isel(time=0).DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot_ppi(ax=ax)
ax.gridlines(draw_labels=True)
[17]:
<cartopy.mpl.gridliner.Gridliner at 0x7fa11f1a84c0>
[18]:
fig = pl.figure(figsize=(10, 8))
proj=ccrs.AzimuthalEquidistant(central_latitude=vol.root.latitude.values,
central_longitude=vol.root.longitude.values)
ax = fig.add_subplot(111, projection=proj)
pm = vol[-1].isel(time=0).DBZH.pipe(wrl.georef.georeference_dataset).wradlib.plot_ppi(ax=ax)
ax.gridlines()
[18]:
<cartopy.mpl.gridliner.Gridliner at 0x7fa11fcf8490>
Inspect radar moments¶
The DataArrays can be accessed by key or by attribute. Each DataArray inherits dimensions and coordinates of it’s parent dataset. There are attributes connected which are defined by Cf/Radial and/or ODIM_H5 standard.
[19]:
vol[-1].isel(time=0).DBZH
[19]:
<xarray.DataArray 'DBZH' (azimuth: 360, range: 240)>
dask.array<getitem, shape=(360, 240), dtype=float32, chunksize=(360, 240), chunktype=numpy.ndarray>
Coordinates:
* azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5
elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray>
rtime (azimuth) datetime64[ns] dask.array<chunksize=(360,), meta=np.ndarray>
* range (range) float32 125.0 375.0 625.0 ... 5.962e+04 5.988e+04
time datetime64[ns] 2022-05-19T11:18:51
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
latitude float64 51.41
altitude float64 185.1
Attributes:
_Undetect: 0.0
long_name: Equivalent reflectivity factor H
units: dBZ
standard_name: radar_equivalent_reflectivity_factor_h[20]:
vol[-1].isel(time=0).sweep_mode
[20]:
<xarray.DataArray 'sweep_mode' ()>
array('azimuth_surveillance', dtype='<U20')
Coordinates:
time datetime64[ns] 2022-05-19T11:18:51
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
latitude float64 51.41
altitude float64 185.1[21]:
vol.root
[21]:
<xarray.Dataset>
Dimensions: (sweep: 10)
Coordinates:
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
altitude float64 185.1
latitude float64 51.41
Dimensions without coordinates: sweep
Data variables:
volume_number int64 0
platform_type <U5 'fixed'
instrument_type <U5 'radar'
primary_axis <U6 'axis_z'
time_coverage_start <U20 '2022-05-19T11:15:35Z'
time_coverage_end <U20 '2022-05-19T11:47:30Z'
sweep_group_name (sweep) <U7 'sweep_0' 'sweep_1' ... 'sweep_8' 'sweep_9'
sweep_fixed_angle (sweep) float64 5.5 4.5 3.5 2.5 ... 8.0 12.0 17.0 25.0
Attributes:
version: None
title: None
institution: None
references: None
source: None
history: None
comment: im/exported using wradlib
instrument_name: None
fixed_angle: 5.5Plot Quasi Vertical Profile¶
[22]:
vol
[22]:
<wradlib.RadarVolume>
Dimension(s): (sweep: 10)
Elevation(s): (5.5, 4.5, 3.5, 2.5, 1.5, 0.5, 8.0, 12.0, 17.0, 25.0)
[23]:
ts = vol[-1]
ts
[23]:
<xarray.Dataset>
Dimensions: (azimuth: 360, time: 6, range: 240)
Coordinates:
* azimuth (azimuth) float64 0.5 1.5 2.5 3.5 ... 356.5 357.5 358.5 359.5
elevation (azimuth) float64 dask.array<chunksize=(360,), meta=np.ndarray>
rtime (time, azimuth) datetime64[ns] dask.array<chunksize=(1, 360), meta=np.ndarray>
* range (range) float32 125.0 375.0 625.0 ... 5.962e+04 5.988e+04
* time (time) datetime64[ns] 2022-05-19T11:18:51 ... 2022-05-19T11:4...
sweep_mode <U20 'azimuth_surveillance'
longitude float64 6.967
latitude float64 51.41
altitude float64 185.1
Data variables:
DBZH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray>
VRADH (time, azimuth, range) float32 dask.array<chunksize=(1, 360, 240), meta=np.ndarray>
Attributes:
fixed_angle: 25.0[24]:
fig = pl.figure(figsize=(10, 4))
ax = fig.add_subplot(111)
ts.DBZH.median('azimuth').plot(x='time', vmin=-10, vmax=30, ax=ax)
ax.set_title(f"{np.datetime_as_string(ts.time[0].values, unit='D')}")
ax.set_ylim(0, 20000)
[24]:
(0.0, 20000.0)
Export to OdimH5¶
This exports the radar volume at given timestep including all moments into one ODIM_H5 compliant data file.
[25]:
vol.to_odim('dwd_odim.h5', timestep=0)
Export to Cf/Radial2¶
This exports the radar volume at given timestep including all moments into one Cf/Radial2 compliant data file.
[26]:
vol.to_cfradial2('dwd_cfradial2.nc', timestep=0)
Import again and check equality¶
[27]:
vol1 = wrl.io.open_odim_dataset('dwd_odim.h5')
vol2 = wrl.io.open_cfradial2_dataset('dwd_cfradial2.nc')
[28]:
xr.testing.assert_equal(vol1.root, vol2.root)
for i in range(len(vol1)):
xr.testing.assert_equal(vol1[i].drop_vars("rtime"), vol2[i].drop_vars("rtime"))