wradlib: An Open Source Library for Weather Radar Data Processing¶

Release:	1.7.0
Date:	2020-06-12
notebooks:	f22965e
docs:	7a3625a

The $\omega radlib$ project has been initiated in order facilitate the use of weather radar data as well as to provide a common platform for research on new algorithms. $\omega radlib$ is an open source library which is well documented and easy to use. It is written in the free programming language Python. As of version 1.3 $\omega radlib$ is Python 3 only.

Note

Please cite $\omega radlib$ as Heistermann, M., Jacobi, S., and Pfaff, T.: Technical Note: An open source library for processing weather radar data (wradlib), Hydrol. Earth Syst. Sci., 17, 863-871, doi:10.5194/hess-17-863-2013, 2013

If you refer to a specific $\omega radlib$ version please cite using $\omega radlib$’s zenodo doi:10.5281/zenodo.1209843 for this version.

Weather radar data is potentially useful in meteorology, hydrology and risk management. Its ability to provide information on precipitation with high spatio-temporal resolution over large areas makes it an invaluable tool for short term weather forecasting or flash flood forecasting.

$\omega radlib$ is designed to assist you in the most important steps of processing weather radar data. These may include: reading common data formats, georeferencing, converting reflectivity to rainfall intensity, identifying and correcting typical error sources (such as clutter or attenuation) and visualising the data.

This documentation is under steady development. It provides a complete library reference as well as a set of tutorials which will get you started in working with $\omega radlib$.

Documentation¶

Installation

Installation¶

Anaconda/Conda¶

In order to run $\omega radlib$, you need to have a Python interpreter installed on your local computer, as well as a number of Python packages (Dependencies). We recommend installing Anaconda as it includes Python, numerous required packages, and other useful tools (e.g. Spyder).

Using Anaconda the installation process is harmonised across platforms. Download and install the latest Anaconda distribution from https://www.anaconda.com/download/ for your specific OS. You might also consider the minimal Miniconda if you do not want to install a full scientific python stack.

We are constantly performing tests with these distributions (for python versions 3.6 and 3.7 respectively).

If your Anaconda Python installation is working, the following command (in a console) should work:

$ python --version
Python 3.7.3 :: Continuum Analytics, Inc.

Now you can use the conda package and environment manager (conda documentation) to setup your $\omega radlib$ installation.

Add the conda-forge channel, where $\omega radlib$ and its dependencies are located. Read more about the community effort conda-forge:
```
$ conda config --add channels conda-forge
```
Use strict channel priority to prevent channel clashes:
```
$ conda config --set channel_priority strict
```

Create a new environment from scratch:

$ conda create --name wradlib python=3.7

Activate the $\omega radlib$ environment:
```
$ conda activate wradlib
```
Install $\omega radlib$ and its dependencies:
```
(wradlib) $ conda install wradlib
```

Now you have a conda environment with a working $\omega radlib$ installation.

Test the integrity of your $\omega radlib$ installation by opening a console window and typing calling the python interpreter:

$ python
Python 3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

The Python prompt should appear. Then type:

>>> import wradlib
>>> wradlib.__version__
'1.4.0'

If everything is ok, this will show the running $\omega radlib$ version. If the $\omega radlib$ package is not found by the interpreter, you will get:

>>> import wradlib
ImportError: No module named wradlib

Alternatively, you can install the Bleeding edge code, but you have to keep track of $\omega radlib's$ dependencies yourself.

Bleeding edge code¶

Warning

The $\omega radlib$ version on PyPI might lag behind the actual developments. You can use the bleeding edge code from the $\omega radlib$ repository. Note, however, that you need to make sure yourself that all Dependencies are met (see below).

Download the source, unzip, and run:

$ python setup.py install

Alternatively, you can add the $\omega radlib$ directory to your environment variable PYTHONPATH.

Installing via pip¶

Although we recommend using the Anaconda Python Environment you can install $\omega radlib$ from PyPi via pip.

Open a terminal and run:

$ pip install wradlib

Depending on your system you might need to be root (or sudo the above command) for this to work. pip will then fetch the source distribution from the Python Package Index and run the installation.

Afterwards it will check for any dependencies not met, yet.

Be aware that using pip we can only look for python-module dependencies. For example the numpy module itself depends on some other libraries, which need to be present in order for the module to compile properly after being downloaded by pip. We have no control over these dependencies and it is rather hard to give a complete overview.

Therefore we recommend trying to satisfy the dependencies using your favorite package management system.

Dependencies¶

$\omega radlib$ was not designed to be a self-contained library. Besides extensive use of Numpy and Scipy, $\omega radlib$ uses additional libraries, which you will need to install before you can use $\omega radlib$.

Package	min	recommended
numpy	>= 1.9	>= 1.16.0
matplotlib	>= 2	>= 3.0.2
scipy	>= 0.9	>= 1.2.0
h5py	>= 2.0.1	>= 2.9.0
netCDF4	>= 1.0	>= 1.4.2
xarray	>= 0.1	>= 0.11.3
xmltodict	>= 0.11	>= 0.11.0
gdal	>= 1.9	>= 2.4.0

You can check whether the required Dependencies are available on your computer by opening a Python console and enter:

>>> import <package_name>
ImportError: No module named <package_name>

This will be the response in case the package is not available.

In case the import is successful, you should also check the version number:

>>> package_name.__version__
some version number

The version number should be consistent with the above Dependencies.

Optional Dependencies¶

Apart from the obligatory Dependencies, some dependencies in $\omega radlib$ are optional. This is because the installation of these dependencies can be somewhat tedious while many $\omega radlib$ users will not need them anyway. In case users use a $\omega radlib$ function that requires an optional dependency, and this dependency is not satisfied in the local environment, $\omega radlib$ will raise an exception.

As for now, the following dependencies are defined as optional:

The speedup module

The speedup module is intended as a collection of Fortran code in order to speed up specific $\omega radlib$ function that are critical for performance. In order to build the speedup module as a shared library, you need to use f2py (https://sysbio.ioc.ee/projects/f2py2e/). f2py usually ships with numpy and should be available via the command line. To test whether f2py is available on your system, execute f2py on the system console. Or, alternatively, f2py.py. If it is available, you should get a bunch of help instructions. Now change to the $\omega radlib$ module directory and execute on the system console:

$ f2py.py -c -m speedup speedup.f

Now the speedup module should be available.

Known Issues¶

Depending on your OS and installation method you may encounter different problems. Here are some guidelines for attacking them.

We strongly recommend using the Anaconda conda package and environment manager (see Installation). Using conda-forge we will maintain the wradlib-feedstock for constant availability of recent $\omega radlib$ versions.

If you can’t use Anaconda/Miniconda, it is generally a good idea to use your systems package manager to install dependencies. This will also take account for other needed bindings, libs etc.

If you encounter problems installing $\omega radlib$, check on your favorite search engine or create an issue here with details on the problem or send an email on the wradlib-users mailing list.

Using Docker¶

Overview¶

Especially for easy integration with servers we setup the wradlib-docker repository. There we provide the Dockerfiles for five different docker images in feature branches. Those docker images are build automatically and are available on docker-hub.

The branch names are the docker tag names:

base
- based on centos7
- plain miniconda installation
min
- based on base
- conda wradlib environment with latest wradlib release including all runtime dependencies
notebook
- based on min
- added notebook packages to conda wradlib environment
full
- based on notebook
- added wradlib build dependencies to conda wradlib environment
training
- based on full
- added additional scientific python packages to conda wradlib environment

Installation¶

In order to run these docker images as containers, you need to have docker installed on your local computer or server. Please refer to the docker documentation for installation guidance. Once you have docker installed and running you can use the provided wradlib-docker images. You can either pull them directly from docker hub or create them locally from the wradlib-docker repository.

Pull from docker hub using the wanted tag:

$ docker pull wradlib/wradlib-docker:min

Build from scratch:

$ git clone https://github.com/wradlib/wradlib-docker.git
$ cd wradlib-docker
$ git checkout min
$ docker build -t wradlib_min .

The only difference is then in the docker run command, where wradlib/wradlib-docker:min is used in the first and wradlib_min in the second case.

Running docker images¶

Which docker image you need to use depends strongly on your preferences. Here only some applications are shown.

Minimal wradlib¶

Here 5 different approaches for running containers are shown.

Start container directly into bash:

$ docker run -ti --name wradlib_min wradlib/wradlib-docker:min /bin/bash
[wradlib@0f994a690230 /]$ source activate wradlib
(wradlib) [wradlib@0f994a690230 /]$ python
Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> wradlib.version.full_version
'1.0.0'
>>>

Start container directly into python (wradlib env):

$ docker run -ti --name wradlib_min wradlib/wradlib-docker:min /opt/conda/envs/wradlib/bin/python
Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Initialize wradlib user with your users id:

$ docker run -ti --name wradlib_min -e LOCAL_USER_ID=$UID wradlib/wradlib-docker:min /bin/bash
(wradlib) [wradlib@0f994a690230 /]$

Mount host-folder into container (rw, assuming we are in current users /home-folder, replace user with your current user):

$ mkdir docker-test
$ cd docker-test
$ echo "Greetings from the host" > greets.txt
$ docker run -ti --name wradlib_min -e LOCAL_USER_ID=$UID -v /home/user/docker-test:/home/docker-test wradlib/wradlib-docker:min /bin/bash
[wradlib@d6ffc3a7bbe5 /]$ cat /home/docker-test/greets.txt
Greetings from the host
$ [wradlib@d6ffc3a7bbe5 /]$ echo "Greetings from the container" >> /home/docker-test/greets.txt
$ [wradlib@d6ffc3a7bbe5 /]$ exit
$ cat greets.txt
Greetings from the host
Greetings from the container

Start container with X11 capabilities (eg. for use of matplotlib):

$ XSOCK=/tmp/.X11-unix/
$ XAUTH=/tmp/.docker.xauth
$ touch $XAUTH
$ xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -
$ GID=`id -g $USER`
$ docker run -ti --name wradlib_min_x11 -e LOCAL_GROUP_ID=$GID -e LOCAL_USER_ID=$UID -e LOCAL_USER_NAME=$USER -v $XSOCK:$XSOCK:rw -v $XAUTH:$XAUTH:rw -e XAUTHORITY=${XAUTH} -e DISPLAY wradlib/wradlib-docker:min /bin/bash
[user@7324bddd3b81 /]$ source activate wradlib
(wradlib) [user@7324bddd3b81 /]$ python
Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import wradlib as wrl
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> data = np.random.random((360,1000))
>>> fig = plt.figure()
>>> wrl.vis.plot_ppi(data, fig=fig)
>>> plt.show()

This will open a matplotlib figure on your current $DISPLAY.

Jupyter Notebook Server¶

You can run a jupyter notebook server using the notebook using the following setup. You would need to download wradlib-notebooks and wradlib-data. The docker run command mounts the host folders containing wradlib-notebooks and wradlib-data into the container:

$ docker run -ti --name wradlib_nb -p 8888:8888 -v /host/path/to/wradlib-notebooks:/home/notebooks -v /host/path/to/wradlib-data:/home/wradlib-data -e LOCAL_USER_ID=$UID -e WRADLIB_DATA=/home/wradlib-data wradlib/wradlib-docker:notebook /opt/conda/envs/wradlib/bin/jupyter notebook --notebook-dir=/home/notebooks --ip='*' --port=8888

[I 08:07:35.865 NotebookApp] Writing notebook server cookie secret to /home/wradlib/.local/share/jupyter/runtime/notebook_cookie_secret
[W 08:07:36.087 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 08:07:36.098 NotebookApp] Serving notebooks from local directory: /home/notebooks
[I 08:07:36.098 NotebookApp] 0 active kernels
[I 08:07:36.098 NotebookApp] The Jupyter Notebook is running at:
[I 08:07:36.098 NotebookApp] http://[all ip addresses on your system]:8888/?token=6673cfb299fb93728c183be8a4590fc77608fb1312bce340
[I 08:07:36.099 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 08:07:36.099 NotebookApp] No web browser found: could not locate runnable browser.
[C 08:07:36.099 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=6673cfb299fb93728c183be8a4590fc77608fb1312bce340

Copy/paste the given url into your local web browser and you are ready to explore the wradlib-notebooks together with wradlib-data.

IDEs for Python¶

Development Environments vs. Notebooks¶

Jupyter notebooks are great for courses and for interactive data exploration. However, they are not suited for developing applications. Once you start to build applications or scripts on top of $\omega radlib$, you should use a suitable Integrated Development Environment (IDE). IDEs give you the opportunity for organising different source files, debugging, available inspection and a lot more.

Some suitable IDEs¶

There is a comprehensive Wiki article on Python IDEs. Maybe a bit too comprehensive… if you already are used to an IDE and happy with it, there is no reason to change it.

However, if you have no idea which IDE to choose, why not start with Spyder? It is included in the Anaconda Python Distribution by default. If you installed Anaconda, just open a shell (Anaconda Prompt on Windows) and enter:

$ spyder

Alternatively, we can warmly recommend PyCharm which has a free, yet powerful Community Edition.

User Guide

Introduction¶

How to use the tutorials and examples?¶

Examples: your entry point to wradlib¶

The documentation of $\omega radlib$ basically consists of two elements: the first element is the library reference which provides a systematic documentation of each function in wradlib’s API. In order to use the library reference properly, you should have an idea what you’re actually looking for. This is why new users mostly start with the second element, the tutorials and examples. Here you can look for application examples that are close to what you actually want to achieve, and then use these as a basis for further development. Once you understood an example, you also know what to look for in the library reference.

Interactive examples with jupyter notebooks¶

All $\omega radlib$ examples are distributed as jupyter notebooks in the wradlib-notebooks repository. This way, you can interactively explore various topics. Each notebook is organized in documented code blocks. You can browse through a notebook block by block, inspect the results, and experiment with the code. However, you can also view the rendered notebooks including the example results on the web pages of this section: Each page was directly generated from $\omega radlib$’s example notebooks. This way, you can copy&paste code snippets directly into your applications.

Note

Are you using $\omega radlib$ on the Open Virtual Machine for Cross-Platform Weather Radar Science? Then skip the rest: just log in, run the command $ ./start_notebook.sh and direct your browser to http://127.0.0.1:8888/tree. That’s it.

Otherwise, you need to make sure to

get the actual notebook files,
get the example data,
and install jupyter

Once these conditions are met, you can open a notebook: Launch a console or bash window in the directory that contains the example notebooks. If you installed $\omega radlib$ (e.g. via conda install wradlib) into Anaconda’s default (root) environment, you just need to execute:

> jupyter notebook

If you installed $\omega radlib$ into a conda environment (as recommended here), you need to activate that environment first. Let’s say you named the environment wradlib:

> conda activate wradlib
> jupyter notebook

Did you forget the name of the environment that contains wradlib? This way, you get an overview over all environments:

> conda info --envs

which will give you something like (example output under Windows):

# conda environments
wradlib      C:\Anaconda3\envs\wradlib
root      *  C:\Anaconda3

In both cases, a browser window will open (typically at http://localhost:8888/tree) which will show the tree of the directory in which you started the jupyter notebook server. Just open any notebook by clicking. Code cells are executed by hitting Shift + Enter or by using the toolbar icons. It’s pretty much self-explaining, and you’ll soon get the hang of it.

How can I get the example notebooks?¶

The notebooks are available in the wradlib-notebooks repository. Alternatively, you can download the latest (“bleeding edge”) notebooks. In both cases, you find everything in the directory /notebooks (after extracting the downloaded archives).

How can I get the example data?¶

Most notebooks use example data. These data are provided in a separate repository and you need to “install” them manually. “Installing” actually means: Download the data archive here and extract it into any arbitrary directory. Now you need to set an environment variable pointing to that directory:

Under Windows, open a console and execute setx WRADLIB_DATA <full path to wradlib-data>.

Under Linux, you need to set the env variable in your current shell profile. In most cases, this will be loaded from ~/.bashrc. Open that file in an editor (e.g. via $EDITOR ~/.bashrc) and add the following line at the end of the file:

export WRADLIB_DATA=/insert/full/path/to/wradlib-data/

After this procedure, the example notebooks will automagically pull the required files from the data archive.

How to install jupyter?¶

As already pointed out above, you can just look at the rendered notebooks online docs. In order to use them interactively, you need to install jupyter. jupyter is shipped with Anaconda’s distribution by default. If you installed $\omega radlib$ in a separate virtual environment (as recommended here), you need to install jupyter in that virtual environment, too:

Under Windows:
> activate wradlib
[wradlib] > conda install jupyter

Under Linux/OSX:
$ source activate wradlib
[wradlib] $ conda install jupyter

If you are not sure which conda environments you have, you can check via conda info --envs.

If you did not install $\omega radlib$ on top of Anaconda, you should first check whether jupyter might already be available on your system (use e.g. jupyter --version). If jupyter is not available, you should check out the jupyter docs for alternative installation options.

I prefer simple Python scripts instead of notebooks¶

No problem. If you downloaded the notebooks directly from the wradlib repository, you can easily convert them to Python scripts yourself (but you need to install jupyter to do the conversion):

$ jupyter nbconvert --to script <name of the notebook>

An incomplete introduction to Python¶

In order to use $\omega radlib$, you need to be able to use Python. The notebooks in this section will give you a glimpse of the Python language and important scientific packages, confront you with some Python exercises, and refer you to further online material. In addition, we will address some issues that are relevant to $\omega radlib$, but do not necessarily address $\omega radlib$ functionality.

As an introduction to scientific python, this section is necessarily incomplete. There are better and more comprehensive courses such as the SciPy Lecture Notes or the Dive into Python book.

Mostly, we will assume that you have some basic knowledge about programming, e.g. from other interpreted languages such as R or Matlab. Otherwise, a fundamental course such as learnpython.org might help you more.

A quick start to Python¶

In order to use $\omega radlib$, you need to be able to use Python.

This notebook will - give you a glimpse of the Python language, - confront you with some Python exercises, - and refer you to further in-depth courses for learning Python.

Hello interpreted world¶

Python is an interpreted language just as e.g. R, Matlab and many others. In contrast to a compiled languages such as C, C++ or Fortran, each command is immediately interpreted by the Python interpreter. In this case, executing the following cell produces an immediate output.

[1]:

print("Hello interpreted world.")

Hello interpreted world.

Writing a program in Python¶

Instead of entering this command interactively in a Python console (or a jupyter cell, as in this case), you can also write a Python script (or, in other words, a Python “program”).

Use a text editor to create a file,
add the line print("Hello interpreted world."),
save the file under the name demoscript.py

(by convention, Python scripts should have the extension .py).

Now execute your script from any shell (or the DOS console on Windows):

$ python demoscript.py

Note that this will work only if you open the shell in the same directory in which you saved your script. Otherwise, you need to provide the full path like e.g.

Under Linux/OSX:

$ python /home/user/path/to/demoscript.py

Under Windows:

> python e:\path\to\demoscript.py

Important data types¶

Python knows various standard data types such as - integer, - float, - bool, - and str.

If you define a variable, Python typically assigns the type of that variable for you.

[2]:

x1 = 2
print("x1 is of %r" % type(x1))

x2 = 2.3
print("x2 is of %r" % type(x2))

x3 = True
print("x3 is of %r" % type(x3))

x4 = x1 * x2
print("x4 is of %r" % type(x4))

x5 = "Hello interpreted world."
print("x5 is of %r" % type(x5))

x1 is of <class 'int'>
x2 is of <class 'float'>
x3 is of <class 'bool'>
x4 is of <class 'float'>
x5 is of <class 'str'>

As you noticed in the above cell, you can inspect the data type of a variable by using function type(). Python has a plethora of data types beyond the standard ones shown above. See for example:

[3]:

print( type(sum) )
print( type(type) )

<class 'builtin_function_or_method'>
<class 'type'>

Other data types: collections¶

Further important data types (and specific to Python) are so called data collections (or containers), namely

lists (list),
dictionaries (dict),
and tuples (tuple).

Lists¶

A list is an ordered collection of objects which may have different types.

[4]:

# a list
l = [1, "Paris", 3.0]
# a list is "mutable" (i.e. you can change the values of its elements)
l[1] = 2
print(l)
print(l[0:2])

[1, 2, 3.0]
[1, 2]

Excursus: indexing in Python¶

You might have noticed that the index 1 refers to the second element of the list.

This is an important rule for all containers in Python: indexing begins with 0

(which is familiar e.g. to C programmers, and new e.g. to R programmers)

Dictionaries¶

a dictionary is basically an efficient table that maps keys to values.
it is an unordered container.

There are two ways to define dictionaries - both are equivalent.

[5]:

# One way to define a dictionary
d1 = {"band": "C", "altitude":1000.}

# Another way to define a dictionary
d2 = dict(band="C", altitude=1000.)

# Both are equivalent
print("d1 equals d2 is a %r statement " % (d1 == d2) )
print("The type of d1 is %r" % type(d1) )

d1 equals d2 is a True statement
The type of d1 is <class 'dict'>

A dictionary can be queried using their keywords. You can also iterate over the keys.

[6]:

print(d1["band"])
print("Printing all keys and values of d1:")
for key in d1.keys():
    print("\t%s: %r" % (key, d1[key]) )

C
Printing all keys and values of d1:
        band: 'C'
        altitude: 1000.0

Control flow¶

Loops or iterations as well as conditional statments are key to data processing workflows.

In this section, we will present the most important syntax for control flow.

“for” loops¶

[7]:

# For loops
for i in range(3):
    print(i)

0
1
2

Indentation in Python

Indentation is a mandatory element of Python syntax - not just a question of style.

Not using indentation will raise an IndentationError.

“while” loops¶

[8]:

# While loops (in case the number of iterations is not known a-priori)
i = 0
while i < 3:
    print(i)
    i += 1

0
1
2

Conditional statements¶

[9]:

# Conditional statements
a = 5
if a < 10:
    print("a is less than 10.")
elif a == 10:
    print("a equals 10.")
else:
    print("a is greater than 10.")

a is less than 10.

You can also exit a loop before it is finished or skip parts of a specific iteration.

[10]:

for i in range(3):
    # Exit the loop already if i equals 1
    if i == 1:
        break
    print(i)

[11]:

for i in range(3):
    # Skip the iteration if i equals 1 (but continue the loop)
    if i == 1:
        continue
    print(i)

0
2

Importing functions from modules¶

For most applications, you will need to use functions that are available in modules. Just think of a module as a collection (or library) of functions. Some are shipped with the Python interpreter (Python Standard Library), others (such as wradlib) need to be installed (e.g. from the Python Package Index) or using the Anaconda Package Manager (conda).

In order to use a function from a module, you need to import the module. See e.g. how to import the os module and use its getcwd function (which retruns the path to the current working directory):

[12]:

import os
mycwd = os.getcwd()
print(mycwd)

/home/travis/build/wradlib/wradlib-notebooks/notebooks/python

Getting help¶

In most environments, you can use the TAB key for code completion and for listing functions available in a module.
Use the following cell, enter os. and hit the TAB key. You can narrow down available functions by typing further characters.
Once you’ve selected (or typed) a function name, jupyter will show you the help content after hitting TAB+Shift.
Alternatively, you can get help for a function in any Python environment by calling help (e.g. help(os.getcwd)). In juypter, you can also use the following syntax: os.getcwd?

[ ]:

Writing your own functions¶

In order to organise applications and write reusable code, you will need to write funcions on you own.

This is how you define and call a function (notice again the indentation):

[13]:

# Define the function...
def demofunction(a, b):
    c = a + b
    return(c)

[14]:

# ...and call the function
demofunction(2,3)

[14]:

Using docstrings, you can document your function as you write it.

[15]:

def demofunction(a, b):
    """This function computes the sum of a and b.

    Parameters
    ----------
    a : numeric
    b : numeric

    Returns
    -------
    output : numeric (the sum of a and b)

    """
    c = a + b
    return(c)

Now try calling help(demofunction).

[ ]:

This should be the output:

Help on function demofunction in module __main__:

demofunction(a, b)
    This function computes the sum of a and b

    Parameters
    ----------
    a : numeric
    b : numeric

    Returns
    -------
    output : numeric (the sum of a and b)

Functions can have keyword arguments with default values. In the following example, we set a default value for parameter b.

[16]:

def demofunction(a, b=2):
    """Adds 2 to a by default.
    """
    c = a + b
    return(c)

[17]:

print(demofunction(3))
print(demofunction(3, b=3))

5
6

File input and output¶

This is the basic way to read from and write to text and binary files (there are a lot of other file formats which have specific interfaces - particularly in the radar world: see file formats supported by wradlib).

[18]:

# Open a file for writing
f = open('testfile.txt', 'w') # opens the workfile file
print(type(f))
f.write('This is a test line,\nand another test line.\n')
f.close()

<class '_io.TextIOWrapper'>

[19]:

# Open a file for reading
f = open('testfile.txt', 'r')
s = f.read()
print(s)
f.close()

This is a test line,
and another test line.

A safer way to handle files is to use Python’s with statement which makes sure that the file is properly closed even if your code breaks inbetween opening and closing the file.

[20]:

with open("testfile.txt") as f:
    s = f.read()
    print(s)

This is a test line,
and another test line.

The building blocks of scientific Python¶

With this, we conclude the basic Python intro.

The following sections introduce, we introduce the most important packages for scientific computing in Python. Unlike Matlab, Scilab or R, Python does not come with a pre-bundled set of modules for scientific computing. Instead, a scientific computing environment can be obtained by combining these building blocks:

Python, a generic and modern computing language, including standard data types and collections, flow control, modules of the standard library, and development tools (automatic testing, documentation generation)
IPython: an interactive Python shell, see our article on how to use notebooks.
Numpy: provides powerful numerical arrays objects, and routines to manipulate them - see our NumPy intro article
Matplotlib : visualization and “publication-ready” plots, see related intro here
Scipy: high-level data processing routines - e.g. optimization, regression, interpolation, etc (https://www.scipy.org)

Please also check out further in-depth courses such as - the SciPy Lecture Notes or - the Dive into Python book.

[1]:

from __future__ import print_function

NumPy: manipulating numerical data¶

NumPy is the key Python package for creating and manipulating (multi-dimensional) numerical arrays. NumPy arrays are also the most important data objects in $\omega radlib$. It has become a convention to import NumPy as follows:

[2]:

import numpy as np

Creating and inspecting NumPy arrays¶

The ndarray, a numerical array, is the most important data type in NumPy.

[3]:

a = np.array([0, 1, 2, 3])
print(a)
print(type(a))

[0 1 2 3]
<class 'numpy.ndarray'>

Inspect the shape (i.e. the number and size of the dimensions of an array).

[4]:

print(a.shape)
# This creates a 2-dimensional array
a2 = np.array([[0, 1], [2, 3]])
print(a2.shape)

(4,)
(2, 2)

There are various ways to create arrays: from lists (as above), using convenience functions, or from file.

[5]:

# From lists
a = np.array([0, 1, 2, 3])
print("a looks like:\n%r\n" % a)

# Convenience functions
b = np.ones( shape=(2,3) )
print("b looks like:\n%r\nand has shape %r\n" % (b, b.shape) )

c = np.zeros( shape=(2,1) )
print("c looks like:\n%r\nand has shape %r\n" % (c, c.shape) )

d = np.arange(2,10)
print("d looks like:\n%r\nand has shape %r\n" % (d, d.shape) )

e = np.linspace(0,10,5)
print("e looks like:\n%r\nand has shape %r\n" % (e, e.shape) )

a looks like:
array([0, 1, 2, 3])

b looks like:
array([[1., 1., 1.],
       [1., 1., 1.]])
and has shape (2, 3)

c looks like:
array([[0.],
       [0.]])
and has shape (2, 1)

d looks like:
array([2, 3, 4, 5, 6, 7, 8, 9])
and has shape (8,)

e looks like:
array([ 0. ,  2.5,  5. ,  7.5, 10. ])
and has shape (5,)

You can change the shape of an array without changing its size.

[6]:

a = np.arange(10)
b = np.reshape(a, (2,5))
print("Array a has shape %r.\nArray b has shape %r" % (a.shape, b.shape))

Array a has shape (10,).
Array b has shape (2, 5)

Indexing and slicing¶

You can index an ndarray in the same way as a list:

[7]:

a = np.arange(10)
print(a)
print(a[0], a[2], a[-1])

[0 1 2 3 4 5 6 7 8 9]
0 2 9

Just follow your intuition for indexing multi-dimensional arrays:

[8]:

a = np.diag(np.arange(3))
print(a, end="\n\n")

print("Second row, second column: %r\n" % a[1, 1])

# Setting an array item
a[2, 1] = 10 # third line, second column
print(a, end="\n\n")

# Acessing a full row
print("Second row:\n%r" % a[1])

[[0 0 0]
 [0 1 0]
 [0 0 2]]

Second row, second column: 1

[[ 0  0  0]
 [ 0  1  0]
 [ 0 10  2]]

Second row:
array([0, 1, 0])

Slicing is just a way to access multiple array items at once:

[9]:

a = np.arange(10)
print(a, end="\n\n")

print("1st:", a[2:9])
print("2nd:", a[2:])
print("3rd:", a[:5])
print("4th:", a[2:9:3]) # [start:end:step]
print("5th:", a[a>5]) # using a mask

[0 1 2 3 4 5 6 7 8 9]

1st: [2 3 4 5 6 7 8]
2nd: [2 3 4 5 6 7 8 9]
3rd: [0 1 2 3 4]
4th: [2 5 8]
5th: [6 7 8 9]

Get further info on NumPy arrays here!

Visualisation and plotting with Matplotlib¶

Matplotlib is the key Python package for producing so called publication-ready plot. It provides the basis for $\omega radlib$’s entire visualisation module, and is typically used together with NumPy - which is the other major $\omega radlib$ dependency.

Different ways to import matplotlib¶

In a Python script¶

import matplotlib.pyplot as pl

In an IPython notebook¶

# This magic just sets up matplotlib's interactive mode
%matplotlib
# So you have to explicitely import the module into the namespace
import matplotlib.pyplot as pl

If you want to enable inline plotting

(mandatory if you use the Virtual Machine for Cross-Platform Weather Radar Science)

# This magic just sets up matplotlib's interactive mode
%matplotlib inline
# So you have to explicitely import the module into the namespace
import matplotlib.pyplot as pl

%matplotlib inline turns on “inline plotting”, where plot graphics will appear in your notebook. This has important implications for interactivity: for inline plotting, commands in cells below the cell that outputs a plot will not affect the plot. For example, changing the color map is not possible from cells below the cell that creates a plot. However, for other backends, such as qt4, that open a separate window, cells below those that create the plot will change the plot - it is a live object in memory. If you are not using matplotlib in interactive mode at all, figures will only appear if you invoke plt.show().

If you do not want to use inline plotting, just use %matplotlib instead of %matplotlib inline. The Kernel has to be restarted for this change to become effective.

If you want to magically import numpy and matplotlib

%pylab
# or
%pylab inline

In the following, we use a sightly different syntax for matplotlib inline. This is because the notebook needs to be convertable to a Python script where IPython magic does not work. Please don’t let this confuse you…

[1]:

# Instead of matplotlib inline
import matplotlib.pyplot as pl
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

Simple plots and decorations¶

After these imports, inline plots should work rightaway, e.g. a simple line plot of the sinus function:

[2]:

x = np.arange(0,4*np.pi,0.1)
y = np.sin(x)
pl.plot(x, y)

[2]:

[<matplotlib.lines.Line2D at 0x7f721fd38dc0>]

_images/notebooks_python_mplintro_15_1.png

More complex plots and fine control¶

A matplotlib plot can be understood from an object-oriented perspective. Each plot consists of a figure object (like a canvas), an axes object (like a subplot or panel), and other objects such as a title, an axis, or a colorbar.

alt text

Accordingly, a plot can be developed by creating or modifying the different objects. For example, the size of the entire plot is controlled by the Figure object. Each subplot in that figure corresponds to an axes object.

[3]:

# Create the figure object
fig = pl.figure(figsize=(12,8))

# Add first axes object (of a multi-panel plot with two rows and one column)
ax = fig.add_subplot(211)
pl.plot(x, np.sin(x))
pl.title("The Sinus Function")
pl.xlabel("This is my x-axis label")

# Add second axes object
ax = fig.add_subplot(212)
pl.plot(x, np.cos(x))
pl.title("The Cosinus Function")
pl.xlabel("This is my x-axis label")

# Make sure the elements of the plot are arranged properly
pl.tight_layout()

_images/notebooks_python_mplintro_20_0.png

Dealing with time series¶

Dealing with radar data typically means implies dealing with time series (of radar records or rain gauge observations). This article gives a brief intro on how to deal with times series and datetimes in Python.

The datetime module¶

The datetime module provides a number of types to deal with dates, times, and time intervals.

[1]:

import datetime as dt

There are different ways to create datetime objects.

[2]:

# This is now (system time)
now = dt.datetime.now()
# Just using the date
birth_van_rossum = dt.datetime(1956, 1, 31)
# Providing both date and time
first_wradlib_commit = dt.datetime(2011, 10, 26, 11, 54, 58)
# Or initialising from a string
erad_2016_begins = dt.datetime.strptime("2016-10-09 09:00:00", "%Y-%m-%d %H:%M:%S")

You can compute the difference between two datetime objects.

[3]:

# Age of Guido van Rossum
age_van_rossum = now - birth_van_rossum
print("This is a %r object.\n" % type(age_van_rossum) )
print("It looks like this: %r" % age_van_rossum )
print("and consists of\n\t%d days,\n\t%d seconds,\n\tand %d microseconds.\n"
      % (age_van_rossum.days, age_van_rossum.seconds, age_van_rossum.microseconds) )
# Age of wradlib
age_wradlib = now - first_wradlib_commit
# Time until (or since) beginning of ERAD 2016 OSS Short course
from_to_erad2016 = now - erad_2016_begins

print("Guido van Rossum is %d seconds old." % age_van_rossum.total_seconds())
print("wradlib's first commit was %d days ago." % age_wradlib.days)
if from_to_erad2016.total_seconds() < 0:
    print("The ERAD 2016 OSS Short course will start in %d days." % -from_to_erad2016.days )
else:
    print("The ERAD 2016 OSS Short course took place %d days ago." % from_to_erad2016.days)

This is a <class 'datetime.timedelta'> object.

It looks like this: datetime.timedelta(days=23509, seconds=24800, microseconds=750626)
and consists of
        23509 days,
        24800 seconds,
        and 750626 microseconds.

Guido van Rossum is 2031202400 seconds old.
wradlib's first commit was 3151 days ago.
The ERAD 2016 OSS Short course took place 1341 days ago.

Or you can create a datetime.timedelta object yourself and add/subtract a time interval from/to a datetime object. You can use any of these keywords: days, seconds, microseconds, milliseconds, minutes, hours, weeks, but datetime.timedelta will always represent the result in days, seconds, microseconds.

[4]:

# This is an interval of two minutes
print(dt.timedelta(minutes=1, seconds=60))
# And this is, too
print(dt.timedelta(minutes=2))
now = dt.datetime.now()
print("This is now: %s" % now)
print("This is two minutes before: %s" % (now - dt.timedelta(minutes=2)) )

0:02:00
0:02:00
This is now: 2020-06-12 06:53:20.770914
This is two minutes before: 2020-06-12 06:51:20.770914

The default string format of a datetime object corresponds to the isoformat. Using the strftime function, however, you can control string formatting yourself. The following example shows this feature together with other features we have learned before. The idea is to loop over time and generate corresponding string representations. We also store the datetime objects in a list.

[5]:

start = dt.datetime(2016, 10, 9)
end = dt.datetime(2016, 10, 14)
interval = dt.timedelta(days=1)
dtimes = []
print("These are the ERAD 2016 conference days (incl. short courses):")
while start <= end:
    print(start.strftime("\t%A, %d. %B %Y"))
    dtimes.append(start)
    start += interval

These are the ERAD 2016 conference days (incl. short courses):
        Sunday, 09. October 2016
        Monday, 10. October 2016
        Tuesday, 11. October 2016
        Wednesday, 12. October 2016
        Thursday, 13. October 2016
        Friday, 14. October 2016

matplotlib generally understands datetime objects and tries to make sense of them in plots.

[6]:

# Instead of %matplotlib inline
import matplotlib.pyplot as pl
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

[7]:

# Create some dummy data
level = np.linspace(100,0,len(dtimes))

# And add a time series plot
fig = pl.figure(figsize=(10,5))
ax = fig.add_subplot(111)
pl.plot(dtimes, level, "bo", linestyle="dashed")
pl.xlabel("Day of the conference", fontsize=15)
pl.ylabel("Relative attentiveness (%)", fontsize=15)
pl.title("Development of participants' attentiveness during the conference", fontsize=15)
pl.tick_params(labelsize=12)

_images/notebooks_python_timeseries_15_0.png

Using virtual environments¶

Virtual environments provide a great opportunity to experiment with different system configuations (e.g. package versions) without putting your entire Python installation at risk. Working with Anaconda makes the use of virtual environments quite easy.

Why should I use virtual environments¶

To be continued…

Why should I NOT use virtual environments¶

To be continued…

[ ]:

Getting started with wradlib¶

This section provides a collection of example code snippets to make users familiar with $\omega radlib$.

Import wradlib and check its version¶

This simple example shows the $\omega radlib$ version at rendering time.

[1]:

import wradlib
print(wradlib.__version__)

1.7.0

A Typical Workflow For Radar-Based Rainfall Estimation¶

Raw, unprocessed reflectivity products can already provide useful visual information about the spatial distribution of rainfall fields. However, in order to use weather radar observations for quantitative studies (e.g. in hydrological modelling or for assimilation into Numerical Weather Prediction models), the data has to be carefully processed in order to account for typical errors sources such as ground echoes (clutter), attenuation of the radar signal, or uncertainties in the Z/R relationship.

Moreover, it might be necessary to transfer the data from polar coordinates to Cartesian grids, or to combine observations from different radar locations in overlapping areas on a common grid (composition). And in the end, you would typically like to visualise the spatial rainfall distribution on a map. Many users also need to quantify the potential error (uncertainty) of their data-based rainfall estimation.

These are just some steps that might be necessary in order to make radar data useful in a specific quantitative application environment. All steps together are typically referred to as a “radar data processing chain”. $\omega radlib$ was designed to support you in establishing your own processing chain, suited to your specific requirements. In the following, we will provide an outline of a typical processing chain, step-by-step. You might not need all steps for your own workflow, or you might need steps which are not yet included here.

Introduction¶

Consider this just as an example. We will not go into detail for each step in this section, but refer to more detailed tutorials (if available) or the corresponding entry in the library reference. Most of the steps have a corresponding $\omega radlib$ module. In order to access the functions of $\omega radlib$, you have to import $\omega radlib$ in your Python environment:

import wradlib as wrl

If you have trouble with that import, please head back to the Getting Started section.

Note

The data used in this tutorial can be found in the wradlib-data repository. Follow these instructions to install and use this data files.

Warning

Be aware that applying an algorithm for error correction does not guarantee that the error is totally removed. Error correction procedures are suceptible to errors, too. Not only might they fail to remove the error. They might also introduce new errors. The trade-off between costs (introduction of new errors) and benefits (error reduction) can turn out differently for different locations, different points in time, or different rainfall situations.

Reading the data¶

The binary encoding of many radar products is a major obstacle for many potential radar users. Often, decoder software is not easily available. $\omega radlib$ supports a couple of formats such as the ODIM_H5 implementation, NetCDF, and some formats used by the German Weather Service. We seek to continuously enhance the range of supported formats.

The basic data type used in $\omega radlib$ is a multi-dimensional array, the numpy.ndarray. Such an array might e.g. represent a polar or Cartesian grid, or a series of rain gage observations. Metadata are normally managed as Python dictionaries. In order to read the content of a data file into a numpy array, you would normally use the wradlib.io module. In the following example, a local PPI from the German Weather Service, a DX file, is read:

[2]:

import pylab as pl
pl.figure(figsize=(10,8))
filename = wrl.util.get_wradlib_data_file('dx/raa00-dx_10908-0806021655-fbg---bin.gz')
data, metadata = wrl.io.read_dx(filename)
ax, pm = wrl.vis.plot_ppi(data) # simple diagnostic plot
cbar = pl.colorbar(pm, shrink=0.75)

<Figure size 720x576 with 0 Axes>

_images/notebooks_basics_wradlib_workflow_11_1.png

The metadata object can be inspected via keywords. The data object contains the actual data, in this case a polar grid with 360 azimuth angles and 128 range bins.

Seealso

Get more info in the section Supported radar data formats and in the library reference section Raw Data I/O.

Clutter removal¶

Clutter are non-meteorological echos. They are caused by the radar beam hitting objects on the earth’s surface (e.g. mountain or hill tops, houses, wind turbines) or in the air (e.g. airplanes, birds). These objects can potentially cause high reflectivities due large scattering cross sections. Static clutter, if not efficiently removed by Doppler filters, can cause permanent echos which could introduce severe bias in quantitative applications. Thus, an efficient identification and removal of clutter is mandatory e.g. for hydrological studies. Clutter removal can be based on static maps or dynamic filters. Normally, static clutter becomes visible more clearly in rainfall accumulation maps over periods of weeks or months. We recommend such accumulations to create static clutter maps which can in turn be used to remove the static clutter from an image and fill the resulting gaps by interpolation.

In the following example, the clutter filter published by Gabella et al., 2002) is applied to the single radar sweep of the above example:

[3]:

clutter = wrl.clutter.filter_gabella(data, tr1=12, n_p=6, tr2=1.1)
pl.figure(figsize=(10,8))
ax, pm = wrl.vis.plot_ppi(clutter, cmap=pl.cm.gray)
pl.title('Clutter Map')

[3]:

Text(0.5, 1.0, 'Clutter Map')

<Figure size 720x576 with 0 Axes>

_images/notebooks_basics_wradlib_workflow_17_2.png

The resulting Boolean array clutter indicates the position of clutter. It can be used to interpolate the values at those positons from non-clutter values, as shown in the following line:

[4]:

data_no_clutter = wrl.ipol.interpolate_polar(data, clutter)
pl.figure(figsize=(10,8))
ax, pm = wrl.vis.plot_ppi(data_no_clutter) # simple diagnostic plot
cbar = pl.colorbar(pm, shrink=0.75)

<Figure size 720x576 with 0 Axes>

_images/notebooks_basics_wradlib_workflow_19_1.png

It is generally recommended to remove the clutter before e.g. gridding the data. Otherwise the clutter signal might be “smeared” over multiple grid cells, resulting into a decrease in detectability.

Seealso

Get more info in the library reference section Clutter Identification.

Attenuation correction¶

Attenuation by wet radome and by heavy rainfall can cause serious underestimation of rainfall for C-Band and X-Band devices. For such radar devices, situations with heavy rainfall require a correction of attenuation effects. The general approach with single-polarized radars is to use a recursive gate-by-gate approach. See Kraemer et al., 2008 for an introduction to this concept. Basically, the specific attenuation k of the first range gate is computed via a so-called k-Z relationship. Based on k, the reflectivity of the second range gate is corrected and then used to compute the specific attenuation for the second range gate (and so on). The concept was first introduced by Hitschfeld et al., 1954). Its main drawback is its suceptibility to instable behaviour. $\omega radlib$ provides different implementations which address this problem.

One example is the algorithm published by Jacobi and Heistermann, 2016:

[5]:

pia = wrl.atten.correct_attenuation_constrained(data_no_clutter, a_max=1.67e-4, a_min=2.33e-5,
                                                n_a=100, b_max=0.7, b_min=0.65, n_b=6,
                                                gate_length=1., constraints=[wrl.atten.constraint_dbz,
                                                                             wrl.atten.constraint_pia],
                                                constraint_args=[[59.0],[20.0]])
data_attcorr = data_no_clutter + pia

The first line computes the path integrated attenuation pia for each radar bin. The second line uses pia to correct the reflectivity values. Let’s inspect the effect of attenuation correction for an azimuth angle of 65 deg:

[6]:

pl.figure(figsize=(10,8))
pl.plot(data_attcorr[65], label="attcorr")
pl.plot(data_no_clutter[65], label="no attcorr")
pl.xlabel("km")
pl.ylabel("dBZ")
pl.legend()

[6]:

<matplotlib.legend.Legend at 0x7fc106f3f5e0>

_images/notebooks_basics_wradlib_workflow_27_1.png

Seealso

Get more info in the library reference section Attenuation Correction. There you will learn to know the algorithms available for attenuation correction and how to manipulate their behaviour by using additonal keyword arguments.

Vertical Profile of Reflectivity¶

Precipitation is 3-dimensional in space. The vertical distribution of precipitation (and thus reflectivity) is typically non-uniform. As the height of the radar beam increases with the distance from the radar location (beam elevation, earth curvature), one sweep samples from different heights. The effects of the non-uniform VPR and the different sampling heights need to be accounted for if we are interested in the precipiation near the ground or in defined altitudes.

Seealso

Get more info in the library reference section Vertical Profile of Reflectivity (VPR). There you will learn how to reference polar volume data, to create CAPPIs and Pseudo CAPPIs, to inspect vertical profiles of reflectivity (UNDER DEVELOPMENT), and to use these for correction (UNDER DEVELOPMENT).

Conversion of Reflectivity into Rainfall¶

Reflectivity (Z) and precipitation rate (R) can be related in form of a power law $R=a*Z^b$. The parameters a and b depend on the type of precipitation in terms of drop size distribution and water temperature. Before applying the Z-R relationship, we need to convert from dBZ to Z:

[7]:

R = wrl.zr.z_to_r(wrl.trafo.idecibel(data_attcorr))

The above line uses the default parameters a=200 and b=1.6 for the Z-R relationship. In order to compute a rainfall depth from rainfall intensity, we have to specify an integration interval in seconds. In this example, we choose five minutes (300 s), corresponding to the scan interval.

[8]:

depths = wrl.trafo.r_to_depth(R, 300)

Seealso

Get more info in the section Converting reflectivity to rainfall and in the library reference sections Z-R Conversions and Data Transformation. Here you will learn about the effects of the Z-R parameters a and b.

Rainfall accumulation¶

For many applications, accumulated rainfall depths over specific time intervals are required, e.g. hourly or daily accumulations. In the following example, we will use a synthetic time series of 5 minute intervals. Just imagine we have repeated the above procedure for one day of five-minute sweeps and combined the arrays of rainfall depth in a 3-dimensional array of shape (number of time steps, number of azimuth angles, number of range gates).

Now we want to compute the daily rainfall sum:

[9]:

sweep_times = wrl.util.from_to("2012-10-26 00:00:00", "2012-10-27 00:00:00", 300)
np.random.seed(1319622840)
depths_5min = np.random.uniform(size=(len(sweep_times)-1, 360, 128))
depth_daily = depths_5min.sum(axis=0)

Check the shape the resulting array for plausibility:

[10]:

print(depth_daily.shape)

(360, 128)

Seealso

For more advanced operations on time series, we recommend the Pandas package.

Georeferencing and Projection¶

In order to define the horizontal and vertical position of the radar bins, we need to retrieve the corresponding 3-dimensional coordinates in terms of longitude, latitude and altitude. This information is required e.g. if the positions should be plotted on a map. It is also required for constructing CAPPIs. The position of a radar bin in 3-dimensional space depends on the position of the radar device, the elevation and azimuth angle of the radar beam, the range of the bin, and the assumed influence of atmospheric refraction on the beam propagation. For the sample data used above, the position of the radar device is the Feldberg in Germany (8.005, 47.8744, 1517):

[11]:

radar_location = (8.005, 47.8744, 1517) # (lon, lat, alt) in decimal degree and meters
elevation = 0.5 # in degree
azimuths = np.arange(0,360) # in degrees
ranges = np.arange(0, 128000., 1000.) # in meters
polargrid = np.meshgrid(ranges, azimuths)

Using georef.spherical_to_xyz we get the cartesian coordinates in xyz-space and the connected azimuthal equidistant projection osr-object rad.

[12]:

coords, rad = wrl.georef.spherical_to_xyz(polargrid[0], polargrid[1],
                                          elevation, radar_location)
x = coords[..., 0]
y = coords[..., 1]

$\omega radlib$ supports the projection between geographical coordinates (lon/lat) and other reference systems. It uses GDAL/OSR Spatial References Objects as function parameters. Basically, you have to create the OSR-object by using GDAL-capabilities or one of the provided helper functions. We recommend the creation using EPSG numbers:

[13]:

# UTM Zone 32, EPSG-Number 32632
utm = wrl.georef.epsg_to_osr(32632)
utm_coords = wrl.georef.reproject(coords, projection_source=rad,
                                  projection_target=utm)

Second, you can provide a string which represents the projection - based on the PROJ.4 library. You can look up projection strings, but for some projections, $\omega radlib$ helps you to define a projection string. In the following example, the target projection is ‘dwd-radolan’:

[14]:

radolan = wrl.georef.create_osr("dwd-radolan")
radolan_coords = wrl.georef.reproject(coords, projection_target=radolan)

Seealso

Get more info in the library reference section Georeferencing.

Gridding¶

Assume you would like to transfer the rainfall intensity from the above example from polar coordinates to a Cartesian grid, or to an arbitrary set of irregular points in space (e.g. centroids of sub-catchments). You already retrieved the Cartesian coordinates of the radar bins in the previous section Georeferencing and Projection. Now you only need to define the target coordinates (e.g. a grid) and apply the togrid function of the wradlib.comp module. In this example, we want our grid only to represent the South-West sector of our radar circle on a 100 x 100 grid. First, we define the target grid coordinates (these must be an array of 100x100 rows with one coordinate pair each):

[15]:

xgrid = np.linspace(x.min(), x.mean(), 100)
ygrid = np.linspace(y.min(), y.mean(), 100)
grid_xy = np.meshgrid(xgrid, ygrid)
grid_xy = np.vstack((grid_xy[0].ravel(), grid_xy[1].ravel())).transpose()

Now we transfer the polar data to the grid and mask out invalid values for plotting (values outside the radar circle receive NaN):

[16]:

xy=np.concatenate([x.ravel()[:,None],y.ravel()[:,None]], axis=1)
gridded = wrl.comp.togrid(xy, grid_xy, 128000., np.array([x.mean(), y.mean()]), data.ravel(), wrl.ipol.Nearest)
gridded = np.ma.masked_invalid(gridded).reshape((len(xgrid), len(ygrid)))

fig = pl.figure(figsize=(10,8))
ax = pl.subplot(111, aspect="equal")
pm = pl.pcolormesh(xgrid, ygrid, gridded)
pl.colorbar(pm, shrink=0.75)
pl.xlabel("Easting (m)")
pl.ylabel("Northing (m)")
pl.xlim(min(xgrid), max(xgrid))
pl.ylim(min(ygrid), max(ygrid))

[16]:

(-126946.43557281807, 0.0)

_images/notebooks_basics_wradlib_workflow_58_1.png

Seealso

Get more info about the function wradlib.comp.togrid().

Adjustment by rain gage observations¶

Adjustment normally refers to using rain gage observations on the ground to correct for errors in the radar-based rainfall estimating. Goudenhoofdt et al., 2009 provide an excellent overview of adjustment procedures. A typical approach is to quantify the error of the radar-based rainfall estimate at the rain gage locations, assuming the rain gage observation to be accurate. The error can be assumed to be additive, multiplicative, or a mixture of both. Most approaches assume the error to be heterogeneous in space. Hence, the error at the rain gage locations will be interpolated to the radar bin (or grid) locations and then used to adjust (correct) the raw radar rainfall estimates.

In the following example, we will use an illustrative one-dimensional example with synthetic data (just imagine radar rainfall estimates and rain gage observations along one radar beam).

First, we create the synthetic “true” rainfall (truth):

[17]:

import numpy as np
radar_coords = np.arange(0,101)
np.random.seed(1319622840)
truth = np.abs(1.5+np.sin(0.075*radar_coords)) + np.random.uniform(-0.1,0.1,len(radar_coords))

The radar rainfall estimate radar is then computed by imprinting a multiplicative error on truth and adding some noise:

[18]:

error = 0.75 + 0.015*radar_coords
np.random.seed(1319622840)
radar = error * truth + np.random.uniform(-0.1,0.1,len(radar_coords))

Synthetic gage observations obs are then created by selecting arbitrary “true” values:

[19]:

obs_coords = np.array([5,10,15,20,30,45,65,70,77,90])
obs = truth[obs_coords]

Now we adjust the radar rainfall estimate by using the gage observations. First, you create an “adjustment object” from the approach you want to use for adjustment. After that, you can call the object with the actual data that is to be adjusted. Here, we use a multiplicative error model with spatially heterogenous error (see wradlib.adjust.AdjustMultiply():

[20]:

adjuster = wrl.adjust.AdjustMultiply(obs_coords, radar_coords, nnear_raws=3)
adjusted = adjuster(obs, radar)

Let’s compare the truth, the radar rainfall estimate and the adjusted product:

[21]:

pl.plot(radar_coords, truth, 'k-', label="True rainfall", linewidth=2.)
pl.xlabel("Distance (km)")
pl.ylabel("Rainfall intensity (mm/h)")
pl.plot(radar_coords, radar, 'k-', label="Raw radar rainfall", linewidth=2., linestyle="dashed")
pl.plot(obs_coords, obs, 'o', label="Gage observation", markersize=10.0, markerfacecolor="grey")
pl.plot(radar_coords, adjusted, '-', color="green", label="Multiplicative adjustment", linewidth=2., )
pl.legend(prop={'size':12})

[21]:

<matplotlib.legend.Legend at 0x7fc1009890a0>

_images/notebooks_basics_wradlib_workflow_70_1.png

Seealso

Get more info in the library reference section Gage Adjustment. There, you will also learn how to use the built-in cross-validation in order to evaluate the performance of the adjustment approach.

Verification and quality control¶

Typically, radar-based precipitation estimation and the effectiveness of the underlying correction and adjustment methods are verified by comparing the results against rain gage observations on the ground. wradlib.verify module provides procedures not only to extract the radar values at specific gauge locations, but also a set of error metrics which are computed from gage observations and the corresponding radar-based precipitation estimates (including standard metrics such as RMSE, mean error, Nash-Sutcliffe Efficiency). In the following, we will illustrate the usage of error metrics by comparing the “true” rainfall against the raw and adjusted radar rainfall estimates from the above example:

[22]:

raw_error  = wrl.verify.ErrorMetrics(truth, radar)
adj_error  = wrl.verify.ErrorMetrics(truth, adjusted)

Error metrics can be reported e.g. as follows:

[23]:

print("Error metrics for unadjusted radar rainfall estimates:")
raw_error.pprint()
print("\nError metrics for adjusted radar rainfall estimates:")
adj_error.pprint()

Error metrics for unadjusted radar rainfall estimates:
{'corr': 0.69,
 'mas': 0.77,
 'meanerr': 0.69,
 'mse': 1.19,
 'nash': -1.54,
 'pbias': 43.4,
 'r2': 0.48,
 'ratio': 1.5,
 'rmse': 1.09,
 'spearman': 0.77,
 'sse': 120.29}

Error metrics for adjusted radar rainfall estimates:
{'corr': 1.0,
 'mas': 0.06,
 'meanerr': 0.03,
 'mse': 0.01,
 'nash': 0.98,
 'pbias': 1.9,
 'r2': 0.99,
 'ratio': 1.01,
 'rmse': 0.1,
 'spearman': 0.99,
 'sse': 0.79}

Seealso

Get more info in the library reference section Verification.

Visualisation and mapping¶

In the above sections Reading the data, Clutter removal, and Gridding you already saw examples of the $\omega radlib's$ plotting capabilities.

Seealso

Get more info in the library reference section Visualisation.

Data export to other applications¶

Once you created a dataset which meets your requirements, you might want to export it to other applications or archives. $\omega radlib$ does not favour or support a specific output format. Basically, you have all the freedom of choice offered by Python and its packages in order to export your data. Arrays can be stored as text or binary files by using numpy functions. You can use the package NetCDF4 to write NetCDF files, and the packages h5py or PyTables to write hdf5 files. At a later stage of development, $\omega radlib$ might support a standardized data export by using the OPERA’s ODIM_H5 data model (see Supported Radar Data Formats). Of course, you can also export data as images. See Visualisation for some options.

Export your data array as a text file:

[24]:

np.savetxt("mydata.txt", data)

Or as a gzip-compressed text file:

[25]:

np.savetxt("mydata.gz", data)

Or as a NetCDF file:

[26]:

import netCDF4
rootgrp = netCDF4.Dataset('test.nc', 'w', format='NETCDF4')
sweep_xy = rootgrp.createGroup('sweep_xy')
dim_azimuth = sweep_xy.createDimension('azimuth', None)
dim_range = sweep_xy.createDimension('range', None)
azimuths_var = sweep_xy.createVariable('azimuths','i4',('azimuth',))
ranges_var = sweep_xy.createVariable('ranges','f4',('range',))
dBZ_var = sweep_xy.createVariable('dBZ','f4',('azimuth','range',))
azimuths_var[:] = np.arange(0,360)
ranges_var[:] = np.arange(0, 128000., 1000.)
dBZ_var[:] = data

You can easily add metadata to the NetCDF file on different group levels:

[27]:

rootgrp.bandwith = "C-Band"
sweep_xy.datetime = "2012-11-02 10:15:00"
rootgrp.close()

Note

An example for hdf5 export will follow.

Converting Reflectivity to Rainfall¶

Reflectivity (Z) and precipitation rate (R) can be related in form of a power law $Z=a \cdot R^b$. The parameters a and b depend on the type of precipitation (i.e. drop size distribution and water temperature). $\omega radlib$ provides a couple of functions that could be useful in this context.

The following example demonstrates the steps to convert from the common unit dBZ (decibel of the reflectivity factor Z) to rainfall intensity (in the unit of mm/h). This is an array of typical reflectivity values (unit: dBZ)

[2]:

dBZ = np.array([20., 30., 40., 45., 50., 55.])
print(dBZ)

[20. 30. 40. 45. 50. 55.]

Convert to reflectivity factor Z (unit: $mm^6/m^3$):

[3]:

Z = wrl.trafo.idecibel(dBZ)
print(Z)

[1.00000000e+02 1.00000000e+03 1.00000000e+04 3.16227766e+04
 1.00000000e+05 3.16227766e+05]

Convert to rainfall intensity (unit: mm/h) using the Marshall-Palmer Z(R) parameters:

[4]:

R = wrl.zr.z_to_r(Z, a=200., b=1.6)
print(np.round(R, 2))

[ 0.65  2.73 11.53 23.68 48.62 99.85]

Convert to rainfall depth (unit: mm) assuming a rainfall duration of five minutes (i.e. 300 seconds)

[5]:

depth = wrl.trafo.r_to_depth(R, 300)
print(np.round(depth, 2))

[0.05 0.23 0.96 1.97 4.05 8.32]

An example with real radar data¶

The following example is based on observations of the DWD C-band radar on mount Feldberg (SW-Germany). The figure shows a 15 minute accumulation of rainfall which was produced from three consecutive radar scans at 5 minute intervals between 17:30 and 17:45 on June 8, 2008.

The radar data are read using wradlib.io.read_dx function which returns an array of dBZ values and a metadata dictionary (see also Reading-DX-Data). The conversion is carried out the same way as in the example above. The plot is produced using the function wradlib.vis.plot_ppi.

[6]:

def read_data(dtimes):
    """Helper function to read raw data for a list of datetimes <dtimes>
    """
    data = np.empty((len(dtimes),360,128))
    for i, dtime in enumerate(dtimes):
        f = wrl.util.get_wradlib_data_file('dx/raa00-dx_10908-{0}-fbg---bin.gz'.format(dtime))
        data[i], attrs = wrl.io.read_dx(f)
    return data

Read data from radar Feldberg for three consecutive 5 minute intervals and compute the accumulated rainfall depth.

[7]:

# Read
dtimes = ["0806021735","0806021740","0806021745"]
dBZ = read_data(dtimes)
# Convert to rainfall intensity (mm/h)
Z = wrl.trafo.idecibel(dBZ)
R = wrl.zr.z_to_r(Z, a=200., b=1.6)
# Convert to rainfall depth (mm)
depth = wrl.trafo.r_to_depth(R, 300)
# Accumulate 15 minute rainfall depth over all three 5 minute intervals
accum = np.sum(depth, axis=0)

Plot PPI of 15 minute rainfall depth

[8]:

pl.figure(figsize=(10,8))
ax, cf = wrl.vis.plot_ppi(accum, cmap="viridis")
pl.xlabel("Easting from radar (km)")
pl.ylabel("Northing from radar (km)")
pl.title("Radar Feldberg\n15 min. rainfall depth, 2008-06-02 17:30-17:45 UTC")
cb = pl.colorbar(cf, shrink=0.8)
cb.set_label("mm")
pl.xlim(-128,128)
pl.ylim(-128,128)
pl.grid(color="grey")

<Figure size 720x576 with 0 Axes>

_images/notebooks_basics_wradlib_get_rainfall_18_1.png

Quick-view a sweep in polar or cartesian reference systems¶

[1]:

import numpy as np
import matplotlib.pyplot as pl
import wradlib
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()

Read a polar data set from the German Weather Service¶

[2]:

filename = wradlib.util.get_wradlib_data_file('dx/raa00-dx_10908-0806021735-fbg---bin.gz')
print(filename)

/home/travis/build/wradlib/wradlib-notebooks/wradlib-data/dx/raa00-dx_10908-0806021735-fbg---bin.gz

[3]:

img, meta = wradlib.io.read_dx(filename)

Inspect the data set a little

[4]:

print("Shape of polar array: %r\n" % (img.shape,))
print("Some meta data of the DX file:")
print("\tdatetime: %r" % (meta["datetime"],))
print("\tRadar ID: %s" % (meta["radarid"],))

Shape of polar array: (360, 128)

Some meta data of the DX file:
        datetime: datetime.datetime(2008, 6, 2, 17, 35, tzinfo=<UTC>)
        Radar ID: 10908

Simple ways to plot this dataset¶

The simplest plot¶

[5]:

ax, pm = wradlib.vis.plot_ppi(img)
txt = pl.title('Simple PPI')

_images/notebooks_visualisation_wradlib_plot_ppi_example_10_0.png

Simple plot with offset¶

[6]:

ax, pm = wradlib.vis.plot_ppi(img, site=(10, 45, 0))
txt = pl.title('Simple PPI with offset')

_images/notebooks_visualisation_wradlib_plot_ppi_example_12_0.png

Plot with azimuth/ranges and in curvelinear grid¶

[7]:

r = np.arange(img.shape[1], dtype=np.float)
r += (r[1] - r[0]) / 2.
r *= 1000
az = np.arange(img.shape[0], dtype=np.float)
az += (az[1] - az[0]) / 2.
fig = pl.figure(figsize=(16, 8))
ax1, pm = wradlib.vis.plot_ppi(img, r=r, az=az, ax=121, fig=fig)
ax2, pm = wradlib.vis.plot_ppi(img, r=r, az=az, proj='cg', ax=122, fig=fig)
t = ax1.set_title('Simple Plot with r/az')
t.set_y(1.05)
t = ax2.set_title('Simple Plot in cg grid')
t.set_y(1.05)

_images/notebooks_visualisation_wradlib_plot_ppi_example_14_0.png

Simple plot with applied range factor ‘rf’¶

[8]:

ax, pm = wradlib.vis.plot_ppi(img, r=r, az=az, rf=1e3)
txt = pl.title('Simple PPI with range factor')

_images/notebooks_visualisation_wradlib_plot_ppi_example_16_0.png

Simple plot in projected coordinate system¶

[9]:

r = np.arange(img.shape[1], dtype=np.float)
r += (r[1] - r[0]) / 2.
r *= 1000
az = np.arange(img.shape[0], dtype=np.float)
az += (az[1] - az[0]) / 2.
epsg=wradlib.georef.epsg_to_osr(31466)

fig = pl.figure(figsize=(10,10))
ax, pm = wradlib.vis.plot_ppi(img, r=r, az=az, site=(10, 45, 0), proj=epsg, fig=fig)
txt = pl.title('Simple PPI in Gauss-Krüger zone 2')

_images/notebooks_visualisation_wradlib_plot_ppi_example_18_0.png

Simple plot in WGS 84 coordinates¶

[10]:

proj=wradlib.georef.get_default_projection()
ax, pm = wradlib.vis.plot_ppi(img, r=r, az=az, site=(10, 45, 0), proj=proj)
txt = pl.title('Simple PPI in WGS84 coordinates')

_images/notebooks_visualisation_wradlib_plot_ppi_example_20_0.png

Plotting just one sector¶

For this purpose, we need to give the ranges and azimuths explicitly…

[11]:

r = np.arange(40, 81)
az = np.arange(200, 251)
ax, pm = wradlib.vis.plot_ppi(img[200:251, 40:81], r, az)
txt = pl.title('Sector PPI')

_images/notebooks_visualisation_wradlib_plot_ppi_example_23_0.png

Adding a crosshair to the PPI¶

[12]:

# We introduce a site offset...
site = (10., 45., 0)
wradlib.vis.plot_ppi(img, site=site)
# ... plot a crosshair over our data...
wradlib.vis.plot_ppi_crosshair(site=site, ranges=[50, 100, 128],
                               angles=[0, 90, 180, 270],
                               line=dict(color='white'),
                               circle={'edgecolor': 'white'},
                               )
pl.title('Offset and Custom Crosshair')
pl.axis("tight")
pl.gca().set_aspect('equal')

_images/notebooks_visualisation_wradlib_plot_ppi_example_25_0.png

Placing the polar data in a projected Cartesian reference system¶

Using the proj keyword we tell the function to: - interpret the site coordinates as longitude/latitude - reproject the coordinates to the given projection (here: dwd-radolan composite coordinate system)

[13]:

site=(10., 45., 0)
r = np.arange(img.shape[1], dtype=np.float)
r += (r[1] - r[0]) / 2.
r *= 1000
az = np.arange(img.shape[0], dtype=np.float)
az += (az[1] - az[0]) / 2.
elev = np.zeros_like(az)
proj_rad = wradlib.georef.create_osr("dwd-radolan")
wradlib.vis.plot_ppi(img, r=r, az=az, site=site, proj=proj_rad)
# Now the crosshair ranges must be given in meters
wradlib.vis.plot_ppi_crosshair(site=site,
                               ranges=[40000, 80000, 128000],
                               line=dict(color='white'),
                               circle={'edgecolor':'white'},
                               proj=proj_rad
                               )
pl.title('Georeferenced/Projected PPI')
pl.axis("tight")
pl.gca().set_aspect('equal')

_images/notebooks_visualisation_wradlib_plot_ppi_example_28_0.png

Some side effects of georeferencing¶

Transplanting the radar virtually moves it away from the central meridian of the projection (which is 10 degrees east). Due north now does not point straight upwards on the map.

The crosshair shows this: for the case that the lines should actually become curved, they are implemented as a piecewise linear curve with 10 vertices. The same is true for the range circles, but with more vertices, of course.

[14]:

site=(45., 7., 0.)
ax, pm = wradlib.vis.plot_ppi(img, r=r, az=az, site=site, proj=proj_rad)
ax = wradlib.vis.plot_ppi_crosshair(site=site,
                               ranges=[64000, 128000],
                               line=dict(color='red'),
                               circle={'edgecolor': 'red'},
                               proj=proj_rad
                               )
txt = pl.title('Projection Side Effects')

_images/notebooks_visualisation_wradlib_plot_ppi_example_31_0.png

Simple Plot on Mercator-Map using cartopy¶

[15]:

import cartopy.crs as ccrs
map_proj = ccrs.Mercator(central_longitude=site[1])

[16]:

site=(7, 45, 0.)
fig = pl.figure(figsize=(10,10))
ax, pm = wradlib.vis.plot_ppi(img, r=r, az=az, site=site, proj=map_proj, fig=fig)
ax.gridlines(draw_labels=True)

[16]:

<cartopy.mpl.gridliner.Gridliner at 0x7f68802a3460>

_images/notebooks_visualisation_wradlib_plot_ppi_example_34_1.png

More decorations and annotations¶

You can annotate these plots by using standard matplotlib methods.

[17]:

ax, pm = wradlib.vis.plot_ppi(img)
ylabel = ax.set_xlabel('easting [km]')
ylabel = ax.set_ylabel('northing [km]')
title = ax.set_title('PPI manipulations/colorbar')
# you can now also zoom - either programmatically or interactively
xlim = ax.set_xlim(-80, -20)
ylim = ax.set_ylim(-80, 0)
# as the function returns the axes- and 'mappable'-objects colorbar needs, adding a colorbar is easy
cb = pl.colorbar(pm, ax=ax)

_images/notebooks_visualisation_wradlib_plot_ppi_example_37_0.png

A one hour tour of wradlib¶

caption

A guided tour of some $\omega radlib$ notebooks.

(find all wradlib notebooks in the `docs <https://docs.wradlib.org/en/1.7.0/notebooks.html>`__.)

Some background, first¶

Development started in 2011…or more precisely:

October 26th, 2011

Key motivation¶

A community platform for collaborative development of algorithms

Your entry points¶

Start out from wradlib.org ¶

Documentation¶

Check out the online docs with tutorials and examples and a comprehensive library reference

User group¶

Get help and connect more than 120 users at the wradlib user group!

For developers¶

Fork us from https://github.com/wradlib/wradlib or raise an issue!

Installation¶

1. Install Anaconda or Miniconda¶

Get it here for Windows, Linux, or Mac.

2. Create environment, add conda-forge, install wradlib¶

$ conda config --add channels conda-forge
$ conda create --name newenv python=3.6
$ source activate newenv
(newenv) $ conda install wradlib

To run our tutorials…¶

Get notebooks
Get sample data
Set environment variable WRADLIB_DATA

Development paradigm¶

Keep the magic to a minimum¶

transparent
flexible, but lower level

Flat (or no) data model¶

pass data as numpy arrays,
and pass metadata as dictionaries.

Import wradlib¶

[1]:

import wradlib

[2]:

# check installed version
print(wradlib.__version__)

1.7.0

In the next cell, type wradlib. and hit Tab.

Inpect the available modules and functions.

[ ]:

Reading and viewing data¶

Zoo of file formats¶

This notebook shows you how to access various file formats.

Addressing observational errors and artefacts¶

Attenuation¶

In this example, we reconstruct path-integrated attenuation from single-pol data of the German Weather Service.

Clutter detection¶

wradlib provides several methods for clutter detection. Here, we look at an example that uses dual-pol moments and a simple fuzzy classification.

Partial beam blockage¶

In this example, wradlib attempts to quantify terrain-induced beam blockage from a DEM.

Integration with other geodata¶

Average precipitation over your river catchment¶

In this example, we compute zonal statistics over polygons imported in a shapefile.

Over and underlay of other geodata¶

Often, you need to present your radar data in context with other geodata (DEM, rivers, gauges, catchments, …).

Merging with other sensors¶

Adjusting radar-based rainfall estimates by rain gauges¶

In this example, we use synthetic radar and rain gauge observations and confront them with different adjustment techniques.

Attenuation correction¶

In this example, we will compare different approaches to constrain the gate-by-gate retrieval of path-integrated attenuation.

Introduction¶

Rainfall-induced attenuation is a major source of underestimation for radar-based precipitation estimation at C-band and X-band. Unconstrained forward gate-by-gate correction is known to be inherently unstable and thus not suited for unsupervised quality control procedures. Ideally, reference measurements (e.g. from microwave links) should be used to constrain gate-by-gate procedures. However, such attenuation references are usually not available. $\omega radlib$ provides a pragmatic approach to constrain gate-by-gate correction procedures, inspired by the work of Kraemer et al., 2008. It turned out that these procedures can effectively reduce the error introduced by attenuation, and, at the same time, minimize instability issues (Jacobi et al., 2016).

The Example Event: June 2nd, 2008, SW-Germany¶

Let’s have a look at the situation in South-West Germany on June 2nd, 2008, at 16:55 UTC, as observed by the DWD C-band radar on mount Feldberg.

The data can be read by the following lines and then visualized by wradlib.vis.plot_ppi():

[2]:

fpath = 'dx/raa00-dx_10908-0806021655-fbg---bin.gz'
f = wrl.util.get_wradlib_data_file(fpath)
data, attrs = wrl.io.read_dx(f)

[3]:

pl.figure(figsize=(10,8))
ax, cf = wrl.vis.plot_ppi(data, cmap="viridis")
pl.xlabel("Easting from radar (km)")
pl.ylabel("Northing from radar (km)")
pl.title("Radar Feldberg, 2008-06-02 16:55 UTC")
cb = pl.colorbar(cf, shrink=0.8)
cb.set_label("dBZ")
pl.plot([0,105.6],[0,73.4],"-", color="white", lw=2)
pl.xlim(-128,128)
pl.ylim(-128,128)
pl.grid(color="grey")

<Figure size 720x576 with 0 Axes>

_images/notebooks_attenuation_wradlib_attenuation_9_1.png

We see a set of convective cells with high rainfall intensity in the NE-sector of the Feldberg radar. Let us examine the reflectivity profile along three beams which at azimuths 53-55 degree (as marked by the white line in the PPI above).

[4]:

# just a little helper function
def plot_beams(data, mybeams, sub=111):
    ax = fig.add_subplot(sub)
    labelsize=13
    for beam in range(mybeams.start, mybeams.stop):
        pl.plot(data[beam], label="{0} deg".format(beam))
    pl.grid()
    pl.text(0.99, 0.88, "Reflectivity along beams",
            horizontalalignment='right',
            transform = ax.transAxes, fontsize="large")
    pl.xlabel("range (km)", fontsize="large")
    pl.ylabel("Reflectivity (dBZ)", fontsize="large")
    pl.legend(loc="upper left")
    ax.tick_params(axis='x', labelsize=labelsize)
    ax.tick_params(axis='y', labelsize=labelsize)
    pl.xlim(0,128)

[5]:

mybeams = slice(53,56)
fig = pl.figure(figsize=(10,3))
plot_beams(data, mybeams)

_images/notebooks_attenuation_wradlib_attenuation_12_0.png

Examining different attenuation correction procedures¶

Hitschfeld and Bordan¶

Unconstrained gate-by-gate retrieval (1954)

First, we examine the behaviour of the “classical” unconstrained forward correction which is typically referred to Hitschfeld et al., 1954, although Hitschfeld and Bordan themselves rejected this approach. The Path Integrated Attenuation (PIA) according to this approach can be obtained as follows:

[6]:

pia_hibo = wrl.atten.correct_attenuation_hb(
    data,
    coefficients = dict(a=8.e-5, b=0.731, gate_length=1.0),
    mode="warn",
    thrs=59.)

corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)
corrected signal over threshold (59.0)

In the coefficients dictionary, we can pass the power law parameters of the A(Z) relation as well as the gate length (in km). If we pass “warn” as the mode argument, we will obtain a warning log in case the corrected reflectivity exceeds the value of argument thrs (dBZ).

Plotting the result below the reflectivity profile, we obtain the following figure.

[7]:

# another little helper function
def plot_pia(pia, sub=111, title=None):
    ax = fig.add_subplot(sub)
    labelsize=13
    pl.plot(pia.T)
    pl.grid()
    pl.ylim(0,30)
    pl.ylabel("PIA (dB)", fontsize="large")
    pl.text(0.01, 0.88, title,
            transform = ax.transAxes, fontsize="large")
    ax.tick_params(axis='x', labelsize=labelsize)
    ax.tick_params(axis='y', labelsize=labelsize)
    pl.xlim(0,128)

[8]:

fig = pl.figure(figsize=(10,6))
plot_beams(data, mybeams, 211)
plot_pia(pia_hibo[mybeams], 212,
         "PIA according to Hitchfeld and Bordan")
pl.tight_layout()

_images/notebooks_attenuation_wradlib_attenuation_19_0.png

Apparently, slight differences in the reflectivity profile can cause a dramatic change in the behaviour. While at 54 and 55 degrees, the retrieval of PIA appears to be fairly stable, the profile of PIA for 53 degree demonstrates a case of instability.

Harrison¶

Harrison et.al. (2000): Cap PIA at a factor of two

Harrison et al., 2000 suggested to simply cap PIA in case it would cause a correction of rainfall intensity by more than a factor of two. Depending on the parameters of the Z(R) relationship, that would correpond to PIA values between 4 and 5 dB (4.8 dB if we assume exponent b=1.6).

One way to implement this approach would be the following:

[9]:

pia_harrison = wrl.atten.correct_attenuation_hb(
    data,
    coefficients = dict(a=4.57e-5, b=0.731, gate_length=1.0),
    mode="warn",
    thrs=59.)
pia_harrison[pia_harrison > 4.8] = 4.8

And the results would look like this:

[10]:

fig = pl.figure(figsize=(10,6))

mybeams = slice(53,56)
labelsize=13

plot_beams(data, mybeams, 211)
plot_pia(pia_harrison[mybeams], 212,
         "PIA according to Harrison")
pl.tight_layout()

_images/notebooks_attenuation_wradlib_attenuation_25_0.png

Kraemer¶

Kraemer et al. (2008): Iterative estimation of A(Z) relationsip

Kraemer et al., 2008 suggested to iteratively determine the power law parameters of the A(Z). In particular, the power law coefficient is interatively decreased until the attenuation correction does not lead to reflectivity values above a given threshold (Kraemer suggested 59 dBZ). Using $\omega radlib$, this would be called by using the function wradlib.atten.correct_attenuation_constrained() with a specific constraints argument:

[11]:

pia_kraemer = wrl.atten.correct_attenuation_constrained(
    data,
    a_max=1.67e-4,
    a_min=2.33e-5,
    n_a=100,
    b_max=0.7,
    b_min=0.65,
    n_b=6,
    gate_length=1.,
    constraints=[wrl.atten.constraint_dbz],
    constraint_args=[[59.0]])

In brief, this call specifies ranges of the power parameters a and b of the A(Z) relation. Beginning from the maximum values (a_max and b_max), the function searches for values of a and b so that the corrected reflectivity will not exceed the dBZ constraint of 59 dBZ. Compared to the previous results, the corresponding profiles of PIA look like this:

[12]:

fig = pl.figure(figsize=(10,6))
plot_beams(data, mybeams, 211)
plot_pia(pia_kraemer[mybeams], 212,
         "PIA according to Kraemer")
pl.tight_layout()

_images/notebooks_attenuation_wradlib_attenuation_30_0.png

Modified Kraemer¶

Generalised Kraemer procedure: adding additional constraints

The function wradlib.atten.correct_attenuation_constrained() allows us to pass any kind of constraint function or lists of constraint functions via the argument constraints. The arguments of these functions are passed via a nested list as argument constraint_args. For example, Jacobi et al., 2016 suggested to constrain both the corrected reflectivity (by a maximum of 59 dBZ) and the resulting path-intgrated attenuation PIA (by a maximum of 20 dB):

[13]:

pia_mkraemer = wrl.atten.correct_attenuation_constrained(
    data,
    a_max=1.67e-4,
    a_min=2.33e-5,
    n_a=100,
    b_max=0.7,
    b_min=0.65,
    n_b=6,
    gate_length=1.,
    constraints=[wrl.atten.constraint_dbz,
                 wrl.atten.constraint_pia],
    constraint_args=[[59.0],[20.0]])

[14]:

fig = pl.figure(figsize=(10,6))
plot_beams(data, mybeams, 211)
plot_pia(pia_mkraemer[mybeams], 212,
         "PIA according to modified Kraemer")
pl.tight_layout()

_images/notebooks_attenuation_wradlib_attenuation_34_0.png

Comparison of all Methods¶

Plotting all of the above methods (Hitschfeld and Bordan, Harrison, Kraemer, Modified Kraemer allows for a better comparison of their behaviour. Please refer to Jacobi et al., 2016 for an in-depth discussion of this example.

[15]:

fig = pl.figure(figsize=(10,12))

plot_beams(data, mybeams, 511)
plot_pia(pia_hibo[mybeams], 512,
         "PIA according to Hitschfeld and Bordan" )
plot_pia(pia_harrison[mybeams], 513,
         "PIA according to Harrison")
plot_pia(pia_kraemer[mybeams], 514,
         "PIA according to Kraemer")
plot_pia(pia_mkraemer[mybeams], 515,
         "PIA according to modified Kraemer")
pl.tight_layout()

_images/notebooks_attenuation_wradlib_attenuation_37_0.png

[ ]:

Clutter and Echo Classification¶

This section provides a collection of example code snippets to show clutter detection and correction as well as echo classification capabilities of $\omega radlib$.

Clutter detection using the Gabella approach¶

[1]:

import matplotlib.pyplot as pl
import numpy as np
import wradlib.vis as vis
import wradlib.clutter as clutter
import wradlib.util as util
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

Read the data¶

[2]:

filename = util.get_wradlib_data_file('misc/polar_dBZ_fbg.gz')
data = np.loadtxt(filename)

Apply filter¶

[3]:

clmap = clutter.filter_gabella(data,
                               wsize=5,
                               thrsnorain=0.,
                               tr1=6.,
                               n_p=8,
                               tr2=1.3)

Plot results¶

[4]:

fig = pl.figure(figsize=(12,8))
ax = fig.add_subplot(121)
ax, pm = vis.plot_ppi(data, ax=ax)
ax.set_title('Reflectivity')
ax = fig.add_subplot(122)
ax, pm = vis.plot_ppi(clmap, ax=ax)
ax.set_title('Cluttermap')

[4]:

Text(0.5, 1.0, 'Cluttermap')

_images/notebooks_classify_wradlib_clutter_gabella_example_8_1.png

[ ]:

Clutter detection by using space-born cloud images¶

[1]:

import numpy as np

import wradlib.vis as vis
import wradlib.clutter as cl
import wradlib.georef as georef
import wradlib.ipol as ipol
import wradlib.io as io
import wradlib.util as util
import matplotlib.pyplot as plt
try:
    get_ipython().magic("matplotlib inline")
except:
    plt.ion()

Read the radar data and count the number of tilts¶

[2]:

# read the radar volume scan
filename = 'hdf5/20130429043000.rad.bewid.pvol.dbzh.scan1.hdf'
filename = util.get_wradlib_data_file(filename)
pvol = io.read_opera_hdf5(filename)

# Count the number of dataset

ntilt = 1
for i in range(100):
    try:
        pvol["dataset%d/what" % ntilt]
        ntilt += 1
    except Exception:
        ntilt -= 1
        break

Reconstruct the radar values¶

[3]:

nrays = int(pvol["dataset1/where"]["nrays"])
nbins = int(pvol["dataset1/where"]["nbins"])
val = np.empty((ntilt, nrays, nbins))
for t in range(ntilt):
    val[t, ...] = pvol["dataset%d/data1/data" % (t + 1)]
gain = float(pvol["dataset1/data1/what"]["gain"])
offset = float(pvol["dataset1/data1/what"]["offset"])
val = val * gain + offset

Construct the corresponding radar coordinates¶

[4]:

rscale = int(pvol["dataset1/where"]["rscale"])
coord = np.empty((ntilt, nrays, nbins, 3))
for t in range(ntilt):
    elangle = pvol["dataset%d/where" % (t + 1)]["elangle"]
    coord[t, ...] = georef.sweep_centroids(nrays, rscale, nbins, elangle)
# ascale = math.pi / nrays
sitecoords = (pvol["where"]["lon"], pvol["where"]["lat"],
              pvol["where"]["height"])

coord, proj_radar = georef.spherical_to_xyz(coord[..., 0],
                                            coord[..., 1],
                                            coord[..., 2],
                                            sitecoords,
                                            re=6370040.,
                                            ke=4./3.)

/home/travis/miniconda/envs/travis_wradlib/lib/python3.8/site-packages/wradlib/georef/polar.py:142: DeprecationWarning: Function `spherical_to_xyz` returns an array of shape (theta, phi, range, 3). Use `squeeze=True` to remove singleton dimensions.
  warnings.warn("Function `spherical_to_xyz` returns an array "

Construct collocated satellite data¶

[5]:

filename = 'hdf5/SAFNWC_MSG3_CT___201304290415_BEL_________.h5'
filename = util.get_wradlib_data_file(filename)
sat_gdal = io.read_safnwc(filename)
val_sat = georef.read_gdal_values(sat_gdal)
coord_sat = georef.read_gdal_coordinates(sat_gdal)
proj_sat = georef.read_gdal_projection(sat_gdal)
coord_sat = georef.reproject(coord_sat, projection_source=proj_sat,
                             projection_target=proj_radar)
coord_radar = coord
interp = ipol.Nearest(coord_sat[..., 0:2].reshape(-1, 2),
                      coord_radar[..., 0:2].reshape(-1, 2))
val_sat = interp(val_sat.ravel()).reshape(val.shape)

Estimate localisation errors¶

[6]:

timelag = 9 * 60
wind = 10
error = np.absolute(timelag) * wind

Identify clutter based on collocated cloudtype¶

[7]:

clutter = cl.filter_cloudtype(val[0, ...], val_sat[0, ...],
                              scale=rscale, smoothing=error)

Plot the results¶

[8]:

fig = plt.figure(figsize=(16,8))

ax = fig.add_subplot(131)
ax, pm = vis.plot_ppi(val[0, ...], ax=ax)
plt.colorbar(pm, shrink=0.5)
plt.title('Radar reflectivity')

ax = fig.add_subplot(132)
ax, pm = vis.plot_ppi(val_sat[0, ...], ax=ax)
plt.colorbar(pm, shrink=0.5)
plt.title('Satellite cloud classification')

ax = fig.add_subplot(133)
ax, pm = vis.plot_ppi(clutter, ax=ax)
plt.title('Detected clutter')

[8]:

Text(0.5, 1.0, 'Detected clutter')

_images/notebooks_classify_wradlib_clutter_cloud_example_16_1.png

Heuristic clutter detection based on distribution properties (“histo cut”)¶

Detects areas with anomalously low or high average reflectivity or precipitation. It is recommended to use long term average or sums (months to year).

[1]:

import wradlib.clutter as clutter
from wradlib.vis import plot_ppi
import wradlib.util as util
import numpy as np
import matplotlib.pyplot as pl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()

Load annual rainfall acummulation example (from DWD radar Feldberg)¶

[2]:

filename = util.get_wradlib_data_file('misc/annual_rainfall_fbg.gz')
yearsum = np.loadtxt(filename)

Apply histo-cut filter to retrieve boolean array that highlights clutter as well as beam blockage¶

[3]:

mask = clutter.histo_cut(yearsum)

Plot results¶

[4]:

fig = pl.figure(figsize=(14,8))
ax = fig.add_subplot(121)
ax, pm = plot_ppi(np.log(yearsum), ax=ax)
pl.title("Logarithm of annual precipitation sum")
pl.colorbar(pm, shrink=0.75)
ax = fig.add_subplot(122)
ax, pm = plot_ppi(mask.astype(np.uint8), ax=ax)
pl.title("Map of execptionally low and high values\n(clutter and beam blockage)")
pl.colorbar(pm, shrink=0.75)

[4]:

<matplotlib.colorbar.Colorbar at 0x7f762cbfb070>

_images/notebooks_classify_wradlib_histo_cut_example_8_1.png

This notebook is part of the $\omega radlib$ documentation: https://docs.wradlib.org.

Copyright (c) $\omega radlib$ developers. Distributed under the MIT License. See LICENSE.txt for more info.

Simple fuzzy echo classification from dual-pol moments¶

[1]:

import wradlib
from wradlib.util import get_wradlib_data_file
import os
import numpy as np
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    plt.ion()

Setting the file paths¶

[2]:

rhofile = get_wradlib_data_file('netcdf/TAG-20120801-140046-02-R.nc')
phifile = get_wradlib_data_file('netcdf/TAG-20120801-140046-02-P.nc')
reffile = get_wradlib_data_file('netcdf/TAG-20120801-140046-02-Z.nc')
dopfile = get_wradlib_data_file('netcdf/TAG-20120801-140046-02-V.nc')
zdrfile = get_wradlib_data_file('netcdf/TAG-20120801-140046-02-D.nc')
mapfile = get_wradlib_data_file('hdf5/TAG_cmap_sweeps_0204050607.hdf5')

Read the data (radar moments and static clutter map)¶

[3]:

# We need to organize our data as a dictionary
dat = {}
dat["rho"], attrs_rho = wradlib.io.read_edge_netcdf(rhofile)
dat["phi"], attrs_phi = wradlib.io.read_edge_netcdf(phifile)
dat["ref"], attrs_ref = wradlib.io.read_edge_netcdf(reffile)
dat["dop"], attrs_dop = wradlib.io.read_edge_netcdf(dopfile)
dat["zdr"], attrs_zdr = wradlib.io.read_edge_netcdf(zdrfile)
dat["map"]            = wradlib.io.from_hdf5(mapfile)[0][0]

Identify non-meteorological echoes using fuzzy echo classification¶

See Crisologo et al. (2015) and Vulpiani et al. (2012) for details.

[4]:

weights = {"zdr": 0.4,
           "rho": 0.4,
           "rho2": 0.4,
           "phi": 0.1,
           "dop": 0.1,
           "map": 0.5}
cmap, nanmask = wradlib.clutter.classify_echo_fuzzy(dat,
                                                    weights=weights,
                                                    thresh=0.5)

View classfication results¶

[5]:

fig = plt.figure(figsize=(18,16))

#   Horizontal reflectivity
ax = plt.subplot(121, aspect="equal")
ax, pm = wradlib.vis.plot_ppi(np.ma.masked_invalid(dat["ref"]), ax=ax)
ax = wradlib.vis.plot_ppi_crosshair(site=(0,0,0),
                                    ranges=[80,160,240])
plt.xlim(-240,240)
plt.ylim(-240,240)
plt.xlabel("# bins from radar")
plt.ylabel("# bins from radar")
cbar = plt.colorbar(pm, shrink=0.3)
cbar.set_label("dBZ", fontsize = "large")

#   Echo classification
ax = plt.subplot(122, aspect="equal")
ax, pm = wradlib.vis.plot_ppi(np.ma.masked_array(cmap.astype(np.uint8),
                                                 np.isnan(dat["ref"])),
                              ax=ax, cmap="bwr")
ax = wradlib.vis.plot_ppi_crosshair(site=(0,0,0),
                                    ranges=[80,160,240])
plt.xlim(-240,240)
plt.ylim(-240,240)
plt.xlabel("# bins from radar")
plt.ylabel("# bins from radar")
cbar = plt.colorbar(pm, shrink=0.3)
cbar.set_label("meterol. echo=0 - non-meteorol. echo=1",
               fontsize = "large")

_images/notebooks_classify_wradlib_fuzzy_echo_classify_11_0.png

This notebook is part of the $\omega radlib$ documentation: https://docs.wradlib.org.

Copyright (c) $\omega radlib$ developers. Distributed under the MIT License. See LICENSE.txt for more info.

Hydrometeorclassification¶

[1]:

import wradlib as wrl
import os
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as pl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
from scipy import interpolate
import datetime as dt
import glob

The hydrometeorclassification code is based on the paper by Zrnic et.al 2001 utilizing 2D trapezoidal membership functions based on the paper by Straka et. al 2000 adapted by Evaristo et. al 2013 for X-Band.

Precipitation Types¶

[2]:

pr_types = wrl.classify.pr_types
for k, v in pr_types.items():
    print(str(k) + ' - '.join(v))

0LR - Light Rain
1MR - Moderate Rain
2HR - Heavy Rain
3LD - Large Drops
4HL - Hail
5RH - Rain/Hail
6GH - Graupel/Hail
7DS - Dry Snow
8WS - Wet Snow
9HC - H Crystals
10VC - V Crystals
11NP - No Precip

Membership Functions¶

Load 2D Membership Functions¶

[3]:

filename = wrl.util.get_wradlib_data_file('misc/msf_xband.gz')
msf = wrl.io.get_membership_functions(filename)

Plot 2D Membership Functions¶

[4]:

obs = ['reflectivity (dBZ)', 'differential reflectivity (dB)',
       'correlation coefficient', 'specific differential phase (degrees/km)',
       'temperature (degrees celsius)']
minmax = [(-10, 100),(-1, 6), (0.3, 1.0), (-5, 35), (-65, 45)]

for i, v in enumerate(msf):
    fig = pl.figure(figsize=(10, 8))
    t = fig.suptitle(' - '.join(pr_types[i]))
    t.set_y(1.02)
    for k, p in enumerate(v):
        ax = fig.add_subplot(3,2,k+1)
        ax.plot(p[:,0],
                np.ma.masked_equal(p[:,1], 0.),
                'k', lw=1.0)
        ax.plot(p[:,0],
                np.ma.masked_equal(p[:,2], 0.),
                'k', lw=2.0)
        ax.plot(p[:,0],
                np.ma.masked_equal(p[:,3], 0.),
                'k', lw=2.0)
        ax.plot(p[:,0],
                np.ma.masked_equal(p[:,4], 0.),
                'k', lw=1.0)

        ax.set_xlim((p[:,0].min(), p[:,0].max()))
        ax.margins(x=0.05, y=0.05)
        ax.set_xlabel('ZH - Reflectivity [dBZ]')
        ax.set_ylabel(obs[k])
        t = ax.set_title(obs[k])
        ax.set_ylim(minmax[k])
    fig.tight_layout()
pl.show()

_images/notebooks_classify_wradlib_2d_hmc_10_0.png

_images/notebooks_classify_wradlib_2d_hmc_10_1.png

_images/notebooks_classify_wradlib_2d_hmc_10_2.png

_images/notebooks_classify_wradlib_2d_hmc_10_3.png

_images/notebooks_classify_wradlib_2d_hmc_10_4.png

_images/notebooks_classify_wradlib_2d_hmc_10_5.png

_images/notebooks_classify_wradlib_2d_hmc_10_6.png

_images/notebooks_classify_wradlib_2d_hmc_10_7.png

_images/notebooks_classify_wradlib_2d_hmc_10_8.png

_images/notebooks_classify_wradlib_2d_hmc_10_9.png

_images/notebooks_classify_wradlib_2d_hmc_10_10.png

Use Sounding Data¶

Retrieve Sounding Data¶

To get the temperature as additional discriminator we use radiosonde data from the University of Wyoming.

The function get_radiosonde tries to find the next next available radiosonde measurement on the given date.

[5]:

rs_time = dt.datetime(2014, 6, 10, 12, 0)
import urllib
try:
    rs_data, rs_meta = wrl.io.get_radiosonde(10410, rs_time)
except urllib.error.HTTPError:
    dataf = wrl.util.get_wradlib_data_file('misc/radiosonde_10410_20140610_1200.h5')
    rs_data, _ = wrl.io.from_hdf5(dataf)
    metaf = wrl.util.get_wradlib_data_file('misc/radiosonde_10410_20140610_1200.json')
    with open(metaf, 'r') as infile:
        import json
        rs_meta = json.load(infile)
rs_meta

[5]:

{'Station identifier': 'EDZE',
 'Station number': 10410,
 'Observation time': datetime.datetime(2014, 6, 10, 12, 0),
 'Station latitude': 51.4,
 'Station longitude': 6.96,
 'Station elevation': 153.0,
 'Showalter index': 1.65,
 'Lifted index': -5.84,
 'LIFT computed using virtual temperature': -6.23,
 'SWEAT index': 89.02,
 'K index': 23.7,
 'Cross totals index': 17.3,
 'Vertical totals index': 31.3,
 'Totals totals index': 48.6,
 'Convective Available Potential Energy': 1536.47,
 'CAPE using virtual temperature': 1637.41,
 'Convective Inhibition': -140.01,
 'CINS using virtual temperature': -80.22,
 'Equilibrum Level': 202.61,
 'Equilibrum Level using virtual temperature': 202.55,
 'Level of Free Convection': 735.93,
 'LFCT using virtual temperature': 772.57,
 'Bulk Richardson Number': 57.19,
 'Bulk Richardson Number using CAPV': 60.94,
 'Temp [K] of the Lifted Condensation Level': 288.2,
 'Pres [hPa] of the Lifted Condensation Level': 882.1,
 'Equivalent potential temp [K] of the LCL': 334.86,
 'Mean mixed layer potential temperature': 298.74,
 'Mean mixed layer mixing ratio': 12.41,
 '1000 hPa to 500 hPa thickness': 5657.0,
 'Precipitable water [mm] for entire sounding': 28.11,
 'quantity': {'PRES': 'hPa',
  'HGHT': 'm',
  'TEMP': 'C',
  'DWPT': 'C',
  'FRPT': 'C',
  'RELH': '%',
  'RELI': '%',
  'MIXR': 'g/kg',
  'DRCT': 'deg',
  'SKNT': 'knot',
  'THTA': 'K',
  'THTE': 'K',
  'THTV': 'K'}}

Extract Temperature and Height¶

[6]:

stemp = rs_data['TEMP']
sheight = rs_data['HGHT']
# remove nans
idx = np.isfinite(stemp)
stemp = stemp[idx]
sheight = sheight[idx]

Interpolate to higher resolution¶

[7]:

# highres height
hmax = 30000.
ht = np.arange(0., hmax)

ipolfunc = interpolate.interp1d(sheight, stemp, kind='linear',
                                bounds_error=False)
itemp = ipolfunc(ht)

Fix Temperature below first measurement¶

[8]:

first = np.nanargmax(itemp)
itemp[0:first] = itemp[first]

Plot Temperature Profile¶

[9]:

fig = pl.figure(figsize=(5,10))
ax = fig.add_subplot(111)
l1 = ax.plot(itemp, ht/1000, 'r.')
l2 = ax.plot(stemp, sheight/1000, 'bo')
ax.set_xlabel('Temperature [°C]')
ax.set_ylabel('Height [km]')
ax.set_ylim(0, hmax/1000)
ax.grid()
pl.show()

Prepare Radar Data¶

Load Radar Data¶

[10]:

# read the radar volume scan
filename = 'hdf5/2014-06-09--185000.rhi.mvol'
filename = wrl.util.get_wradlib_data_file(filename)

Extract data for georeferencing¶

[11]:

data, meta = wrl.io.read_gamic_hdf5(filename)
r = meta['SCAN0']['r'] - meta['SCAN0']['bin_range'] / 2.
el = meta['SCAN0']['el']
az = meta['SCAN0']['az']
rays = meta['SCAN0']['ray_count']
bins = meta['SCAN0']['bin_count']
sitecoords = (meta['VOL']['Longitude'], meta['VOL']['Latitude'],
              meta['VOL']['Height'])

Get Heights of Radar Bins¶

[12]:

re = wrl.georef.get_earth_radius(sitecoords[1])
print(r.shape, el.shape)
bheight = wrl.georef.bin_altitude(r[:, np.newaxis], el, sitecoords[2], re).T
print(bheight.shape)

(667,) (450,)
(450, 667)

Plot RHI of Heights¶

[13]:

fig = pl.figure(figsize=(8, 7))
cmap = mpl.cm.viridis
cgax, im = wrl.vis.plot_rhi(bheight/1000., r=r/1000., th=el,
                            proj='cg', cmap=cmap,
                            ax=111, fig=fig)
cbar = pl.colorbar(im, fraction=0.046, pad=0.1)
cbar.set_label('Height [km]')
caax = cgax.parasites[0]
caax.set_xlabel('Range [km]')
caax.set_ylabel('Range [km]')
pl.show()

_images/notebooks_classify_wradlib_2d_hmc_30_0.png

Get Index into High Res Height Array¶

[14]:

idx = np.digitize(bheight, ht)
print(idx.shape, ht.shape, itemp.shape)
rtemp = itemp[idx-1]
print(rtemp.min())

(450, 667) (30000,) (30000,)
-66.1

Plot RHI of Temperatures¶

[15]:

fig = pl.figure(figsize=(10,5))
cgax, im = wrl.vis.plot_rhi(rtemp, r=r/1000., th=el,
                            proj='cg', cmap=cmap,
                            ax=111, fig=fig)
cbar = pl.colorbar(im, fraction=0.046, pad=0.1)
cbar.set_label('Temperature [°C]')
caax = cgax.parasites[0]
caax.set_xlabel('Range [km]')
caax.set_ylabel('Range [km]')
cgax.set_ylim(0, hmax/1000)
pl.show()

_images/notebooks_classify_wradlib_2d_hmc_34_0.png

HMC Workflow¶

Stack Observables¶

[16]:

hmca = np.vstack((data['SCAN0']['ZH']['data'][np.newaxis,...],
                  data['SCAN0']['ZDR']['data'][np.newaxis,...],
                  data['SCAN0']['RHOHV']['data'][np.newaxis,...],
                  data['SCAN0']['KDP']['data'][np.newaxis,...],
                  rtemp[np.newaxis,...]))
print(hmca.shape)

(5, 450, 667)

Setup Independent Observable $Z_H$¶

[17]:

msf_idp = msf[0, 0, :, 0]
msf_obs = msf[..., 1:]

Retrieve membership function values based on independent observable¶

[18]:

msf_val = wrl.classify.msf_index_indep(msf_obs, msf_idp,
                                       hmca[0])

Fuzzyfication¶

[19]:

fu = wrl.classify.fuzzyfi(msf_val, hmca)

Probability¶

[20]:

# Weights
w = np.array([2., 1., 1., 1., 1.])
prob = wrl.classify.probability(fu, w)

Classification¶

[21]:

hmc_idx, hmc_vals = wrl.classify.classify(prob, threshold=0.)

Plot HMC Results¶

Plot Probability of HMC Types¶

[22]:

fig = pl.figure(figsize=(30,20))
import matplotlib.gridspec as gridspec
gs = gridspec.GridSpec(3, 4, wspace=0.4, hspace=0.4)
for i, c in enumerate(prob):
    cgax, im = wrl.vis.plot_rhi(np.ma.masked_less_equal(c, 0.),
                                r=r/1000., th=el,
                                proj='cg',
                                ax=gs[i], fig=fig)
    cbar = pl.colorbar(im, fraction=0.046, pad=0.15)
    cbar.set_label('Probability')
    caax = cgax.parasites[0]
    caax.set_xlabel('Range [km]')
    caax.set_ylabel('Range [km]')
    t = cgax.set_title(' - '.join(pr_types[i]))
    t.set_y(1.1)
    cgax.set_xlim(0,40)
    cgax.set_ylim(0,14)
pl.show()

_images/notebooks_classify_wradlib_2d_hmc_50_0.png

Plot maximum probability¶

[23]:

fig = pl.figure(figsize=(10, 8))
cgax, im = wrl.vis.plot_rhi(hmc_vals[11], ax=111,
                            proj='cg',
                            r=r/1000., th=el,
                            cmap='cubehelix', fig=fig)
cbar = pl.colorbar(im, fraction=0.046, pad=0.1)
cbar.set_label('Probability')
caax = cgax.parasites[0]
caax.set_xlabel('Range [km]')
caax.set_ylabel('Range [km]')
t = cgax.set_title('Hydrometeorclassification')
t.set_y(1.05)
cgax.set_xlim(0,40)
cgax.set_ylim(0,14)
pl.show()

_images/notebooks_classify_wradlib_2d_hmc_52_0.png

Plot classification result¶

[24]:

bounds = np.arange(-0.5, prob.shape[0] + 0.6, 1)
ticks=np.arange(0, prob.shape[0] + 1)
cmap = mpl.cm.get_cmap('cubehelix', len(ticks))
norm = mpl.colors.BoundaryNorm(bounds, cmap.N)

[25]:

fig = pl.figure(figsize=(10,8))
cgax, im = wrl.vis.plot_rhi(hmc_idx[11], ax=111,
                            proj={'angular_spacing': 20.,
                                  'radial_spacing': 12.,
                                  'latmin': 2.5},
                            r=r/1000., th=el,
                            norm=norm,
                            cmap=cmap,
                            fig=fig)
cbar = pl.colorbar(im, ticks=ticks, fraction=0.046, norm=norm, pad=0.1)
cbar.set_label('Hydrometeorclass')
caax = cgax.parasites[0]
caax.set_xlabel('Range [km]')
caax.set_ylabel('Range [km]')
labels = [pr_types[i][1] for i, _ in enumerate(pr_types)]
labels = cbar.ax.set_yticklabels(labels)
t = cgax.set_title('Hydrometeorclassification')
t.set_y(1.05)
cgax.set_xlim(0,40)
cgax.set_ylim(0,14)
pl.show()

_images/notebooks_classify_wradlib_2d_hmc_55_0.png

Georeferencing¶

This section provides a collection of example code snippets to show how georeferencing of radar and ancillary data is done in $\omega radlib$.

Computing cartesian and geographical coordinates for polar data¶

[1]:

import numpy as np
import wradlib.georef as georef
import wradlib.io as io
import wradlib.util as util
import warnings
warnings.filterwarnings('ignore')

Read the data¶

Here, we use an OPERA hdf5 dataset.

[2]:

filename = 'hdf5/20130429043000.rad.bewid.pvol.dbzh.scan1.hdf'
filename = util.get_wradlib_data_file(filename)
pvol = io.read_opera_hdf5(filename)

Count the number of datasets¶

[3]:

ntilt = 1
for i in range(100):
    try:
        pvol["dataset%d/what" % ntilt]
        ntilt += 1
    except Exception:
        ntilt -= 1
        break

Define radar location and scan geometry¶

[4]:

nrays = int(pvol["dataset1/where"]["nrays"])
nbins = int(pvol["dataset1/where"]["nbins"])
rscale = int(pvol["dataset1/where"]["rscale"])
coord = np.empty((ntilt, nrays, nbins, 3))
for t in range(ntilt):
    elangle = pvol["dataset%d/where" % (t + 1)]["elangle"]
    coord[t, ...] = georef.sweep_centroids(nrays, rscale, nbins, elangle)
sitecoords = (pvol["where"]["lon"], pvol["where"]["lat"],
              pvol["where"]["height"])
print(coord.shape)

(5, 360, 960, 3)

Retrieve azimuthal equidistant coordinates and projection¶

[5]:

coords, proj_radar = georef.spherical_to_xyz(coord[..., 0],
                                             coord[..., 1],
                                             coord[..., 2], sitecoords,
                                             squeeze=True)
test = coords[0, 90, 0:960:60, 0]
print(test)

[1.24984800e+02 1.51230048e+04 3.01206529e+04 4.51178353e+04
 6.01144584e+04 7.51104287e+04 9.01056525e+04 1.05100036e+05
 1.20093487e+05 1.35085910e+05 1.50077213e+05 1.65067302e+05
 1.80056083e+05 1.95043465e+05 2.10029352e+05 2.25013652e+05]

Retrieve geographic coordinates (longitude and latitude)¶

Using convenience function spherical_to_proj.¶

[6]:

lonlatalt = georef.spherical_to_proj(coord[..., 0],
                                     coord[..., 1],
                                     coord[..., 2], sitecoords)
test = lonlatalt[0, 90, 0:960:60, 0]
print(test)

[5.50734017 5.71615328 5.92494778 6.13371911 6.34246274 6.55117413
 6.75984874 6.96848204 7.17706951 7.38560664 7.5940889  7.8025118
 8.01087084 8.21916154 8.42737941 8.63552   ]

Using reproject¶

[7]:

lonlatalt1 = georef.reproject(coords, projection_source=proj_radar,
                             projection_target=georef.get_default_projection())

test = lonlatalt1[0, 90, 0:960:60, 0]
print(test)

[5.50734017 5.71615328 5.92494778 6.13371911 6.34246274 6.55117413
 6.75984874 6.96848204 7.17706951 7.38560664 7.5940889  7.8025118
 8.01087084 8.21916154 8.42737941 8.63552   ]

Example for georeferencing a radar dataset¶

[1]:

import wradlib.georef as georef
import numpy as np
import matplotlib.pyplot as pl
import matplotlib as mpl
from matplotlib.patches import Rectangle
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()

1st step: Compute centroid coordinates and vertices of all radar bins in WGS84 (longitude and latitude).

[2]:

# Define the polar coordinates and the site coordinates in lat/lon
r = np.arange(1, 129) * 1000
az = np.linspace(0, 360, 361)[0:-1]

# Site coordinates for different DWD radar locations (you choose)
# LAT:  drs:  51.12527778 ; fbg: 47.87444444 ; tur: 48.58611111 ; # muc: 48.3372222
# LON:  drs:  13.76972222 ; fbg: 8.005 ; tur: 9.783888889 ; muc: 11.61277778
sitecoords = (9.7839, 48.5861)

We can now generate the polgon vertices of the radar bins - with each vertex in lon/lat coordinates.

[3]:

proj_wgs84 = georef.epsg_to_osr(4326)
polygons = georef.spherical_to_polyvert(r, az, 0, sitecoords,
                                        proj=proj_wgs84)
polygons = polygons[..., 0:2]
polygons.shape

[3]:

(46080, 5, 2)

… or we can compute the corresponding centroids of all bins - - with each centroid in lon/lat coordinates.

[4]:

cent_coords = georef.spherical_to_centroids(r, az, 0, sitecoords,
                                            proj=proj_wgs84)
cent_coords = np.squeeze(cent_coords)
cent_lon = cent_coords[..., 0]
cent_lat = cent_coords[..., 1]

In order to understand how vertices and centroids correspond, we can plot them together.

[5]:

fig = pl.figure(figsize=(16, 16))
aspect = ( cent_lon.max()-cent_lon.min() ) / ( cent_lat.max()-cent_lat.min() )
ax = fig.add_subplot(121, aspect = aspect )
polycoll = mpl.collections.PolyCollection(polygons, closed=True,
                                          facecolors='None', linewidth=0.1)
ax.add_collection(polycoll, autolim=True)
#ax.plot(cent_lon, cent_lat, 'r+')
pl.title('Zoom in\n(only possible for interactive plots).')
ax.add_patch(Rectangle((sitecoords[0]+0.25, sitecoords[1]+0.25), 0.2, 0.2/aspect, edgecolor="red", facecolor="None", zorder=3))
pl.xlim(cent_lon.min(), cent_lon.max())
pl.ylim(cent_lat.min(), cent_lat.max())

ax = fig.add_subplot(122, aspect = aspect)
polycoll = mpl.collections.PolyCollection(polygons, closed=True,
                                          facecolors='None')
ax.add_collection(polycoll, autolim=True)
ax.plot(cent_lon, cent_lat, 'r+')
pl.title('Zoom into red box of left plot')
pl.xlim(sitecoords[0]+0.25, sitecoords[0]+0.25+0.2)
pl.ylim(sitecoords[1]+0.25, sitecoords[1]+0.25+0.2/aspect)

[5]:

(48.8361, 48.96877939454404)

_images/notebooks_georeferencing_wradlib_georef_example_10_1.png

2nd step: Reproject the centroid coordinates to Gauss-Krueger Zone 3 (i.e. EPSG-Code 31467).

[6]:

proj_gk3 = georef.epsg_to_osr(31467)
x, y = georef.reproject(cent_lon, cent_lat, projection_targe=proj_gk3)

RADOLAN Grid¶

Polar Stereographic Projection¶

The projected composite raster is equidistant with a grid-spacing of 1.0 km in most cases. There are composites which have 2.0 km grid-spacing (e.g. PC).

There are five different grid sizes, the well-known 900 rows by 900 columns (normal, 1km grid spacinf), 1500 rows by 1400 columns (extended, 1km grid spacing), 460 rows by 460 columns (small, 2km grid spacing) and the legacy 450 rows by 450 rows (2km grid spacing). Since the RADSYS-E project is finalized an extended national composite with 1100 rows by 900 columns (normal_wx, 1km grid spacing) is available, too.

Common to all is that the plane of projection intersects the earth sphere at $\phi_0 = 60.0^{\circ}N$. The cartesian co-ordinate system is aligned parallel to the $\lambda_0 = 10.0^{\circ}E$ meridian.

The reference point ($\lambda_m$, $\phi_m$) is $9.0^{\circ}E$ and $51.0^{\circ}N$, which is the center of the two smaller grids. The extended european grid has an offset in respect to this reference point of 350km by 150km, the extended national grid 100km by -80km.

The earth as sphere with an radius of 6370.04 km is used for all calculations.

With formulas (1), (2) and (3) the geographic reference points ($\lambda$, $\phi$) can be converted to projected cartesian coordinates. The calculated (x y) is the distance vector to the origign of the cartesian coordinate system (north pole).

$\begin{equation} x = R * M(\phi) * cos(\phi) * sin(\lambda - \lambda_0) \tag{1} \end{equation}$

$\begin{equation} y = -R * M(\phi) * cos(\phi) * cos(\lambda - \lambda_0) \tag{2} \end{equation}$

$\begin{equation} M(\phi) = \frac {1 + sin(\phi_0)} {1 + sin(\phi)} \tag{3} \end{equation}$

Assumed the point ($10.0^{\circ}E$, $90.0^{\circ}N$) is defined as coordinate system origin. Then all ccordinates can be calculated with the known grid-spacing d as:

$\begin{equation} x = x_0 + d * (j - j_0) \tag{4} \end{equation}$

$\begin{equation} y = y_0 + d * (i - i_0) \tag{5} \end{equation}$

with i, j as cartesian indices.

$\omega radlib$ provides the convenience function wradlib.georef.get_radolan_grid() which returns the radolan grid for further processing. It takes nrows and ncols as parameters and returns the projected cartesian coordinates or the wgs84 coordinates (keyword arg wgs84=True) as numpy ndarray (nrows x ncols x 2).

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

900x900, 1km grid box¶

[2]:

radolan_grid_xy = wrl.georef.get_radolan_grid(900,900)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_xy.shape, *radolan_grid_xy[0,0,:]))
radolan_grid_ll = wrl.georef.get_radolan_grid(900,900, wgs84=True)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_ll.shape, *radolan_grid_ll[0,0,:]))

(900, 900, 2), (-523.4622, -4658.6447)
(900, 900, 2), (3.5889, 46.9526)

1100x900, 1km grid box¶

[3]:

radolan_grid_xy = wrl.georef.get_radolan_grid(1100,900)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_xy.shape, *radolan_grid_xy[0,0,:]))
radolan_grid_ll = wrl.georef.get_radolan_grid(1100,900, wgs84=True)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_ll.shape, *radolan_grid_ll[0,0,:]))

(1100, 900, 2), (-443.4622, -4758.6447)
(1100, 900, 2), (4.6759, 46.1929)

1500x1400, 1km grid box¶

[4]:

radolan_grid_xy = wrl.georef.get_radolan_grid(1500,1400)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_xy.shape, *radolan_grid_xy[0,0,:]))
radolan_grid_ll = wrl.georef.get_radolan_grid(1500,1400, wgs84=True)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_ll.shape, *radolan_grid_ll[0,0,:]))

(1500, 1400, 2), (-673.4622, -5008.6447)
(1500, 1400, 2), (2.3419, 43.9336)

460x460, 2km grid box¶

[5]:

radolan_grid_xy = wrl.georef.get_radolan_grid(460,460)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_xy.shape, *radolan_grid_xy[0,0,:]))
radolan_grid_ll = wrl.georef.get_radolan_grid(460,460, wgs84=True)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_ll.shape, *radolan_grid_ll[0,0,:]))

(460, 460, 2), (-533.4622, -4668.6447)
(460, 460, 2), (3.4814, 46.8603)

450x450, 2km grid box¶

[6]:

radolan_grid_xy = wrl.georef.get_radolan_grid(450,450)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_xy.shape, *radolan_grid_xy[0,0,:]))
radolan_grid_ll = wrl.georef.get_radolan_grid(450,450, wgs84=True)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_ll.shape, *radolan_grid_ll[0,0,:]))

(450, 450, 2), (-523.4622, -4658.6447)
(450, 450, 2), (3.5889, 46.9526)

Inverse Polar Stereographic Projection¶

The geographic coordinates of specific datapoints can be calculated by using the cartesian coordinates (x,y) and the following formulas:

$\begin{equation} \lambda = \arctan\left(\frac {-x} {y}\right) + \lambda_0 \tag{6} \end{equation}$

$\begin{equation} \phi = \arcsin\left(\frac {R^2 * \left(1 + \sin\phi_0\right)^2 - \left(x^2 + y^2\right)} {R^2 * \left(1 + \sin\phi_0\right)^2 + \left(x^2 + y^2\right)}\right) \tag{7} \end{equation}$

Standard Formats¶

WKT-String¶

The German Weather Service provides a WKT-string. This WKT (well known text) is used to create the osr-object representation of the radolan projection.

For the scale_factor the intersection of the projection plane with the earth sphere at $60.0^{\circ}N$ has to be taken into account:

$\begin{equation} scale\_factor = \frac {1 + \sin\left(60.^{\circ}\right)} {1 + \sin\left(90.^{\circ}\right)} = 0.93301270189 \tag{8} \end{equation}$

Also, the PROJECTION["Stereographic_North_Pole"] isn’t known within GDAL/OSR. It has to be changed to the known PROJECTION["polar_stereographic"].

With these adaptions we finally yield the Radolan Projection as WKT-string. This WKT-string is used within $\omega radlib$ to create the osr-object by using the helper-function wradlib.georef.create_osr().

[7]:

proj_stereo = wrl.georef.create_osr("dwd-radolan")
print(proj_stereo)

PROJCS["Radolan Projection",
    GEOGCS["Radolan Coordinate System",
        DATUM["Radolan_Kugel",
            SPHEROID["Erdkugel",6370040,0]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]]],
    PROJECTION["Polar_Stereographic"],
    PARAMETER["latitude_of_origin",90],
    PARAMETER["central_meridian",10],
    PARAMETER["scale_factor",0.933012701892],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["kilometre",1000,
        AUTHORITY["EPSG","9036"]],
    AXIS["Easting",SOUTH],
    AXIS["Northing",SOUTH]]

PROJ.4¶

Using the above WKT-String the PROJ.4 representation can be derived as:

+proj=stere +lat_0=90 +lat_ts=90 +lon_0=10 +k=0.93301270189
+x_0=0 +y_0=0 +a=6370040 +b=6370040 +to_meter=1000 +no_defs

This PROJ.4-string can also be used to create the osr-object by using the helper-function wradlib.georef.proj4_to_osr():

[8]:

# create radolan projection osr object
dwd_string = ('+proj=stere +lat_0=90 +lat_ts=90 +lon_0=10 +k=0.93301270189 '
              '+x_0=0 +y_0=0 +a=6370040 +b=6370040 +to_meter=1000 +no_defs')
proj_stereo = wrl.georef.proj4_to_osr(dwd_string)
print(proj_stereo)

PROJCS["unknown",
    GEOGCS["unknown",
        DATUM["unknown",
            SPHEROID["unknown",6370040,0]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]]],
    PROJECTION["Polar_Stereographic"],
    PARAMETER["latitude_of_origin",90],
    PARAMETER["central_meridian",10],
    PARAMETER["scale_factor",0.93301270189],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["kilometre",1000,
        AUTHORITY["EPSG","9036"]],
    AXIS["Easting",SOUTH],
    AXIS["Northing",SOUTH]]

Grid Reprojection¶

Within $\omega radlib$ the wradlib.georef.reproject() function can be used to convert the radolan grid data from xy-space to lonlat-space and back. First, we need to create the necessary Spatial Reference Objects for the RADOLAN-projection and wgs84.

[9]:

from osgeo import osr
proj_stereo = wrl.georef.create_osr("dwd-radolan")
print(proj_stereo)
proj_wgs = osr.SpatialReference()
proj_wgs.ImportFromEPSG(4326)
print(proj_wgs)

PROJCS["Radolan Projection",
    GEOGCS["Radolan Coordinate System",
        DATUM["Radolan_Kugel",
            SPHEROID["Erdkugel",6370040,0]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]]],
    PROJECTION["Polar_Stereographic"],
    PARAMETER["latitude_of_origin",90],
    PARAMETER["central_meridian",10],
    PARAMETER["scale_factor",0.933012701892],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["kilometre",1000,
        AUTHORITY["EPSG","9036"]],
    AXIS["Easting",SOUTH],
    AXIS["Northing",SOUTH]]
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9122"]],
    AXIS["Latitude",NORTH],
    AXIS["Longitude",EAST],
    AUTHORITY["EPSG","4326"]]

Then, we call reproject with the osr-objects as projection_source and projection_target parameters.

[10]:

radolan_grid_xy = wrl.georef.get_radolan_grid(900,900)
radolan_grid_ll = wrl.georef.reproject(radolan_grid_xy, projection_source=proj_stereo, projection_target=proj_wgs)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_ll.shape, *radolan_grid_ll[0,0,:]))

(900, 900, 2), (3.5889, 46.9526)

And the other way round.

[11]:

radolan_grid_xy = wrl.georef.reproject(radolan_grid_ll, projection_source=proj_wgs, projection_target=proj_stereo)
print("{0}, ({1:.4f}, {2:.4f})".format(radolan_grid_xy.shape, *radolan_grid_xy[0,0,:]))

(900, 900, 2), (-523.4622, -4658.6447)

In the following example the RADOLAN grid is projected to wgs84 and GaussKrüger Zone3.

[12]:

# create UTM zone 32 projection osr object
proj_utm32 = osr.SpatialReference()
proj_utm32.ImportFromEPSG(32632)

# transform radolan polar stereographic projection to wgs84 and then to utm zone 32
radolan_grid_ll = wrl.georef.reproject(radolan_grid_xy,
                                       projection_source=proj_stereo,
                                       projection_target=proj_wgs)
radolan_grid_utm32 = wrl.georef.reproject(radolan_grid_ll,
                                       projection_source=proj_wgs,
                                       projection_target=proj_utm32)

lon_wgs0 = radolan_grid_ll[:, :, 0]
lat_wgs0 = radolan_grid_ll[:, :, 1]

x_utm32 = radolan_grid_utm32[:, :, 0]
y_utm32 = radolan_grid_utm32[:, :, 1]

x_rad = radolan_grid_xy[:, :, 0]
y_rad = radolan_grid_xy[:, :, 1]

print("\n------------------------------")
print("source radolan x,y-coordinates")
print(u"       {0}      {1} ".format('x [km]', 'y [km]'))
print("ll: {:10.4f} {:10.3f} ".format(x_rad[0, 0], y_rad[0, 0]))
print("lr: {:10.4f} {:10.3f} ".format(x_rad[0, -1], y_rad[0, -1]))
print("ur: {:10.4f} {:10.3f} ".format(x_rad[-1, -1], y_rad[-1, -1]))
print("ul: {:10.4f} {:10.3f} ".format(x_rad[-1, 0], y_rad[-1, 0]))
print("\n--------------------------------------")
print("transformed radolan lonlat-coordinates")
print(u"      {0}  {1} ".format('lon [degE]', 'lat [degN]'))
print("ll: {:10.4f}  {:10.4f} ".format(lon_wgs0[0, 0], lat_wgs0[0, 0]))
print("lr: {:10.4f}  {:10.4f} ".format(lon_wgs0[0, -1], lat_wgs0[0, -1]))
print("ur: {:10.4f}  {:10.4f} ".format(lon_wgs0[-1, -1], lat_wgs0[-1, -1]))
print("ul: {:10.4f}  {:10.4f} ".format(lon_wgs0[-1, 0], lat_wgs0[-1, 0]))
print("\n-----------------------------------")
print("transformed radolan utm32-coordinates")
print(u"     {0}   {1} ".format('easting [m]', 'northing [m]'))
print("ll: {:10.0f}   {:10.0f} ".format(x_utm32[0, 0], y_utm32[0, 0]))
print("lr: {:10.0f}   {:10.0f} ".format(x_utm32[0, -1], y_utm32[0, -1]))
print("ur: {:10.0f}   {:10.0f} ".format(x_utm32[-1, -1], y_utm32[-1, -1]))
print("ul: {:10.0f}   {:10.0f} ".format(x_utm32[-1, 0], y_utm32[-1, 0]))


------------------------------
source radolan x,y-coordinates
       x [km]      y [km]
ll:  -523.4622  -4658.645
lr:   375.5378  -4658.645
ur:   375.5378  -3759.645
ul:  -523.4622  -3759.645

--------------------------------------
transformed radolan lonlat-coordinates
      lon [degE]  lat [degN]
ll:     3.5889     46.9526
lr:    14.6087     47.0711
ur:    15.7042     54.7327
ul:     2.0736     54.5790

-----------------------------------
transformed radolan utm32-coordinates
     easting [m]   northing [m]
ll:      88298      5214122
lr:     925789      5228352
ur:     931372      6085693
ul:      52659      6070030

Plot additional geodata¶

underlay e.g. terrain data from a Digital Elevation Model (DEM)
overlay features such as administrative borders, rivers, catchments, rain gauges, cities, …

Here, we create a map without radar data to concentrate on the other layers.

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np
# Some more matplotlib tools we will need...
import matplotlib.ticker as ticker
from matplotlib.colors import LogNorm
from mpl_toolkits.axes_grid1 import make_axes_locatable

Plotting a Digital Elevation Model (DEM)¶

We use a preprocessed geotiff which was created from **SRTM** data via gdal

gdalwarp -te 88. 20. 93. 27. srtm_54_07.tif srtm_55_07.tif srtm_54_08.tif srtm_55_08.tif bangladesh.tif

Here we - read the DEM via wradlib.io.open_raster and extracted via wradlib.georef.extract_raster_dataset. - resample the data to a (lon/lat) grid with spacing=0.005.

Note: we organise the code in functions which we can re-use in this notebook.

[2]:

def plot_dem(ax):
    filename = wrl.util.get_wradlib_data_file('geo/bangladesh.tif')
    ds = wrl.io.open_raster(filename)
    # pixel_spacing is in output units (lonlat)
    ds = wrl.georef.reproject_raster_dataset(ds, spacing=0.005)
    rastervalues, rastercoords, proj = wrl.georef.extract_raster_dataset(ds)
    # specify kwargs for plotting, using terrain colormap and LogNorm
    dem = ax.pcolormesh(rastercoords[..., 0], rastercoords[..., 1],
                        rastervalues, cmap=pl.cm.terrain, norm=LogNorm(),
                        vmin=1, vmax=3000)
    # make some space on the right for colorbar axis
    div1 = make_axes_locatable(ax)
    cax1 = div1.append_axes("right", size="5%", pad=0.1)
    # add colorbar and title
    # we use LogLocator for colorbar
    cb = pl.gcf().colorbar(dem, cax=cax1,
                           ticks=ticker.LogLocator(subs=range(10)))
    cb.set_label('terrain height [m]')

[3]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_dem(ax)

_images/notebooks_visualisation_wradlib_overlay_10_0.png

Plotting borders¶

For country borders, we use ESRI Shapfiles from Natural Earth Data.

We extract features using - the OGR.Layer AttributeFilter and - the wradlib.georef.get_vector_coordinates function.

The plot overlay is done via wradlib.vis.add_lines.

[4]:

def plot_borders(ax):
    # country list
    countries = ['India', 'Nepal', 'Bhutan', 'Myanmar']
    # open the input data source and get the layer
    filename = wrl.util.get_wradlib_data_file('geo/ne_10m_admin_0_boundary_'
                                              'lines_land.shp')
    dataset, inLayer = wrl.io.open_vector(filename)
    # iterate over countries, filter accordingly, get coordinates and plot
    for item in countries:
        # SQL-like selection syntax
        fattr = "(adm0_left = '" + item + "' or adm0_right = '" + item + "')"
        inLayer.SetAttributeFilter(fattr)
        # get borders and names
        borders, keys = wrl.georef.get_vector_coordinates(inLayer, key='name')
        wrl.vis.add_lines(ax, borders, color='black', lw=2, zorder=4)
    ax.autoscale()

[5]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_dem(ax)
plot_borders(ax)
ax.set_xlim((88, 93))
ax.set_ylim((20, 27))

[5]:

(20.0, 27.0)

_images/notebooks_visualisation_wradlib_overlay_15_1.png

Plotting Rivers¶

For rivers, we use ESRI Shapfiles from FAO and Natural Earth Data.

We extract features using - the OGR.Layer SpatialFilter and - the wradlib.georef.get_vector_coordinates function.

Then we use wradlib.vis.add_lines again for the overlay.

[6]:

def plot_rivers(ax):
    # plot rivers from esri vector shape, filter spatially
    # http://www.fao.org/geonetwork/srv/en/metadata.show?id=37331

    # open the input data source and get the layer
    filename = wrl.util.get_wradlib_data_file('geo/rivers_asia_37331.shp')
    dataset, inLayer = wrl.io.open_vector(filename)

    # do spatial filtering to get only geometries inside bounding box
    inLayer.SetSpatialFilterRect(88, 20, 93, 27)
    rivers, keys = wrl.georef.get_vector_coordinates(inLayer, key='MAJ_NAME')

    # plot on ax1, and ax4
    wrl.vis.add_lines(ax, rivers, color=pl.cm.terrain(0.), lw=0.5, zorder=3)

[7]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_dem(ax)
plot_borders(ax)
plot_rivers(ax)
ax.set_xlim((88, 93))
ax.set_ylim((20, 27))

[7]:

(20.0, 27.0)

_images/notebooks_visualisation_wradlib_overlay_20_1.png

[8]:

def plot_water(ax):
    # plot rivers from esri vector shape, filter spatially
    # plot rivers from NED
    # open the input data source and get the layer
    filename = wrl.util.get_wradlib_data_file('geo/ne_10m_rivers_lake_'
                                              'centerlines.shp')
    dataset, inLayer = wrl.io.open_vector(filename)
    inLayer.SetSpatialFilterRect(88, 20, 93, 27)
    rivers, keys = wrl.georef.get_vector_coordinates(inLayer)
    wrl.vis.add_lines(ax, rivers, color=pl.cm.terrain(0.), lw=0.5, zorder=3)

[9]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_dem(ax)
plot_borders(ax)
plot_rivers(ax)
plot_water(ax)
ax.set_xlim((88, 93))
ax.set_ylim((20, 27))

[9]:

(20.0, 27.0)

_images/notebooks_visualisation_wradlib_overlay_22_1.png

Plotting Cities¶

The 5 biggest cities of bangladesh are added using simple matplotlib functions.

[10]:

def plot_cities(ax):
    # plot city dots with annotation, finalize plot
    # lat/lon coordinates of five cities in Bangladesh
    lats = [23.73, 22.32, 22.83, 24.37, 24.90]
    lons = [90.40, 91.82, 89.55, 88.60, 91.87]
    cities = ['Dhaka', 'Chittagong', 'Khulna', 'Rajshahi', 'Sylhet']
    for lon, lat, city in zip(lons, lats, cities):
        ax.plot(lon, lat, 'ro', zorder=5)
        ax.text(lon + 0.01, lat + 0.01, city, fontsize='large')

[11]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_dem(ax)
plot_borders(ax)
plot_rivers(ax)
plot_water(ax)
plot_cities(ax)

ax.set_xlim((88, 93))
ax.set_ylim((20, 27))
ax.set_xlabel('Longitude')
ax.set_ylabel('Latitude')
ax.set_aspect('equal')
ax.set_title('Bangladesh')

[11]:

Text(0.5, 1.0, 'Bangladesh')

_images/notebooks_visualisation_wradlib_overlay_26_1.png

Plotting country patches¶

Plotting in “geographic projection” (WGS84)¶

Here, we plot countries as patches on a lat/lon (WGS84) map (data from Natural Earth Data again).

We again extract the features using - the OGR.Layer SpatialFilter and - wradlib.georef.get_vector_coordinates.

Then the patches are added one by one via wradlib.vis.add_patches.

[12]:

def plot_wgs84(ax):
    from osgeo import osr
    wgs84 = osr.SpatialReference()
    wgs84.ImportFromEPSG(4326)
    # some testing on additional axes
    # add Bangladesh to countries
    countries = ['India', 'Nepal', 'Bhutan', 'Myanmar', 'Bangladesh']
    # create colors for country-patches
    cm = pl.cm.jet
    colors = []
    for i in range(len(countries)):
        colors.append(cm(1. * i / len(countries)))

    # open the input data source and get the layer
    filename = wrl.util.get_wradlib_data_file('geo/ne_10m_admin_0_'
                                              'countries.shp')
    dataset, layer = wrl.io.open_vector(filename)

    # filter spatially and plot as PatchCollection on ax3
    layer.SetSpatialFilterRect(88, 20, 93, 27)
    patches, keys = wrl.georef.get_vector_coordinates(layer,
                                                      dest_srs=wgs84,
                                                      key='name')
    i = 0
    for name, patch in zip(keys, patches):
        # why comes the US in here?
        if name in countries:
            wrl.vis.add_patches(ax, patch, facecolor=colors[i],
                                cmap=pl.cm.viridis, alpha=0.4)
            i += 1
    ax.autoscale(True)
    ax.set_aspect('equal')
    ax.set_xlabel('Longitude')
    ax.set_ylabel('Latitude')
    ax.set_title('South Asia - WGS 84')

[13]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_wgs84(ax)

_images/notebooks_visualisation_wradlib_overlay_33_0.png

Plotting with a map projection¶

Here, we plot countries as patches on a projected map.

We extract the features using - the OGR.Layer AttributeFilter and - the wradlib.georef.get_vector_coordinates function.

The coordinates of the features are reprojected on the fly using the dest_srs keyword of wradlib.georef.get_vector_coordinates.

Then, the patches are added to the map via wradlib.vis.add_patches.

[14]:

def plot_mercator(ax):
    from osgeo import osr
    proj = osr.SpatialReference()
    # "Web Mercator" projection (used by GoogleMaps, OSM, ...)
    proj.ImportFromEPSG(3857)

    # add Bangladesh to countries
    countries = ['India', 'Nepal', 'Bhutan', 'Myanmar', 'Bangladesh']
    # create colors for country-patches
    cm = pl.cm.jet
    colors = []
    for i in range(len(countries)):
        colors.append(cm(1. * i / len(countries)))

    # open the input data source and get the layer
    filename = wrl.util.get_wradlib_data_file('geo/ne_10m_admin_0_'
                                              'countries.shp')
    dataset, layer = wrl.io.open_vector(filename)
    # iterate over countries, filter by attribute,
    # plot single patches on ax2
    for i, item in enumerate(countries):
        fattr = "name = '" + item + "'"
        layer.SetAttributeFilter(fattr)
        # get country patches and geotransform to destination srs
        patches, keys = wrl.georef.get_vector_coordinates(layer,
                                                          dest_srs=proj,
                                                          key='name')
        wrl.vis.add_patches(pl.gca(), patches, facecolor=colors[i])

    ax.autoscale(True)
    ax.set_aspect('equal')
    ax.set_xlabel('X - Coordinate')
    ax.set_ylabel('Y - Coordinate')
    ax.ticklabel_format(style='sci', scilimits=(0, 0))
    ax.set_title('South Asia in Web Mercator Projection ')

[15]:

fig = pl.figure(figsize=(10,10))
ax = fig.add_subplot(111, aspect='equal')
plot_mercator(ax)

_images/notebooks_visualisation_wradlib_overlay_39_0.png

Zonal Statistics¶

The wradlib.zonalstats module provides classes and functions for calculation of zonal statistics for data on arbitrary grids and projections.

It provides classes for:

managing georeferenced data (grid points or grid polygons, zonal polygons),
calculation of geographic intersections and managing resulting vector data
calculation of zonal statistics and managing result data as vector attributes
output to vector and raster files available within ogr/gdal

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

DataSource¶

The wradlib.zonalstats.DataSource class handles point or polygon vector data by wrapping ogr.DataSource with special functions.

The following example shows how to create different DataSource objects:

[2]:

from osgeo import osr

# create utm zone 32 projection osr object
proj_utm = osr.SpatialReference()
proj_utm.ImportFromEPSG(32632)

# Setting up DataSource
box0 = np.array([[2600000., 5630000.],[2600000., 5640000.],
                 [2610000., 5640000.],[2610000., 5630000.],
                 [2600000., 5630000.]])
box1 = np.array([[2610000., 5630000.],[2610000., 5640000.],
                 [2620000., 5640000.],[2620000., 5630000.],
                 [2610000., 5630000.]])
box2 = np.array([[2600000., 5640000.],[2600000., 5650000.],
                 [2610000., 5650000.],[2610000., 5640000.],
                 [2600000., 5640000.]])
box3 = np.array([[2610000., 5640000.],[2610000., 5650000.],
                 [2620000., 5650000.],[2620000., 5640000.],
                 [2610000., 5640000.]])

point0 = np.array(wrl.georef.get_centroid(box0))
point1 = np.array(wrl.georef.get_centroid(box1))
point2 = np.array(wrl.georef.get_centroid(box2))
point3 = np.array(wrl.georef.get_centroid(box3))

# creates Polygons in Datasource
poly = wrl.zonalstats.DataSource(np.array([box0, box1, box2, box3]), srs=proj_utm, name='poly')

# creates Points in Datasource
point = wrl.zonalstats.DataSource(np.vstack((point0, point1, point2, point3)),
                                  srs=proj_utm, name='point')

Let’s have a look at the data, which will be exported as numpy arrays. The property data exports all available data:

[3]:

print(poly.data)
print(point.data)

[[[2600000. 5630000.]
  [2600000. 5640000.]
  [2610000. 5640000.]
  [2610000. 5630000.]
  [2600000. 5630000.]]

 [[2610000. 5630000.]
  [2610000. 5640000.]
  [2620000. 5640000.]
  [2620000. 5630000.]
  [2610000. 5630000.]]

 [[2600000. 5640000.]
  [2600000. 5650000.]
  [2610000. 5650000.]
  [2610000. 5640000.]
  [2600000. 5640000.]]

 [[2610000. 5640000.]
  [2610000. 5650000.]
  [2620000. 5650000.]
  [2620000. 5640000.]
  [2610000. 5640000.]]]
[[2605000. 5635000.]
 [2615000. 5635000.]
 [2605000. 5645000.]
 [2615000. 5645000.]]

Currently data can also be retrieved by:

index - wradlib.zonalstats.DataSource.get_data_by_idx(),
attribute - wradlib.zonalstats.DataSource.get_data_by_att() and
geometry - wradlib.zonalstats.DataSource.get_data_by_geom().

Now, with the DataSource being created, we can add/set attribute data of the features:

[4]:

# add attribute
poly.set_attribute('mean', np.array([10.1, 20.2, 30.3, 40.4]))
point.set_attribute('mean', np.array([10.1, 20.2, 30.3, 40.4]))

Attributes associated with features can also be retrieved:

[5]:

# get attributes
print(poly.get_attributes(['mean']))
# get attributes filtered
print(poly.get_attributes(['mean'], filt=('index', 2)))

[[10.1, 20.2, 30.3, 40.4]]
[[30.3]]

Finally, we can export the contained data to OGR/GDAL supported vector and raster files:

[6]:

# dump as 'ESRI Shapefile', default
poly.dump_vector('test_poly.shp')
point.dump_vector('test_point.shp')
# dump as 'GeoJSON'
poly.dump_vector('test_poly.geojson', 'GeoJSON')
point.dump_vector('test_point.geojson', 'GeoJSON')
# dump as 'GTiff', default
poly.dump_raster('test_poly_raster.tif', attr='mean', pixel_size=100.)
# dump as 'netCDF'
poly.dump_raster('test_poly_raster.nc', 'netCDF', attr='mean', pixel_size=100.)

ZonalData¶

ZonalData is usually available as georeferenced regular gridded data. Here the wradlib.zonalstats.ZonalDataBase class manages the grid data, the zonal data (target polygons) and the intersection data of source grid and target polygons. Because the calculation of intersection is different for point grids and polygon grids, we have subclasses wradlib.zonalstats.ZonalDataPoly and wradlib.zonalstats.ZonalDataPoint.

Basically, wradlib.zonalstats.ZonalDataBase encapsulates three wradlib.zonalstats.DataSource objects:

source grid (points/polygons)
target polygons
destination (intersection) (points/polygons)

The destination DataSource object is created from the provided source grid and target polygons at initialisation time.

As an example the creation of a wradlib.zonalstats.ZonalDataPoly class instance is shown:

[7]:

# setup test grid and catchment
lon = 7.071664
lat = 50.730521
r = np.array(range(50, 100*1000 + 50 , 100))
a = np.array(range(0, 90, 1))
rays = a.shape[0]
bins = r.shape[0]

# setup OSR objects
proj_utm = osr.SpatialReference()
proj_utm.ImportFromEPSG(32632)

# create polar grid polygon vertices in UTM
radar_utm = wrl.georef.spherical_to_polyvert(r, a, 0, (lon, lat),
                                             proj=proj_utm)
radar_utm = radar_utm[..., 0:2]
# reshape
radar_utm.shape = (rays * bins, 5, 2)

box0 = np.array([[390000., 5630000.],[390000., 5640000.],
                 [400000., 5640000.],[400000., 5630000.],
                 [390000., 5630000.]])

box1 = np.array([[400000., 5630000.],[400000., 5640000.],
                 [410000., 5640000.],[410000., 5630000.],
                 [400000., 5630000.]])

targets = np.array([box0, box1])

zdpoly = wrl.zonalstats.ZonalDataPoly(radar_utm, targets, srs=proj_utm)

When calculationg the intersection, also weights are calculated for every source grid feature and attributed to the destination features.

With the property isecs it is possible to retrieve the intersection geometries as numpy array, further get-functions add to the functionality:

[8]:

# get intersections as numpy array
isecs = zdpoly.isecs
# get intersections for target polygon 0
isec0 = zdpoly.get_isec(0)
# get source indices referring to target polygon 0
ind0 = zdpoly.get_source_index(0)

print(isecs.shape, isec0.shape, ind0.shape)

(2,) (2053,) (2053,)

There are import/export functions using ESRI-Shapfile Format as data format. Next export and import is shown: wradlib.zonalstats.ZonalDataBase

[9]:

zdpoly.dump_vector('test_zdpoly')
zdpoly_new = wrl.zonalstats.ZonalDataPoly('test_zdpoly')

ZonalStats¶

For ZonalStats the wradlib.zonalstats.ZonalStatsBase class and the two subclasses wradlib.zonalstats.ZonalStatsPoly and wradlib.zonalstats.ZonalStatsPoint are available. ZonalStatsBase encapsulates one ZonalData object. Properties for simple access of ZonalData, intersection indices and weights are provided. The following code will add mean and var attributes to the target DataSource:

[10]:

# create ZonalStatsPoly instance
gc = wrl.zonalstats.ZonalStatsPoly(zdpoly_new)
# create some artificial data for processing using the features indices
count = radar_utm.shape[0]
data = 1000000. / np.array(range(count))
# calculate mean and variance
mean = gc.mean(data)
var = gc.var(data)

print("Average:", mean)
print("Variance:", var)

Average: [13.25806535 12.65538485]
Variance: [4.99591556 2.5116987 ]

Next we can export the resulting zonal statistics to vector and raster files:

[11]:

# export to vector GeoJSON
gc.zdata.trg.dump_vector('test_zonal_json.geojson', 'GeoJSON')
# export 'mean' to raster netCDF
gc.zdata.trg.dump_raster('test_zonal_hdr.nc', 'netCDF', 'mean', pixel_size=100.)

The ZonalStats classes can also be used without any ZonalData by instantiating with precalculated index and weight values. Be sure to use matching ix, w and data arrays:

[12]:

# get ix, and weight arrays
ix = gc.ix
w = gc.w
# instantiate new ZonlaStats object
gc1 = wrl.zonalstats.ZonalStatsPoly(ix=ix, w=w)
# caclulate statistics
avg = gc1.mean(data)
var = gc1.var(data)

print("Average:", avg)
print("Variance:", var)

Average: [13.25806535 12.65538485]
Variance: [4.99591556 2.5116987 ]

Examples¶

Examples of using Zonal Statistics working with rectangular grids as well as polar grids is shown in the Zonal Statistics Example notebook.

How to use wradlib’s ipol module for interpolation tasks?¶

[1]:

import wradlib.ipol as ipol
from wradlib.util import get_wradlib_data_file
from wradlib.vis import plot_ppi
import numpy as np
import matplotlib.pyplot as pl
import datetime as dt
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()

1-dimensional example¶

Includes Nearest Neighbours, Inverse Distance Weighting, and Ordinary Kriging.

[2]:

# Synthetic observations
xsrc = np.arange(10)[:, None]
vals = np.sin(xsrc).ravel()

# Define target coordinates
xtrg = np.linspace(0, 20, 100)[:, None]

# Set up interpolation objects
#   IDW
idw = ipol.Idw(xsrc, xtrg)
#   Nearest Neighbours
nn = ipol.Nearest(xsrc, xtrg)
#   Linear
ok = ipol.OrdinaryKriging(xsrc, xtrg)

# Plot results
pl.figure(figsize=(10,5))
pl.plot(xsrc.ravel(), vals, 'bo', label="Observation")
pl.plot(xtrg.ravel(), idw(vals), 'r-', label="IDW interpolation")
pl.plot(xtrg.ravel(), nn(vals), 'k-', label="Nearest Neighbour interpolation")
pl.plot(xtrg.ravel(), ok(vals), 'g-', label="Ordinary Kriging")
pl.xlabel("Distance", fontsize="large")
pl.ylabel("Value", fontsize="large")
pl.legend(loc="bottomright")

[2]:

<matplotlib.legend.Legend at 0x7fa56f26b100>

_images/notebooks_interpolation_wradlib_ipol_example_4_1.png

2-dimensional example¶

Includes Nearest Neighbours, Inverse Distance Weighting, Linear Interpolation, and Ordinary Kriging.

[3]:

# Synthetic observations and source coordinates
src = np.vstack( (np.array([4, 7, 3, 15]), np.array([8, 18, 17, 3]))).transpose()
np.random.seed(1319622840)
vals = np.random.uniform(size=len(src))

# Target coordinates
xtrg = np.linspace(0, 20, 40)
ytrg = np.linspace(0, 20, 40)
trg = np.meshgrid(xtrg, ytrg)
trg = np.vstack( (trg[0].ravel(), trg[1].ravel()) ).T

# Interpolation objects
idw = ipol.Idw(src, trg)
nn = ipol.Nearest(src, trg)
linear = ipol.Linear(src, trg)
ok = ipol.OrdinaryKriging(src, trg)

# Subplot layout
def gridplot(interpolated, title=""):
    pm = ax.pcolormesh(xtrg, ytrg, interpolated.reshape( (len(xtrg), len(ytrg)) ) )
    pl.axis("tight")
    ax.scatter(src[:, 0], src[:, 1], facecolor="None", s=50, marker='s')
    pl.title(title)
    pl.xlabel("x coordinate")
    pl.ylabel("y coordinate")

# Plot results
fig = pl.figure(figsize=(8,8))
ax = fig.add_subplot(221, aspect="equal")
gridplot(idw(vals), "IDW")
ax = fig.add_subplot(222, aspect="equal")
gridplot(nn(vals), "Nearest Neighbours")
ax = fig.add_subplot(223, aspect="equal")
gridplot(np.ma.masked_invalid(linear(vals)), "Linear interpolation")
ax = fig.add_subplot(224, aspect="equal")
gridplot(ok(vals), "Ordinary Kriging")
pl.tight_layout()

_images/notebooks_interpolation_wradlib_ipol_example_6_0.png

Using the convenience function ipol.interpolation in order to deal with missing values¶

(1) Exemplified for one dimension in space and two dimensions of the source value array (could e.g. be two time steps).

[4]:

# Synthetic observations (e.g. two time steps)
src = np.arange(10)[:, None]
vals = np.hstack((1.+np.sin(src), 5. + 2.*np.sin(src)))
# Target coordinates
trg = np.linspace(0, 20, 100)[:, None]
# Here we introduce missing values in the second dimension of the source value array
vals[3:5, 1] = np.nan
# interpolation using the convenience function "interpolate"
idw_result = ipol.interpolate(src, trg, vals, ipol.Idw, nnearest=4)
nn_result = ipol.interpolate(src, trg, vals, ipol.Nearest)
# Plot results
fig = pl.figure(figsize=(10,5))
ax = fig.add_subplot(111)
pl1 = ax.plot(trg, idw_result, 'b-', label="IDW")
pl2 = ax.plot(trg, nn_result, 'k-', label="Nearest Neighbour")
pl3 = ax.plot(src, vals, 'ro', label="Observations")

_images/notebooks_interpolation_wradlib_ipol_example_8_0.png

(2) Exemplified for two dimensions in space and two dimensions of the source value array (e.g. time steps), containing also NaN values (here we only use IDW interpolation)

[5]:

# Just a helper function for repeated subplots
def plotall(ax, trgx, trgy, src, interp, pts, title, vmin, vmax):
    ix = np.where(np.isfinite(pts))
    ax.pcolormesh(trgx, trgy, interp.reshape( (len(trgx),len(trgy) ) ), vmin=vmin, vmax=vmax )
    ax.scatter(src[ix, 0].ravel(), src[ix, 1].ravel(), c=pts.ravel()[ix], s=20, marker='s',
               vmin=vmin, vmax=vmax)
    ax.set_title(title)
    pl.axis("tight")

[6]:

# Synthetic observations
src = np.vstack( (np.array([4, 7, 3, 15]), np.array([8, 18, 17, 3])) ).T
np.random.seed(1319622840 + 1)
vals = np.round(np.random.uniform(size=(len(src), 2)), 1)

# Target coordinates
trgx = np.linspace(0, 20, 100)
trgy = np.linspace(0, 20, 100)
trg = np.meshgrid(trgx, trgy)
trg = np.vstack((trg[0].ravel(), trg[1].ravel())).transpose()

result = ipol.interpolate(src, trg, vals, ipol.Idw, nnearest=4)

# Now introduce NaNs in the observations
vals_with_nan = vals.copy()
vals_with_nan[1, 0] = np.nan
vals_with_nan[1:3, 1] = np.nan
result_with_nan = ipol.interpolate(src, trg, vals_with_nan, ipol.Idw, nnearest=4)
vmin = np.concatenate((vals.ravel(), result.ravel())).min()
vmax = np.concatenate((vals.ravel(), result.ravel())).max()

fig = pl.figure(figsize=(8,8))
ax = fig.add_subplot(221)
plotall(ax, trgx, trgy, src, result[:, 0], vals[:, 0], '1st dim: no NaNs', vmin, vmax)
ax = fig.add_subplot(222)
plotall(ax, trgx, trgy, src, result[:, 1], vals[:, 1], '2nd dim: no NaNs', vmin, vmax)
ax = fig.add_subplot(223)
plotall(ax, trgx, trgy, src, result_with_nan[:, 0], vals_with_nan[:, 0], '1st dim: one NaN', vmin, vmax)
ax = fig.add_subplot(224)
plotall(ax, trgx, trgy, src, result_with_nan[:, 1], vals_with_nan[:, 1], '2nd dim: two NaN', vmin, vmax)
pl.tight_layout()

_images/notebooks_interpolation_wradlib_ipol_example_11_0.png

How to use interpolation for gridding data in polar coordinates?¶

Read polar coordinates and corresponding rainfall intensity from file

[7]:

filename = get_wradlib_data_file('misc/bin_coords_tur.gz')
src = np.loadtxt(filename)

filename = get_wradlib_data_file('misc/polar_R_tur.gz')
vals = np.loadtxt(filename)

[8]:

src.shape

[8]:

(46080, 2)

Define target grid coordinates

[9]:

xtrg = np.linspace(src[:,0].min(), src[:,0].max(), 200)
ytrg = np.linspace(src[:,1].min(), src[:,1].max(), 200)
trg = np.meshgrid(xtrg, ytrg)
trg = np.vstack((trg[0].ravel(), trg[1].ravel())).T

Linear Interpolation

[10]:

ip_lin = ipol.Linear(src, trg)
result_lin = ip_lin(vals.ravel(), fill_value=np.nan)

IDW interpolation

[11]:

ip_near = ipol.Nearest(src, trg)
maxdist = trg[1,0] - trg[0,0]
result_near = ip_near(vals.ravel(), maxdist=maxdist)

Plot results

[12]:

fig = pl.figure(figsize=(15, 6))
fig.subplots_adjust(wspace=0.4)
ax = fig.add_subplot(131, aspect="equal")
plot_ppi(vals, ax=ax)
ax = fig.add_subplot(132, aspect="equal")
pl.pcolormesh(xtrg, ytrg, result_lin.reshape( (len(xtrg), len(ytrg)) ) )
ax = fig.add_subplot(133, aspect="equal")
pl.pcolormesh(xtrg, ytrg, result_near.reshape( (len(xtrg), len(ytrg)) ) )

[12]:

<matplotlib.collections.QuadMesh at 0x7fa5602b0f10>

_images/notebooks_interpolation_wradlib_ipol_example_23_1.png

Adjusting radar-base rainfall estimates by rain gauge observations¶

Background¶

There are various ways to correct specific errors and artifacts in radar-based quantitative precipitation estimates (radar QPE). Alternatively, you might want to correct your radar QPE regardless of the error source - by using ground truth, or, more specifically, rain gauge observations. Basically, you define the error of your radar QPE at a rain gauge location by the discrepancy between rain gauge observation (considered as “the truth”) and radar QPE at that very location. Whether you consider this “discrepancy” as an additive or multiplicative error is somehow arbitrary - typically, it’s a mix of both. If you quantify this error at various locations (i.e. rain gauges), you can go ahead and construct correction fields for your radar QPE. You might compute a single correction factor for your entire radar domain (which would e.g. make sense in case of hardware miscalibration), or you might want to compute a spatially variable correction field. This typically implies to interpolate the error in space.

$\omega radlib$ provides different error models and different spatial interpolation methods to address the adjustment problem. For details, please refer to $\omega radlib's$ library reference.

[1]:

import wradlib.adjust as adjust
import wradlib.verify as verify
import wradlib.util as util
import numpy as np
import matplotlib.pyplot as pl

try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()

Example for the 1-dimensional case¶

Looking at the 1-D (instead of 2-D) case is more illustrative.

Create synthetic data¶

First, we create synthetic data: - true rainfall, - point observations of the truth, - radar observations of the truth.

The latter is disturbed by some kind of error, e.g. a combination between systemtic and random error.

[2]:

# gage and radar coordinates
obs_coords = np.array([5, 10, 15, 20, 30, 45, 65, 70, 77, 90])
radar_coords = np.arange(0, 101)

# true rainfall
np.random.seed(1319622840)
truth = np.abs(1.5 + np.sin(0.075 * radar_coords)) + np.random.uniform(
    -0.1, 0.1, len(radar_coords))

# radar error
erroradd = 0.7 * np.sin(0.2 * radar_coords + 10.)
errormult = 0.75 + 0.015 * radar_coords
noise = np.random.uniform(-0.05, 0.05, len(radar_coords))

# radar observation
radar = errormult * truth + erroradd + noise

# gage observations are assumed to be perfect
obs = truth[obs_coords]

# add a missing value to observations (just for testing)
obs[1] = np.nan

Apply different adjustment methods¶

additive error, spatially variable (AdjustAdd)
multiplicative error, spatially variable (AdjustMultiply)
mixed error, spatially variable (AdjustMixed)
multiplicative error, spatially uniform (AdjustMFB)

[3]:

# number of neighbours to be used
nnear_raws = 3

# adjust the radar observation by additive model
add_adjuster = adjust.AdjustAdd(obs_coords, radar_coords,
                                nnear_raws=nnear_raws)
add_adjusted = add_adjuster(obs, radar)

# adjust the radar observation by multiplicative model
mult_adjuster = adjust.AdjustMultiply(obs_coords, radar_coords,
                                      nnear_raws=nnear_raws)
mult_adjusted = mult_adjuster(obs, radar)

# adjust the radar observation by AdjustMixed
mixed_adjuster = adjust.AdjustMixed(obs_coords, radar_coords,
                                    nnear_raws=nnear_raws)
mixed_adjusted = mixed_adjuster(obs, radar)

# adjust the radar observation by MFB
mfb_adjuster = adjust.AdjustMFB(obs_coords, radar_coords,
                                nnear_raws=nnear_raws,
                                mfb_args = dict(method="median"))
mfb_adjusted = mfb_adjuster(obs, radar)

/home/travis/miniconda/envs/travis_wradlib/lib/python3.8/site-packages/wradlib/ipol.py:375: RuntimeWarning: divide by zero encountered in true_divide
  weights = 1.0 / self.dists ** self.p
/home/travis/miniconda/envs/travis_wradlib/lib/python3.8/site-packages/numpy/core/fromnumeric.py:746: UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
  a.partition(kth, axis=axis, kind=kind, order=order)

Plot adjustment results¶

[4]:

# Enlarge all label fonts
font = {'size'   : 15}
pl.rc('font', **font)

pl.figure(figsize=(10,5))
pl.plot(radar_coords, radar,         'k-', linewidth=2., linestyle="dashed", label="Unadjusted radar", )
pl.plot(radar_coords, truth,         'k-', linewidth=2., label="True rainfall", )
pl.plot(obs_coords,   obs,           'o',  markersize=10.0, markerfacecolor="grey", label="Gage observation")
pl.plot(radar_coords, add_adjusted,  '-',  color="red",   label="Additive adjustment")
pl.plot(radar_coords, mult_adjusted, '-',  color="green", label="Multiplicative adjustment")
pl.plot(radar_coords, mfb_adjusted,  '-',  color="orange", label="Mean Field Bias adjustment")
pl.plot(radar_coords, mixed_adjusted,'-',  color="blue", label="Mixed (mult./add.) adjustment")
pl.xlabel("Distance (km)")
pl.ylabel("Rainfall intensity (mm/h)")
leg = pl.legend(prop={'size': 10})

_images/notebooks_multisensor_wradlib_adjust_example_14_0.png

Verification¶

We use the verify module to compare the errors of different adjustment approaches.

Here, we compare the adjustment to the “truth”. In practice, we would carry out a cross validation.

[5]:

# Verification for this example
rawerror = verify.ErrorMetrics(truth, radar)
mfberror = verify.ErrorMetrics(truth, mfb_adjusted)
adderror = verify.ErrorMetrics(truth, add_adjusted)
multerror = verify.ErrorMetrics(truth, mult_adjusted)
mixerror = verify.ErrorMetrics(truth, mixed_adjusted)

[6]:

# Helper function for scatter plot
def scatterplot(x, y, title=""):
    """Quick and dirty helper function to produce scatter plots
    """
    pl.scatter(x, y)
    pl.plot([0, 1.2 * maxval], [0, 1.2 * maxval], '-', color='grey')
    pl.xlabel("True rainfall (mm)")
    pl.ylabel("Estimated rainfall (mm)")
    pl.xlim(0, maxval + 0.1 * maxval)
    pl.ylim(0, maxval + 0.1 * maxval)
    pl.title(title)

[7]:

# Verification reports
maxval = 4.
# Enlarge all label fonts
font = {'size'   : 10}
pl.rc('font', **font)
fig = pl.figure(figsize=(14, 8))
ax = fig.add_subplot(231, aspect=1.)
scatterplot(rawerror.obs, rawerror.est, title="Unadjusted radar")
ax.text(0.2, maxval, "Nash=%.1f" % rawerror.nash(), fontsize=12)
ax = fig.add_subplot(232, aspect=1.)
scatterplot(adderror.obs, adderror.est, title="Additive adjustment")
ax.text(0.2, maxval, "Nash=%.1f" % adderror.nash(), fontsize=12)
ax = fig.add_subplot(233, aspect=1.)
scatterplot(multerror.obs, multerror.est, title="Multiplicative adjustment")
ax.text(0.2, maxval, "Nash=%.1f" % multerror.nash(), fontsize=12)
ax = fig.add_subplot(234, aspect=1.)
scatterplot(mixerror.obs, mixerror.est, title="Mixed (mult./add.) adjustment")
ax.text(0.2, maxval, "Nash=%.1f" % mixerror.nash(), fontsize=12)
ax = fig.add_subplot(235, aspect=1.)
scatterplot(mfberror.obs, mfberror.est, title="Mean Field Bias adjustment")
ax.text(0.2, maxval, "Nash=%.1f" % mfberror.nash(), fontsize=12)
pl.tight_layout()

_images/notebooks_multisensor_wradlib_adjust_example_19_0.png

Example for the 2-dimensional case¶

For the 2-D case, we follow the same approach as before:

create synthetic data: truth, rain gauge observations, radar-based rainfall estimates
apply adjustment methods
verification

The way these synthetic data are created is totally arbitrary - it’s just to show how the methods are applied.

Create 2-D synthetic data¶

[8]:

# grid axes
xgrid = np.arange(0, 10)
ygrid = np.arange(20, 30)

# number of observations
num_obs = 10

# create grid
gridshape = len(xgrid), len(ygrid)
grid_coords = util.gridaspoints(ygrid, xgrid)

# Synthetic true rainfall
truth = np.abs(10. * np.sin(0.1 * grid_coords).sum(axis=1))

# Creating radar data by perturbing truth with multiplicative and
# additive error
# YOU CAN EXPERIMENT WITH THE ERROR STRUCTURE
np.random.seed(1319622840)
radar = 0.6 * truth + 1. * np.random.uniform(low=-1., high=1,
                                             size=len(truth))
radar[radar < 0.] = 0.

# indices for creating obs from raw (random placement of gauges)
obs_ix = np.random.uniform(low=0, high=len(grid_coords),
                           size=num_obs).astype('i4')

# creating obs_coordinates
obs_coords = grid_coords[obs_ix]

# creating gauge observations from truth
obs = truth[obs_ix]

Apply different adjustment methods¶

[9]:

# Mean Field Bias Adjustment
mfbadjuster = adjust.AdjustMFB(obs_coords, grid_coords)
mfbadjusted = mfbadjuster(obs, radar)

# Additive Error Model
addadjuster = adjust.AdjustAdd(obs_coords, grid_coords)
addadjusted = addadjuster(obs, radar)

# Multiplicative Error Model
multadjuster = adjust.AdjustMultiply(obs_coords, grid_coords)
multadjusted = multadjuster(obs, radar)

/home/travis/miniconda/envs/travis_wradlib/lib/python3.8/site-packages/wradlib/adjust.py:619: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  slope, _, _, _ = np.linalg.lstsq(x, y)
/home/travis/miniconda/envs/travis_wradlib/lib/python3.8/site-packages/wradlib/ipol.py:375: RuntimeWarning: divide by zero encountered in true_divide
  weights = 1.0 / self.dists ** self.p

Plot 2-D adjustment results¶

[10]:

# Helper functions for grid plots
def gridplot(data, title):
    """Quick and dirty helper function to produce a grid plot
    """
    xplot = np.append(xgrid, xgrid[-1] + 1.) - 0.5
    yplot = np.append(ygrid, ygrid[-1] + 1.) - 0.5
    grd = ax.pcolormesh(xplot, yplot, data.reshape(gridshape), vmin=0,
                        vmax=maxval)
    ax.scatter(obs_coords[:, 0], obs_coords[:, 1], c=obs.ravel(),
               marker='s', s=50, vmin=0, vmax=maxval)
    #pl.colorbar(grd, shrink=0.5)
    pl.title(title)

[11]:

# Maximum value (used for normalisation of colorscales)
maxval = np.max(np.concatenate((truth, radar, obs, addadjusted)).ravel())

# open figure
fig = pl.figure(figsize=(10, 6))

# True rainfall
ax = fig.add_subplot(231, aspect='equal')
gridplot(truth, 'True rainfall')

# Unadjusted radar rainfall
ax = fig.add_subplot(232, aspect='equal')
gridplot(radar, 'Radar rainfall')

# Adjusted radar rainfall (MFB)
ax = fig.add_subplot(234, aspect='equal')
gridplot(mfbadjusted, 'Adjusted (MFB)')

# Adjusted radar rainfall (additive)
ax = fig.add_subplot(235, aspect='equal')
gridplot(addadjusted, 'Adjusted (Add.)')

# Adjusted radar rainfall (multiplicative)
ax = fig.add_subplot(236, aspect='equal')
gridplot(multadjusted, 'Adjusted (Mult.)')

pl.tight_layout()

_images/notebooks_multisensor_wradlib_adjust_example_28_0.png

[12]:

# Open figure
fig = pl.figure(figsize=(6, 6))

# Scatter plot radar vs. observations
ax = fig.add_subplot(221, aspect='equal')
scatterplot(truth, radar, 'Radar vs. Truth (red: Gauges)')
pl.plot(obs, radar[obs_ix], linestyle="None", marker="o", color="red")

# Adjusted (MFB) vs. radar (for control purposes)
ax = fig.add_subplot(222, aspect='equal')
scatterplot(truth, mfbadjusted, 'Adjusted (MFB) vs. Truth')

# Adjusted (Add) vs. radar (for control purposes)
ax = fig.add_subplot(223, aspect='equal')
scatterplot(truth, addadjusted, 'Adjusted (Add.) vs. Truth')

# Adjusted (Mult.) vs. radar (for control purposes)
ax = fig.add_subplot(224, aspect='equal')
scatterplot(truth, multadjusted, 'Adjusted (Mult.) vs. Truth')

pl.tight_layout()

_images/notebooks_multisensor_wradlib_adjust_example_29_0.png

Zonal Statistics¶

A quick start to zonal statistics¶

Zonal statistics can be used to compute e.g. the areal average precipitation over a catchment.

Here, we show a brief example using RADOLAN composite data from the German Weather Service (DWD).

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np
from osgeo import osr

Preparing the RADOLAN data¶

Preparing the radar composite data includes to - read the data, - geoference the data in native RADOLAN projection, - reproject the data to Germany Zone 2 projection.

[2]:

# Read and preprocess the RADOLAN data
fpath = 'radolan/misc/raa01-sf_10000-1406100050-dwd---bin.gz'
f = wrl.util.get_wradlib_data_file(fpath)
data, attrs = wrl.io.read_radolan_composite(f, missing=np.nan)
sec = attrs['secondary']
data.flat[sec] = np.nan

[3]:

# Get RADOLAN grid coordinates (lon/lat)
grid_xy_radolan = wrl.georef.get_radolan_grid(900, 900)
x_radolan = grid_xy_radolan[:, :, 0]
y_radolan = grid_xy_radolan[:, :, 1]

[4]:

# This is the native RADOLAN projection
# (polar stereographic projection)
proj_stereo = wrl.georef.create_osr("dwd-radolan")

# This is our target projection (UTM Zone 32)
proj_utm = osr.SpatialReference()
_ = proj_utm.ImportFromEPSG(32632)

[5]:

# Reproject the RADOLAN coordinates
xy = wrl.georef.reproject(grid_xy_radolan,
                          projection_source=proj_stereo,
                          projection_target=proj_utm)

Import catchment boundaries from ESRI shapefile¶

[6]:

# Open shapefile (source is in GK2)
proj_gk2 = osr.SpatialReference()
proj_gk2.ImportFromEPSG(31466)
shpfile = wrl.util.get_wradlib_data_file(
    'shapefiles/agger/agger_merge.shp')
dataset, inLayer = wrl.io.open_vector(shpfile)
cats, keys = wrl.georef.get_vector_coordinates(inLayer, dest_srs=proj_utm, source_srs=proj_gk2)
print("Found %d sub-catchments in shapefile." % len(cats))

Found 13 sub-catchments in shapefile.

Clip subgrid from RADOLAN grid¶

This is just to speed up the computation (so we don’t have to deal with the full grid).

[7]:

bbox = inLayer.GetExtent()
bbox = wrl.georef.reproject([[bbox[0], bbox[2]], [bbox[1], bbox[3]]],
                            projection_source=proj_gk2,
                            projection_target=proj_utm)
bbox = (bbox[0, 0], bbox[1, 0], bbox[0, 1], bbox[1, 1])
buffer = 5000.
bbox = dict(left=bbox[0] - buffer, right=bbox[1] + buffer,
            bottom=bbox[2] - buffer, top=bbox[3] + buffer)
mask = (((xy[..., 1] > bbox['bottom']) & (xy[..., 1] < bbox['top'])) &
        ((xy[..., 0] > bbox['left']) & (xy[..., 0] < bbox['right'])))
xy_ = np.dstack((xy[..., 0][mask].ravel(), xy[..., 1][mask].ravel()))
data_ = data[mask]

Compute the average precipitation for each catchment¶

To compute the zonal average, we have to understand the the grid cells as polygons defined by a set of vertices.

[8]:

# Create vertices for each grid cell
# (MUST BE DONE IN NATIVE RADOLAN COORDINATES)
grdverts = wrl.zonalstats.grid_centers_to_vertices(x_radolan[mask],
                                                   y_radolan[mask],
                                                   1., 1.)
# And reproject to our target projection (here: UTM)
grdverts = wrl.georef.reproject(grdverts,
                                projection_source=proj_stereo,
                                projection_target=proj_utm)

Based on the overlap of these polygons with the catchment area, we can then compute a weighted average.

[9]:

# This object collects our source and target data
#   and computes the intersections
zd = wrl.zonalstats.ZonalDataPoly(grdverts, cats, srs=proj_utm)

# This object can actually compute the statistics
obj = wrl.zonalstats.ZonalStatsPoly(zd)

# We just call this object with any set of radar data
avg = obj.mean(data_.ravel())

Plot results in map¶

[10]:

from matplotlib.collections import PatchCollection
from matplotlib.colors import from_levels_and_colors
import matplotlib.patches as patches

# Create discrete colormap
levels=np.arange(0,30,2.5)
colors = pl.cm.inferno(np.linspace(0, 1, len(levels)))
mycmap, mynorm = from_levels_and_colors(levels, colors, extend="max")

fig = pl.figure(figsize=(10, 10))

# Average rainfall sum
ax = fig.add_subplot(121, aspect="equal")
patches = [patches.Polygon(item, True) for item in obj.zdata.trg.data]
coll = PatchCollection(patches, array=avg, cmap=mycmap, norm=mynorm,
                       edgecolors='white', lw=0.5)
ax.add_collection(coll)
pl.colorbar(coll, ax=ax, orientation="horizontal", pad=0.05)
pl.xlabel("UTM Zone 32 Easting (m)")
pl.ylabel("UTM Zone 32 Northing (m)")
pl.title("Catchment areal average")
pl.xlim(bbox["left"], bbox["right"])
pl.ylim(bbox["bottom"], bbox["top"])
pl.grid()

# Original radar data
ax = fig.add_subplot(122, aspect="equal")
pm = pl.pcolormesh(xy[:, :, 0], xy[:, :, 1],
                   np.ma.masked_invalid(data),
                   cmap=mycmap, norm=mynorm)
wrl.vis.add_patches(ax=ax, patch_array=cats,
                    facecolor="None", edgecolor="white")
cb = pl.colorbar(pm, ax=ax, orientation="horizontal", pad=0.05)
cb.set_label("(mm/h)")
pl.xlabel("UTM Zone 32 Easting (m)")
pl.ylabel("UTM Zone 32 Northing (m)")
pl.title("RADOLAN rain depth")
pl.xlim(bbox["left"], bbox["right"])
pl.ylim(bbox["bottom"], bbox["top"])
pl.grid(color="white")
pl.tight_layout()

Save time by reading the weights from a file¶

The computational expensive part is the computation of intersections and weights. You only need to do it once.

You can dump the results on disk and read them from there when required. Let’s do a little benchmark:

[11]:

import datetime as dt

# dump to file
zd.dump_vector('test_zonal_poly_cart')

t1 = dt.datetime.now()
# Create instance of type ZonalStatsPoly from zonal data file
obj = wrl.zonalstats.ZonalStatsPoly('test_zonal_poly_cart')
t2 = dt.datetime.now()

# Create instance of type ZonalStatsPoly from sratch
zd = wrl.zonalstats.ZonalDataPoly(grdverts, cats, srs=proj_utm)
obj = wrl.zonalstats.ZonalStatsPoly(zd)
t3 = dt.datetime.now()

# Calling the object
avg = obj.mean(data_.ravel())
t4 = dt.datetime.now()

print("\nCreate object from file: %f seconds" % (t2 - t1).total_seconds())
print("Create object from sratch: %f seconds" % (t3 - t2).total_seconds())
print("Compute stats: %f seconds" % (t4 - t3).total_seconds())


Create object from file: 0.125561 seconds
Create object from sratch: 2.829500 seconds
Compute stats: 0.002523 seconds

_images/notebooks_zonalstats_wradlib_zonalstats_quickstart_25_1.png

Zonal Statistics Example¶

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

Setup Examples¶

[2]:

def testplot(cats, catsavg, xy, data,
             levels=[0, 1, 2, 3, 4, 5, 10, 15, 20, 25, 30, 40, 50, 100],
             title=""):
    """Quick test plot layout for this example file
    """
    colors = pl.cm.viridis(np.linspace(0, 1, len(levels)))
    mycmap, mynorm = from_levels_and_colors(levels, colors, extend="max")

    radolevels = [0, 1, 2, 3, 4, 5, 10, 15, 20, 25, 30, 40, 50, 100]
    radocolors = pl.cm.viridis(np.linspace(0, 1, len(radolevels)))
    radocmap, radonorm = from_levels_and_colors(radolevels, radocolors,
                                                extend="max")

    fig = pl.figure(figsize=(10, 16))

    # Average rainfall sum
    ax = fig.add_subplot(211, aspect="equal")
    coll = PatchCollection(cats, array=catsavg, cmap=mycmap, norm=mynorm,
                           edgecolors='white', lw=0.5)
    ax.add_collection(coll)
    ax.autoscale()
    pl.colorbar(coll, ax=ax, shrink=0.5)
    pl.xlabel("UTM Zone 32 Easting")
    pl.ylabel("UTM Zone 32 Northing")
    pl.title(title)
    pl.draw()

    # Original radar data
    ax1 = fig.add_subplot(212, aspect="equal")
    pm = pl.pcolormesh(xy[:, :, 0], xy[:, :, 1], np.ma.masked_invalid(data),
                        cmap=radocmap, norm=radonorm)
    coll = PatchCollection(cats, facecolor='None', edgecolor='white', lw=0.5)
    ax1.add_collection(coll)
    cb = pl.colorbar(pm, ax=ax1, shrink=0.5)
    cb.set_label("(mm/h)")
    pl.xlabel("UTM Zone 32 Easting")
    pl.ylabel("UTM Zone 32 Northing")
    pl.title("Original radar rain sums")
    pl.draw()
    pl.tight_layout()

Zonal Stats Rectangular Grid¶

[3]:

from matplotlib.collections import PatchCollection
from matplotlib.colors import from_levels_and_colors
import matplotlib.patches as patches
import datetime as dt
from osgeo import osr

[4]:

# check for GEOS enabled GDAL
if not wrl.util.has_geos():
    print("NO GEOS support within GDAL, aborting...")
    exit(0)

[5]:

# Get RADOLAN grid coordinates
grid_xy_radolan = wrl.georef.get_radolan_grid(900, 900)
x_radolan = grid_xy_radolan[:, :, 0]
y_radolan = grid_xy_radolan[:, :, 1]

# create radolan projection osr object
proj_stereo = wrl.georef.create_osr("dwd-radolan")

# create UTM Zone 32 projection osr object
proj_utm = osr.SpatialReference()
proj_utm.ImportFromEPSG(32632)

# transform radolan polar stereographic projection to UTM Zone 32
xy = wrl.georef.reproject(grid_xy_radolan,
                          projection_source=proj_stereo,
                          projection_target=proj_utm)

# Open shapefile (in GK2)
proj_gk2 = osr.SpatialReference()
proj_gk2.ImportFromEPSG(31466)
shpfile = wrl.util.get_wradlib_data_file('shapefiles/agger/agger_merge.shp')
dataset, inLayer = wrl.io.open_vector(shpfile)
cats, keys = wrl.georef.get_vector_coordinates(inLayer, dest_srs=proj_utm, source_srs=proj_gk2)

# Read and prepare the actual data (RADOLAN)
f = wrl.util.get_wradlib_data_file(
    'radolan/misc/raa01-sf_10000-1406100050-dwd---bin.gz')
data, attrs = wrl.io.read_radolan_composite(f, missing=np.nan)
sec = attrs['secondary']
data.flat[sec] = np.nan

# Reduce grid size using a bounding box (to enhancing performance)
bbox = inLayer.GetExtent()
bbox = wrl.georef.reproject([[bbox[0], bbox[2]], [bbox[1], bbox[3]]],
                            projection_source=proj_gk2,
                            projection_target=proj_utm)
bbox = (bbox[0, 0], bbox[1, 0], bbox[0, 1], bbox[1, 1])

buffer = 5000.
bbox = dict(left=bbox[0] - buffer, right=bbox[1] + buffer,
            bottom=bbox[2] - buffer, top=bbox[3] + buffer)
mask = (((xy[..., 1] > bbox['bottom']) & (xy[..., 1] < bbox['top'])) &
        ((xy[..., 0] > bbox['left']) & (xy[..., 0] < bbox['right'])))
xy_ = np.vstack((xy[..., 0][mask].ravel(), xy[..., 1][mask].ravel())).T
data_ = data[mask]

[6]:

###########################################################################
# Approach #1: Assign grid points to each polygon and compute the average.
#
# - Uses matplotlib.path.Path
# - Each point is weighted equally (assumption: polygon >> grid cell)
# - this is quick, but theoretically dirty
###########################################################################

t1 = dt.datetime.now()

# Create instance of type ZonalDataPoint from source grid and
# catchment array
zd = wrl.zonalstats.ZonalDataPoint(xy_, cats, srs=proj_utm, buf=500.)
# dump to file (for later use - see below)
zd.dump_vector('test_zonal_points_cart')
# Create instance of type ZonalStatsPoint from zonal data object
obj1 = wrl.zonalstats.ZonalStatsPoint(zd)

isecs1 = obj1.zdata.isecs  # for plotting (see below)

t2 = dt.datetime.now()

# Compute stats for target polygons
avg1 = obj1.mean(data_.ravel())
var1 = obj1.var(data_.ravel())

t3 = dt.datetime.now()

# Create instance of type ZonalStatsPoint from zonal data file
# (much faster)
obj1 = wrl.zonalstats.ZonalStatsPoint('test_zonal_points_cart')

t4 = dt.datetime.now()

print("Approach #1 computation time:")
print("\tCreate object from scratch: %f "
      "seconds" % (t2 - t1).total_seconds())
print("\tCreate object from dumped file: %f "
      "seconds" % (t4 - t3).total_seconds())
print("\tCompute stats using object: %f "
      "seconds" % (t3 - t2).total_seconds())

# PLOTTING Approach #1

# Just a test for plotting results with zero buffer
zd2 = wrl.zonalstats.ZonalDataPoint(xy_, cats, buf=0)
# Create instance of type ZonalStatsPoint from zonal data object
obj2 = wrl.zonalstats.ZonalStatsPoint(zd2)
isecs2 = obj2.zdata.isecs

Approach #1 computation time:
        Create object from scratch: 3.306843 seconds
        Create object from dumped file: 0.115608 seconds
        Compute stats using object: 0.007780 seconds

[7]:

# Illustrate results for an example catchment i
i = 6  # try e.g. 48, 100
fig = pl.figure(figsize=(10,8))
ax = fig.add_subplot(111, aspect="equal")

# Target polygon patches
trg_patches = [patches.Polygon(item, True) for item in obj1.zdata.trg.data]
trg_patch = [trg_patches[i]]
p = PatchCollection(trg_patch, facecolor="None", edgecolor="black",
                    linewidth=2)
ax.add_collection(p)

# pips
sources = obj1.zdata.src.data
pl.scatter(sources[:, 0], sources[:, 1], s=200, c="grey",
            edgecolor="None", label="all points")
pl.scatter(isecs2[i][:, 0], isecs2[i][:, 1], s=200, c="green",
            edgecolor="None", label="buffer=0 m")
pl.scatter(isecs1[i][:, 0], isecs1[i][:, 1], s=50, c="red",
            edgecolor="None", label="buffer=500 m")
bbox = wrl.zonalstats.get_bbox(cats[i][:, 0], cats[i][:, 1])
pl.xlim(bbox["left"] - 2000, bbox["right"] + 2000)
pl.ylim(bbox["bottom"] - 2000, bbox["top"] + 2000)
pl.legend()
pl.title("Catchment #%d: Points considered for stats" % i)

[7]:

Text(0.5, 1.0, 'Catchment #6: Points considered for stats')

[8]:

# Plot average rainfall and original data
testplot(trg_patches, avg1, xy, data,
         title="Catchment rainfall mean (ZonalStatsPoint)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_11_0.png

_images/notebooks_zonalstats_wradlib_zonalstats_example_11_1.png

[9]:

testplot(trg_patches, var1, xy, data,
         levels=np.arange(0, np.max(var1), 1.),
         title="Catchment rainfall variance (ZonalStatsPoint)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_12_0.png

[10]:

###########################################################################
# Approach #2: Compute weighted mean based on fraction of source polygons
# in target polygons
#
# - This is more accurate (no assumptions), but probably slower...
###########################################################################

# Create vertices for each grid cell
# (MUST BE DONE IN NATIVE RADOLAN COORDINATES)
grdverts = wrl.zonalstats.grid_centers_to_vertices(x_radolan[mask],
                                                   y_radolan[mask], 1.,
                                                   1.)
# And reproject to Cartesian reference system (here: UTM Zone 32)
grdverts = wrl.georef.reproject(grdverts,
                                projection_source=proj_stereo,
                                projection_target=proj_utm)

t1 = dt.datetime.now()

# Create instance of type ZonalDataPoly from source grid and
# catchment array
zd = wrl.zonalstats.ZonalDataPoly(grdverts, cats, srs=proj_utm)
# dump to file
zd.dump_vector('test_zonal_poly_cart')
# Create instance of type ZonalStatsPoint from zonal data object
obj3 = wrl.zonalstats.ZonalStatsPoly(zd)

t2 = dt.datetime.now()

# Compute stats for target polygons
avg3 = obj3.mean(data_.ravel())
var3 = obj3.var(data_.ravel())

t3 = dt.datetime.now()

# Create instance of type ZonalStatsPoly from zonal data file
obj3 = wrl.zonalstats.ZonalStatsPoly('test_zonal_poly_cart')

t4 = dt.datetime.now()

print("Approach #2 computation time:")
print("\tCreate object from scratch: %f "
      "seconds" % (t2 - t1).total_seconds())
print("\tCreate object from dumped file: %f "
      "seconds" % (t4 - t3).total_seconds())
print("\tCompute stats using object: %f "
      "seconds" % (t3 - t2).total_seconds())

Approach #2 computation time:
        Create object from scratch: 2.940716 seconds
        Create object from dumped file: 0.151661 seconds
        Compute stats using object: 0.006372 seconds

[11]:

# PLOTTING Approach #2

# Target polygon patches
trg_patches = [patches.Polygon(item, True) for item in obj3.zdata.trg.data]

# Plot average rainfall and original data
testplot(trg_patches, avg3, xy, data,
         title="Catchment rainfall mean (ZonalStatsPoly)")

[12]:

testplot(trg_patches, var3, xy, data,
         levels=np.arange(0, np.max(var3), 1.),
         title="Catchment rainfall variance (ZonalStatsPoly)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_15_0.png

_images/notebooks_zonalstats_wradlib_zonalstats_example_15_1.png

[13]:

# Illustrate results for an example catchment i
i = 6  # try any index between 0 and 13
fig = pl.figure(figsize=(10,8))
ax = fig.add_subplot(111, aspect="equal")

# Grid cell patches
src_index = obj3.zdata.get_source_index(i)
grd_patches = [patches.Polygon(item)
               for item in obj3.zdata.src.get_data_by_idx(src_index)]
p = PatchCollection(grd_patches, facecolor="None", edgecolor="black")
ax.add_collection(p)

# Target polygon patches
trg_patch = [trg_patches[i]]
p = PatchCollection(trg_patch, facecolor="None", edgecolor="red",
                    linewidth=2)
ax.add_collection(p)

# View the actual intersections
isecs = obj3.zdata.get_isec(i)
isec_patches = wrl.zonalstats.numpy_to_pathpatch(isecs)
colors = 100 * np.linspace(0, 1., len(isec_patches))
p = PatchCollection(isec_patches, cmap=pl.cm.plasma, alpha=0.5)
p.set_array(np.array(colors))
ax.add_collection(p)

bbox = wrl.zonalstats.get_bbox(cats[i][:, 0], cats[i][:, 1])
pl.xlim(bbox["left"] - 2000, bbox["right"] + 2000)
pl.ylim(bbox["bottom"] - 2000, bbox["top"] + 2000)
pl.draw()

[14]:

# Compare estimates
maxlim = np.max(np.concatenate((avg1, avg3)))
fig = pl.figure(figsize=(10, 8))
ax = fig.add_subplot(111, aspect="equal")
pl.scatter(avg1, avg3, edgecolor="None", alpha=0.5)
pl.xlabel("Average of points in or close to polygon (mm)")
pl.ylabel("Area-weighted average (mm)")
pl.xlim(0, maxlim)
pl.ylim(0, maxlim)
pl.plot([-1, maxlim + 1], [-1, maxlim + 1], color="black")
pl.show()

_images/notebooks_zonalstats_wradlib_zonalstats_example_17_0.png

_images/notebooks_zonalstats_wradlib_zonalstats_example_17_1.png

Zonal Stats Polar Grid¶

[15]:

filename = wrl.util.get_wradlib_data_file('hdf5/rainsum_boxpol_20140609.h5')
data, attrib = wrl.io.from_hdf5(filename)

# get Lat, Lon, range, azimuth, rays, bins out of radar data
lat1 = attrib['Latitude']
lon1 = attrib['Longitude']
r1 = attrib['r']
a1 = attrib['az']

rays = a1.shape[0]
bins = r1.shape[0]

# setup OSR objects
# create UTM Zone 32 projection osr object
proj_utm = osr.SpatialReference()
proj_utm.ImportFromEPSG(32632)

# create polar grid polygon vertices in GK2
radar_utm = wrl.georef.spherical_to_polyvert(r1, a1, 0, (lon1, lat1),
                                             proj=proj_utm)
radar_utm = radar_utm[..., 0:2]

# create polar grid centroids in GK2
radar_utmc = wrl.georef.spherical_to_centroids(r1, a1, 0, (lon1, lat1),
                                               proj=proj_utm)
radar_utmc = radar_utmc[..., 0:2]

# reshape
radar_utm.shape = (rays, bins, 5, 2)
radar_utmc.shape = (rays, bins, 2)
shpfile = wrl.util.get_wradlib_data_file('shapefiles/agger/agger_merge.shp')
proj_gk = osr.SpatialReference()
proj_gk.ImportFromEPSG(31466)
dataset, inLayer = wrl.io.open_vector(shpfile)

cats, keys = wrl.georef.get_vector_coordinates(inLayer, dest_srs=proj_utm, source_srs=proj_gk2)


# create synthetic box
box = np.array([[390000., 5630000.], [390000., 5640000.],
                [400000., 5640000.], [400000., 5630000.],
                [390000., 5630000.]])
l = list(cats)

# Todo: remove after testing
l = l[0:3]

l.append(box)
cats = np.array(l)
bbox = inLayer.GetExtent()
bbox = wrl.georef.reproject([[bbox[0], bbox[2]], [bbox[1], bbox[3]]],
                            projection_source=proj_gk2,
                            projection_target=proj_utm)
bbox = (bbox[0, 0], bbox[1, 0], bbox[0, 1], bbox[1, 1])

# create catchment bounding box
buffer = 5000.
bbox = dict(left=bbox[0]-buffer, right=bbox[1]+buffer,
            bottom=bbox[2]-buffer, top=bbox[3]+buffer)

mask = (((radar_utmc[..., 1] > bbox['bottom']) & (radar_utmc[..., 1] < bbox['top'])) &
        ((radar_utmc[..., 0] > bbox['left']) & (radar_utmc[..., 0] < bbox['right'])))

radar_utmc_ = radar_utmc[mask, :]
radar_utm_ = radar_utm[mask]
data_ = data[mask]

[16]:

###########################################################################
# Approach #1: Assign grid points to each polygon and compute the average.
#
# - Uses matplotlib.path.Path
# - Each point is weighted equally (assumption: polygon >> grid cell)
# - this is quick, but theoretically dirty
# - for polar grids a range-area dependency has to be taken into account
###########################################################################

t1 = dt.datetime.now()

# Create instance of type ZonalDataPoint from source grid and
# catchment array
zd = wrl.zonalstats.ZonalDataPoint(radar_utmc_, cats, srs=proj_utm,
                                   buf=500.)
# dump to file
zd.dump_vector('test_zonal_points')
# Create instance of type ZonalStatsPoint from zonal data object
obj1 = wrl.zonalstats.ZonalStatsPoint(zd)

isecs1 = obj1.zdata.isecs
t2 = dt.datetime.now()

# Compute stats for target polygons
avg1 = obj1.mean(data_.ravel())
var1 = obj1.var(data_.ravel())

t3 = dt.datetime.now()

# Create instance of type ZonalStatsPoint from zonal data file
obj1 = wrl.zonalstats.ZonalStatsPoint('test_zonal_points')

t4 = dt.datetime.now()

print ("Approach #1 computation time:")
print(
"\tCreate object from scratch: %f seconds" % (t2 - t1).total_seconds())
print(
"\tCreate object from dumped file: %f seconds" % (t4 - t3).total_seconds())
print(
"\tCompute stats using object: %f seconds" % (t3 - t2).total_seconds())

Approach #1 computation time:
        Create object from scratch: 2.071067 seconds
        Create object from dumped file: 0.380302 seconds
        Compute stats using object: 0.001848 seconds

[17]:

# PLOTTING Approach #2

# Just a test for plotting results with zero buffer
zd = wrl.zonalstats.ZonalDataPoint(radar_utmc_, cats, buf=0)
# Create instance of type ZonalStatsPoint from zonal data object
obj2 = wrl.zonalstats.ZonalStatsPoint(zd)
isecs2 = obj2.zdata.isecs

[18]:

# Illustrate results for an example catchment i
i = 0  # try e.g. 6, 12
fig = pl.figure(figsize=(10,8))
ax = fig.add_subplot(111, aspect="equal")

# Target polygon patches
trg_patches = [patches.Polygon(item, True) for item in obj1.zdata.trg.data]
trg_patch = [trg_patches[i]]
p = PatchCollection(trg_patch, facecolor="None", edgecolor="black",
                    linewidth=2)
ax.add_collection(p)

# pips
sources = obj1.zdata.src.data
pl.scatter(sources[:, 0], sources[:, 1], s=200, c="grey",
           edgecolor="None", label="all points")
pl.scatter(isecs2[i][:, 0], isecs2[i][:, 1], s=200, c="green",
           edgecolor="None", label="buffer=0 m")
pl.scatter(isecs1[i][:, 0], isecs1[i][:, 1], s=50, c="red",
           edgecolor="None", label="buffer=500 m")
bbox = wrl.zonalstats.get_bbox(cats[i][:, 0], cats[i][:, 1])
pl.xlim(bbox["left"] - 2000, bbox["right"] + 2000)
pl.ylim(bbox["bottom"] - 2000, bbox["top"] + 2000)
pl.legend()
pl.title("Catchment #%d: Points considered for stats" % i)

[18]:

Text(0.5, 1.0, 'Catchment #0: Points considered for stats')

[19]:

# Plot average rainfall and original data
testplot(trg_patches, avg1, radar_utmc, data,
         title="Catchment rainfall mean (ZonalStatsPoint)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_23_0.png

_images/notebooks_zonalstats_wradlib_zonalstats_example_23_1.png

[20]:

testplot(trg_patches, var1, radar_utmc, data, levels=np.arange(0, 20, 1.0),
         title="Catchment rainfall variance (ZonalStatsPoint)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_24_0.png

[21]:

###########################################################################
# Approach #2: Compute weighted mean based on fraction of source polygons
# in target polygons
#
# - This is more accurate (no assumptions), but probably slower...
###########################################################################

t1 = dt.datetime.now()

# Create instance of type ZonalDataPoly from source grid and
# catchment array
zd = wrl.zonalstats.ZonalDataPoly(radar_utm_, cats, srs=proj_utm)
# dump to file
zd.dump_vector('test_zonal_poly')
# Create instance of type ZonalStatsPoint from zonal data object
obj3 = wrl.zonalstats.ZonalStatsPoly(zd)

obj3.zdata.dump_vector('test_zonal_poly')
t2 = dt.datetime.now()

avg3 = obj3.mean(data_.ravel())
var3 = obj3.var(data_.ravel())

t3 = dt.datetime.now()

# Create instance of type ZonalStatsPoly from zonal data file
obj4 = wrl.zonalstats.ZonalStatsPoly('test_zonal_poly')

t4 = dt.datetime.now()

print ("Approach #2 computation time:")
print(
"\tCreate object from scratch: %f seconds" % (t2 - t1).total_seconds())
print(
"\tCreate object from dumped file: %f seconds" % (t4 - t3).total_seconds())
print(
"\tCompute stats using object: %f seconds" % (t3 - t2).total_seconds())

obj3.zdata.trg.dump_raster('test_zonal_hdr.nc', 'netCDF', 'mean',
                           pixel_size=100.)

obj3.zdata.trg.dump_vector('test_zonal_shp')
obj3.zdata.trg.dump_vector('test_zonal_json.geojson', 'GeoJSON')

# Target polygon patches
trg_patches = [patches.Polygon(item, True) for item in obj3.zdata.trg.data]

Approach #2 computation time:
        Create object from scratch: 4.099174 seconds
        Create object from dumped file: 0.373083 seconds
        Compute stats using object: 0.001833 seconds

[22]:

# Plot average rainfall and original data
testplot(trg_patches, avg3, radar_utmc, data,
         title="Catchment rainfall mean (PolarZonalStatsPoly)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_26_0.png

[23]:

testplot(trg_patches, var3, radar_utmc, data, levels=np.arange(0, 20, 1.0),
         title="Catchment rainfall variance (PolarZonalStatsPoly)")

_images/notebooks_zonalstats_wradlib_zonalstats_example_27_0.png

[24]:

# Illustrate results for an example catchment i
i = 0  # try any index between 0 and 13
fig = pl.figure(figsize=(10,8))
ax = fig.add_subplot(111, aspect="equal")

# Grid cell patches
src_index = obj3.zdata.get_source_index(i)
grd_patches = [patches.Polygon(item) for item in
               obj3.zdata.src.get_data_by_idx(src_index)]
p = PatchCollection(grd_patches, facecolor="None", edgecolor="black")
ax.add_collection(p)

# Target polygon patches
trg_patch = [trg_patches[i]]
p = PatchCollection(trg_patch, facecolor="None", edgecolor="red",
                    linewidth=2)
ax.add_collection(p)

# View the actual intersections
t1 = dt.datetime.now()
isecs = obj3.zdata.get_isec(i)
isec_patches = wrl.zonalstats.numpy_to_pathpatch(isecs)
colors = 100 * np.linspace(0, 1., len(isec_patches))
p = PatchCollection(isec_patches, cmap=pl.cm.plasma, alpha=0.5)
p.set_array(np.array(colors))
ax.add_collection(p)

bbox = wrl.zonalstats.get_bbox(cats[i][:, 0], cats[i][:, 1])
pl.xlim(bbox["left"] - 2000, bbox["right"] + 2000)
pl.ylim(bbox["bottom"] - 2000, bbox["top"] + 2000)
pl.draw()

t2 = dt.datetime.now()
print("plot intersection takes: %f seconds" % (t2 - t1).total_seconds())

plot intersection takes: 0.172328 seconds

[25]:

# Compare estimates
maxlim = np.max(np.concatenate((avg1, avg3)))
fig = pl.figure(figsize=(10, 8))
ax = fig.add_subplot(111, aspect="equal")
pl.scatter(avg1, avg3, edgecolor="None", alpha=0.5)
pl.xlabel("Average of points in or close to polygon (mm)")
pl.ylabel("Area-weighted average (mm)")
pl.xlim(0, maxlim)
pl.ylim(0, maxlim)
pl.plot([-1, maxlim + 1], [-1, maxlim + 1], color="black")
pl.show()

_images/notebooks_zonalstats_wradlib_zonalstats_example_29_0.png

_images/notebooks_zonalstats_wradlib_zonalstats_example_29_1.png

Specials¶

Beam Blockage Calculation using a DEM¶

Here, we derive (partial) beam-blockage (PBB) from a Digital Elevation Model (DEM).

We require - the local radar setup (sitecoords, number of rays, number of bins, antenna elevation, beamwidth, and the range resolution); - a DEM with a adequate resolution.

Here we use pre-processed data from the GTOPO30 and SRTM missions.

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

Setup for Bonn radar¶

First, we need to define some radar specifications (here: University of Bonn).

[2]:

sitecoords = (7.071663, 50.73052, 99.5)
nrays = 360 # number of rays
nbins = 1000 # number of range bins
el = 1.0 # vertical antenna pointing angle (deg)
bw = 1.0 # half power beam width (deg)
range_res = 100. # range resolution (meters)

Create the range, azimuth, and beamradius arrays.

[3]:

r = np.arange(nbins) * range_res
beamradius = wrl.util.half_power_radius(r, bw)

We use

to calculate the spherical coordinates of the bin centroids and their longitude, latitude and altitude.

[4]:

coord = wrl.georef.sweep_centroids(nrays, range_res, nbins, el)
coords = wrl.georef.spherical_to_proj(coord[..., 0],
                                      coord[..., 1],
                                      coord[..., 2], sitecoords)
lon = coords[..., 0]
lat = coords[..., 1]
alt = coords[..., 2]

[5]:

polcoords = coords[..., :2]
print("lon,lat,alt:", coords.shape)

lon,lat,alt: (360, 1000, 3)

[6]:

rlimits = (lon.min(), lat.min(), lon.max(), lat.max())
print("Radar bounding box:\n\t%.2f\n%.2f             %.2f\n\t%.2f" %
      (lat.max(), lon.min(), lon.max(), lat.min()))

Radar bounding box:
        51.63
5.66             8.49
        49.83

Preprocessing the digitial elevation model¶

Read the DEM from a geotiff file (in WRADLIB_DATA);
clip the region inside the bounding box;
map the DEM values to the polar grid points.

Note: You can choose between the coarser resolution bonn_gtopo.tif (from GTOPO30) and the finer resolution bonn_new.tif (from the SRTM mission).

The DEM raster data is opened via wradlib.io.open_raster and extracted via wradlib.georef.extract_raster_dataset.

[7]:

#rasterfile = wrl.util.get_wradlib_data_file('geo/bonn_gtopo.tif')
rasterfile = wrl.util.get_wradlib_data_file('geo/bonn_new.tif')

ds = wrl.io.open_raster(rasterfile)
rastervalues, rastercoords, proj = wrl.georef.extract_raster_dataset(ds, nodata=-32768.)

# Clip the region inside our bounding box
ind = wrl.util.find_bbox_indices(rastercoords, rlimits)
rastercoords = rastercoords[ind[1]:ind[3], ind[0]:ind[2], ...]
rastervalues = rastervalues[ind[1]:ind[3], ind[0]:ind[2]]

# Map rastervalues to polar grid points
polarvalues = wrl.ipol.cart_to_irregular_spline(rastercoords, rastervalues,
                                             polcoords, order=3,
                                             prefilter=False)

Calculate Beam-Blockage¶

Now we can finally apply the wradlib.qual.beam_block_frac function to calculate the PBB.

[8]:

PBB = wrl.qual.beam_block_frac(polarvalues, alt, beamradius)
PBB = np.ma.masked_invalid(PBB)
print(PBB.shape)

(360, 1000)

So far, we calculated the fraction of beam blockage for each bin.

But we need to into account that the radar signal travels along a beam. Cumulative beam blockage (CBB) in one bin along a beam will always be at least as high as the maximum PBB of the preceeding bins (see wradlib.qual.cum_beam_block_frac)

[9]:

CBB = wrl.qual.cum_beam_block_frac(PBB)
print(CBB.shape)

(360, 1000)

Visualize Beamblockage¶

Now we visualize - the average terrain altitude per radar bin - a beam blockage map - interaction with terrain along a single beam

[10]:

# just a little helper function to style x and y axes of our maps
def annotate_map(ax, cm=None, title=""):
    ticks = (ax.get_xticks()/1000).astype(np.int)
    ax.set_xticklabels(ticks)
    ticks = (ax.get_yticks()/1000).astype(np.int)
    ax.set_yticklabels(ticks)
    ax.set_xlabel("Kilometers")
    ax.set_ylabel("Kilometers")
    if not cm is None:
        pl.colorbar(cm, ax=ax)
    if not title=="":
        ax.set_title(title)
    ax.grid()

[11]:

fig = pl.figure(figsize=(10, 8))

# create subplots
ax1 = pl.subplot2grid((2, 2), (0, 0))
ax2 = pl.subplot2grid((2, 2), (0, 1))
ax3 = pl.subplot2grid((2, 2), (1, 0), colspan=2, rowspan=1)

# azimuth angle
angle = 225

# Plot terrain (on ax1)
ax1, dem = wrl.vis.plot_ppi(polarvalues,
                            ax=ax1, r=r,
                            az=coord[:,0,1],
                            cmap=mpl.cm.terrain, vmin=0.)
ax1.plot([0,np.sin(np.radians(angle))*1e5],
         [0,np.cos(np.radians(angle))*1e5],"r-")
ax1.plot(sitecoords[0], sitecoords[1], 'ro')
annotate_map(ax1, dem, 'Terrain within {0} km range'.format(np.max(r / 1000.) + 0.1))

# Plot CBB (on ax2)
ax2, cbb = wrl.vis.plot_ppi(CBB, ax=ax2, r=r,
                            az=coord[:,0,1],
                            cmap=mpl.cm.PuRd, vmin=0, vmax=1)
annotate_map(ax2, cbb, 'Beam-Blockage Fraction')

# Plot single ray terrain profile on ax3
bc, = ax3.plot(r / 1000., alt[angle, :], '-b',
               linewidth=3, label='Beam Center')
b3db, = ax3.plot(r / 1000., (alt[angle, :] + beamradius), ':b',
                 linewidth=1.5, label='3 dB Beam width')
ax3.plot(r / 1000., (alt[angle, :] - beamradius), ':b')
ax3.fill_between(r / 1000., 0.,
                 polarvalues[angle, :],
                 color='0.75')
ax3.set_xlim(0., np.max(r / 1000.) + 0.1)
ax3.set_ylim(0., 3000)
ax3.set_xlabel('Range (km)')
ax3.set_ylabel('Altitude (m)')
ax3.grid()

axb = ax3.twinx()
bbf, = axb.plot(r / 1000., CBB[angle, :], '-k',
                label='BBF')
axb.set_ylabel('Beam-blockage fraction')
axb.set_ylim(0., 1.)
axb.set_xlim(0., np.max(r / 1000.) + 0.1)


legend = ax3.legend((bc, b3db, bbf),
                    ('Beam Center', '3 dB Beam width', 'BBF'),
                    loc='upper left', fontsize=10)

_images/notebooks_beamblockage_wradlib_beamblock_25_0.png

Visualize Beam Propagation showing earth curvature¶

Now we visualize - interaction with terrain along a single beam

In this representation the earth curvature is shown. For this we assume the earth a sphere with exactly 6370000 m radius. This is needed to get the height ticks at nice position.

[12]:

def height_formatter(x, pos):
    x = (x - 6370000) / 1000
    fmt_str = '{:g}'.format(x)
    return fmt_str

def range_formatter(x, pos):
    x = x / 1000.
    fmt_str = '{:g}'.format(x)
    return fmt_str

The wradlib.vis.create_cg-function is facilitated to create the curved geometries.
The actual data is plottet as (theta, range) on the parasite axis.
Some tweaking is needed to get the final plot look nice.

[13]:

fig = pl.figure(figsize=(10, 6))

cgax, caax, paax = wrl.vis.create_cg(fig=fig, rot=0, scale=1)

# azimuth angle
angle = 225

# fix grid_helper
er = 6370000
gh = cgax.get_grid_helper()
gh.grid_finder.grid_locator2._nbins=80
gh.grid_finder.grid_locator2._steps=[1,2,4,5,10]

# calculate beam_height and arc_distance for ke=1
# means line of sight
bhe = wrl.georef.bin_altitude(r, 0, sitecoords[2], re=er, ke=1.)
ade = wrl.georef.bin_distance(r, 0, sitecoords[2], re=er, ke=1.)
nn0 = np.zeros_like(r)
# for nice plotting we assume earth_radius = 6370000 m
ecp = nn0 + er
# theta (arc_distance sector angle)
thetap = - np.degrees(ade/er) + 90.0

# zero degree elevation with standard refraction
bh0 = wrl.georef.bin_altitude(r, 0, sitecoords[2], re=er)

# plot (ecp is earth surface normal null)
bes, = paax.plot(thetap, ecp, '-k', linewidth=3, label='Earth Surface NN')
bc, = paax.plot(thetap, ecp + alt[angle, :], '-b', linewidth=3, label='Beam Center')
bc0r, = paax.plot(thetap, ecp + bh0 + alt[angle, 0] , '-g', label='0 deg Refraction')
bc0n, = paax.plot(thetap, ecp + bhe + alt[angle, 0], '-r', label='0 deg line of sight')
b3db, = paax.plot(thetap, ecp + alt[angle, :]  + beamradius, ':b', label='+3 dB Beam width')
paax.plot(thetap, ecp + alt[angle, :] - beamradius, ':b', label='-3 dB Beam width')

# orography
paax.fill_between(thetap, ecp,
                  ecp + polarvalues[angle, :],
                  color='0.75')

# shape axes
cgax.set_xlim(0, np.max(ade))
cgax.set_ylim([ecp.min()-1000, ecp.max()+2500])
caax.grid(True, axis='x')
cgax.grid(True, axis='y')
cgax.axis['top'].toggle(all=False)
caax.yaxis.set_major_locator(mpl.ticker.MaxNLocator(steps=[1,2,4,5,10], nbins=20, prune='both'))
caax.xaxis.set_major_locator(mpl.ticker.MaxNLocator())
caax.yaxis.set_major_formatter(mpl.ticker.FuncFormatter(height_formatter))
caax.xaxis.set_major_formatter(mpl.ticker.FuncFormatter(range_formatter))

caax.set_xlabel('Range (km)')
caax.set_ylabel('Altitude (km)')

legend = paax.legend((bes, bc0n, bc0r, bc, b3db),
                    ('Earth Surface NN', '0 deg line of sight', '0 deg std refraction', 'Beam Center', '3 dB Beam width'),
                    loc='upper left', fontsize=10)

_images/notebooks_beamblockage_wradlib_beamblock_30_0.png

Go back to Read DEM Raster Data, change the rasterfile to use the other resolution DEM and process again.

Match spaceborn SR (GPM/TRMM) with ground radars GR¶

The idea is to match ground radar (GR) and space-born radar (SR) measurements in order to create spatially and temporally coicident samples without interpolation. The procedure had been suggested by Schwaller and Morris (2011) and is based on the adaption by Warren, et. al. (2017).

The basic principle is illustrated in Fig. 2 of the original paper of Schwaller and Morris (2011):

Quote Warren, et.al: “[…] In this approach, intersections between indi vidual SR beams and GR elevation sweeps are identified and the reflectivity values from both instruments are averaged within a spatial neighborhood around the intersection. Specifically, SR data are averaged in range over the width of the GR beam at the GR range of the intersection, while GR data are averaged in the range–azimuth plane within the footprint of the SR beam. The result is a pair of reflectivity measurements corresponding to approximately the same volume of atmosphere. […]”.

This becomes clearer in Fig. 3:

Schwaller, MR, and Morris, KR. 2011. A ground validation network for the Global Precipitation Measurement mission. J. Atmos. Oceanic Technol., 28, 301-319.

Warren, R.A., A. Protat, S.T. Siems, H.A. Ramsay, V. Louf, M.J. Manton, and T.A. Kane, 0: Calibrating ground-based radars against TRMM and GPM. J. Atmos. Oceanic Technol., 0,

Conventions¶

This code is based on the following conventions:

gr indicates ground radar
sr indicates space-born precipitation radar (TRMM or GPM)

The base routines are designed to process one GR sweep at a time. If a full GR volume with nelev of sweeps is available, you can iterate over each sweep. In this code, ee is an index that points to one of the nelev sweeps/elevation angles. Accordingly, a GR data set will be organised as an array of shape (nelev_gr, nray_gr, ngate_gr).

A SR data set is typically organised as arrays with dimensions (nscan_sr, nray_sr, ngate_sr).

Please note: the GR and SR datasets are managed by using convenience functions from the module file io_func.py located in the same directory as this notebook.

[1]:

import warnings
warnings.filterwarnings('ignore')
import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
from matplotlib.collections import PatchCollection
from matplotlib.colors import from_levels_and_colors
from matplotlib.path import Path
import matplotlib.patches as patches
import matplotlib.cm as cm
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np
import datetime as dt
from osgeo import osr

[2]:

# define GPM data set
gpm_file = wrl.util.get_wradlib_data_file('gpm/2A-CS-151E24S154E30S.GPM.Ku.V7-20170308.20141206-S095002-E095137.004383.V05A.HDF5')
# define matching ground radar file
gr2gpm_file = wrl.util.get_wradlib_data_file('hdf5/IDR66_20141206_094829.vol.h5')

# define TRMM data sets
trmm_2a23_file = wrl.util.get_wradlib_data_file('trmm/2A-CS-151E24S154E30S.TRMM.PR.2A23.20100206-S111425-E111526.069662.7.HDF')
trmm_2a25_file = wrl.util.get_wradlib_data_file('trmm/2A-CS-151E24S154E30S.TRMM.PR.2A25.20100206-S111425-E111526.069662.7.HDF')

# define matching ground radar file
gr2trmm_file = wrl.util.get_wradlib_data_file('hdf5/IDR66_20100206_111233.vol.h5')

[3]:

# Space-born precipitation radar parameters
sr_pars = {"trmm": {
    "zt": 402500.,  # orbital height of TRMM (post boost)   APPROXIMATION!
    "dr": 250.,     # gate spacing of TRMM
    "gr_file": gr2trmm_file,
}, "gpm": {
    "zt": 407000.,  # orbital height of GPM                 APPROXIMATION!
    "dr": 125.,      # gate spacing of GPM
    "gr_file": gr2gpm_file,
}}

[4]:

# Set parameters for this procedure
bw_sr = 0.71                  # SR beam width
platf = "gpm"                 # SR platform/product: one out of ["gpm", "trmm"]
zt = sr_pars[platf]["zt"]     # SR orbit height (meters)
dr_sr = sr_pars[platf]["dr"]  # SR gate length (meters)
gr_file = sr_pars[platf]["gr_file"]
ee = 2                        # Index that points to the GR elevation angle to be used

Data Input¶

Read and organize the GR data¶

The following code reads data in ODIM H5 format. If your GR data is in some other format respective adaptions are needed.

[5]:

def _get_tilts(dic):
    i = 0
    for k in dic.keys():
        if 'dataset' in k:
            i += 1
    return i


def read_gr(filename, loaddata=True):

    gr_data = wrl.io.read_generic_netcdf(filename)
    dat = gr_data['what']['date']
    tim = gr_data['what']['time']
    date = dt.datetime.strptime(dat + tim, "%Y%d%m%H%M%S")
    source = gr_data['what']['source']

    lon = gr_data['where']['lon']
    lat = gr_data['where']['lat']
    alt = gr_data['where']['height']

    if gr_data['what']['object'] == 'PVOL':
        ntilt = _get_tilts(gr_data)
    else:
        raise ValueError('GR file is no PPI/Volume File')

    ngate = np.zeros(ntilt, dtype=np.int16)
    nbeam = np.zeros(ntilt)
    elang = np.zeros(ntilt)
    r0 = np.zeros(ntilt)
    dr = np.zeros(ntilt)
    a0 = np.zeros(ntilt)

    for i in range(0, ntilt):
        dset = gr_data['dataset{0}'.format(i+1)]
        a0[i] = dset['how']['astart']
        elang[i] = dset['where']['elangle']
        ngate[i] = dset['where']['nbins']
        r0[i] = dset['where']['rstart']
        dr[i] = dset['where']['rscale']
        nbeam[i] = dset['where']['nrays']

    if ((len(np.unique(r0)) != 1) |
            (len(np.unique(dr)) != 1) |
            (len(np.unique(a0)) != 1) |
            (len(np.unique(nbeam)) != 1) |
            (nbeam[0] != 360)):
        raise ValueError('GroundRadar Data layout dos not match')

    gr_dict = {}
    gr_dict.update({'source': source, 'date': date, 'lon': lon, 'lat': lat,
                    'alt': alt, 'ngate': ngate, 'nbeam': nbeam, 'ntilt': ntilt,
                    'r0': r0, 'dr': dr, 'a0': a0, 'elang': elang})
    if not loaddata:
        return gr_dict

    sdate = []
    refl = []
    for i in range(0, ntilt):
        dset = gr_data['dataset{0}'.format(i+1)]
        dat = dset['what']['startdate']
        tim = dset['what']['starttime']
        date = dt.datetime.strptime(dat + tim, "%Y%d%m%H%M%S")
        sdate.append(date)
        data = dset['data1']
        quantity = data['what']['quantity']
        factor = data['what']['gain']
        offset = data['what']['offset']
        if quantity == 'DBZH':
            dat = data['variables']['data']['data'] * factor + offset
            refl.append(dat)

    sdate = np.array(sdate)
    refl = np.array(refl)

    gr_dict.update({'sdate': sdate, 'refl': refl})

    return gr_dict

[6]:

# read matching GR data
gr_data = read_gr(gr_file)

Data handling¶

Extract relevant GR data and meta-data¶

[7]:

# number of rays in gr sweep
nray_gr = gr_data['nbeam'].astype("i4")[ee]
# number of gates in gr beam
ngate_gr = gr_data['ngate'].astype("i4")[ee]
# number of sweeps
nelev = gr_data['ntilt']
# elevation of sweep (degree)
elev_gr = gr_data['elang'][ee]
# gate length (meters)
dr_gr = gr_data['dr'][ee]
# reflectivity array of sweep
ref_gr = gr_data['refl'][ee]
# sweep datetime stamp
date_gr = gr_data['sdate'][ee]
# range of first gate
r0_gr = gr_data['r0'][ee]
# azimuth angle of first beam
a0_gr = gr_data['a0'][ee]
# Longitude of GR
lon0_gr = gr_data['lon']
# Latitude of GR
lat0_gr = gr_data['lat']
# Altitude of GR (meters)
alt0_gr = gr_data['alt']
# Beam width of GR (degree)
bw_gr = 1.
print(elev_gr, lon0_gr)

1.2999999523162842 153.24000549316406

Calculate GR Bounding Box¶

[8]:

coord = wrl.georef.sweep_centroids(nray_gr, dr_gr, ngate_gr, elev_gr)
coords = wrl.georef.spherical_to_proj(coord[..., 0],
                                      coord[..., 1],
                                      coord[..., 2],
                                      (lon0_gr, lat0_gr, alt0_gr))
lon = coords[..., 0]
lat = coords[..., 1]
alt = coords[..., 2]
bbox = wrl.zonalstats.get_bbox(lon, lat)
print("Radar bounding box:\n\t%.2f\n%.2f           %.2f\n\t%.2f" %
      (bbox['top'], bbox['left'], bbox['right'], bbox['bottom']))

Radar bounding box:
        -26.37
151.72           154.76
        -29.07

Read and organize the SR data¶

[9]:

# read spaceborn SR data
if platf == "gpm":
    sr_data = wrl.io.read_gpm(gpm_file, bbox)
elif platf == "trmm":
    sr_data = wrl.io.read_trmm(trmm_2a23_file, trmm_2a25_file, bbox)
else:
    raise("Invalid platform")

[10]:

refl = sr_data['refl']
print(refl)

[[[-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  ...
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]]

 [[-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  ...
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]]

 [[-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  ...
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]]

 ...

 [[-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  ...
  [-- -- -- ... -- -- --]
  [15.90999984741211 15.930000305175781 15.920000076293945 ... -- -- --]
  [-- 15.0 15.020000457763672 ... -- -- --]]

 [[-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  ...
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]]

 [[-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  ...
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]
  [-- -- -- ... -- -- --]]]

Extract relevant SR data and meta-data¶

[11]:

# Longitudes of SR scans
sr_lon = sr_data['lon']
# Latitudes of SR scans
sr_lat = sr_data['lat']
# Precip flag
pflag = sr_data['pflag']
# Number of scans on SR data
nscan_sr= sr_data['nscan']
# Number of rays in one SR scan
nray_sr = sr_data['nray']
# Number of gates in one SR ray
ngate_sr = sr_data['nbin']

[12]:

print(sr_lon.shape)

(82, 49)

Georeferencing¶

Set fundamental georeferencing parameters¶

[13]:

# Calculate equivalent earth radius
wgs84 = wrl.georef.get_default_projection()
re1 = wrl.georef.get_earth_radius(lat0_gr, wgs84)
print("Earth radius 1:", re1)
a = wgs84.GetSemiMajor()
b = wgs84.GetSemiMinor()
print("SemiMajor, SemiMinor:", a, b)

# Set up aeqd-projection gr-centered
rad = wrl.georef.proj4_to_osr(('+proj=aeqd +lon_0={lon:f} ' +
                                   '+lat_0={lat:f} +a={a:f} ' +
                                   '+b={b:f}').format(lon=lon0_gr,
                                                      lat=lat0_gr,
                                                      a=a, b=b))
re2 = wrl.georef.get_earth_radius(lat0_gr, rad)
print("Earth radius 2:", re2)

Earth radius 1: 6373541.039814419
SemiMajor, SemiMinor: 6378137.0 6356752.314245179
Earth radius 2: 6373541.039814381

Georeference GR data¶

[14]:

# create gr range and azimuth arrays
rmax_gr = r0_gr + ngate_gr * dr_gr
r_gr = np.arange(0, ngate_gr) * dr_gr + dr_gr/2.
az_gr = np.arange(0, nray_gr) - a0_gr
print("Range/Azi-Shape:", r_gr.shape, az_gr.shape)

# create gr polar grid and calculate aeqd-xyz coordinates
gr_polargrid = np.meshgrid(r_gr, az_gr)
gr_xyz, rad = wrl.georef.spherical_to_xyz(gr_polargrid[0], gr_polargrid[1], elev_gr, (lon0_gr, lat0_gr, alt0_gr ),
                                          squeeze=True)
print("XYZ-Grid-Shape:", gr_xyz.shape)

# create gr poygon array in aeqd-xyz-coordinates
gr_poly, rad1 = wrl.georef.spherical_to_polyvert(r_gr, az_gr, elev_gr, (lon0_gr, lat0_gr, alt0_gr))
print(gr_poly.shape, 360 * 600)
gr_poly.shape = (nray_gr, ngate_gr, 5, 3)

# get radar domain (outer ring)
gr_domain = gr_xyz[:,-1,0:2]
gr_domain = np.vstack((gr_domain, gr_domain[0]))
print("Domain-Shape:", gr_domain.shape)

Range/Azi-Shape: (600,) (360,)
XYZ-Grid-Shape: (360, 600, 3)
(216000, 5, 3) 216000
Domain-Shape: (361, 2)

Georeference SR data¶

[15]:

sr_x, sr_y = wrl.georef.reproject(sr_lon, sr_lat,
                                      projection_source=wgs84,
                                      projection_target=rad)
sr_xy = np.dstack((sr_x, sr_y))
print("SR-GRID-Shapes:", sr_x.shape, sr_y.shape, sr_xy.shape)

SR-GRID-Shapes: (82, 49) (82, 49) (82, 49, 2)

Subset relevant SR data¶

[16]:

# Create ZonalData for spatial subsetting (inside GR range domain)

# get precip indexes
print(sr_xy.shape)
print(pflag.shape)

precip_mask = (pflag == 2) & wrl.zonalstats.get_clip_mask(sr_xy, gr_domain, rad)

# get iscan/iray boolean arrays
print(precip_mask.shape, sr_xy.shape)
print(pflag.shape, sr_xy.reshape(-1, sr_xy.shape[-1]).shape)
pl.imshow(precip_mask)

(82, 49, 2)
(82, 49)
(82, 49) (82, 49, 2)
(82, 49) (4018, 2)

[16]:

<matplotlib.image.AxesImage at 0x7fcbabd69730>

_images/notebooks_match3d_wradlib_match_workflow_31_2.png

SR Parallax Correction¶

[17]:

print("NRAY", nray_sr)
print("NBIN", ngate_sr)

# use localZenith Angle
alpha = sr_data['zenith']
beta = abs(-17.04 + np.arange(nray_sr) * bw_sr)

# Correct for parallax, get 3D-XYZ-Array
#   xyzp_sr: Parallax corrected xyz coordinates
#   r_sr_inv: range array from ground to SR platform
#   zp: SR bin altitudes
xyp_sr, r_sr_inv, z_sr = wrl.georef.correct_parallax(sr_xy, ngate_sr, dr_sr, alpha)
print(xyp_sr.shape, r_sr_inv.shape, z_sr.shape)
xyzp_sr = np.concatenate((xyp_sr, z_sr[..., np.newaxis]),
                   axis=-1)
print(sr_xy.shape)
print("SR_XYP:", xyp_sr.shape, xyzp_sr.shape, r_sr_inv.shape, z_sr.shape)

NRAY 49
NBIN 176
(82, 49, 176, 2) (176,) (82, 49, 176)
(82, 49, 2)
SR_XYP: (82, 49, 176, 2) (82, 49, 176, 3) (176,) (82, 49, 176)

Compute spherical coordinates of SR bins with regard to GR¶

[18]:

r_sr, az_sr, elev_sr = wrl.georef.xyz_to_spherical(xyzp_sr, alt0_gr, proj=rad)
#TODO: hardcoded 1.0
mask = (elev_sr > (1.0 - bw_gr/2.)) & (elev_sr < (1.0 + bw_gr/2.))
pl.figure()
pl.pcolormesh(mask[40,:,:].T)

[18]:

<matplotlib.collections.QuadMesh at 0x7fcbbb3a1e20>

_images/notebooks_match3d_wradlib_match_workflow_35_1.png

Compute SR and GR pulse volumes¶

Calculate distance from orbit $r_s$¶

[19]:

rs = wrl.georef.dist_from_orbit(zt, alpha, beta, r_sr_inv, re1)

SR pulse volume¶

[20]:

# Small anngle approximation
vol_sr2  = np.pi * dr_sr * rs**2 * np.radians(bw_sr / 2.)**2

# Or using wradlib's native function
vol_sr = wrl.qual.pulse_volume(rs, dr_sr, bw_sr)
# Evaluate difference between both approaches
print("Min. difference (m3):", (vol_sr - vol_sr2).min())
print("Max. difference (m3): ", (vol_sr - vol_sr2).max())
print("Average rel. difference (%):", round(np.mean(vol_sr-vol_sr2)*100./np.mean(np.mean(vol_sr2)), 4))

# Verdict: differences are negligble - use wradlibs's native function!

Min. difference (m3): 57223.421278476715
Max. difference (m3):  70339.5594573021
Average rel. difference (%): 0.0026

GR pulse volume¶

[21]:

# GR pulse volumes
#   along one beam
vol_gr = wrl.qual.pulse_volume(r_gr, dr_gr, bw_gr)
#   with shape (nray_gr, ngate_gr)
vol_gr = np.repeat(vol_gr, nray_gr).reshape((nray_gr, ngate_gr), order="F")

Calculate horizontal and vertical dimensions $R_s$ and $D_s$ of SR bins¶

[22]:

Rs = 0.5 * (1 +  np.cos(np.radians(alpha)))[:,:,np.newaxis] * rs * np.tan(np.radians(bw_sr/2.))
Ds = dr_sr / np.cos(np.radians(alpha))
Ds = np.broadcast_to(Ds[..., np.newaxis], Rs.shape)

Median Brightband Width/Height¶

[23]:

print(z_sr.shape)
print(sr_data['zbb'].shape, sr_data['bbwidth'].shape, sr_data['quality'].shape)
ratio, ibb = wrl.qual.get_bb_ratio(sr_data['zbb'], sr_data['bbwidth'], sr_data['quality'], z_sr)
print(ratio.shape)
zbb = sr_data['zbb'].copy()
zbb[~ibb] = np.nan
print(np.nanmin(ratio[..., 9]), np.nanmax(ratio[..., 9]))
#pl.imshow(ratio, vmin=-10, vmax=30)

(82, 49, 176)
(82, 49) (82, 49) (82, 49)
(82, 49, 176)
-4.183126378193563 -4.0904391409528404

Convert SR Ku reflectivities to S-band¶

Based on Cao et.al (2013)

[24]:

ref_sr = sr_data['refl'].filled(np.nan)
print(ref_sr.shape, ratio.shape)
ref_sr_ss = np.zeros_like(ref_sr) * np.nan
ref_sr_sh = np.zeros_like(ref_sr) * np.nan

a_s, a_h = (wrl.trafo.KuBandToS.snow, wrl.trafo.KuBandToS.hail)

ia = (ratio >= 1)
ref_sr_ss[ia] = ref_sr[ia] + wrl.util.calculate_polynomial(ref_sr[ia], a_s[:,10])
ref_sr_sh[ia] = ref_sr[ia] + wrl.util.calculate_polynomial(ref_sr[ia], a_h[:,10])
ib = (ratio <= 0)
ref_sr_ss[ib] = ref_sr[ib] + wrl.util.calculate_polynomial(ref_sr[ib], a_s[:,0])
ref_sr_sh[ib] = ref_sr[ib] + wrl.util.calculate_polynomial(ref_sr[ib], a_h[:,0])
im = (ratio > 0) & (ratio < 1)
ind = np.round(ratio[im] * 10).astype(np.int)
ref_sr_ss[im] = ref_sr[im] + wrl.util.calculate_polynomial(ref_sr[im], a_s[:,ind])
ref_sr_sh[im] = ref_sr[im] + wrl.util.calculate_polynomial(ref_sr[im], a_h[:,ind])

# Jackson Tan's fix for C-band
is_cband = False
if (is_cband):
    deltas = (ref_sr_ss - ref_sr) * 5.3 / 10.0
    ref_sr_ss = outref_sr + deltas
    deltah = (ref_sr_sh - ref_sr) * 5.3 / 10.0
    ref_sr_sh = ref_sr + deltah

ref_sr_ss[ref_sr < 0] = np.nan

(82, 49, 176) (82, 49, 176)

Convert S-band GR reflectivities to Ku-band¶

Using the method of Liao and Meneghini (2009)

[25]:

# Convert S-band GR reflectivities to Ku-band using method of Liao and Meneghini (2009)
ref_gr_ku = np.zeros_like(ref_gr) * np.nan

# Which zbb value should we take here???
#    Q'n'Dirty: just take the mean of all SR profiles
#    TODO: Consider zbb for each profile during the matching process

# Snow
ia = ( gr_xyz[...,2] >= np.nanmean(zbb) )
#ref_gr_ku[ia] = wrl.trafo.KuBandToS.snow[0] + wrl.trafo.KuBandToS.snow[1]*ref_gr[ia] + wrl.trafo.KuBandToS.snow[2]*ref_gr[ia]**2
ref_gr_ku[ia] = wrl.util.calculate_polynomial(ref_gr[ia], wrl.trafo.SBandToKu.snow)
# Rain
ib = ( gr_xyz[...,2] < np.nanmean(zbb) )
#ref_gr_ku[ib] = wrl.trafo.KuBandToS.rain[0] + wrl.trafo.KuBandToS.rain[1]*ref_gr[ia] + wrl.trafo.KuBandToS.rain[2]*ref_gr[ia]**2
ref_gr_ku[ib] = wrl.util.calculate_polynomial(ref_gr[ib], wrl.trafo.SBandToKu.rain)

# Jackson Tan's fix for C-band
is_cband = False
if (is_cband):
    delta = (ref_gr_ku - ref_gr) * 5.3/10.0
    ref_gr_ku = ref_gr + delta

Matching SR/GR¶

Identify which SR rays actually intersect with the GR sweep¶

Based on the above criteria (in radar range, precipitating SR profile) and based on SR elevation angle (with regard to GR).

[26]:

# First assumption: no valid SR bins (all False)
valid = np.asarray(elev_sr, dtype=np.bool)==False
print(valid.shape, precip_mask.shape)
# SR is inside GR range and is precipitating
iscan = precip_mask.nonzero()[0]
iray = precip_mask.nonzero()[1]
valid[iscan,iray] = True
# SR bins intersect with GR sweep
valid = valid & (elev_sr >= (elev_gr-bw_gr/2.)) & (elev_sr <= (elev_gr+bw_gr/2.))
# Number of matching SR bins per profile
nvalids = np.sum(valid, axis=2)
# scan and ray indices for profiles with at least one valid bin
vscan, vray = np.where(nvalids>0)
# number of profiles with at least one valid bin
nprof = len(vscan)
print(vscan.shape)
print(valid.shape)

(82, 49, 176) (82, 49)
(1218,)
(82, 49, 176)

Averaging SR parameters¶

[27]:

# average coordinates
xyz_v1 = xyzp_sr.copy()
print(xyz_v1.shape)
xyz_v1[~valid] = np.nan
xyz_c1 = xyzp_sr.filled(0)
xyz_c1[~valid] = 0
c = np.count_nonzero(xyz_c1[..., 0], axis=2)
ntotsr = c[vscan, vray]
xyz_m1 = np.nanmean(xyz_v1,axis=2)
xyz = xyz_m1[vscan, vray]
print(xyz.shape, c.shape)

# approximate Rs
rs_v1 = Rs.copy()
rs_v1[~valid] = np.nan
rs_m1 = np.nanmax(rs_v1, axis=2)
rs_prof = rs_m1[vscan, vray]
ds = rs_prof

# approximate Ds
ds_v1 = Ds.copy()
ds_v1[~valid] = np.nan
ds_m1 = np.nansum(ds_v1, axis=2)
ds_prof = ds_m1[vscan, vray]
dz = ds_prof

# approximate Vs
vs_v1 = vol_sr.copy()
vs_v1[~valid] = np.nan
vs_m1 = np.nansum(vs_v1, axis=2)
vs_prof = vs_m1[vscan, vray]
volsr1 = vs_prof

from mpl_toolkits.mplot3d import Axes3D
fig = pl.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(xyz_m1[..., 0], xyz_m1[..., 1], xyz_m1[..., 2], c=c.ravel())

(82, 49, 176, 3)
(1218, 3) (82, 49)

[27]:

<mpl_toolkits.mplot3d.art3d.Path3DCollection at 0x7fcbbb31efa0>

_images/notebooks_match3d_wradlib_match_workflow_55_2.png

Calculate spherical coordinates of SR sample volume with respect to GR¶

[28]:

r_sr, az_sr, el_rs = wrl.georef.xyz_to_spherical(xyz, alt0_gr, proj=rad)

Average SR reflectivity¶

[29]:

ref_sr_1 = wrl.trafo.idecibel(ref_sr)
ref_sr_1[~valid] = np.nan
refsr1a = np.nanmean(ref_sr_1, axis=2)[vscan,vray]
refsr1a = wrl.trafo.decibel(refsr1a)

ref_sr_2 = wrl.trafo.idecibel(ref_sr_ss)
ref_sr_2[~valid] = np.nan
refsr2a = np.nanmean(ref_sr_2, axis=2)[vscan,vray]
refsr2a = wrl.trafo.decibel(refsr2a)

ref_sr_3 = wrl.trafo.idecibel(ref_sr_sh)
ref_sr_3[~valid] = np.nan
refsr3a = np.nanmean(ref_sr_3, axis=2)[vscan,vray]
refsr3a = wrl.trafo.decibel(refsr3a)


print(refsr1a.shape)

(1218,)

Identify which GR bins actually intersect with the SR footprint¶

Create Zonal Data Source of matching SR profiles¶

[30]:

zds = wrl.zonalstats.DataSource(xyz[:, 0:2].reshape(-1, 2), rad)
print(zds.ds.GetLayer().GetFeatureCount())

Create SR beam polygons using $R_s$¶

[31]:

tmp_trg_lyr = zds.ds.GetLayer()
trg_poly = []
for i, feat in enumerate(tmp_trg_lyr):
    geom = feat.GetGeometryRef().Buffer(rs_prof[i])
    poly = wrl.georef.ogr_to_numpy(geom)
    trg_poly.append(poly)

Create ZonalDataPoly for GR and SR profiles¶

[32]:

#%%time
print("creating")
zdp = wrl.zonalstats.ZonalDataPoly(gr_poly[..., 0:2].reshape(-1, 5, 2), trg_poly, srs=rad)
zdp.dump_vector('m3d_zonal_poly_{0}'.format(platf))

creating

[33]:

#%%time
print("loading")
obj3 = wrl.zonalstats.ZonalStatsPoly('m3d_zonal_poly_{0}'.format(platf))

loading

Average GR volume and reflectivity¶

[34]:

#%%time
print(obj3.ix.shape)
volgr1 = np.array([np.sum(vol_gr.ravel()[obj3.ix[i]])
                   for i in np.arange(len(obj3.ix))[~obj3.check_empty()]])
print(volgr1.shape)

ref_gr_i = wrl.trafo.idecibel(ref_gr.ravel())
ref_gr_ku_i = wrl.trafo.idecibel(ref_gr_ku.ravel())
refgr1a = np.array([np.nanmean(ref_gr_i[obj3.ix[i]])
             for i in np.arange(len(obj3.ix))[~obj3.check_empty()]])
refgr1a = wrl.trafo.decibel(refgr1a)
refgr2a = np.array([np.nanmean(ref_gr_ku_i[obj3.ix[i]])
             for i in np.arange(len(obj3.ix))[~obj3.check_empty()]])
refgr2a = wrl.trafo.decibel(refgr2a)

(1218,)
(1218,)

Diagnostic Plots¶

[35]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr1a, refsr1a, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr1a[refsr1a>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr1a[refsr1a>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("uncorrected GR vs uncorrected SR")

[35]:

Text(0.5, 0.98, 'uncorrected GR vs uncorrected SR')

_images/notebooks_match3d_wradlib_match_workflow_71_1.png

[36]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr2a, refsr1a, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr1a[refsr1a>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr1a[refsr1a>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("corrected GR vs uncorrected SR")

[36]:

Text(0.5, 0.98, 'corrected GR vs uncorrected SR')

_images/notebooks_match3d_wradlib_match_workflow_72_1.png

[37]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr2a, refsr2a, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr2a[refsr2a>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr2a[refsr2a>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("corrected GR vs corrected SR snow")

[37]:

Text(0.5, 0.98, 'corrected GR vs corrected SR snow')

_images/notebooks_match3d_wradlib_match_workflow_73_1.png

[38]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr2a, refsr3a, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr2a[refsr3a>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr3a[refsr3a>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("corrected GR vs corrected SR hail")

[38]:

Text(0.5, 0.98, 'corrected GR vs corrected SR hail')

_images/notebooks_match3d_wradlib_match_workflow_74_1.png

[39]:

fig = pl.figure(figsize=(12,8))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(xyz[..., 0], xyz[...,1], c=refsr1a, cmap=pl.cm.viridis, vmin=10, vmax=50, edgecolor="None")
pl.title("SR reflectivity")
pl.xlim(-100000, 150000)
pl.ylim(-150000, 150000)
pl.grid()
ax = fig.add_subplot(122, aspect="equal")
pl.scatter(xyz[..., 0], xyz[...,1], c=refgr1a, cmap=pl.cm.viridis, vmin=10, vmax=50, edgecolor="None")
pl.title("GR reflectivity")
pl.xlim(-100000, 150000)
pl.ylim(-150000, 150000)
pl.grid()
fig.suptitle("uncorrected GR vs uncorrected SR")
pl.tight_layout()

_images/notebooks_match3d_wradlib_match_workflow_75_0.png

Former Method using loop¶

[40]:

# Lots of containers to store samples (only for one GR sweep angle!)
x = np.zeros(nprof)*np.nan        # x coordinate of sample
y = np.zeros(nprof)*np.nan        # y coordinate of sample
z = np.zeros(nprof)*np.nan        # z coordinate of sample
dz = np.zeros(nprof)*np.nan       # depth of sample
ds = np.zeros(nprof)*np.nan       # width of sample
rs = np.zeros(nprof)*np.nan       # range of sample from GR
refsr1 = np.zeros(nprof)*np.nan     # SR reflectivity
refsr2 = np.zeros(nprof)*np.nan     # SR reflectivity (S-band, snow)
refsr3 = np.zeros(nprof)*np.nan     # SR reflectivity (S-band, hail)
refgr1 = np.zeros(nprof)*np.nan     # GR reflectivity
refgr2 = np.zeros(nprof)*np.nan     # GR reflectivity (Ku-band)
ntotpr = np.zeros(nprof,dtype="i4")# total number of SR bins in sample
nrej1 = np.zeros(nprof,dtype="i4")# number of rejected SR bins in sample
ntotgr = np.zeros(nprof,dtype="i4")# total number of GR bins in sample
nrej2 = np.zeros(nprof,dtype="i4")# number of rejected GR bins in sample
iref1 = np.zeros(nprof)*np.nan    # path-integrated SR reflectivity
iref2 = np.zeros(nprof)*np.nan    # path-integrated GR reflectivity
stdv1 = np.zeros(nprof)*np.nan    # std. dev. of SR reflectivity in sample
stdv2 = np.zeros(nprof)*np.nan    # std. dev. of GR reflectivity in sample
volsr = np.zeros(nprof)*np.nan     # total volume of SR bins in sample
volgr = np.zeros(nprof)*np.nan     # total volume of GR bins in sample

[41]:

#%%time
# Loop over relevant SR profiles
for ii, (ss, rr)  in enumerate(zip(vscan,vray)):
    # Index and count valid bins in each profile
    ip = np.where(valid[ss,rr])[0]
    numbins = len(ip)
    ntotpr[ii]=numbins
    if numbins == 0:
        continue
    # Compute the mean position of these bins
    x[ii]=np.mean(xyzp_sr[ss,rr,ip,0])
    y[ii]=np.mean(xyzp_sr[ss,rr,ip,1])
    z[ii]=np.mean(xyzp_sr[ss,rr,ip,2])

    # Thickness of the layer
    dz[ii]=(numbins * dr_sr) / np.cos( np.radians(alpha[ss, rr]) )

    # SR averaging volume
    volsr[ii]=np.sum(vol_sr[ss, rr, ip])

    # Note mean TRMM beam diameter
    ds[ii]=np.radians(bw_sr) * np.mean( ( (zt-z[ii]) /
                                         np.cos( np.radians(alpha[ss, rr]) ) ) )
    # Note distance from radar
    s=np.sqrt(x[ii]**2+y[ii]**2)
    rs[ii]=(re2+z[ii]) * np.sin(s/re2) / np.cos(np.radians(elev_gr))

    # This should not be required because we applied ZonalData
    ### Check that sample is within radar range
    ##if r[ii,jj]+ds[ii,jj]/2. gt rmax then continue

    ## THIS IS THE ORIGINAL IDL CODE - IS THIS A BUG???
    ##ref1[ii,jj]=MEAN(ref_sr1,/nan)
    ##ref3[ii,jj]=MEAN(ref_sr2,/nan)
    ##ref4[ii,jj]=MEAN(ref_sr3,/nan)

    # Simple linear average of reflectivity
    #   - we can become fancier in the next step
    # ATTENTION: NEED TO FLIP ARRAY

    refsr1[ii]=np.nanmean(ref_sr[ss,rr,ip])
    refsr2[ii]=np.nanmean(ref_sr_ss[ss,rr,ip])
    refsr3[ii]=np.nanmean(ref_sr_sh[ss,rr,ip])

    ## Not sure why we need this...
    ### Note the number of rejected bins
    ##nrej1[ii,jj]=ROUND(TOTAL(FINITE(ref_sr1,/nan)))
    ##if FINITE(stdv1[ii,jj]) eq 0 and np-nrej1[ii,jj] gt 1 then STOP

    # SHOULD WE USE ZONALDATA INSTEAD? COULD BE MORE ACCURATE, BUT ALSO SLOWER
    # WE COULD BASICALLY START A NEW LOOP HERE AND RUN ZONALDATA BEFORE

    # Compute the horizontal distance to all the GR bins
    d = np.sqrt((gr_xyz[...,0]-x[ii])**2 + (gr_xyz[...,1]-y[ii])**2)

    # Find all GR bins within the SR beam
    aa, bb = np.where(d <= ds[ii]/2.)

    # Store the number of bins
    ntotgr[ii] = len(aa)

    if len(aa) == 0:
        continue

    # Extract the relevant GR bins

    # Compute the GR averaging volume
    volgr[ii]=np.sum(vol_gr[aa,bb])

    # Average over those bins that exceed the reflectivity threshold
    #   IDL code does exponential distance and volume weighting
    #   Let's try simple mean first,
    #   THEN ZonalStats!
    refgr1[ii]=np.nanmean(ref_gr[aa,bb])
    refgr2[ii]=np.nanmean(ref_gr_ku[aa,bb])

[42]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr1, refsr1, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr1[refsr1>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr1[refsr1>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("uncorrected GR vs uncorrected SR")

[42]:

Text(0.5, 0.98, 'uncorrected GR vs uncorrected SR')

_images/notebooks_match3d_wradlib_match_workflow_79_1.png

[43]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr2, refsr1, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr1[refsr1>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr1[refsr1>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("corrected GR vs uncorrected SR")

[43]:

Text(0.5, 0.98, 'corrected GR vs uncorrected SR')

_images/notebooks_match3d_wradlib_match_workflow_80_1.png

[44]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr2, refsr2, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr2[refsr2>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr2[refsr2>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("corrected GR vs corrected SR snow")

[44]:

Text(0.5, 0.98, 'corrected GR vs corrected SR snow')

_images/notebooks_match3d_wradlib_match_workflow_81_1.png

[45]:

fig = pl.figure(figsize=(12,5))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(refgr2, refsr3, marker="+", c="black")
pl.plot([0,60],[0,60], linestyle="solid", color="black")
pl.xlim(10,50)
pl.ylim(10,50)
pl.xlabel("GR reflectivity (dBZ)")
pl.ylabel("SR reflectivity (dBZ)")
ax = fig.add_subplot(122)
pl.hist(refgr2[refsr3>10], bins=np.arange(-10,50,5), edgecolor="None", label="GR")
pl.hist(refsr3[refsr3>-10], bins=np.arange(-10,50,5), edgecolor="red", facecolor="None", label="SR")
pl.xlabel("Reflectivity (dBZ)")
pl.legend()
fig.suptitle("corrected GR vs corrected SR hail")

[45]:

Text(0.5, 0.98, 'corrected GR vs corrected SR hail')

_images/notebooks_match3d_wradlib_match_workflow_82_1.png

[46]:

fig = pl.figure(figsize=(12,8))
ax = fig.add_subplot(121, aspect="equal")
pl.scatter(x, y, c=refsr1, cmap=pl.cm.viridis, vmin=10, vmax=50, edgecolor="None")
pl.title("SR reflectivity")
pl.xlim(-100000, 150000)
pl.ylim(-150000, 150000)
pl.grid()
ax = fig.add_subplot(122, aspect="equal")
pl.scatter(x, y, c=refgr1, cmap=pl.cm.viridis, vmin=10, vmax=50, edgecolor="None")
pl.title("GR reflectivity")
pl.xlim(-100000, 150000)
pl.ylim(-150000, 150000)
pl.grid()
fig.suptitle("uncorrected GR vs uncorrected SR")
pl.tight_layout()

_images/notebooks_match3d_wradlib_match_workflow_83_0.png

[1]:

# flake8: noqa

Routine verification measures for radar-based precipitation estimates¶

[2]:

import wradlib
import os
import numpy as np
import matplotlib.pyplot as pl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()

Extract bin values from a polar radar data set at rain gage locations¶

Read polar data set¶

[3]:

filename = wradlib.util.get_wradlib_data_file('misc/polar_R_tur.gz')
data = np.loadtxt(filename)

Define site coordinates (lon/lat) and polar coordinate system¶

[4]:

r = np.arange(1,129)
az = np.linspace(0,360,361)[0:-1]
sitecoords = (9.7839, 48.5861)

Make up two rain gauge locations (say we want to work in Gaus Krueger zone 3)¶

[5]:

# Define the projection via epsg-code
proj = wradlib.georef.epsg_to_osr(31467)
# Coordinates of the rain gages in Gauss-Krueger 3 coordinates
x, y = np.array([3557880, 3557890]), np.array([5383379, 5383375])

Now extract the radar values at those bins that are closest to our rain gauges¶

For this purppose, we use the PolarNeighbours class from wraldib’s verify module. Here, we extract the 9 nearest bins…

[6]:

polarneighbs = wradlib.verify.PolarNeighbours(r, az, sitecoords, proj, x, y, nnear=9)
radar_at_gages = polarneighbs.extract(data)
print( "Radar values at rain gauge #1: %r" % radar_at_gages[0].tolist())
print( "Radar values at rain gauge #2: %r" % radar_at_gages[1].tolist())

Radar values at rain gauge #1: [0.01, 0.02, 0.01, 0.01, 0.02, 0.05, 0.01, 0.01, 0.01]
Radar values at rain gauge #2: [0.2, 0.06, 0.15, 0.69, 0.06, 0.26, 0.05, 0.99, 0.32]

Retrieve the bin coordinates (all of them or those at the rain gauges)¶

[7]:

binx, biny = polarneighbs.get_bincoords()
binx_nn, biny_nn = polarneighbs.get_bincoords_at_points()

Plot the entire radar domain and zoom into the surrounding of the rain gauge locations¶

[8]:

fig = pl.figure(figsize=(12,12))
ax = fig.add_subplot(121)
ax.plot(binx, biny, 'r+')
ax.plot(binx_nn, biny_nn, 'b+', markersize=10)
ax.plot(x, y, 'bo')
ax.axis('tight')
ax.set_aspect("equal")
pl.title("Full view")
ax = fig.add_subplot(122)
ax.plot(binx, biny, 'r+')
ax.plot(binx_nn, biny_nn, 'b+', markersize=10)
ax.plot(x, y, 'bo')
pl.xlim(binx_nn.min()-5, binx_nn.max()+5)
pl.ylim(biny_nn.min()-7, biny_nn.max()+8)
ax.set_aspect("equal")
txt = pl.title("Zoom into rain gauge locations")
pl.tight_layout()

_images/notebooks_verification_wradlib_verify_example_16_0.png

Create a verification report¶

In this example, we make up a true Kdp profile and verify our reconstructed Kdp.

Create synthetic data and reconstruct KDP¶

[9]:

# Synthetic truth
dr = 0.5
r = np.arange(0, 100, dr)
kdp_true = np.sin(0.3*r)
kdp_true[kdp_true<0] = 0.
phidp_true = np.cumsum(kdp_true)*2*dr
# Synthetic observation of PhiDP with a random noise and gaps
np.random.seed(1319622840)
phidp_raw = phidp_true + np.random.uniform(-2, 2, len(phidp_true))
gaps = np.random.uniform(0, len(r), 20).astype("int")
phidp_raw[gaps] = np.nan

# linearly interpolate nan
nans = np.isnan(phidp_raw)
phidp_ipol = phidp_raw.copy()
phidp_ipol[nans] = np.interp(r[nans], r[~nans], phidp_raw[~nans])

# Reconstruct PhiDP and KDP
phidp_rawre, kdp_rawre = wradlib.dp.process_raw_phidp_vulpiani(phidp_raw, dr=dr)
phidp_ipre, kdp_ipre = wradlib.dp.process_raw_phidp_vulpiani(phidp_ipol, dr=dr)

# Plot results
fig = pl.figure(figsize=(12,8))
ax = fig.add_subplot(211)
pl.plot(kdp_true, "g-", label="True KDP")
pl.plot(kdp_rawre, "r-", label="Reconstructed Raw KDP")
pl.plot(kdp_ipre, "b-", label="Reconstructed Ipol KDP")
pl.grid()
lg = pl.legend()

ax = fig.add_subplot(212)
pl.plot(r, phidp_true, "b--", label="True PhiDP")
pl.plot(r, np.ma.masked_invalid(phidp_raw), "b-", label="Raw PhiDP")
pl.plot(r, phidp_rawre, "r-", label="Reconstructed Raw PhiDP")
pl.plot(r, phidp_ipre, "g-", label="Reconstructed Ipol PhiDP")
pl.grid()
lg = pl.legend(loc="lower right")
txt = pl.xlabel("Range (km)")

_images/notebooks_verification_wradlib_verify_example_19_0.png

Create verification report¶

[10]:

metrics_raw = wradlib.verify.ErrorMetrics(kdp_true, kdp_rawre)
metrics_raw.pprint()
metrics_ip = wradlib.verify.ErrorMetrics(kdp_true, kdp_ipre)
metrics_ip.pprint()

pl.subplots_adjust(wspace=0.5)
ax = pl.subplot(121, aspect=1.)
ax.plot(metrics_raw.obs, metrics_raw.est, "bo")
ax.plot([-1, 2], [-1, 2], "k--")
pl.xlim(-0.3, 1.1)
pl.ylim(-0.3, 1.1)
xlabel = ax.set_xlabel("True KDP (deg/km)")
ylabel = ax.set_ylabel("Reconstructed Raw KDP (deg/km)")
ax = pl.subplot(122, aspect=1.)
ax.plot(metrics_ip.obs, metrics_ip.est, "bo")
ax.plot([-1, 2], [-1, 2], "k--")
pl.xlim(-0.3, 1.1)
pl.ylim(-0.3, 1.1)
xlabel = ax.set_xlabel("True KDP (deg/km)")
ylabel = ax.set_ylabel("Reconstructed Ipol KDP (deg/km)")

{'corr': 0.95,
 'mas': 0.1,
 'meanerr': -0.0,
 'mse': 0.01,
 'nash': 0.93,
 'pbias': -0.0,
 'r2': 0.9,
 'ratio': nan,
 'rmse': 0.1,
 'spearman': 0.92,
 'sse': 2.96}
{'corr': 0.95,
 'mas': 0.1,
 'meanerr': -0.01,
 'mse': 0.01,
 'nash': 0.93,
 'pbias': -3.0,
 'r2': 0.91,
 'ratio': nan,
 'rmse': 0.1,
 'spearman': 0.92,
 'sse': 2.91}

_images/notebooks_verification_wradlib_verify_example_21_1.png

RADOLAN¶

RADOLAN Product Showcase¶

In this notebook an overview over the existing RADOLAN products is given.

Setup Environment¶

Import modules, filter warnings to avoid cluttering output with DeprecationWarnings and use matplotlib inline or interactive mode if running in ipython or python respectively.

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

Get RADOLAN Grid and RADOLAN Extended Grid.

[2]:

# Get coordinates
radolan_grid_xy = wrl.georef.get_radolan_grid(900,900)
radolan_egrid_xy = wrl.georef.get_radolan_grid(1500,1400)
radolan_wgrid_xy = wrl.georef.get_radolan_grid(1100, 900)
x = radolan_grid_xy[:,:,0]
y = radolan_grid_xy[:,:,1]

xe = radolan_egrid_xy[:,:,0]
ye = radolan_egrid_xy[:,:,1]

xw = radolan_wgrid_xy[:,:,0]
yw = radolan_wgrid_xy[:,:,1]

Define data reading function and plotting function.

[3]:

def read_radolan(radfile):
    radfile = wrl.util.get_wradlib_data_file('radolan/showcase/' + radfile)
    return wrl.io.read_radolan_composite(radfile)

[4]:

def plot_radolan(data, attrs, grid, clabel=None):
    fig = pl.figure(figsize=(10,8))
    ax = fig.add_subplot(111, aspect='equal')
    x = grid[:,:,0]
    y = grid[:,:,1]
    pm = ax.pcolormesh(x, y, data, cmap='viridis')
    cb = fig.colorbar(pm, shrink=0.75)
    cb.set_label(clabel)
    pl.xlabel("x [km]")
    pl.ylabel("y [km]")
    pl.title('{0} Product\n{1}'.format(attrs['producttype'],
                                       attrs['datetime'].isoformat()))
    pl.xlim((x[0,0],x[-1,-1]))
    pl.ylim((y[0,0],y[-1,-1]))
    pl.grid(color='r')

RADOLAN Composite¶

A few products including RW and SF are available free of charge at this DWD FTP Server. A full list of RADOLAN products can be found in the DWD RADOLAN Produktübersicht. Specific details on the RADOLAN Format can be retrieved from the DWD RADOLAN/RADVOR-OP Kompositformat.

Currently, most of the RADOLAN composites have a spatial resolution of 1km x 1km, with the National Composites (R-, S- and W-series) being 900 x 900 km grids, and the European Composites 1500 x 1400 km grids. The polar-stereographic projection is described in the chapter RADOLAN Grid.

One difference is the extended National Composite (only WX) with a 1100 x 900 km grid.

Also the PG/PC-Product with 460 x 460 km grid and runlength-coding is shortly described.

National Composites¶

ID	INT	avail	Description
RX/WX	5 min	5 min	original radardata in qualitative RVP6-units (1 byte coded)
RZ	5 min	5 min	radardata after correction of PBB converted to rainrate with improved Z-R-relation
RY	5 min	5 min	radardata after correction with Quality-composit (QY)
RH	1 h	5 min	1 h summation of RZ-composit
RB	1 h	hh:50	1 h summation with preadjustment
RW	1 h	hh:50	1 h summation with standard adjustment “best of two”
RL	1 h	hh:50	1 h summation with adjustment by Merging
RU	1 h	hh:50	1 h summation with standard and merging adjustment “best of three”
SQ	6 h	hh:50	6 h summation of RW
SH	12 h	hh:50	12 h summation of RW
SF	24 h	hh:50	24 h summation of RW
W1	7 d	05:50	7 d summation of RW
W2	14 d	05:50	14 d summation of RW
W3	21 d	05:50	21 d summation of RW
W4	30 d	05:50	30 d summation of RW

RADOLAN RX Product¶

Load data from data source.

[5]:

data, attrs = read_radolan('raa01-rx_10000-1408102050-dwd---bin.gz')

Mask data and apply scale and offset

[6]:

data = np.ma.masked_equal(data, -9999) / 2 - 32.5

[7]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='dBZ')

_images/notebooks_radolan_radolan_showcase_20_0.png

RADOLAN RZ Product¶

Load data from data source

[8]:

data, attrs = read_radolan('raa01-rz_10000-1408102050-dwd---bin.gz')

Mask data

[9]:

data = np.ma.masked_equal(data, -9999)

[10]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 5min-1')

_images/notebooks_radolan_radolan_showcase_26_0.png

RADOLAN RY Product¶

Load data from data source

[11]:

data, attrs = read_radolan('raa01-ry_10000-1408102050-dwd---bin.gz')

Mask data

[12]:

data = np.ma.masked_equal(data, -9999)

[13]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 5min-1')

_images/notebooks_radolan_radolan_showcase_32_0.png

RADOLAN RH Product¶

Load data from data source

[14]:

data, attrs = read_radolan('raa01-rh_10000-1408102050-dwd---bin.gz')

Mask data

[15]:

data = np.ma.masked_equal(data, -9999)

[16]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_38_0.png

RADOLAN RB Product¶

Load data from data source

[17]:

data, attrs = read_radolan('raa01-rb_10000-1408102050-dwd---bin.gz')

Mask data

[18]:

data = np.ma.masked_equal(data, -9999)

[19]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_44_0.png

RADOLAN RL Product¶

Load data from data source

[20]:

data, attrs = read_radolan('raa01-rl_10000-1408102050-dwd---bin.gz')

Mask data

[21]:

data = np.ma.masked_equal(data, -9999)

[22]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_50_0.png

RADOLAN RW Product¶

Load data from data source

[23]:

data, attrs = read_radolan('raa01-rw_10000-1408102050-dwd---bin.gz')

Mask data

[24]:

data = np.ma.masked_equal(data, -9999)

[25]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_56_0.png

RADOLAN RU Product¶

Load data from data source

[26]:

data, attrs = read_radolan('raa01-ru_10000-1408102050-dwd---bin.gz')

Mask data

[27]:

data = np.ma.masked_equal(data, -9999)

[28]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_62_0.png

RADOLAN SQ Product¶

Load data from data source

[29]:

data, attrs = read_radolan('raa01-sq_10000-1408102050-dwd---bin.gz')

Mask data

[30]:

data = np.ma.masked_equal(data, -9999)

[31]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 6h-1')

_images/notebooks_radolan_radolan_showcase_68_0.png

RADOLAN SH Product¶

Load data from data source

[32]:

data, attrs = read_radolan('raa01-sh_10000-1408102050-dwd---bin.gz')

Mask data

[33]:

data = np.ma.masked_equal(data, -9999)

[34]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 12h-1')

_images/notebooks_radolan_radolan_showcase_74_0.png

RADOLAN SF Product¶

Load data from data source

[35]:

data, attrs = read_radolan('raa01-sf_10000-1408102050-dwd---bin.gz')

Mask data

[36]:

data = np.ma.masked_equal(data, -9999)

[37]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 24h-1')

_images/notebooks_radolan_radolan_showcase_80_0.png

RADOLAN W1 Product¶

Load data from data source

[38]:

data, attrs = read_radolan('raa01-w1_10000-1408110550-dwd---bin.gz')

Mask data

[39]:

data = np.ma.masked_equal(data, -9999)

[40]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 7d-1')

_images/notebooks_radolan_radolan_showcase_86_0.png

RADOLAN W2 Product¶

Load data from data source

[41]:

data, attrs = read_radolan('raa01-w2_10000-1408110550-dwd---bin.gz')

Mask data

[42]:

data = np.ma.masked_equal(data, -9999)

[43]:

plot_radolan(data, attrs, radolan_grid_xy, clabel='mm * 14d-1')

_images/notebooks_radolan_radolan_showcase_92_0.png

RADOLAN WX Product¶

[44]:

data, attrs = read_radolan('raa01-wx_10000-1408102050-dwd---bin.gz')

Mask data

[45]:

data = np.ma.masked_equal(data, -9999) / 2 - 32.5

[46]:

plot_radolan(data, attrs, radolan_wgrid_xy, clabel='dBZ')

_images/notebooks_radolan_radolan_showcase_97_0.png

Extended RADOLAN Composites¶

The common central european products with a range of 1500 km by 1400 km are presented in the following table:

ID	INT	avail	Description
EX	5 min	5 min	analogue RX
EZ	5 min	5 min	analogue RZ
EY	5 min	5 min	analogue EY after correction with Quality-composit
EH	1 h	hh:50	analogue RH (no preadjustment) 1 h summation of EY-composite
EB	1 h	hh:50	analogue RB (with preadjustment) 1 h summation
EW	1 h	hh:50	analogue RW (full adjustment) 1 h summation

RADOLAN EX Product¶

Load data from data source

[47]:

data, attrs = read_radolan('raa01-ex_10000-1408102050-dwd---bin.gz')

Mask data and apply scale and offset

[48]:

data = np.ma.masked_equal(data, -9999) / 2 - 32.5

[49]:

plot_radolan(data, attrs, radolan_egrid_xy, clabel='dBZ')

_images/notebooks_radolan_radolan_showcase_105_0.png

RADOLAN EZ Product¶

Load data from data source

[50]:

data, attrs = read_radolan('raa01-ez_10000-1408102050-dwd---bin.gz')

Mask data

[51]:

data = np.ma.masked_equal(data, -9999)

[52]:

plot_radolan(data, attrs, radolan_egrid_xy, clabel='mm * 5min-1')

_images/notebooks_radolan_radolan_showcase_111_0.png

RADOLAN EY Product¶

Load data from data source

[53]:

data, attrs = read_radolan('raa01-ey_10000-1408102050-dwd---bin.gz')

Mask data

[54]:

data = np.ma.masked_equal(data, -9999)

[55]:

plot_radolan(data, attrs, radolan_egrid_xy, clabel='mm * 5min-1')

_images/notebooks_radolan_radolan_showcase_117_0.png

RADOLAN EH Product¶

Load data from data source

[56]:

data, attrs = read_radolan('raa01-eh_10000-1408102050-dwd---bin.gz')

Mask data

[57]:

data = np.ma.masked_equal(data, -9999)

[58]:

plot_radolan(data, attrs, radolan_egrid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_123_0.png

RADOLAN EB Product¶

Load data from data source

[59]:

data, attrs = read_radolan('raa01-eb_10000-1408102050-dwd---bin.gz')

Mask data

[60]:

data = np.ma.masked_equal(data, -9999)

[61]:

plot_radolan(data, attrs, radolan_egrid_xy, clabel='mm * h-1')

_images/notebooks_radolan_radolan_showcase_129_0.png

RADOLAN PG/PC Product¶

The PG/PC product is a bit different from the normal RADOLAN formats. The header is actually the same, but the data is runlength encoded. Also the RADOLAN grid cells have 2km edge length (460x460 cells).

Load data from data source

[62]:

radfile = 'raa00-pc_10015-1408030905-dwd---bin.gz'
radfile = wrl.util.get_wradlib_data_file('radolan/misc/' + radfile)
data, attrs = wrl.io.read_radolan_composite(radfile, missing=255)
radolan_grid_pc = wrl.georef.get_radolan_grid(460,460)

Mask data

[63]:

data = np.ma.masked_equal(data, 255)
print(data.shape)

(460, 460)

[64]:

# plot the images side by side
pl.figure(figsize=(10, 8))
pl.subplot(111, aspect='equal')
X = radolan_grid_pc[:,:,0]
Y = radolan_grid_pc[:,:,1]
# color-scheme taken from DWD "legend_radar_products_pc.pdf"
colors = ['lightgrey', 'yellow', 'lightblue', 'magenta', 'green',
          'red', 'darkblue', 'darkred']
cmap = mpl.colors.ListedColormap(colors, name=u'DWD-pc-scheme')
bounds = np.arange(len(colors) + 1)
norm = mpl.colors.BoundaryNorm(bounds, cmap.N)
pl.pcolormesh(X, Y, data, cmap=cmap, norm=norm)
pl.xlim((X[0,0], X[-1,-1]))
pl.ylim((Y[0,0], Y[-1,-1]))

# add colorbar and do some magic for proper visualisation
cb = pl.colorbar(shrink=0.75, norm=norm, boundaries=bounds)
loc = bounds + .5
cb.set_ticks(loc)
labels = bounds[:-1]
cb.set_ticklabels(labels)
cl = cb.ax.get_yticklabels()
cl[-1].set_text('9')
cb.ax.set_yticklabels([elem.get_text() for elem in cl])
pl.title('RADOLAN PG Product \n' + attrs['datetime'].isoformat())
pl.grid(color='r')

RADOLAN Radar Network¶

In this chapter the RW-product is shown in WGS84 and the RADOLAN Polar Stereographic Projection. All for the compositing process used radars are extracted from the metadata and plotted with their respective maximum range rings and location information.

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import matplotlib as mpl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np
from osgeo import osr

[2]:

def get_radar_locations():
    radars = {}
    radar = {'name': 'ASR Dresden', 'wmo': 10487, 'lon': 13.76347,
             'lat': 51.12404, 'alt': 261}
    radars['ASD'] = radar

    radar = {'name': 'Boostedt', 'wmo': 10132, 'lon': 10.04687,
             'lat': 54.00438, 'alt': 124.56}
    radars['BOO'] = radar

    radar = {'name': 'Dresden', 'wmo': 10488, 'lon': 13.76865, 'lat': 51.12465,
             'alt': 263.36}
    radars['DRS'] = radar

    radar = {'name': 'Eisberg', 'wmo': 10780, 'lon': 12.40278, 'lat': 49.54066,
             'alt': 798.79}
    radars['EIS'] = radar

    radar = {'name': 'Emden', 'wmo': 10204, 'lon': 7.02377, 'lat': 53.33872,
             'alt': 58}
    radars['EMD'] = radar

    radar = {'name': 'Essen', 'wmo': 10410, 'lon': 6.96712, 'lat': 51.40563,
             'alt': 185.10}
    radars['ESS'] = radar

    radar = {'name': 'Feldberg', 'wmo': 10908, 'lon': 8.00361, 'lat': 47.87361,
             'alt': 1516.10}
    radars['FBG'] = radar

    radar = {'name': 'Flechtdorf', 'wmo': 10440, 'lon': 8.802, 'lat': 51.3112,
             'alt': 627.88}
    radars['FLD'] = radar

    radar = {'name': 'Hannover', 'wmo': 10339, 'lon': 9.69452, 'lat': 52.46008,
             'alt': 97.66}
    radars['HNR'] = radar

    radar = {'name': 'Neuhaus', 'wmo': 10557, 'lon': 11.13504, 'lat': 50.50012,
             'alt': 878.04}
    radars['NEU'] = radar

    radar = {'name': 'Neuheilenbach', 'wmo': 10605, 'lon': 6.54853,
             'lat': 50.10965, 'alt': 585.84}
    radars['NHB'] = radar

    radar = {'name': 'Offenthal', 'wmo': 10629, 'lon': 8.71293, 'lat': 49.9847,
             'alt': 245.80}
    radars['OFT'] = radar

    radar = {'name': 'Proetzel', 'wmo': 10392, 'lon': 13.85821,
             'lat': 52.64867, 'alt': 193.92}
    radars['PRO'] = radar

    radar = {'name': 'Memmingen', 'wmo': 10950, 'lon': 10.21924,
             'lat': 48.04214, 'alt': 724.40}
    radars['MEM'] = radar

    radar = {'name': 'Rostock', 'wmo': 10169, 'lon': 12.05808, 'lat': 54.17566,
             'alt': 37}
    radars['ROS'] = radar

    radar = {'name': 'Isen', 'wmo': 10873, 'lon': 12.10177, 'lat': 48.1747,
             'alt': 677.77}
    radars['ISN'] = radar

    radar = {'name': 'Tuerkheim', 'wmo': 10832, 'lon': 9.78278,
             'lat': 48.58528, 'alt': 767.62}
    radars['TUR'] = radar

    radar = {'name': 'Ummendorf', 'wmo': 10356, 'lon': 11.17609,
             'lat': 52.16009, 'alt': 183}
    radars['UMM'] = radar

    return radars

[3]:

# load radolan file
rw_filename = wrl.util.get_wradlib_data_file('radolan/showcase/raa01-rw_10000-1408102050-dwd---bin.gz')
rwdata, rwattrs = wrl.io.read_radolan_composite(rw_filename)

[4]:

# print the available attributes
print("RW Attributes:", rwattrs)

RW Attributes: {'producttype': 'RW', 'datetime': datetime.datetime(2014, 8, 10, 20, 50), 'radarid': '10000', 'datasize': 1620000, 'maxrange': '150 km', 'radolanversion': '2.13.1', 'precision': 0.1, 'intervalseconds': 3600, 'nrow': 900, 'ncol': 900, 'radarlocations': ['boo', 'ros', 'emd', 'hnr', 'umd', 'pro', 'ess', 'asd', 'neu', 'nhb', 'oft', 'tur', 'isn', 'fbg', 'mem'], 'nodataflag': -9999, 'secondary': array([   799,    800,    801, ..., 806263, 806264, 807163]), 'nodatamask': array([     0,      1,      2, ..., 809997, 809998, 809999]), 'cluttermask': array([], dtype=int64)}

[5]:

# mask data
sec = rwattrs['secondary']
rwdata.flat[sec] = -9999
rwdata = np.ma.masked_equal(rwdata, -9999)

# create radolan projection object
proj_stereo = wrl.georef.create_osr("dwd-radolan")

# create wgs84 projection object
proj_wgs = osr.SpatialReference()
proj_wgs.ImportFromEPSG(4326)

# get radolan grid
radolan_grid_xy = wrl.georef.get_radolan_grid(900, 900)
x1 = radolan_grid_xy[:, :, 0]
y1 = radolan_grid_xy[:, :, 1]

# convert to lonlat
radolan_grid_ll = wrl.georef.reproject(radolan_grid_xy,
                                       projection_source=proj_stereo,
                                       projection_target=proj_wgs)
lon1 = radolan_grid_ll[:, :, 0]
lat1 = radolan_grid_ll[:, :, 1]

[6]:

# range array 150 km
print("Max Range: ", rwattrs['maxrange'])
r = np.arange(1, 151) * 1000
# azimuth array 1 degree spacing
az = np.linspace(0, 360, 361)[0:-1]

Max Range:  150 km

[7]:

# get radar dict
radars = get_radar_locations()

[8]:

def plot_radar(radar, ax, proj):

    site = (radar['lon'], radar['lat'], radar['alt'] )

    # build polygons for maxrange rangering
    polygons = wrl.georef.spherical_to_polyvert(r, az, 0,
                                                site,
                                                proj=proj)
    polygons = polygons[..., 0:2]
    polygons.shape = (len(az), len(r), 5, 2)
    polygons = polygons[:, -1, :, :]

    x_loc, y_loc = wrl.georef.reproject(site[0], site[1],
                                        projection_source=proj_wgs,
                                        projection_target=proj)


    # create PolyCollections and add to respective axes
    polycoll = mpl.collections.PolyCollection(polygons, closed=True,
                                              edgecolors='r',
                                              facecolors='r')
    ax.add_collection(polycoll, autolim=True)

    # plot radar location and information text
    ax.plot(x_loc, y_loc, 'r+')
    ax.text(x_loc, y_loc, radar['name'], color='r')

[9]:

# plot two projections side by side
fig1 = pl.figure(figsize=(10,8))
ax1 = fig1.add_subplot(111, aspect='equal')
pm = ax1.pcolormesh(lon1, lat1, rwdata, cmap='viridis')
cb = fig1.colorbar(pm, shrink=0.75)
cb.set_label("mm/h")
pl.xlabel("Longitude ")
pl.ylabel("Latitude")
pl.title(
    'RADOLAN RW Product \n' + rwattrs['datetime'].isoformat() + '\n WGS84')
pl.xlim((lon1[0, 0], lon1[-1, -1]))
pl.ylim((lat1[0, 0], lat1[-1, -1]))
pl.grid(color='r')
for radar_id in rwattrs['radarlocations']:
    # get radar coords etc from dict
    # repair Ummendorf ID
    if radar_id == 'umd':
        radar_id = 'umm'
    radar = radars[radar_id.upper()]
    plot_radar(radar, ax1, proj_wgs)

_images/notebooks_radolan_radolan_network_11_0.png

[10]:

fig2 = pl.figure(figsize=(10,8))
ax2 = fig2.add_subplot(111, aspect='equal')
pm = ax2.pcolormesh(x1, y1, rwdata, cmap='viridis')
cb = fig2.colorbar(pm, shrink=0.75)
cb.set_label("mm/h")
pl.xlabel("x [km]")
pl.ylabel("y [km]")
pl.title('RADOLAN RW Product \n' + rwattrs[
    'datetime'].isoformat() + '\n Polar Stereographic Projection')
pl.xlim((x1[0, 0], x1[-1, -1]))
pl.ylim((y1[0, 0], y1[-1, -1]))
pl.grid(color='r')
for radar_id in rwattrs['radarlocations']:
    # get radar coords etc from dict
    # repair Ummendorf ID
    if radar_id == 'umd':
        radar_id = 'umm'
    radar = radars[radar_id.upper()]
    plot_radar(radar, ax2, proj_stereo)

_images/notebooks_radolan_radolan_network_12_0.png

RADOLAN is abbreviated from the german RAdar-OnLine-ANeichung, which means Radar-Online-Adjustment.

Using it’s network of 17 weather radar the German Weather Service provides many products for high resolution precipitation analysis and forecast. A comprehensive product list can be found in chapter RADOLAN Product Showcase.

These composite products are distributed in the RADOLAN Binary Data Format with an ASCII header. All composites are available in Polar Stereographic Projection which will be discussed in the chapter RADOLAN Grid.

This notebook tutorial was prepared with material from the DWD RADOLAN/RADVOR-OP Kompositformat. We also wish to thank Elmar Weigl, German Weather Service, for providing the extensive set of example data and his valuable information about the RADOLAN products.

Recipes¶

Recipe #1: Clutter and attenuation correction plus composition for two DWD radars¶

This recipe shows a workflow to process radar data provided by the German Weather Service (DWD). The processing includes:

(1) Reading local DX-Data for radars Feldberg and Tuerkheim.
(2) Clutter correction using the Gabella filter algorithm.
(3) Attenuation correction using the modified Kraemer algorithm.
(4) Conversion from reflectivity to rainfall using the Z-R Conversions module.
(5) Accumulation of rainfall depths over the entire event.
(6) Composition of data from both radars to a common Cartesian grid (UTM Zone 32). Composition is based on a weighted combination, using the sampling volume as a quality criterion
(7) Plotting a rainfall map using cartesian_plot

[1]:

import wradlib as wrl
import matplotlib.pyplot as pl
import warnings
warnings.filterwarnings('ignore')
try:
    get_ipython().magic("matplotlib inline")
except:
    pl.ion()
import numpy as np

[2]:

import glob
import os
def process_polar_level_data(radarname):
    """Reading and processing polar level data (DX) for radar <radarname>
    """
    print("Polar level processing for radar %s..." % radarname)
    # preparations for loading sample data in source directory
    files = glob.glob(os.path.join(wrl.util.get_wradlib_data_path(),
                                   'dx/recipe1_data/raa*%s*bin' % radarname))

    if len(files) == 0:
        print("WARNING: No data files found - maybe you did not extract "
              "the data from data/recipe1_data.zip?")
    data = np.empty((len(files), 360, 128))
    # loading the data (two hours of 5-minute images)
    for i, f in enumerate(files):
        #print(i, f)
        data[i], attrs = wrl.io.read_dx(f)
    # Clutter filter on an event base
    clmap = wrl.clutter.filter_gabella(data.mean(axis=0), tr1=12, n_p=6,
                                       tr2=1.1)
    for i, scan in enumerate(data):
        data[i] = wrl.ipol.interpolate_polar(scan, clmap)
    # correcting for attenuation
    pia = wrl.atten.correct_attenuation_constrained(
        data,
        a_max=1.67e-4,
        a_min=2.33e-5,
        n_a=100,
        b_max=0.7,
        b_min=0.65,
        n_b=6,
        gate_length=1.,
        constraints=[wrl.atten.constraint_dbz,
                     wrl.atten.constraint_pia],
        constraint_args=[[59.0],[10.0]])
    data = data + pia
    # converting to precipitation depth
    R = wrl.zr.z_to_r(wrl.trafo.idecibel(data), a=256, b=1.4)
    depth = wrl.trafo.r_to_depth(R, 300.)
    # calculate hourly accumulation
    accum = depth.sum(axis=0)

    return accum

[3]:

def bbox(*args):
    """Get bounding box from a set of radar bin coordinates
    """
    x = np.array([])
    y = np.array([])
    for arg in args:
        x = np.append(x, arg[:, 0])
        y = np.append(y, arg[:, 1])
    xmin = x.min()
    xmax = x.max()
    ymin = y.min()
    ymax = y.max()

    return xmin, xmax, ymin, ymax

[4]:

import zipfile
import shutil
import datetime as dt

# set timer
start = dt.datetime.now()
# unzip data
filename = wrl.util.get_wradlib_data_file('dx/recipe1_data.zip')
targetdir = wrl.util.get_wradlib_data_path() + '/dx/recipe1_data'
with zipfile.ZipFile(filename, 'r') as z:
    z.extractall(targetdir)

# set scan geometry and radar coordinates
r = np.arange(500., 128500., 1000.)
az = np.arange(0, 360)
tur_sitecoords = (9.7839, 48.5861)
fbg_sitecoords = (8.005, 47.8744)

# processing polar level radar data
#   Tuerkheim
tur_accum = process_polar_level_data("tur")
#   Feldberg
fbg_accum = process_polar_level_data("fbg")

# remove unzipped files
if os.path.exists(targetdir):
    try:
        shutil.rmtree(targetdir)
    except Exception:
        print("WARNING: Could not remove directory data/recipe1_data")

# derive UTM Zone 32 coordinates of range-bin centroids
# create osr projection using epsg number for UTM Zone 32
proj_utm = wrl.georef.epsg_to_osr(32632)

#  for Tuerkheim radar
tur_coord = wrl.georef.spherical_to_centroids(r, az, 0, tur_sitecoords,
                                              proj=proj_utm)
tur_coord = tur_coord[..., 0:2]
tur_coord = tur_coord.reshape(-1, tur_coord.shape[-1])

# for Feldberg radar
fbg_coord = wrl.georef.spherical_to_centroids(r, az, 0, fbg_sitecoords,
                                              proj=proj_utm)
fbg_coord = fbg_coord[..., 0:2]
fbg_coord = fbg_coord.reshape(-1, fbg_coord.shape[-1])

# define target grid for composition
xmin, xmax, ymin, ymax = bbox(tur_coord, fbg_coord)
x = np.linspace(xmin, xmax + 1000., 1000)
y = np.linspace(ymin, ymax + 1000., 1000)
grid_coords = wrl.util.gridaspoints(y, x)

# derive quality information - in this case, the pulse volume
pulse_volumes = np.tile(wrl.qual.pulse_volume(r, 1000., 1.), 360)
# interpolate polar radar-data and quality data to the grid
print("Gridding Tuerkheim data...")
tur_quality_gridded = wrl.comp.togrid(tur_coord, grid_coords,
                                      r.max() + 500.,
                                      tur_coord.mean(axis=0),
                                      pulse_volumes,
                                      wrl.ipol.Nearest)
tur_gridded = wrl.comp.togrid(tur_coord, grid_coords, r.max() + 500.,
                              tur_coord.mean(axis=0),
                              tur_accum.ravel(),
                              wrl.ipol.Nearest)

print("Gridding Feldberg data...")
fbg_quality_gridded = wrl.comp.togrid(fbg_coord, grid_coords,
                                      r.max() + 500.,
                                      fbg_coord.mean(axis=0),
                                      pulse_volumes,
                                      wrl.ipol.Nearest)
fbg_gridded = wrl.comp.togrid(fbg_coord, grid_coords, r.max() + 500.,
                              fbg_coord.mean(axis=0),
                              fbg_accum.ravel(),
                              wrl.ipol.Nearest)

# compose the both radar-data based on the quality information
# calculated above
print("Composing Tuerkheim and Feldbarg data on a common grid...")
composite = wrl.comp.compose_weighted([tur_gridded, fbg_gridded],
                                      [1. / (tur_quality_gridded + 0.001),
                                       1. / (fbg_quality_gridded + 0.001)])
composite = np.ma.masked_invalid(composite)

print("Processing took:", dt.datetime.now() - start)

Polar level processing for radar tur...
Polar level processing for radar fbg...
Gridding Tuerkheim data...
Gridding Feldberg data...
Composing Tuerkheim and Feldbarg data on a common grid...
Processing took: 0:00:13.068088

[5]:

# Plotting rainfall map
pl.figure(figsize=(10,8))
pl.subplot(111, aspect="equal")
pm = pl.pcolormesh(x, y, composite.reshape((len(x), len(y))),
                   cmap="viridis")
pl.grid()
pl.xlim(min(x), max(x))
pl.ylim(min(y), max(y))
pl.colorbar(pm, shrink=0.85)

[5]:

<matplotlib.colorbar.Colorbar at 0x7faf5123d9a0>

Download required data at the wradlib-data repository.

Note

In order to run the recipe code, you need to extract the sample data into a directory pointed to by environment variable WRADLIB_DATA.

Recipe #2: Reading and visualizing an ODIM_H5 polar volume¶

This recipe shows how extract the polar volume data from an ODIM_H5 hdf5 file (KNMI example file from OPERA), contruct a 3-dimensional Cartesian volume and produce a diagnostic plot. The challenge for this file is that for each elevation angle, the scan strategy is different.

[2]:

import datetime as dt
from osgeo import osr

# read the data (sample file in WRADLIB_DATA)
filename = wrl.util.get_wradlib_data_file('hdf5/knmi_polar_volume.h5')
raw = wrl.io.read_opera_hdf5(filename)
# this is the radar position tuple (longitude, latitude, altitude)
sitecoords = (raw["where"]["lon"][0], raw["where"]["lat"][0],
              raw["where"]["height"][0])
# define your cartesian reference system
# proj = wradlib.georef.create_osr(32632)
proj = osr.SpatialReference()
proj.ImportFromEPSG(32632)
# containers to hold Cartesian bin coordinates and data
xyz, data = np.array([]).reshape((-1, 3)), np.array([])
# iterate over 14 elevation angles
for i in range(14):
    # get the scan metadata for each elevation
    where = raw["dataset%d/where" % (i + 1)]
    what = raw["dataset%d/data1/what" % (i + 1)]
    # define arrays of polar coordinate arrays (azimuth and range)
    az = np.arange(0., 360., 360. / where["nrays"])
    # rstart is given in km, so multiply by 1000.
    rstart = where["rstart"] * 1000.
    r = np.arange(rstart,
                  rstart + where["nbins"] * where["rscale"],
                  where["rscale"])
    # derive 3-D Cartesian coordinate tuples
    xyz_ = wrl.vpr.volcoords_from_polar(sitecoords, where["elangle"],
                                        az, r, proj)
    # get the scan data for this elevation
    #   here, you can do all the processing on the 2-D polar level
    #   e.g. clutter elimination, attenuation correction, ...
    data_ = what["offset"] + what["gain"] * raw[
        "dataset%d/data1/data" % (i + 1)]
    # transfer to containers
    xyz, data = np.vstack((xyz, xyz_)), np.append(data, data_.ravel())

# generate 3-D Cartesian target grid coordinates
maxrange = 200000.
minelev = 0.1
maxelev = 25.
maxalt = 5000.
horiz_res = 2000.
vert_res = 250.
trgxyz, trgshape = wrl.vpr.make_3d_grid(sitecoords, proj, maxrange,
                                        maxalt, horiz_res, vert_res)

# interpolate to Cartesian 3-D volume grid
tstart = dt.datetime.now()
gridder = wrl.vpr.CAPPI(xyz, trgxyz, trgshape, maxrange, minelev,
                        maxelev)
vol = np.ma.masked_invalid(gridder(data).reshape(trgshape))
print("3-D interpolation took:", dt.datetime.now() - tstart)

# diagnostic plot
trgx = trgxyz[:, 0].reshape(trgshape)[0, 0, :]
trgy = trgxyz[:, 1].reshape(trgshape)[0, :, 0]
trgz = trgxyz[:, 2].reshape(trgshape)[:, 0, 0]
wrl.vis.plot_max_plan_and_vert(trgx, trgy, trgz, vol, unit="dBZH",
                               levels=range(-32, 60))

3-D interpolation took: 0:00:01.503522

Note

In order to run the recipe code, you need to extract the sample data into a directory pointed to by environment variable WRADLIB_DATA.

This recipe section provides a collection of code snippets for exemplary $\omega radlib$ applications. Compared to the other notebooks, the level of documentation is a bit lower. For each recipe, we provide a short summary and a link to the example code.

Please feel to send us your recipes so we can include them on this page. Please send an e-mail to wradlib-dev@googlegroups.com containing:

optional: name and affiliation
a suggested title of the recipe
a short decription of the recipe (max. 100 words)
the recipe code (please add comments within the code!)
optional: the data needed to run the code (or a hyperlink to the data)
optional: some test output of the recipe (e.g. an image file)

Help & reference

Library Reference¶

Gage adjustment¶

Concept¶

The objective of this module is the adjustment of radar-based rainfall estimates by rain gage observations. However, this module could also be applied to adjust satellite rainfall by rain gage observations, remotely sensed soil moisture patterns by ground truthing moisture sensors, or any dense spatial point pattern which could be adjusted by sparse point measurements (ground truth).

Basically, we only need two data sources:

point observations (e.g. rain gage observations)
set of (potentially irregular) unadjusted point values (e.g. remotely sensed rainfall)

[Goudenhoofdt2009] provide an excellent overview of adjustment procedures. The general idea is that we quantify the error of the remotely sensed rainfall at the rain gage locations, assuming the rain gage observation to be accurate.

The error can be assumed to be purely additive (AdjustAdd), purely multiplicative (AdjustMultiply, AdjustMFB) or a mixture of both (AdjustMixed). If the error is assumed to be heterogeneous in space (AdjustAdd, AdjustMultiply, AdjustMixed), the error at the rain gage locations is interpolated to the radar bin locations and then used to adjust (correct) the raw radar rainfall estimates. In case of the AdjustMFB approach, though, the multiplicative error is assumed to be homogeneous in space.

Quick start¶

The basic procedure consists of creating an adjustment object from the class you want to use for adjustment. After that, you can call the object with the actual data that is to be adjusted. The following example is using the additive error model with default settings. obs_coords and raw_coords represent arrays with coordinate pairs for the gage observations and the radar bins, respectively. obs and raw are arrays containing the actual data:

adjuster = AdjustAdd(obs_coords, raw_coords)
adjusted = adjuster(obs, raw)

Both obs and raw need to be flat (1-dimensional) arrays of shape (n,) that have the same length as the the obs_coords and raw_coords arrays, respectively.

The user can specify the approach that should be used to interpolate the error in space, as well as the keyword arguments which control the behaviour of the interpolation approach. For this purpose, all interpolation classes from the wradlib.ipol module are available and can be passed by using the ipclass argument. The default interpolation class is Inverse Distance Weighting (Idw). If you want to use e.g. linear barycentric interpolation:

import wradlib.ipol as ipol
adjuster = AdjustAdd(obs_coords, raw_coords, ipclass=ipol.Linear)
adjusted = adjuster(obs, raw)

Warning

Be aware that there are a lot of control parameters that can dramatically influence the behaviour of the adjustment (which gauges are considered, how is an error interpolation carried out, …). Read the docs carefully and try to experiment with the effects of the different control parameters. There might be situations in which the algorithms decides - based on the control parameter - not to do an adjustment and just return the unadjusted values.

Cross validation¶

Another helpful feature is an easy-to-use method for leave-one-out cross-validation [Cross-validation]. Cross validation is a standard procedure for verifying rain gage adjustment or interpolation procedures. You can start the cross validation in the same way as you start the actual adjustment, however, you call the xvalidate method instead. The result of the cross validation are pairs of observation and the corresponding adjustment result at the observation location. Using the wradlib.verify module, you can compute error metrics for the cross validation results:

adjuster = AdjustAdd(obs_coords, raw_coords)
observed, estimated = adjuster.xvalidate(obs, raw)
from wradlib.verify import ErrorMetrics
metrics = ErrorMetrics(observed, estimated)
metrics.report()

`AdjustBase`	The basic adjustment class that inherits to all other classes.
`AdjustMFB`	Multiplicative gage adjustment using one correction factor for the entire domain.
`AdjustMultiply`	Gage adjustment using a multiplicative error model
`AdjustAdd`	Gage adjustment using an additive error model.
`AdjustMixed`	Gage adjustment using a mixed error model (additive and multiplicative).
`RawAtObs`	Get the raw values in the neighbourhood of the observation points
`GageOnly`	Same behaviour as the other adjustment classes, but returns an interpolation of rain gage observations
`AdjustNone`	Same behaviour as the other adjustment classes, but simply returns the unadjusted data.

Attenuation Correction¶

`correct_attenuation_hb`	Gate-by-Gate attenuation correction according to [Hitschfeld1954]
`constraint_dbz`	Constraint callback function for correct_attenuation_constrained.
`constraint_pia`	Constraint callback function for correct_attenuation_constrained.
`correct_attenuation_constrained`	Gate-by-Gate attenuation correction based on the iterative approach of [Kraemer2008] and [Jacobi2016] with a generalized and scalable number of constraints.
`correct_radome_attenuation_empirical`	Estimate two-way wet radome losses.
`pia_from_kdp`	Retrieving path integrated attenuation from specific differential phase (Kdp).

Hydrometeor Classification (HMC)¶

`msf_index_indep`	Retrieve membership function values based on independent observable
`trapezoid`	Calculates membership of obs using trapezoidal membership functions
`fuzzyfi`	Iterate over all hmc-classes and retrieve memberships
`probability`	Calculate probability of hmc-class for every data bin.
`classify`	Calculate probability of hmc-class for every data bin.

Clutter Identification¶

`filter_gabella`	Clutter identification filter developed by [Gabella2002].
`filter_gabella_a`	First part of the Gabella filter looking for large reflectivity gradients.
`filter_gabella_b`	Second part of the Gabella filter comparing area to circumference of contiguous echo regions.
`filter_cloudtype`	Identification of non-meteorological echoes based on cloud type.
`filter_window_distance`	2d filter looking for large reflectivity gradients.
`histo_cut`	Histogram based clutter identification.
`classify_echo_fuzzy`	Fuzzy echo classification and clutter identification based on polarimetric moments.

Composition¶

Combine data from different radar locations on one common set of locations

`extract_circle`	Extract the indices of `coords` which fall within a circle defined by `center` and `radius`.
`togrid`	Interpolate data from a radar location to the composite grid or set of locations
`compose_ko`	Composes grids according to quality information using quality information as a knockout criterion.
`compose_weighted`	Composes grids according to quality information using a weighted averaging approach.

Dual-Pol and Differential Phase¶

Overview¶

This module provides algorithms to process polarimetric radar moments, namely the differential phase, $Phi_{DP}$, and, based on successful $Phi_{DP}$ retrieval, also the specific differential phase, $K_{DP}$. Please note that the actual application of polarimetric moments is implemented in the corresponding wradlib modules, e.g.:

fuzzy echo classification from polarimetric moments (wradlib.clutter.classify_echo_fuzzy)

attenuation correction (wradlib.atten.pia_from_kdp)

direct precipitation retrieval from Kdp (wradlib.trafo.kdp_to_r)

Establishing a valid $Phi_{DP}$ profile for $K_{DP}$ retrieval involves despeckling (linear_despeckle), phase unfolding, and iterative retrieval of $Phi_{DP}$ form $K_{DP}$. The main workflow and its single steps is based on a publication by [Vulpiani2012]. For convenience, the entire workflow has been put together in the function wradlib.dp.process_raw_phidp_vulpiani.

Once a valid $Phi_{DP}$ profile has been established, the kdp_from_phidp functions can be used to retrieve $K_{DP}$.

Please note that so far, the functions in this module were designed to increase performance. This was mainly achieved by allowing the simultaneous application of functions over multiple array dimensions. The only requirement to apply these function is that the range dimension must be the last dimension of all input arrays.

`process_raw_phidp_vulpiani`	Establish consistent $Phi_{DP}$ profiles from raw data.
`kdp_from_phidp`	Retrieves $K_{DP}$ from $Phi_{DP}$.
`unfold_phi_vulpiani`	Alternative phase unfolding which completely relies on $K_{DP}$.
`unfold_phi`	Unfolds differential phase by adjusting values that exceeded maximum ambiguous range.
`linear_despeckle`	Remove floating pixels in between NaNs in a multi-dimensional array.
`texture`	Compute the texture of data.
`depolarization`	Compute the depolarization ration.

Georeferencing¶

Miscellaneous¶

`bin_altitude`	Calculates the height of a radar bin taking the refractivity of the atmosphere into account.
`bin_distance`	Calculates great circle distance from radar site to radar bin over spherical earth, taking the refractivity of the atmosphere into account.
`site_distance`	Calculates great circle distance from bin at certain altitude to the radar site over spherical earth, taking the refractivity of the atmosphere into account.

Polar Grid Functions¶

`spherical_to_xyz`	Transforms spherical coordinates (r, phi, theta) to cartesian coordinates (x, y, z) centered at sitecoords (aeqd).
`spherical_to_proj`	Transforms spherical coordinates (r, phi, theta) to projected coordinates centered at sitecoords in given projection.
`spherical_to_polyvert`	Generate 3-D polygon vertices directly from spherical coordinates (r, phi, theta).
`spherical_to_centroids`	Generate 3-D centroids of the radar bins from the sperical coordinates (r, phi, theta).
`centroid_to_polyvert`	Calculates the 2-D Polygon vertices necessary to form a rectangular polygon around the centroid’s coordinates.
`sweep_centroids`	Construct sweep centroids native coordinates.
`maximum_intensity_projection`	Computes the maximum intensity projection along an arbitrary cut through the ppi from polar data.

Projection Functions¶

`reproject`	Transform coordinates from a source projection to a target projection.
`create_osr`	Conveniently supports the construction of osr spatial reference objects
`proj4_to_osr`	Transform a proj4 string to an osr spatial reference object
`epsg_to_osr`	Create osr spatial reference object from EPSG number
`wkt_to_osr`	Create osr spatial reference object from WKT string
`get_default_projection`	Create a default projection object (wgs84)
`get_earth_radius`	Get the radius of the Earth (in km) for a given Spheroid model (sr) at a given position.
`get_radar_projection`	Get the native radar projection which is an azimuthal equidistant projection centered at the site using WGS84.
`get_earth_projection`	Get a default earth projection based on WGS
`get_extent`	Get the extent of 2d coordinates

Raster Functions¶

`read_gdal_values`	Read values from a gdal object.
`read_gdal_projection`	Get a projection (OSR object) from a GDAL dataset.
`read_gdal_coordinates`	Get the projected coordinates from a GDAL dataset.
`extract_raster_dataset`	Extract data, coordinates and projection information
`get_raster_extent`	Get the coordinates of the 4 corners of the raster dataset
`get_raster_elevation`	Return surface elevation corresponding to raster dataset
`reproject_raster_dataset`	Reproject/Resample given dataset according to keyword arguments
`merge_raster_datasets`	Merge rasters.
`create_raster_dataset`	Create In-Memory Raster Dataset
`set_raster_origin`	Converts Data and Coordinates Origin
`set_raster_indexing`	Sets Data and Coordinates Indexing Scheme
`set_coordinate_indexing`	Sets Coordinates Indexing Scheme
`raster_to_polyvert`	Get raster polygonal vertices from gdal dataset.

Rectangular Grid Functions¶

`get_radolan_coords`	Calculates x,y coordinates of radolan grid from lon, lat
`get_radolan_grid`	Calculates x/y coordinates of radolan grid of the German Weather Service
`xyz_to_spherical`	Returns spherical representation (r, theta, phi) of given cartesian coordinates (x, y, z) with respect to the reference altitude (asl) considering earth’s geometry (proj).
`grid_to_polyvert`	Get polygonal vertices from rectangular grid coordinates.

Satellite Functions¶

`correct_parallax`	Adjust the geo-locations of the SR pixels
`dist_from_orbit`	Returns range distances of SR bins (in meters) as seen from the orbit

Vector Functions (GDAL)¶

`get_vector_coordinates`	Function iterates over gdal ogr layer features and packs extracted vector coordinate points into nested ndarray
`get_vector_points`	Extract coordinate points from given ogr geometry as generator object
`transform_geometry`	Perform geotransformation to given destination SpatialReferenceSystem
`ogr_create_layer`	Creates OGR.Layer objects in gdal.Dataset object.
`ogr_copy_layer`	Copy OGR.Layer object.
`ogr_copy_layer_by_name`	Copy OGR.Layer object.
`ogr_add_feature`	Creates OGR.Feature objects in OGR.Layer object.
`ogr_add_geometry`	Copies single OGR.Geometry object to an OGR.Layer object.
`numpy_to_ogr`	Convert a vertex array to gdal/ogr geometry.
`ogr_to_numpy`	Backconvert a gdal/ogr geometry to a numpy vertex array.
`ogr_geocol_to_numpy`	Backconvert a gdal/ogr geometry Collection to a numpy vertex array.
`get_centroid`	Return centroid of a polygon

Xarray Functions¶

`as_xarray_dataarray`	Create Xarray DataArray from NumPy Array
`create_xarray_dataarray`	Create Xarray DataArray from Polar Radar Data
`georeference_dataset`	Georeference Dataset.

Raw Data I/O¶

Please have a look at the tutorial Supported radar data formats for an introduction on how to deal with different file formats.

Digital Elevation Model I/O¶

Provide surface/terrain elevation information from SRTM data

`download_srtm`	Download NASA SRTM elevation data Version 3 is only available with a login and a token
`get_srtm`	Get NASA SRTM elevation data

Raster and Vector I/O using GDAL¶

`open_vector`	Open vector file, return gdal.Dataset and OGR.Layer
`open_raster`	Open raster file, return gdal.Dataset
`read_safnwc`	Read MSG SAFNWC hdf5 file into a gdal georeferenced object
`gdal_create_dataset`	Creates GDAL.DataSet object.
`write_raster_dataset`	Write raster dataset to file format

HDF Data I/O¶

`read_generic_hdf5`	Reads hdf5 files according to their structure
`read_opera_hdf5`	Reads hdf5 files according to OPERA conventions
`read_gamic_hdf5`	Data reader for hdf5 files produced by the commercial GAMIC Enigma V3 MURAN software
`to_hdf5`	Quick storage of one <data> array and a <metadata> dict in an hdf5 file
`from_hdf5`	Loading data from hdf5 files that was stored by `to_hdf5`
`read_gpm`	Reads GPM files for matching with GR
`read_trmm`	Reads TRMM files for matching with GR

Read IRIS Data Format¶

Reads data from Vaisala’s IRIS data formats

IRIS (Vaisala Sigmet Interactive Radar Information System)

See M211318EN-F Programming Guide ftp://ftp.sigmet.com/outgoing/manuals/

To read from IRIS files numpy.memmap is used to get access to the data. The IRIS header (PRODUCT_HDR, INGEST_HEADER) is read in any case into dedicated OrderedDict’s. Reading sweep data can be skipped by setting loaddata=False. By default the data is decoded on the fly. Using rawdata=True the data will be kept undecoded.

`IrisRecord`	Class holding a single record from a Sigmet IRIS file.
`IrisHeaderBase`	Base Class for Iris Headers.
`IrisStructureHeader`	Iris Structure Header class.
`IrisIngestHeader`	Iris Ingest Header class.
`IrisProductHeader`	Iris Product Header class.
`IrisIngestDataHeader`	Iris Ingest Data Header class.
`IrisFileBase`	Base class for Iris Files.
`IrisFile`	IrisFile class
`IrisIngestHeaderFile`	Iris Ingest Header File class.
`IrisIngestDataFile`	Iris Ingest Data File class.
`IrisRecordFile`	Iris Record File class
`IrisRawFile`	Iris Raw File class.
`IrisProductFile`	Class for retrieving data from Sigmet IRIS Product files.
`IrisCartesianProductFile`	Class for retrieving data from Sigmet IRIS Cartesian Product files.
`read_iris`	Read Iris file and return dictionary.

Miscellaneous Data I/O¶

`write_polygon_to_text`	Writes Polygons to a Text file which can be interpreted by ESRI ArcGIS’s “Create Features from Text File (Samples)” tool.
`to_pickle`	Pickle object <obj> to file <fpath>
`from_pickle`	Return pickled object from file <fpath>
`get_radiosonde`	Download radiosonde data from internet.
`get_membership_functions`	Reads membership function parameters from wradlib-data file.

Read NetCDF¶

`read_edge_netcdf`	Data reader for netCDF files exported by the EDGE radar software
`read_generic_netcdf`	Reads netcdf files and returns a dictionary with corresponding structure.

Read RADOLAN and DX¶

Reading DX and RADOLAN data from German Weather Service

`read_dx`	Data reader for German Weather Service DX product raw radar data files.
`read_radolan_composite`	Read quantitative radar composite format of the German Weather Service
`get_radolan_filehandle`	Opens radolan file and returns file handle
`read_radolan_header`	Reads radolan ASCII header and returns it as string
`parse_dwd_composite_header`	Parses the ASCII header of a DWD quantitative composite file
`read_radolan_binary_array`	Read binary data from file given by filehandle
`decode_radolan_runlength_array`	Decodes the binary runlength coded section from DWD composite file and return decoded numpy array with correct shape
`radolan_to_xarray`	Converts RADOLAN data to xarray Dataset

Read Rainbow¶

read_rainbow Reads Rainbow files files according to their structure

Xarray powered Data I/O¶

Reads data from netcdf-based CfRadial1, CfRadial2 and hdf5-based ODIM_H5 and other hdf5-flavours (GAMIC).

Writes data to CfRadial2 and ODIM_H5 files.

This reader implementation uses

Currently there are two different approaches.

In the first approach the data is claimed using netcdf4-Dataset in a diskless non-persistent mode:

nch = nc.Dataset(filename, diskless=True, persist=False)

Further the different netcdf/hdf groups are accessed via xarray open_dataset and the NetCDF4DataStore:

xr.open_dataset(xr.backends.NetCDF4DataStore(nch), mask_and_scale=True)

For hdf5 data scaling/masking properties will be added to the datasets before decoding. For GAMIC data compound data will be read via h5py.

The data structure holds one or many [‘sweep_X’] xarray datasets, holding the sweep data. The root group xarray dataset which corresponds to the CfRadial2 root-group is available via the .root-object.

The writer implementation uses xarray for CfRadial2 output and relies on h5py for the ODIM_H5 output.

The second approach reads ODIM files (metadata) into a simple accessible structure:

vol = wradlib.io.open_odim(paths, loader='netcdf4', **kwargs)

All datafiles are accessed via the given loader (‘netcdf4’, ‘h5py’, ‘h5netcdf’). Only absolutely neccessary data is actually read in this process, eg. acquisition time and elevation, to fill the structure accordingly. All subsequent metadata retrievals are cached to further improve performance. Actual data access is realised via xarray using engine ‘netcdf4’ or ‘h5netcdf’, depending on the loader.

Since for data handling xarray is utilized all xarray features can be exploited, like lazy-loading, pandas-like indexing on N-dimensional data and vectorized mathematical operations across multiple dimensions.

Examples

See Multi File OdimH5 reader.

Warning

This implementation is considered experimental. Changes in the API should be expected.

`XRadVol`	BaseClass for xarray based RadarVolumes
`CfRadial`	Class for xarray based retrieval of CfRadial data files
`OdimH5`	Class for xarray based retrieval of ODIM_H5 data files
`to_cfradial2`	Save XRadVol/XRadVolume to CfRadial2.0 compliant file.
`to_odim`	Save XRadVol/XRadVolume to ODIM_H5/V2_2 compliant file.
`open_odim`	Open multiple ODIM files as a XRadVolume structure.
`XRadSweep`	Class for holding one radar sweep
`XRadMoment`	Class for holding one radar moment
`XRadTimeSeries`	Class for holding a timeseries of radar sweeps
`XRadVolume`	Class for holding a volume of radar sweeps
`create_xarray_dataarray`	Deprecated since version 1.5.

Interpolation¶

Interpolation allows to transfer data from one set of locations to another. This includes for example:

interpolating the data from a polar grid to a cartesian grid or irregular points
interpolating point observations to a grid or a set of irregular points
filling missing values, e.g. filling clutters

`Nearest`	Nearest-neighbour interpolation in N dimensions.
`Idw`	Inverse distance weighting interpolation in N dimensions.
`Linear`	Interface to the `scipy.interpolate.LinearNDInterpolator` class.
`OrdinaryKriging`	Interpolate using Ordinary Kriging
`ExternalDriftKriging`	ExternalDriftKriging(src, trg, cov=‘1.0 Exp(10000.)’, nnearest=12,
`RectGrid`	Interpolation on a 2d grid in arbitrary dimensions.
`RectBin`	Bin points values to regular grid cells
`QuadriArea`	Map values representing quadrilateral grid cells to another quadrilateral grid.
`interpolate`	Convenience function to use the interpolation classes in an efficient way
`interpolate_polar`	Convenience function to interpolate polar data
`cart_to_irregular_interp`	Interpolate array `values` defined by cartesian coordinate array `cartgrid` to new coordinates defined by `newgrid` using nearest neighbour, linear or cubic interpolation
`cart_to_irregular_spline`	Map array `values` defined by cartesian coordinate array `cartgrid` to new coordinates defined by `newgrid` using spline interpolation.

Data Quality¶

This module will serve two purposes:

provide routines to create simple radar data quality related fields.
provide routines to decide which radar pixel to choose based on the competing information in different quality fields.

Data is supposed to be stored in ‘aligned’ arrays. Aligned here means that all fields are structured such that in each field the data for a certain index is representative for the same physical target.

Therefore no assumptions are made on the dimensions or shape of the input fields except that they exhibit the numpy ndarray interface.

`pulse_volume`	Calculates the sampling volume of the radar beam per bin depending on range and aperture.
`beam_block_frac`	Partial beam blockage fraction.
`cum_beam_block_frac`	Cumulative beam blockage fraction along a beam.
`get_bb_ratio`	Returns the Bright Band ratio of each PR bin

Data Transformation¶

Module <trafo> transforms data e.g. from RVP-units to dBZ-values to Z-values and vice versa.

`rvp_to_dbz`	Calculates dBZ-values from DWD RVP6 values as given in DX-product files.
`decibel`	Calculates the decibel representation of the input values
`idecibel`	Calculates the inverse of input decibel values
`r_to_depth`	Computes rainfall depth (mm) from rainfall intensity (mm/h)
`kdp_to_r`	Estimating rainfall intensity directly from specific differential phase.
`si_to_kmh`	Conversion from SI wind speed units to km/hr.
`si_to_mph`	Conversion from SI wind speed units to miles/hr
`si_2_kts`	Conversion from SI wind speed units to knots
`kmh_to_si`	Conversion from km/hr to SI wind speed units
`mph_to_si`	Conversion from miles/hr to SI wind speed units
`kts_to_si`	Conversion from knots to SI wind speed units
`KuBandToS`	Class to hold coefficients for Radar Reflectivity Conversion
`SBandToKu`	Class to hold coefficients for Radar Reflectivity Conversion

Utility functions¶

Module util provides a set of useful helpers which are currently not attributable to the other modules

`from_to`	Return a list of timesteps from <tstart> to <tend> of length <tdelta>
`filter_window_polar`	Apply a filter of an approximated square window of half size fsize on a given polar image img.
`filter_window_cartesian`	Apply a filter of square window size fsize on a given cartesian image img.
`find_bbox_indices`	Find min/max-indices for NxMx2 array coords using bbox-values.
`get_raster_origin`	Return raster origin
`calculate_polynomial`	Calculate Polynomial
`derivate`	Calculates derivative of data using window of length winlen.
`despeckle`	Remove floating pixels in between NaNs in a multi-dimensional array.

Verification¶

Verification mainly refers to the comparison of radar-based precipitation estimates to ground truth.

`ErrorMetrics`	Compute quality metrics from a set of observations (`obs`) and estimates (`est`).
`PolarNeighbours`	For a set of projected point coordinates, extract the neighbouring bin values from a data set in polar coordinates.

Visualisation¶

Standard plotting and mapping procedures.

`plot_ppi`	Plots a Plan Position Indicator (PPI).
`plot_ppi_crosshair`	Plots a Crosshair for a Plan Position Indicator (PPI).
`plot_rhi`	Plots a Range Height Indicator (RHI).
`WradlibAccessor`	Dataarray Accessor for plotting radar moments
`create_cg`	Helper function to create curvelinear grid
`plot_scan_strategy`	Plot the vertical scanning strategy
`plot_plan_and_vert`	Plot 2-D plan view of `dataxy` together with vertical sections `dataxz` and `datazy`
`plot_max_plan_and_vert`	Plot according to <plot_plan_and_vert> with the maximum values along the three axes of `data`
`add_lines`	Add lines (points in the form Nx2) to axes
`add_patches`	Add patches (points in the form Nx2) to axes

Vertical Profile of Reflectivity (VPR)¶

Precipitation is 3-dimensional in space. The vertical distribution of precipitation (and thus reflectivity) is typically non-uniform. As the height of the radar beam increases with the distance from the radar location (beam elevation, earth curvature), one sweep samples from different heights. The effects of the non-uniform VPR and the different sampling heights need to be accounted for if we are interested in the precipitation near the ground or in defined heights. This module is intended to provide a set of tools to account for these effects.

The first step will normally be to reference the polar volume data in a 3-dimensional Cartesian coordinate system. The three dimensional Cartesian coordinates of the original polar volume data can be computed using wradlib.vpr.volcoords_from_polar.

Then, we can create regular 3-D grids in order to analyse the vertical profile of reflectivity or rainfall intensity. For some applications you might want to create so-called Constant Altitude Plan Position Indicators (CAPPI) in order to make radar observations at different distances from the radar more comparable. Basically, a CAPPI is simply one slice out of a 3-D volume grid. Analoguous, we will refer to the elements in a three dimensional Cartesian grid as voxels. In wradlib, you can create CAPPIS (CAPPI) and Pseudo CAPPIs (PseudoCAPPI) for different altitudes at once.

Here’s an example how a set of CAPPIs can be created from synthetic polar volume data:

import wradlib
import numpy as np

# define elevation and azimuth angles, ranges, radar site coordinates,
# projection
elevs  = np.array([0.5,1.5,2.4,3.4,4.3,5.3,6.2,7.5,8.7,10,12,14,16.7,19.5])
azims  = np.arange(0., 360., 1.)
ranges = np.arange(0., 120000., 1000.)
sitecoords = (14.924218,120.255547,500.)
proj = osr.SpatialReference()
proj.ImportFromEPSG(32651)

# create Cartesian coordinates corresponding the location of the
# polar volume bins
polxyz  = wradlib.vpr.volcoords_from_polar(sitecoords, elevs,
                                           azims, ranges, proj)  # noqa
poldata = wradlib.vpr.synthetic_polar_volume(polxyz)
# this is the shape of our polar volume
polshape = (len(elevs),len(azims),len(ranges))

# now we define the coordinates for the 3-D grid (the CAPPI layers)
x = np.linspace(polxyz[:,0].min(), polxyz[:,0].max(), 120)
y = np.linspace(polxyz[:,1].min(), polxyz[:,1].max(), 120)
z = np.arange(500.,10500.,500.)
xyz = wradlib.util.gridaspoints(x, y, z)
gridshape = (len(x), len(y), len(z))

# create an instance of the CAPPI class and
# use it to create a series of CAPPIs
gridder = wradlib.vpr.CAPPI(polxyz, xyz, maxrange=ranges.max(),
                            gridshape=gridshape, ipclass=wradlib.ipol.Idw)
gridded = np.ma.masked_invalid( gridder(poldata) ).reshape(gridshape)

# plot results
levels = np.linspace(0,100,25)
wradlib.vis.plot_max_plan_and_vert(x, y, z, gridded, levels=levels,
                                   cmap=pl.cm.viridis)

`volcoords_from_polar`	Create Cartesian coordinates for regular polar volumes
`make_3d_grid`	Generate Cartesian coordinates for a regular 3-D grid based on radar specs.
`CartesianVolume`	Create 3-D regular volume grid in Cartesian coordinates from polar data with multiple elevation angles
`CAPPI`	Create a Constant Altitude Plan Position Indicator (CAPPI)
`PseudoCAPPI`	Create a Pseudo-CAPPI Constant Altitude Plan Position Indicator (CAPPI)

Zonal Statistics¶

This module supports you in computing statistics over spatial zones. A typical application would be to compute mean areal precipitation for a catchment by using precipitation estimates from a radar grid in polar coordinates or from precipitation estimates in a Cartesian grid.

The general usage is similar to the wradlib.ipol and wradlib.adjust:

You have to create an instance of a class (derived from ZonalDataBase) by using the spatial information of your source and target objects (e.g. radar bins and catchment polygons). The Zonal Data within this object can be saved eg. as an ESRI Shapefile.

This object is then called with another class to compute zonal statistics for your target objects by calling the class instance with an array of values (one for each source object).

Typically, creating the instance of the ZonalData class will be computationally expensive, but only has to be done once (as long as the geometries do not change).

Calling the objects with actual data, however, will be very fast.

Note

Right now we only support a limited set of 2-dimensional zonal statistics. In the future, we plan to extend this to three dimensions.

`DataSource`	DataSource class for handling ogr/gdal vector data
`ZonalDataBase`	Base class for managing 2-dimensional zonal data.
`ZonalDataPoint`	ZonalData object for source points
`ZonalDataPoly`	ZonalData object for source polygons
`ZonalStatsBase`	Base class for all 2-dimensional zonal statistics.
`ZonalStatsPoly`	Compute weighted average for target polygons based on areal weights.
`ZonalStatsPoint`	Compute zonal average from all points in or close to the target polygon.
`mask_from_bbox`	Return 2-d index array based on spatial selection from a bounding box.
`get_bbox`	Return bbox dictionary that represents the extent of the points.
`grid_centers_to_vertices`	Produces array of vertices from grid’s center point coordinates.
`get_clip_mask`	Returns boolean mask of points `coords` inside polygon `clippoly`

Z-R Conversions¶

Module zr takes care of transforming reflectivity into rainfall rates and vice versa

`z_to_r`	Conversion from reflectivities to rain rates.
`r_to_z`	Calculates reflectivity from rain rates using a power law Z/R relationship Z = aR*b
`z_to_r_enhanced`	Calculates rainrates from radar reflectivities using the enhanced three-part Z-R-relationship used by the DWD (as of 2009)

Community¶

Users¶

$\omega radlib$ is intended to be a community effort, and community needs communication.

The key communication platform for $\omega radlib$ is the wradlib-users mailing list and forum. Through this forum, you can help to improve $\omega radlib$ by reporting bugs, proposing enhancements, or by contributing code snippets (in any programming language) and documentation of algorithms.

You can also ask other users and developers for help, or use your own knowledge and experience to help other users. We strongly encourage you to subscribe to this list. Check it out!

How to contribute to $\omega radlib$?¶

We need your help to enhance the capabilities of $\omega radlib$! Is there an algorithm you would consider useful? Please provide code (in any programming language) or documentation. We will it a try to integrate this algorithm in $\omega radlib$.

However, there is an even better way: Start a Pull Request!

Step 1: Fork your own $\omega radlib$ repository from the $\omega radlib$ main repo.

Step 2: Implement your changes into the forked repository. Test your code.

Step 3: Now you want to feed these changes back into the main $\omega radlib$ development branch? Start a Pull Request!

Step 4: We will review your changes. On approval, we will merge your fork back into the main $\omega radlib$ branch.

Step 5: Now everyone can benefit from your improvements.

A step-by-step tutorial for a pull request can be found here.

Finally, you are welcome to contribute examples of your own $\boldsymbol \omega radlib$ applications in our recipes section. There, you will also find a guide on how to submit your recipes to the $\omega radlib$ documentation pages.

Developer Guide¶

Setup¶

The section Installation will provide you with detailed guidance on how to install $\omega radlib$ and the required dependencies for different operating systems (MS Windows, Linux, Mac OS). $\omega radlib$ 1.7.0 has been tested with the latest Anaconda Python on linux-64. It is available via conda-forge channel on linux-64, osx-64 and MS Windows 64 versions.

As a developer, though, you should rather link into $\omega radlib$’s version control. This way, it will be easier for you to track changes and to contribute your changes to $\omega radlib$’s main respository (see next section). Just install Git, then clone the $\omega radlib$ repository to your local system by executing the following command in your shell: git clone https://github.com/wradlib/wradlib.git. Do not forget to set the PYTHONPATH to point to the corresponding directory.

Contributing to $\omega radlib$¶

Everyone can contribute to the developement of $\omega radlib$ by using the Fork and Pull model. For this purpose, you need to set up Git (see section Setup). Then see section Community for further instructions on how to create a Pull Request.

Building the docs¶

In order to build the documentation, you need to download the wradlib-docs repository. You need to satisfy a few more dependencies which are mainly related to Sphinx. These are specified in the readthedocs_environment.yml.

Once these requirements are met, you can open a console window within the wradlib-docs-repository and execute sphinx-build -v -b html source/ doc-build. This will give you the latest documentation under the wradlib-docs/doc-build directory. Simply open the index.html file to view the documentation.

Testing¶

$\omega radlib$ uses the unittest framework. New functions should come with corresponding unittests in the wradlib/wradlib/tests directory. Just have a look at available tests to get an idea. In addition, examples and docstrings are a good way to combine testing and documentation. Have a look at the wradlib-notebooks in order to get an idea on how to set these up correctly. In the docstrings, the Examples section will be tested by our testing framework. This could look like this:

def foo(a):
    """Docstring to be evaluated by doctest

    Examples
    --------
    >>> from wradlib.some_module import foo
    >>> print(foo(3))
    4
    """
    return a + 1

Continuous Integration¶

We use Travis_CI for Continuous Integration (CI). CI means, in our case, that each commit pushed to $\omega radlib$’s main repository will trigger the test suites on Travis-CI. If all tests pass successfully, a new documentation will be built on https://readthedocs.org and published on https://docs.wradlib.org. In case a new release tag is associated with a commit, a new release will be distributed via PyPI.

Bibliography¶

References

[Goudenhoofdt et al., 2009]

E. Goudenhoofdt and L. Delobbe. Evaluation of radar-gauge merging methods for quantitative precipitation estimates. Hydrology and Earth System Sciences, 13(2):195–203, 2009. doi:10.5194/hess-13-195-2009.

[Hitschfeld et al., 1954]

Walter Hitschfeld and Jack Bordan. Errors inherent in the radar measurement of rainfall at attenuating wavelengths. Journal of Meteorology, 11(1):58–67, Feb 1954. doi:10.1175/1520-0469(1954)011<0058:EIITRM>2.0.CO;2.

[Kraemer et al., 2008]

S. Kraemer and H. R. Verworn. 11th International Conference on Urban Drainage, Edinburgh, Scotland, UK, 2008 Improved C-band radar data processing for real time control of urban drainage systems. 2008. URL: https://web.sbe.hw.ac.uk/staffprofiles/bdgsa/11th_International_Conference_on_Urban_Drainage_CD/ICUD08/pdfs/105.pdf.

[Jacobi et al., 2016]

S. Jacobi and M. Heistermann. Benchmarking attenuation correction procedures for six years of single-polarised c-band weather radar observations in south-west germany. Geomat. Nat. Haz. Risk., 2016. doi:10.1080/19475705.2016.1155080.

[Gabella et al., 2002]

Marco Gabella and Riccardo Notarpietro. Ground clutter characterization and elimination in mountainous terrain. In Use of radar observations in hydrological and NWP models, 305–311. Katlenburg-Lindau, 2002. Copernicus. URL: https://porto.polito.it/1411995/.

[Vulpiani et al., 2012]

Gianfranco Vulpiani, Mario Montopoli, Luca Delli Passeri, Antonio G. Gioia, Pietro Giordano, and Frank S. Marzano. On the use of dual-polarized c-band radar for operational rainfall retrieval in mountainous areas. Journal of Applied Meteorology and Climatology, 51(2):405–425, Feb 2012. doi:10.1175/JAMC-D-10-05024.1.

[Pfaff, 2010]

T. Pfaff. Radargestuetzte Schaetzung von Niederschlagsensembles (in German). Bronstert et al. (Eds.). Operationelle Abfluss- und Hochwasservorhersage in Quellgebieten. Final Project Report, pp. 113-118., 2010. URL: http://www.rimax-hochwasser.de/fileadmin/user_uploads/RIMAX_PUB_22_0015_Abschlussbericht\ OPAQUE_final.pdf.

[Merceret et al., 2000]

Francis J. Merceret and Jennifer G. Ward. Attenuation of Weather Radar Signals Due to Wetting of the Radome by Rainwater or Incomplete Filling of the Beam Volume. Final Report, NASA Kennedy Space Center; Cocoa Beach, FL United States, Apr 09 2000. No Copyright; Unclassified; Publicly available; Unlimited. URL: https://ntrs.nasa.gov/search.jsp?R=20020043890.

[Carey et al., 2000]

Lawrence D. Carey, Steven A. Rutledge, David A. Ahijevych, and Tom D. Keenan. Correcting propagation effects in c-band polarimetric radar observations of tropical convection using differential propagation phase. Journal of Applied Meteorology, 39(9):1405–1433, Sep 2000. doi:10.1175/1520-0450(2000)039<1405:CPEICB>2.0.CO;2.

[Hubbert et al., 2009a]

J. C. Hubbert, M. Dixon, S. M. Ellis, and G. Meymaris. Weather radar ground clutter. part i: identification, modeling, and simulation. Journal of Atmospheric and Oceanic Technology, 26(7):1165–1180, 2009. URL: https://doi.org/10.1175/2009JTECHA1159.1, doi:10.1175/2009JTECHA1159.1.

[Hubbert et al., 2009b]

J. C. Hubbert, M. Dixon, and S. M. Ellis. Weather radar ground clutter. part ii: real-time identification and filtering. Journal of Atmospheric and Oceanic Technology, 26(7):1181–1197, 2009. URL: https://doi.org/10.1175/2009JTECHA1160.1, doi:10.1175/2009JTECHA1160.1.

[Kilambi et al., 2018]

Alamelu Kilambi, FrÃ©dÃ©ric Fabry, and VÃ©ronique Meunier. A simple and effective method for separating meteorological from nonmeteorological targets using dual-polarization data. Journal of Atmospheric and Oceanic Technology, 35(7):1415–1424, 2018. URL: https://doi.org/10.1175/JTECH-D-17-0175.1, doi:10.1175/JTECH-D-17-0175.1.

[Melnikov et al., 2013]

V. Melnikov and S. Matrosov. Radar measurements of the axis ratios of cloud particles. 2013. URL: https://ams.confex.com/ams/36Radar/webprogram/Manuscript/Paper228291/AxisRatio.pdf.

[Ryzhkov et al., 2017]

Alexander Ryzhkov, Sergey Y. Matrosov, Valery Melnikov, Dusan Zrnic, Pengfei Zhang, Qing Cao, Michael Knight, Clemens Simmer, and Silke Troemel. Estimation of depolarization ratio using weather radars with simultaneous transmission/reception. Journal of Applied Meteorology and Climatology, 56(7):1797–1816, 2017. URL: https://doi.org/10.1175/JAMC-D-16-0098.1, doi:10.1175/JAMC-D-16-0098.1.

[Gourley et al., 2007]

Jonathan J Gourley, Pierre Tabary, and Jacques Parent du Chatelet. A fuzzy logic algorithm for the separation of precipitating from nonprecipitating echoes using polarimetric radar observations. Journal of Atmospheric and Oceanic Technology, 24(8):1439–1451, 2007.

[Wang et al., 2009]

Yanting Wang and V. Chandrasekar. Algorithm for estimation of the specific differential phase. Journal of Atmospheric and Oceanic Technology, 26(12):2565–2578, Dec 2009. doi:10.1175/2009JTECHA1358.1.

[Doviak et al., 1993]

R.J. Doviak and D.S. Zrnić. Doppler Radar and Weather Observations. Dover Books on Engineering Series. Dover Publications, 1993. ISBN 9780486450605. URL: https://books.google.de/books?id=ispLkPX9n2UC.

[DrWeigl et al., 2004]

Elmar Dr. Weigl, Thomas Dr. Reich, Peter Lang, Andreas Wagner, Otfried Kohler, Nicole Gerlach, and MitarbeiterInnen der MeteoSolutions GmbH. Projekt RADOLAN - Routineverfahren zur Online-Aneichung der Radarniederschlagsdaten mit Hilfe von automatischen Bodenniederschlagsstationen (Ombrometer). Final Report, German Weather Service, Offenbach, Germany, 2004. in german. URL: https://www.dwd.de/DE/leistungen/radolan/radolan_info/abschlussbericht_pdf.pdf?__blob=publicationFile&v=2.

[DWD, 2009]

RADOLAN/RADVOR-OP Beschreibung des Kompositformats. 2009. in German. URL: https://www.dwd.de/DE/leistungen/radolan/radolan.html.

[Cao et al., 2013]

Qing Cao, Yang Hong, Youcun Qi, Yixin Wen, Jian Zhang, Jonathan J. Gourley, and Liang Liao. Empirical conversion of the vertical profile of reflectivity from ku-band to s-band frequency. Journal of Geophysical Research: Atmospheres, 118(4):1814–1825, 2013. URL: https://dx.doi.org/10.1002/jgrd.50138, doi:10.1002/jgrd.50138.

[Liao et al., 2009]

Liang Liao and Robert Meneghini. Validation of trmm precipitation radar through comparison of its multiyear measurements with ground-based radar. Journal of Applied Meteorology and Climatology, 48(4):804–817, 2009. URL: https://dx.doi.org/10.1175/2008JAMC1974.1, doi:10.1175/2008JAMC1974.1.

[Ryzhkov et al., 2005]

Alexander V. Ryzhkov, Scott E. Giangrande, and Terry J. Schuur. Rainfall estimation with a polarimetric prototype of wsr-88d. Journal of Applied Meteorology, 44(4):502–515, Apr 2005. doi:10.1175/JAM2213.1.

[Bringi et al., 2001]

VN Bringi and V Chandrasekar. Polarimetric Doppler weather radar: principles and applications. Cambridge University Press, 2001.

[Harrison et al., 2000]

D L Harrison, S J Driscoll, and M Kitchen. Improving precipitation estimates from weather radar using quality control and correction techniques. Meteorological Applications, 7(2):135–144, 2000. doi:10.1017/S1350482700001468.

[Jacobi et al., 2012]

S. Jacobi, M. Heistermann, and T. Pfaff. Evaluation and improvement of C-band radar attenuation correction for operational flash flood forecasting. In Proceedings of weather radar and hydrology symposium in Exeter, UK, number 351, 33–38. IAHS, 2012.

[Collier, 1996]

Christopher G. Collier. Applications of weather radar systems : a guide to uses of radar data in meteorology and hydrology. J. Wiley & sons ; Praxis publ., Chichester; New York; Brisbane [etc.]; Chichester, 1996.

[Schwaller et al., 2011]

Mathew R. Schwaller and K. Robert Morris. A ground validation network for the global precipitation measurement mission. Journal of Atmospheric and Oceanic Technology, 28(3):301–319, 2011. URL: https://dx.doi.org/10.1175/2010JTECHA1403.1, doi:10.1175/2010JTECHA1403.1.

Links

[B1]	EUMETNET OPERA. Eumetnet opera weather radar information model for implementation with the hdf5 file format. URL: https://www.eol.ucar.edu/system/files/OPERA_2008_03_WP2.1b_ODIM_H5_v2.1.pdf.

[B2]	StackOverflow. Find large number of consecutive values fulfilling condition in a numpy array. URL: https://stackoverflow.com/questions/4494404/find-large-number-of-consecutive-values-fulfilling-condition-in-a-numpy-array.

[B3]	StackOverflow. Performing a moving linear fit to 1d data in python. URL: https://stackoverflow.com/questions/7288125/performing-a-moving-linear-fit-to-1d-data-in-python.

[B4]	Wikipedia. Cross-validation (statistics). URL: https://en.wikipedia.org/wiki/Cross-validation_(statistics).

[B5]	Wikipedia. Esri grid. URL: https://en.wikipedia.org/wiki/Esri_grid.

[B6]	Wikipedia. Geotiff. URL: https://en.wikipedia.org/wiki/GeoTIFF.

Release Notes¶

Please note that $\omega radlib$ releases follow semantic versioning. API breaks will be announced via deprecation warnings. All $\omega radlib$ releases come without any warranty. Release notes might be incomplete. See here for a complete record of changes.

You can install the latest $\omega radlib$ release from PyPI via $ pip install wradlib or specific version via $ pip install wradlib==x.y.z. The recommended installation process is described in Installation.

Bleeding Edge¶

Nothing, so far.

Version 1.7.0¶

Highlights

implement generalized util.derivate function with improved NaN-handling (PR419, PR423) by @kmuehlbauer
complete rework of phidp/kdp estimation code (Vulpiani) including new keyword-parameters, handling of ndimensional arrays, using scipy.integrate.cumtrapz instead of np.cumsum (PR412, PR422) by @kmuehlbauer
new interpolators on regular grids (PR390, PR429, PR430) by @egouden and @kmuehlbauer

New features

reimplement dp.linear_despeckle as util.despeckle (PR420) by @kmuehlbauer
read RHI in ODIM reader (PR424) by @kmuehlbauer
new get_earth_projection and get_radar_projection functions (PR379) by @egouden
new convenience functions set_raster_indexing and set_coordinate_indexing (PR429) by @kmuehlbauer
implement rainrate decoder to iris reader (PR428) by @tsmsalper

Bugfixes

correct padding and nan-filling for multidimensional arrays in dp.texture (PR418) by @kmuehlbauer
introduce call_kwargs in comp.togrid (GH373) reported by @jorahu (PR425) by @kmuehlbauer

Version 1.6.2¶

re-add removed IRIS features (GH415, PR416) by @kmuehlbauer

Version 1.6.1¶

use LooseVersion to check for dependency matching (GH413, PR414) by @kmuehlbauer

Version 1.6.0¶

Highlights

improvements of georef.raster module (PR376) by @egouden
implement multi-file ODIMH5-reader/writer (PR397, PR409 and PR410) by @kmuehlbauer and @egouden
simplify zr-module, add handling of multidimensional arrays (PR408) by @kmuehlbauer
use __all__ in submodules (georef, io) to specify exported/documented functions (GH386, PR388) by @kmuehlbauer

New features

add STATUS product to Iris/Sigmet reader (PR378) by @kmuehlbauer
improvements of georef.raster module (PR376) by @egouden
add PRF and NSAMPLES to ODIM reader (PR393) by @kmuehlbauer
refactor code into assign_root-function (PR393) by @egouden
add ODIM WRAD moment (PR406) by @kmuehlbauer

Bugfixes

apply correct decoding of VEL, WIDTH and KDP IrisCartesianProductFile (PR378) by @kmuehlbauer
add missing requests dependency to CI (PR374) by @s-m-e
correct error in documentation of sweep_centroids (PR383) by @ElmerJeanpierreLopez
adapt georef.polar.sweep_centroids to only use angles in degrees (PR385) by @kmuehlbauer
work around issue, where ODIM startime == endtime (PR391) by @kmuehlbauer
improve handling of equal sized dimensions (PR393) by @kmuehlbauer
use xarray Dataset.drop_vars instead of deprecated Dataset.drop (PR398) by @kmuehlbauer
use xarray.Dataset.rename instead of rename_dims (PR402) by @kmuehlbauer
add missing +-sign in projection string (PR405) by @kmuehlbauer
fix filter_cloudtype (low cloud switch removes everything) (PR401) by @egouden
use Dataset.swap_dims instead of rename (PR406) by @kmuehlbauer

Version 1.5.0¶

Highlights

consolidation of xarray based functionality, bugfixing, feature adding
speedup zonal statistics by using /vsimem, by creation of spatial and attribute index files as well as by faster reading of attributes and properties

New features

make OdimH5 reader accept list of files and merge into one structure
make chunks available for transparently use dask with OdimH5 and CfRadial readers
make gdal3 compatible (added by @egouden)
use loaddata=’xarray’ kwargs to output data as Xarray Dataset in read_radolan_composite
CI: add Appveyor to run test-suite under Windows OS

Bugfixes

use importlib in import_optional, correct multidimensional calling in gradient_along_axis
several fixes for OdimH5 and Cf/Radial readers/writers
set destination projection to destination dataset in reproject_raster_dataset (spotted by wradlib-forum)

Version 1.4.0¶

Highlights

read sigmet/iris ingest files, redesign of sigmet reader (suggested by @aschueth)
enhance/rewrite fuzzy echo classifier (implemented with @overeem11)

New features

parametrize xarray based OdimH5-reader (suggested by @egouden)
add depolarization ratio calculation (implemented by @overeem11)
add script for test profiling (added by @egouden)

Bugfixes

remove unnecessary seek in radolan-reader (suggested by @PPazderski)
correct handling of edge cases in dp.texture processing (spotted by @overeem11)
correct decoding of DB_FLIQUID2 (sigmet-reader) (implemented by @ckaradavut)
correct handling of non-precip in 2D hmc (spotted by and fixed with @bzohidov)
fix semver handling and install process (suggested by @s-m-e)
fix import for MutableMapping (added by @zssherman)

Version 1.3.0¶

Highlights

wradlib is considered Python3 only working with Python >= 3.6
xarray-powered reader/writer for Cf/Radial 1.X and 2.0 as well as ODIM_H5
xarray-powered plotting using DataArray via xarray-DataArray Accessor

New features

creation of xarray DataArray from spherical coordinates and radar data
update test machinery to use pytest (mainly CI use)
correctly apply semver

Bugfixes

beamblockage calculation, precisely detect clear or blocked beam
catch HTTPError in test_radiosonde, graceful skip test
spherical_to_xyz better aligns with input dimensions

Version 1.2.0¶

Highlights

significantly speed up functions using interpolation classes
add classify module including 2d membershipfunctions hydrometeor classification
fix conformance, correctness and consistency issues in wradlib-docs (thanks @CAM-Gerlach)

New features

add new header token VR and U to radolan header parser
add load_vector-method to zonaldata.DataSource
enable zonaldata.ZonaldataBase to take DataSource objects as parameters
add get_radiosonde to io.misc to retrieve radiosonde data from internet
add minalt keyword argument to vpr.make_3d_grid

Bugfixes

update links, fix typos, improve CI workflow
fix bug in all adjustment classes when checking for None
show angle axis curvelinear grid again
align docstring with actual code and use sweep in iris-reader

Version 1.1.0¶

Highlights

use with-statement in rainbow-reader
fix in gpm-reader and rainbow_reader
fix issues with cg-plot in vis-module
fix in gdal/ogr exception handling
update in versioning/release procedure
automatic build of devel-docs

Version 1.0.0¶

Highlights

export notebooks into dedicated wradlib-notebooks
export doc into dedicated wradlib-docs
complete rewrite of CI-integration
complete rework of modules

Pre 1.0.0 Versions¶

Versions before 1.0.0 are available from the wradlib-old repository.

License¶

wradlib is available under the open source MIT License.