Package: verif
Verif is a command-line tool that lets you verify the quality of weather forecasts for point
locations. It can also compare forecasts from different forecasting systems (that have different
models, post-processing methods, etc).The program reads files with observations and forecasts in a specific format (see “Input files”
below). The input files contain information about dates, forecast lead times, and locations such
that statistics can be aggregated across different dimensions. To ensure a fair comparison among
files, Verif will discard data points where one or more forecast systems have missing forecasts.
Since Verif is a command-line tool, it can be used in scripts to automatically create
verification figures.Verif version 1.2 has been released (see “Installation Instruction” below). We welcome suggestions
for improvements. Verif is developed by Thomas Nipen (thomasn@met.no), David Siuta, and Tim Chui.DocumentationFor more information on how to use Verif, check out the wiki athttps://github.com/WFRT/verif/wiki.FeaturesDeterministic metrics such as MAE, bias, correlation, RMSE (e.g.-mmae)Threshold-based metrics such as the false alarm rate, ETS, EDI, Yule’s Q (e.g.-mets)Probabilistic metrics such as brier score, PIT-histogram, reliability diagrams (e.g.-mbs)Special plots like Taylor diagrams (-mtaylor), quantile-quantile plots (-mqq).Plot scores as a function of date, lead time, station altitude/lat/longitude (e.g.-xdate)Show scores on maps (-typemap)Subset the data by specifying a date range and lat/lon range (-latrange58,60)Export to text (-typetext)Options to adjust font sizes, label positions, tick marks, legends, etc (-labfs14)Anomaly statistics relative to a baseline like climatology (-cclimfile.txt)Output to png, jpeg, eps, etc and specify dimensions and resolution (-fimage.png-dpi300)Installing on UbuntuPrerequisitesVerif requires a pip installation and Python 3. The python package “cartopy” is optional, but provides
a background map when verification scores are plotted on a map. To install Cartopy, with its GEOS and
PROJ4 dependencies, do the following:sudoapt-getupdatesudoapt-getinstalllibgeos-devlibproj-devsudopip3installcartopyInstalling using pipAfter this, the easiest is to install the lastest version of Verif using pip:sudopipinstallverifVerif should then be accessible by typingverifon the command-line. If you do not have
sudo-rights, then install verif as follows:pipinstallverif--userThis will create the executable~/.local/bin/verif. Add this to your PATH environment
variable if necessary (i.e addexportPATH=$PATH:~/.local/binto~/.bashrc).To upgrade to a newer version of Verif, run the following:pipinstallverif--upgradeInstalling from sourceAlternatively, to install from source, download the source code of the latest version:https://github.com/WFRT/verif/releases/. Unzip the file and navigate into the extracted folder.Then install Verif by executing the following inside the extracted folder:sudopipinstall-rrequirements.txtsudopythonsetup.pyinstallThis will create the executable/usr/local/bin/verif. Add/usr/local/binto your PATH environment
variable if necessary. If you do not have sudo privileges do:pipinstall-rrequirements.txt--userpythonsetup.pyinstall--userThis will create the executable~/.local/bin/verif. Add~/.local/binto your PATH environment
variable.Installing on Mac OSXFollow the proceedure as for Ubuntu (either installing with pip or from source). If installing from
source, then look for the line “Installing verif script to <some directory>”, as this will indicate
what folder Verif is installed into. Add the folder to your PATH environment variable if necessary.ExampleA sample dataset for testing the program is found in./examples/. There is one “raw” forecast file and
one “calibrated” forecast file (where statistical methods have been applied). For more information
about the dataset check out the wiki. Here are some example commands to test out:# Shows mean absolute error as a function of lead-timeverifexamples/raw.txtexamples/kf.txt-mmae# Shows average observed and forecasted values as a function on timeverifexamples/raw.txtexamples/kf.txt-mobsfcst-xtime# Shows equitable threat score as a function of thresholdverifexamples/raw.txtexamples/kf.txt-mets# Shows a reliability diagram for a threshold of 0 °Cverifexamples/raw.txtexamples/kf.txt-mreliability-r0# Shows Brier skill score as a function of thresholdverifexamples/raw.txtexamples/kf.txt-mbss-xthresholdCopyright and licenseCopyright © 2015-2020 UBC Weather Forecast Research Team. Verif is licensed under the 3-clause
BSD license. See LICENSE file.