dataesr/scientific-paragraphs-categorization huggingface dataset:
Validation In order to ensure that the parallel version produces results with the same accuracy as the serial version, we have performed a set of tests and validation experiments. A direct comparison between the versions can only be performed in statistical terms because FLEXPART uses Gaussiandistributed random numbers for calculating the turbulent velocities of the particles. For the parallel version we let each process independently calculate a set of random numbers, which leads to small numeric differences (arising from the random "noise") between the versions. To confirm that the only source of differences between the serial and parallel code is in the random number generation, we first observe that when the parallel executable is run using only one process, it produces results identical to the serial version.
ECMWF data retrieval ECMWF data can be comprised of analysis and/or forecast data from the operational IFS data stream or specific reanalysis projects. For operational data, the meteorological fields can currently have a maximal temporal resolution of 1 h (more frequent data are not available), a vertical resolution of 137 model levels and 0. 1 • ×0. 1 • horizontal resolution on a regular latitude-longitude grid. Other ECMWF datasets are not available at such high horizontal resolution. For example, ERA-Interim reanalysis data  (Dee et al. , 2011)  with 1 • × 1 • latitude-longitude resolution and 60 vertical levels can be retrieved 3-hourly by mixing 6-hourly analysis and 3 h forecast fields, but higher resolution is not available from the standard archive. The new Copernicus reanalysis ERA5 provides 1-hourly analysis fields with 137 model levels and a horizontal resolution of 31 km (0.
28125 • ). Notice that access to some datasets, in particular the operational forecasts, is restricted and requires specific access (https://www. ecmwf. int/ en/forecasts/accessing-forecasts, last access: 23 June 2018). However, reanalysis data (https://software. ecmwf. int/wiki/ display/WEBAPI/Available+ECMWF+Public+Datasets, last access: 23 June 2018) are publicly available. The IFS is a global model that uses spectral representation with spherical harmonics for the dynamical part and a grid-point representation on a reduced Gaussian grid for the physical part. However, FLEXPART needs the input data on a regular latitude-longitude grid, and thus IFS data have to be preprocessed. With respect to the vertical coordinate system, the data need to be on the native ECMWF model levels (η levels), which are subsequently transformed within FLEX-PART to a terrain-following vertical coordinate system.
As explained above, each ECMWF dataset has its own specific temporal and spatial resolution, and the meteorological parameters provided can be different from dataset to dataset. To produce meteorological GRIB files suitable for FLEXPART input from these different datasets, a software called flex_extract (current version 7. 0. 4) has been developed specifically for this purpose. In order to prepare the GRIB files from the ECMWF Meteorological Archival and Retrieval System (MARS; https://software. ecmwf. int/ wiki/download/attachments/45759146/mars. pdf, last access: 24 June 2018), several retrieval requests using the MARS command language and some further processing steps are needed. Since all ECMWF datasets need to be handled differently and some may not even contain all information needed for FLEXPART, flex_extract has a focus on some of the most important ones for driving FLEXPART.
These are, in particular, the reanalysis datasets ERA-Interim  (Dee et al. , 2011) , CERA-20C (the coupled climate reanalysis of the 20th century;  Laloyaux et al. , 2018)  and the latest reanalysis ERA5, as well as data from the operational IFS stream. Each file (one for each time step) prepared by flex_extract for FLEXPART consists of a set of model-level and surface data as a combination of analysis and forecast fields depending on availability. For example, certain variables such as precipitation may only be available in forecast fields, whereas other data are also contained in analysis fields; flex_extract seeks an optimum combination of such data. Note that some parameters are stored as time-accumulated fields in the ECMWF archives and flex_extract calculates the instantaneous fluxes out of them (e.
Users from other countries worldwide can self-register at the ECMWF for a public account to be able to retrieve the public datasets (i. e. , most reanalysis products); flex_extract v7. 0. 4 makes use of the WebAPI (https://software. ecmwf. int/wiki/display/WEBAPI/ ECMWF+Web+API+Home, last access: 24 June 2018) tool provided by the ECMWF to access the data from outside their systems. This tool can distinguish between public and member state users. Therefore, it is also a convenient option for member and cooperating state users who only need data older than a few days from the operational stream or reanalysis data. A full-access account to ECMWF servers is no longer needed in this case. The flex_extract software v7. 0. 4 is a set of Python routines combined with a Fortran program for faster computation of grid transformations and vertical velocity calculation.
A Python 2. 7 interpreter with several common modules, such as NumPy and date-time, are required and usually included in the Anaconda distribution (https://www. anaconda. com/download/, last access: 25 June 2018). Additionally, a Fortran compiler, the ECMWF WebAPI tool, the GRIB-API or ecCodes module, and the Emoslib interpolation library have to be available. Note that the GRIB-API (or ecCodes) module has to be available for Python as well as for Fortran. Installation instructions can be found at ECMWF websites directly or in the Software Installation Plan for flex_extract. Knowledge of Python, although helpful, is not necessary for using the retrieval scripts. A certain knowledge of the ECMWF dataset to be retrieved is useful to understand the composition of retrievals, but many basic examples of CONTROL files are provided in the flex_extract distribution.
These CONTROL files determine the key parameters for the flex_extract MARS retrievals and can be adapted to change domain as well as spatial and temporal resolution. Even for these few parameters the user should check for availability upfront. For example, ERA-Interim data have a maximum grid resolution of 0. 75 • × 0. 75 • and 6-hourly temporal resolution for the public dataset. The flex_extract v7. 0. 4 software is included in the FLEXPART v10. 4 file tree under the directory preprocess (see Table  1 ). It can also be downloaded from https://flexpart. eu/ (last access: 30 October 2019) (https: //flexpart. eu/downloads/62, last access: 30 October 2019) as a tarball. For more details the reader is referred to the flex_extract v7. 0. 4 user documentation (e. g. , the Software Installation Plan -SIP.
NCEP data retrieval Meteorological data from the NCEP GFS are freely available, easily accessible and ingested by FLEXPART on pressure levels, unlike ECMWF data. These pressure-level data have lower resolution than model-level data but offer the advantage of great consistency between different datasets. Therefore, preprocessing NCEP data is much simpler than ECMWF data and limited to precipitation data, which are available only in forecast fields. Both operational analysis data and several reanalysis datasets are available. Notice that NCEP also provides forecast data for free, which are not available from the ECMWF even for member state users except for national meteorological services or users with a special contract. The data retrieval from NCEP is described in a wiki page on the FLEXPART website (https://flexpart.
eu/wiki/FpInputMetGfs, last access: 8 July 2018), where a script for downloading NCEP data can also be found. Operational GFS data can be downloaded by simple FTP or wget from a rolling archive of the meteorological forecast and analysis data (http://www. ftp. ncep. noaa. gov/data/nccf/com/gfs/prod/, last access: 8 July 2018) under the catalog gfs. YYYYMMDDHH, which contains fields in GRIB-2 format. Six-hourly NCEP FNL (Final) Operational Model Global Tropospheric Analyses (http://rda. ucar. edu/ datasets/ds083. 2/, last access: 30 October 2019) are available in near-real time since July 1999. These data are similar to the operational analyses, but NCEP also ingests lateincoming observation data for their production. Archived re-analysis datasets are also available from NCEP, e. g. , the Climate Forecast System Reanalysis (CFSR) Selected Hourly Time-Series Products (http://rda.
Finally, all four variables are written out to the unformatted output file. This format replaces the compression used up to version 7 (the smallest of a full dump and a simple sparse matrix format), saving up to 60 % of disk space. The sparse matrix data can be read, for example, with the functions readgrid. f (Fortran) and flex_read. m (MATLAB) described in Sect. 6. 4 6. 3 NetCDF output FLEXPART v10. 4 can also support output in NetCDF format if the NetCDF libraries are available. To activate NetCDF support, append ncf=yes to the make command. If FLEXPART is compiled and linked to the NetCDF libraries, output files in NetCDF format can be produced by adding 8 to the IOUT parameter in the input file COMMAND, e. g. , IOUT=9 corresponds to IOUT=1 with the standard binary output; see Table  11  and Sect.
Post-processing routines For the NetCDF output of FLEXPART, standard visualization tools, for example Panoply, can be used. For the sparse matrix binary output, several post-processing routines (MAT-LAB, Fortran, R, Python and IDL) have been developed in order to assist in the usage and analysis of these data. A number of post-processing tools are available online (https: //flexpart. eu/wiki/FpOutput, last access: 16 August 2018). Note that some of these tools require reading a text string containing the model version. Since the length of this string changed in FLEXPART v9. 2, the post-processing routines now require the allocation of a longer string. Table  12. Physical units of the input (in file RELEASES) and output data for forward (files grid_conc_date_nnn) and backward (files grid_time_date_nnn) runs for the various settings of the unit switches ind_source and ind_receptor (for both switches, 1 refers to mass units, 2 to mass mixing ratio units).
Routines are also available to plot trajectories on a map from the file trajectories. txt and to plot time series of concentrations (or mixing ratios) from the file receptor_conc (or receptor_pptv). Several Python tools are available for reading FLEXPART data from release 8. 0 and above. The module reflexible, available from the FLEXPART website and also at https: //github. com/spectraphilic/reflexible (last access: 6 August 2018), enables the user to easily read and access the header and grid output data of the FLEXPART model runs. It provides a simple tool that facilitates consistent reading of both the original sparse matrix output files and the NetCDF output. Some basic plotting functionality is provided to quickly assess and validate runs or to look at the input parameters. An alternative Python tool is Quicklook that can be also downloaded from the https://flexpart.
eu website. All examples are variations of a default example case, which uses the settings in the user input files as distributed with the FLEXPART v10. 4 code package. These default input files are located in the directory options (Sect. 5) and are consistent with the default meteorological data retrieved from the ECMWF by the flex_extract package (Appendices A5 and B1). An AVAILABLE file fitting with these input data is also distributed with FLEXPART. These default settings are in detail in Appendix B2.
This is done by the bash script gen_options_all. sh. For instance, the example "bwd" is generated by changing the line containing the parameter LDIRECT to -1 in the file COMMAND. After the input data files are generated, all examples can be executed interactively from the command line. Alternatively, the script gen_batch_jobs_cl. sh generates a batch script for each case (to be run from the command line or using a workload manager such as SLURM). This procedure automates the sample output generation. Once the output files are created, they can be read using the tools in the directory postprocess. They can be plotted and analyzed with, e. g. , the reading routines described in Sect. 6. 4. In addition, some testing capabilities have been added. These are presented in Appendix B.
Final remarks, outlook and future code development In this paper, we have described the Lagrangian particle dispersion model FLEXPART v10. 4; 2 decades ago, the model code was developed mainly by one person, with specific code input from a few other researchers. At that time, no specific measures were needed to ensure code consistency, track code changes or identify coding bugs. However, as the number of FLEXPART users has grown substantially in recent years, more and more people have started to develop the code, contributed code snippets, and reported or identified bugs. The resulting code changes range from the adaptation of more modern coding standards, parallelization and efficiency enhancements, the improvement of the model functionality, and As FLEXPART is developed further, updates will continue to be made available on the FLEXPART website at https://flexpart.
sudo apt-get install cmake Python is not required for FLEXPART itself but is necessary for some preprocessing and post-processing tools, in particular flex_extract for retrieving ECMWF wind fields. Git is recommended to access the code repositories. An editor (e. g. , vim) is usually also necessary. gunzip grib_api-X. X. X. tar. gz tar -xf grib_api-X. X. X. tar. /configure [--prefix=grib_api_dir]  [--with_jasper=<jasper installation path>] make make check make install If you have no root privileges in your system, give the full path of grib_api_dir to the prefix option. If jasper is in a nonstandard location, is has to be passed to the grib_api configuration script. Please note that GRIB-API is no longer maintained. The primary GRIB encoding-decoding package used at the ECMWF is currently ecCodes.
Any new features for the handling of GRIB files will only be developed in ec-Codes. However, for FLEXPART v10. 4 grib_api is sufficient. We keep the grib_api instructions for backward consistency. For future versions, ensure that the path /usr/local/lib/ is in the environment variable $PATH; otherwise, ecCodes may not find it. Obtain and unpack ecCodes. curl https://software. ecmwf. int/wiki/ download/attachments/45757960/ eccodes-2. 7. 3-Source. tar. gz \ --output eccodes-2. 7. 3-Source. tar. gz tar -xvf eccodes-2. 7. 3-Source. tar. gz The ecCodes requires CMake. The installation procedure is described on the ECMWF ecCodes web page.
A5 Installing flex_extract A short description of the installation steps for this software is given for the public user mode (other modes are described in the flex_extract documentation). For this mode, the user does not need to be a member state user (https://www. ecmwf. int/en/about/who-we-are/member-states, last access: 13 October 2018) but can simply register at the ECMWF website. For the other operating modes and a more detailed explanation, see the README. md file of the python directory in the flex_extract distribution or the documentation files SIP. pdf and SUT_ondemand. pdf. First of all, the user should register at the ECMWF website (https://www. ecmwf. int/en/forecasts/accessing-forecasts/ order-historical-datasets, last access: 13 October 2018). To access public datasets each dataset license has to be accepted separately before the account can be used for retrieval of these data.
This can be done at the following website: https://software. ecmwf. int/wiki/display/WEBAPI/ Available+ECMWF+Public+Datasets (last access: 13 October 2018). A5. 1 System preparation for flex_extract flex_extract requires a Python environment and a Fortran compiler. See Sect. A1 for installation instructions. To prepare the environment for the flex_extract installation, it is advisable to consider the official documentation and information from the ECMWF websites. We recommend the following steps. It is important to use the same compiler and compiler version for all libraries and the Fortran program CONVERT2.
A5. 2 Building flex_extract To install flex_extract a script install. py was prepared. The user can find it in the python directory of the flex_extract distribution. The public user mode requires a local installation of flex_extract. Hence, we recommend adapting the paths to ecCodes, Emoslib or grib_api in one of the prepared makefiles, such as Makefile. local. gfortran, which can be found in the src directory. If a different compiler is used, this must also be adapted in the makefile. Then the installation script can be called as follows. /install. py --target=local --makefile=Makefile. local. gfortran With this setting flex_extract is installed within the current flex_extract directory. To install it in a different place, e. g. within a FLEXPART distribution, the user can set the path with the parameter flexpart_root_scripts.
A5. 3 Running flex_extract flex_extract is controlled by providing CONTROL files that contain a list of parameter settings. These parameters are described in detail in the Software User Tutorial (SUT. pdf) in the docs directory. The CONTROL files specify which ECMWF dataset is to be retrieved, the time and spatial resolution, the format of the GRIB file, and other options. In the Python directory are some example CONTROL files for the different datasets and access modes. They can be used as templates. CONTROL files with a. public ending are usable for the public access mode. The main difference is the parameter dataset, which explicitly specifies the public datasets. Note that not all meteorological fields, times and parameters were archived in the public datasets. This is already considered in the public CONTROL files.
To run flex_extract, the main program submit. py must be called. It retrieves the ECMWF data and generates the FLEXPART input files. To show all possible parameter options one can use the -h option. The script must be called www. geosci-model-dev. net/12/4955/2019/ Geosci. Model Dev. , 12, 4955-4997, 2019 from the python directory of the flex_extract distribution. From the -h output it is clear that most parameters have default values or were already set via a CONTROL file parameter, except for the date. To retrieve just one day, one only needs to provide the start date. The rest will be done by flex_extract. This leads to the following script call for an arbitrary date. /submit. py --controlfile=CONTROL_EI. public \ --start_date=20120101 \ --public=1 The program now displays each MARS request and some messages for the preparation of the FLEXPART input files.
B1 Meteorological input for the examples Appendix A describes how to build the flex_extract version included in the source code. Here, we describe the settings to produce the meteorological input data required for running the default (Sect. B2) and derived (Sect. B3) cases. The instructions are for ECMWF ERA5 reanalysis, which is a publicly available dataset (https://confluence. ecmwf. int/display/WEBAPI/ Access+ECMWF+Public+Datasets, last access: 30 October 2019). Therefore, the data can be obtained via ecmwfapi and no special access rights to the ECMWF are needed. However, in order to retrieve the data the user needs to register, obtain a personal Secure Shell (SSH) key and properly configure the file. ecmwfapirc. The execution of the retrieval requires the Python packages ecmwfapi (for access) and grib_api or eccodes (for processing).
B3 Generating variations of the default example To demonstrate more functionalities, a set of shell scripts generating different FLEXPART setups are provided in $flexhome/tests/examples. The script set_default_example. sh takes the content of the options directory and pathnames file from Sect. B2 as a basis, and then gen_options_all. sh creates new options_suffix directories for all of the cases described in Table  14. Here, the suffix corresponds to the example name as given in column 2 in Table  14. Finally, the script gen_pathnames. sh generates corresponding pathnames_suffix files pointing to all the options_suffix directories. With this, all example cases in Table  14  are ready to run. the existing Emanuel-based convection parameterization, a convection parameterization based on the Tiedtke scheme, which is identical to the one implemented in COSMO itself, was introduced.
A possibility for offline nesting of a FLEXPART-COSMO run into a FLEXPART-ECMWF run for backward simulations was developed that only requires minor modifications of the FLEXPART-ECMWF version and allows particles to leave the limited COSMO domain. The OpenMP shared memory parallelization to the model allows for asynchronous reading of input data. The code is available on request from dominik. brunner@empa. ch and stephan. henne@empa. ch.
For this, download zlib (version 1. 2. 8) from the zlib website (https: //www. zlib. net/, last access: 30 October 2019) and install it. Download the latest stable version of NetCDF-C from the Unidata website (https://www. unidata. ucar. edu/downloads/ netcdf/, last access: 30 October 2019) and install it. This mirrors https://git. nilu. no/flexpart/flexpart. Additional mirrors exist, e. g. , at Bitbucket (https://bitbucket. org/ flexpart/flexpart, last access 30 October 2019) and GitHub (https://github. com/flexpart/flexpart, last access 30 October 2019.
Acknowledgements. The work was performed at the  Nordic Center of Excellence eSTICC. Pirmin Kaufmann  and  Martin Schraner  (MeteoSwiss) are acknowledged for code reformatting from fixed Fortran  77  to free Fortran 90 format. We thank  Mariëlle Mulder  for comments on an early version of this paper. The resources for the numerical simulations were provided by  UNINETT Sigma2  (the  National Infrastructure for High Performance Computing and Data Storage in Norway)  under projects  NN9419K  and  NS9419K. Input wind fields were provided by the  ECMWF. Financial support. This research has been supported by  NordForsk  (the  Nordic Center of Excellence eSTICC , grant no. 57001 ), the  European Research Council  (project  COMTESSA , grant no. 670462 ), and the CTBTO (Enhancements of the FLEXPART software on a call-off basis).
Code and data availability. The code described in this work is archived as flexpart10. 4. tar at https://doi. org/10. 5281/zenodo. 3542278 (Pisso et al. , 2019). FLEXPART downloads are available at https://www. flexpart. eu/ downloads (last access: 22 November 2019) and FLEXPART releases are available at https://git. nilu. no/flexpart/flexpart/-/releases (last access: 22 November 2019). The working git repository for this version 10. 4 (branches master, dev and release) can be accessed at https://git. nilu. no/flexpart/flexpart (last access: 22 November 2019) as well as the mirrors https://www. flexpart. eu/gitmob/flexpart (last access: 22 November 2019), https://github. com/flexpart/flexpart (last access:.
For this, download zlib (version 1. 2. 8) from the zlib website (https: //www. zlib. net/, last access: 30 October 2019) and install it. Download the latest stable version of NetCDF-C from the Unidata website (https://www. unidata. ucar. edu/downloads/ netcdf/, last access: 30 October 2019) and install it. This mirrors https://git. nilu. no/flexpart/flexpart. Additional mirrors exist, e. g. , at Bitbucket (https://bitbucket. org/ flexpart/flexpart, last access 30 October 2019) and GitHub (https://github. com/flexpart/flexpart, last access 30 October 2019 tar -xzvf zlib-1. 2. 8. tar. gz git clone https://www. flexpart. eu/gitmob/ cd zlib-1. 2. 8/. /configure [--prefix=<installation path>] flexpart make make install Download HDF5 from the HDF Group website (https: //www. hdfgroup. org/HDF5/release/obtainsrc.