ecosys 0.0.2
ECOSYS Input file wrapper

# ECOSYS-util

This code repositry is the set of utilities for running ECOSYS including these features.

* Generate ECOSYS input files from other datasets.
* Wrap XML style input files for improving readability.
* Provides a quick viewer for ECOSYS outputs.
* Generate figures automatically after running ECOSYS model.

## Environment setup

### conda

You can set up an environment for this repository. This repository uses anaconda.

```bash
conda env create -f environment.yml
```

Some issues are remains. I have issue (`ECCODES ERROR`) with running `ecosys_util/preprocess/site_forcing_prepare_mpi_NLDAS.py` file on Windows PyCharm. In my environment, I can run it with Anaconda prompt manually. 

* [Cannot find ECCODES_DEFINITON PATH using sample grib file in Windows](https://gitmemory.com/issue/ecmwf/cfgrib/87/499399878)

## Get started

### XML

#### `ecosys_util/load_input_soil_xml.py`

This code uses the sample file, `input/runfile_sample/mesoil`. It generate `output/runscript/mesoil.xml` and `/output/runscript/mesoil_generated`.
You can see the identical between input file `input/runfile_sample/mesoil` and output file `/output/runscript/mesoil_generated`.

This code also demonstrates that `output/runscript/mesoil_generated_from_xml` is the same as `input/runfile_sample/mesoil`. That means that the XML file converts to original ECOSYS input file style well.

This module is good to run without any settings. All settings are included in this repository.

#### `ecosys_util/load_input_xml.py`

This code uses the sample file, `input/runfile_sample/runfile_yyf`. It generate `output/runscript/runfile_yyf.xml` and `/output/runscript/runfile_yyf_generated`.

#### `ecosys_util/load_weather_xml.py`


This code uses the sample file, `input/runfile_sample/me14w.csv`.

Also this code uses the input file which name is `me15w.csv` and it is generated from `ecosys_util/preprocess/write_climate_data_Ecosys.py`.

### Preprocess

This all files are located in `ecosys_util/preprocess`. This module needs environment settings because NLDAS files are large, so NLDAS files place in specific location at the first.

The `site_forcing_prepare_mpi_NLDAS.py`, `convert_npy_to_csv.py`, `write_climate_data_Ecosys.py` should be executed in the order.

Integrated version is `generate_weather_NLDAS.py`. This version use NLDAS on NAS, so users need to mount NAS and modify location for own environment. But the setting is not much.

#### `site_forcing_prepare_mpi_NLDAS.py`

Original version is developed by Zhou, Wang. This code doesn't use MPI, it can run standalone.

#### `convert_npy_to_csv.py`

Original code, which extracts data from NLDAS, is designed to produce numpy array, but csv is easier to read for researcher. This code converted npy binary to csv text file.

#### `write_climate_data_Ecosys.py`

This module generate ECOSYS weather input file.

#### `generate_weather_NLDAS.py`

This module is the integrated version of previous three files above. This module uses NLDAS files on NAS. Currently entire NLDAS dataset is updating every morning. And cropped NLDAS is provided and is generated manually with scripts.

Scripts for processing NLDAS dataset are located at `ecosys_util.server` in this repository.

### Data preparation

`ecosys_util.server` has two scripts that download NLDAS and generate cropped NLDAS.

#### `download_NLDAS.py`

This script download recent files from the server. Login needs `.urs_cookies` file, so generate this cookie file before running this script. See the official tutorial about wget download.

Taegon's BBE desktop runs this script every morning at 7:30 am. The files are downloaded in the NAS.

#### `crop_NLDAS-linux.py`

The original NLDAS dataset covers whole US. If `generate_weather_NLDAS.py` uses this database directly, it takes ~1 hours per one year dataset preparation.

This script generates cropped dataset, especially of Minnesota. This script use `cdo` command, it's easy to install on WSL. But it's complicated on Windows 10 with cygwin. This script runs manually and it also runs on Taegon's BBE desktop.

#### gSSURGO

Extract the gSSURGO soil dataset in subfield or point scales to generate input soil datasets for the Ecosys model

* [`gSSURGO_extract.py`](ecosys_util/preprocess/gSSURGO/gSSURGO_extract.py) is the main code  
* [`tailor.py`](ecosys_util/preprocess/gSSURGO/tailor.py) is to extract gSSURGO file based on outlines (fields or points)  
* [`ext.py`](ecosys_util/preprocess/gSSURGO/ext.py) is to extract soil properities of gSSURGO and then transform it into input soil datasets in the Ecosystem model  
* [`soil.py`](ecosys_util/preprocess/gSSURGO/soil.py) is the defined soil class

### Postprocess

#### `ecosys_util/report`

This module uses only for the test. This module provides web API and demonstrates that users can see the ECOSYS results in the web browser.

This module also requires a few settings. If you have ECOSYS output, you can replace the path of output folder with yours.