timepath 0.1.0
Agent based simulation framework for population models with discrete events
Requires Python: >=3.9

# Time Path

![build](https://github.com/flo-schu/timepath/actions/workflows/python-app.yml/badge.svg)

A python implementation of DEBkiss (Jager, 2018). Involving stochastic proccesses around death and offspring. With this model we try to include experimental realism into the model, by including external discrete processes that are omnipresent in scientific experimental designs but also in nature. 

**Features:**

- [X] object oriented implementation of reserve less DEB model
- [X] Simulation of life history parameters of Daphnia Magna
  - [X] Individuals
  - [X] Populations
  - [X] Offspring
  - [X] Survival
  - [X] Toxic effects of pesticides
  - [X] Effect of aging
  - [ ] Effects of environmental stressors
- [X] simulation based inference (SBI) <https://github.com/mackelab/sbi> of parameters of the stochasitc model
- [X] submit scripts for execution of large scale models on high performance clusters with SLURM scheduling engine
- [X] implementation of highly flexible feeding and filtration schemes with recurring and non-recurring patterns
- [ ] implement GUTS as a survival model on top of DEB currently a stochastic
death model is implemented with plenty of parameters, which may only be informative
with prior information.

## Installation  

When running the development version, include ./model in PYTHONPATH

### PYMC and JAX

for running bayesian inference on timeseries or alike install pymc
additionally install jax for increased performance.

On linux just pip install jaxlib; pip install jax
On windows download a prebuilt wheel on <https://whls.blob.core.windows.net/unstable/index.html> and install with `pip install <version_downloaded_wheel.whl>`
then `pip install jax==version` where version is replaced with the same version
as jaxlib.

Make sure to choose the same python version as running on your system/repository

## Folder Structure

```
timepath
│   README.md
│   requirements.txt
│
└───data
│   └───sensitivity
│   |   └───sobol
│   |   |   │   file112.txt
│   |   |   │   ...
|   |   |
|   |   └───morris
|   |
│   └───optimization
|       └───...  
|
└───model
|   | [directory for app contents]
|   └───cluster
|   |   |   ...
|   |
|   └───helpers
|   |   |   ...
|   |
|   └───objects
|   |   |   build.py (model classes and methods)
|   |   |   build_classic_deb.py (unfinished)
|   |
|   └───optimization
|   |   |   ...
|   |
|   └───sensitivity
|   |   |   ...
|   |
|   └───sims
|       |   simulation.py
|
└───docs
|
└───scripts
|   | [directory for executables for the model]
|   │   run_individuals.py
|   │   run_sim.py
|   |

```

## Sensitivity analsis

## Optimization

use <model/optimization_indy.py> to 

## Custom Simulations

It is possible to create custom simulations with this package.
Steps necessary for this are:

1. Overwrite class methods of Animals and World in order to adapt to specific needs
2. Overwrite class methods of Simulation class to adapt Simulation process.
3. It is highly recommended to write new tests for new simulation to ensure that it is numerically stable and ensure that expected model results are met

## State of the model

v01 is a good candidate for nanomodel. It works with individuals and populations
optimization functions and sensitivity analyses have been written for this
branch and tests exist as well and at least the commit log says they were all
passed.

- flowmodel, should be further pursued once nanomodel is in a good state, but
  it has no priority. It would require too much work with too little gain.

- What should be done with v01 is
  - read issues on github. Can the model be easily upgraded to v1?
  - to improve the package structure (carefully, in order not to loose commit
    history because of too many renames)
  - re-run tests
  - get a quick overview how flowmodel interacts with the basic model, to assess
    whether it is possible to continue developing v01 without loosing the
    ability to later merge flow-model (I suspect it is not so easy)
  - once everything is done, import the package in nanomodel. Here class methods
    can be overwritten without affecting the main model-development-process
  - write submodels as scripts where class methods are overwritten!! In this
    way it should be possible to maintain the model structure for future work
    on flow-model. __Think about this before__. A submodel should include all
    necessary changes and information for the model to work and where all
    assumptions stemming from the hypothesis are integrated.

## TODOs

- [ ] revise units of toxicants and volume (ml for world vs concentration mg/L?!)
- [ ] set halflife for environmental decay (8 days)
- [x] check internal balances
- [ ] calibrate on individual timeseries
- [ ] set up continous integration
  - [x] dependencies
  - [x] lint
  - [ ] black
  - [ ] tests
  - [ ] python package
- [ ] 

## Instructions

- Order of columns in events table determines execution order.
- Input of an event file always overrides the `time` configuration in `simulation`

### set up

- install python 3.9
- vscode as a texteditor is recommended but not necessary scripts can be run from the commandline, line-by-line or in a debugging environment
- if working locally or remote clone the git repository in a place where you want to start simulating.
- then install requirements.txt (there might still be some errors)
- install any missing packages with pip install <package-name>
- to test if everything works as expected run tests (very easy in vscode)
  - currently only test_ad_libitum
- add the file <config/settings.json> with the content `{"localhost":"YOURHOSTNAME"}` where you replace `YOURHOSTNAME` by the actual name of your computer. On Windows you can get it with the command `hostname` in powershell. This file makes sure the data are correctly stored whether you are working remotely or not. This File must exist on the remote and locally, but always contain the local hostname

## calibration of the model

- the model uses simulation based inference (SBI) for parameter estimation and inference on the model.
- to start out some 1000s of simulations should be generated (depending on the amount of parameters)
- for this use the script generate_sims.py
- if a High performance computation facility is available. This script is ready to be used on a cluster
- generate_sims.py draws parameters from a multiple independent distribution to get candidates that are later compared against model results

### calibrating the basic (control) organism

- the model rests on high quality calibration of parameters of the basic organism. For this about 1e6 simulations should be run and a neural network an estimator for the posterior density trained on input parameter samples and output summary statistics. A shell script that does all necessary involved steps from generating samples to training input can be executed on a SLURM scheduling
engine with (arguments denote arrays and batch size (n_sim for each job))

All relevant parameters are set in SBI Config files stored under config/parameters/sbi/

`scripts/bash/simulate_and_train.sh config/parameters/sbi/expo_control.json 1-1000 1000 1-10 1000`

in detail the script executes the following steps (these can be run locally as well)

1. execute model/generate_sims.py 
2. execute model/process_simulations.py (will generate 1 mio simulations)
3. execute model/train_network.py (twice 1 x SNLE 1x SNPE)
4. execute model/sbi_snle_sample_posterior.py will generate 10000 samples from the SNLE posterior

### experimental treatments
- I think for each 


## roadmap