remade 21.6.11
REMADE — REMADE Estimates (Meta)genomic Ancient Damage Efficiently
License: MIT
Requires Python: >=3.8,<4.0
Keywords: remade,metadamage,metagenomics,ancient damage,DNA damage

# REMADE — REMADE Estimates (Meta)genomic Ancient Damage Efficiently

![PyPI](https://img.shields.io/pypi/v/remade)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/remade)
![PyPI - License](https://img.shields.io/pypi/l/remade)
![GitHub last commit](https://img.shields.io/github/last-commit/christianmichelsen/remade)

---

#### Work in progress. Please contact christianmichelsen@gmail.com for further information.

---


Personal recommendations for this project:

<!-- - Python Version: [Pyenv](https://github.com/pyenv/pyenv) -->
<!-- - Virtual Environment:  [Virtualenv](https://github.com/pyenv/pyenv-virtualenv) -->
- Package management and virtual environment: [Poetry](https://python-poetry.org/)

This project requires a recent Python version (>=3.8) installed.

## <a name="install"></a> Installation:

With Poetry:
```console
$ poetry add remade
```
or, if you prefer regular pip:
```console
$ pip install remade
```


## <a name="dependencies"></a> Dependencies:

`remade` depends on having ngslca (metadamage) installed somewhere on your system.
Personally, I have it installed here: `/Users/christianmichelsen/work/thorfinn_metaLCA/metadamage`.

---

## <a name="cli"></a> REMADE CLI:

For help on the CLI interface, `remade` provides a `help` function:

```console
$ remade --help
```

Example for fitting a single file using `remade`:
```console
$ remade compute raw_data/subs.original.bam \
                 --names raw_data/names.dmp.gz \
                 --nodes raw_data/nodes.dmp.gz \
                 --acc2tax raw_data/combined_taxid_accssionNO_20200425.gz \
                 --ngslca-command /Users/christianmichelsen/work/thorfinn_metaLCA/metadamage
```

## <a name="options"></a> `remade compute` CLI options:

The `remade compute` CLI first takes `alignment_files` as an argument. This argument is either a single (or multiple) alignment-files or a directory containing alignment-file(s). In addition, `remade compute` also takes the following options.

- Input files:
  - `--names`: Path to the (NCBI) `names.dmp.gz`.
  - `--nodes`: Path to the (NCBI) `nodes.dmp.gz`.
  - `--acc2tax`: Path to the (NCBI) `acc2tax.gz`.

- ngsLCA parameters:
  - `--ngsLCA-command`: The command needed to run the ngsLCA-program.
  - `--simscorelow`: Normalized edit distance (read to reference similarity) minimum. Number between 0-1 (default is 0.95).
  - `--simscorehigh`: Normalized edit distance (read to reference similarity) maximum. Number between 0-1 (default is 1.0).
  - `--editdistmin`: Minimum edit distance (read to reference similarity). Number between 0-10 (default is 0).
  - `--editdistmax`: Maximum edit distance (read to reference similarity). Number between 0-10 (default is 10).
  - `--minmapq`: Minimum mapping quality.
  - `--max-position`: Maximum position in the sequence to include. Default is +/- 15 (forward/reverse).
  - `--lca-rank`: The LCA rank used in ngsLCA. Can be either `family`, `genus`, `species` or `""` (everything). Default is `""`.

- General parameters:
  - `--storage_path`: Path where the generated output files and folders are stored.
  - `--max-cores`: The maximum number of cores to use in the ancient damage estimation.

- Boolean flags (does not take any values, only the flag). Default is false.
  - `--bayesian`: Include a fully Bayesian model (probably better, but also _a lot_ slower, about a factor of 100).
  - `--forced`: Ignore previously generated results.

- Other:
  - `--help`: Print the help documentation and exit.
  - `--version`: Print the current version of the program and exit.


Example for fitting a single file using `remade`:
```console
$ remade compute raw_data/subs.original.bam \
                 --names raw_data/names.dmp.gz \
                 --nodes raw_data/nodes.dmp.gz \
                 --acc2tax raw_data/combined_taxid_accssionNO_20200425.gz \
                 --ngslca-command /Users/christianmichelsen/work/thorfinn_metaLCA/metadamage
```


<!-- [tex-image-link-generator](https://tex-image-link-generator.herokuapp.com/) -->
<!-- https://gist.github.com/a-rodin/fef3f543412d6e1ec5b6cf55bf197d7b -->

---

## <a name="setup"></a> Setup Local Python Environment:

Make sure you have a decent Python version (>=3.8) installed.
Personally, I recommend using Poetry to manage your virtual environments and to locate them inside the project:

```console
$ poetry config virtualenvs.in-project true
```


We now use Poetry to setup a new project which uses REMADE:
```console
$ poetry new remade-poetry
$ cd remade-poetry
```

We now have a working local, virtual Python environment where the packages are managed by Poetry, so we can now add metadamamage to our project:
```console
$ poetry add remade
```

Now install and activate the new environment:
```console
$ poetry install
$ source $(poetry env info --path)/bin/activate
```

Now if you just write:

```console
$ remade --help
```

you should see the following:

```console
$ remade compute
Usage: remade [OPTIONS] COMMAND [ARGS]...

  REMADE — REMADE Estimates (Meta)genomic Ancient Damage
  Efficiently.

  ...
```

which shows that it is working and installed.

Next time that you are going to use REMADE (from a new terminal window) remember to run `source $(poetry env info --path)/bin/activate` again to activate the virtual environment.

---


## <a name="dash"></a> Dashboard:

To make use of the new, interactive dashboard, run the following command (after having [computed](#cli) the fit results):

```console
$ remade dashboard
```

And then open a browser and go to `127.0.0.1:8050` (if it did not open automatically). For more information, use:
```console
$ remade dashboard --help
```

## <a name="dash-server"></a> Dashboard on a server:

If you are running remade on a server and want to use the dashboard locally, you can setup a SSH tunnel. First, on the server, run `remade dashboard` with the relevant options (and keep it open, e.g. with [TMUX](https://github.com/tmux/tmux/wiki)). Afterwards, on your local machine, run:

```console
$ ssh -L 8050:127.0.0.1:8050 -N user@remote
```
Now you can open a browser and go to [`http://0.0.0.0:8050`](http://0.0.0.0:8050).

In case you're connecting through a jump host, you can use the the `-J` option:
```console
$ ssh -L 8050:127.0.0.1:8050 -N -J user@jumphost user@remote
```

For an easier method, you can setup your ssh config (usually at `~/.ssh/config`) in the following way:

```ssh-config
Host jumphost
    User your-jumphost-username-here
    HostName your-jumphost-address-here

Host remote

Host dashboard
    Port 22
    LocalForward 8050 localhost:8050
    RemoteCommand echo "Connecting to dashboard ... CTRL+C to terminate"; sleep infinity

Host remote dashboard
    ProxyJump jumphost
    User your-remote-username-here
    HostName your-remote-address-here
```

Now if you simply run the following on your own computer (in a new terminal session):

```console
$ ssh dashboard
```
you can open open a browser and go to [`http://0.0.0.0:8050`](http://0.0.0.0:8050).

<!-- ssh -L 8050:127.0.0.1:8050 -N -J willerslev mnv794@wonton-snm -->
<!-- ssh -L 8050:127.0.0.1:8050 -N -J mnv794@ssh-snm-willerslev.science.ku.dk mnv794@wonton-snm -->

<!-- ssh -L 8050:127.0.0.1:8050 -N hep -->



---

## <a name="update"></a> Update:

With Poetry:
```console
$ poetry update remade
```
or, if you prefer regular pip:
```console
$ pip install remade --upgrade
```

---

## <a name="dev_branch"></a> Development Branch:

You can also use a newer version directly from Github:
```console
$ poetry add git+https://github.com/ChristianMichelsen/REMADE.git
```
or a specific branch (named `BRANCH`):
```console
$ poetry add git+https://github.com/ChristianMichelsen/REMADE.git#BRANCH
```

---

## Conda:

If you prefer using Conda, you can also install `remade` (via pip). First create a folder:
```console
$ mkdir remade-conda
$ cd remade-conda
```

To install `remade`:
```console
$ wget https://raw.githubusercontent.com/ChristianMichelsen/REMADE/main/environment.yaml
$ conda env create -f environment.yaml
```

To update it to a new, released version of `remade`:
```console
$ wget https://raw.githubusercontent.com/ChristianMichelsen/REMADE/main/environment.yaml
$ conda env update --file environment.yaml
```

Finally remember to activate the environment:
```console
$ conda activate remade
```

<!-- poetry add git+https://github.com/christianmichelsen/remade.git -->
<!-- poetry add git+https://github.com/christianmichelsen/remade.git#develop -->



## <a name="contribute"></a> Contribute:

To contribute, clone the entire project and install with `poetry`:
```console
$ git clone https://github.com/ChristianMichelsen/REMADE.git remade-git
$ poetry install
```