primal-page 1.12.0
Tooling and schemas for the primerschemes index
Requires Python: <3.14,>=3.10

# Primal-Page

[![CI](https://github.com/ChrisgKent/primal-page/actions/workflows/pytest.yml/badge.svg)](https://github.com/ChrisgKent/primal-page/actions/workflows/pytest.yml)


Tooling and schemas for the primerschemes index

## Contents

[Installation](https://github.com/ChrisgKent/primal-page?tab=readme-ov-file#installation)

[Quick Start](https://github.com/ChrisgKent/primal-page?tab=readme-ov-file#quick-start)

[Schemas](https://github.com/ChrisgKent/primal-page?tab=readme-ov-file#new-schemas)

[CLI](https://github.com/ChrisgKent/primal-page?tab=readme-ov-file#cli)

# Installation

```
pip install primal-page
```

# Quick Start

Each version of a primerscheme has three parts; `{schemename}/{ampliconsize}/{version}`, which when combined these form the schemes unique identifier.

For a scheme to be added to the repo it requires three essential files. 
- `primer.bed`: Contains the primer information.   
- `reference.fasta`: Contains the reference genomes.
- `info.json`: Contains key metadata for the scheme.

The `primal-page create` command will generates the `info.json` and parses a bed file and a fasta file into `primer.bed` and `reference.fasta`

Additional files are copied into the schemes work directory. 

## Example
### Adding a minimal scheme

For a simple scheme, only fasta file (--reference) and a bed file (--primerbed) are required. Alongside metadata of --schemename, --algorithmversion, --species, --authors.

```
primal-page create \
    --schemename example-scheme \
    --algorithmversion ps:100 \
    --ampliconsize 500 \
    --species 123  \
    --schemeversion v1.0.0 \
    --primerbed ~/'minimal-scheme/minimal.bed' \
    --reference '~/minimal-scheme/ref.fasta' \
    --authors "me" \
    --authors "you" /
    '~/minimal-scheme' 
```

### Adding a custom scheme to quick-lab/primerschemes


> create a local fork of https://github.com/quick-lab/primerschemes

> Add the files to the forks primerschemes folder
```
primal-page create primal-page create ... --output ~/primerschemes/primerschemes
```

> Create a pull request 


# New Schemas

## info.json

This is the main metadata file for each primerscheme.

- `ampliconsize`: PositiveInt
- `schemeversion`: Following format listed below
- `schemename`: Following format listed below
- `primer_bed_md5`: MD5 hash of the schemes primer.bed file
- `reference_fasta_md5`: MD5 hash of the schemes reference.fasta file
- `status`: 
    - `withdrawn`: Removed due to major issue
    - `deprecated`: Newer scheme is recommended
    - `autogenerated`: Scheme has been autogenerated using species-agnostic pipelines
    - `draft`: Scheme has been inspected _in silico_
    - `tested`: Scheme has been tested in the laboratory
    - `validated`: Scheme has been validated and/or published
- `citations`: How the scheme should be cited if used. DOI links are recommended, however, tweets/blogs are all allowed
- `authors`: The person or organization who generated the scheme. It is recommended that only corresponding/primary authors are included, with all other contributors recognized in the `citations` field
- `algorithmversion`: The algorithm and the version used to generate the scheme
- `species`: A list of organisms targeted by this scheme. NCBI TaxIds are recommend
- `license`: The name of the license the primerscheme is offered under
- `primerclass`: The class of scheme. Only `primerscheme` at the moment
- `infoschema`: The version of the info.json 
- `articbedversion`: The version of the primer.bed (See below)
- `description`: A free text description to describe the primerscheme
- `derivedfrom`: To show if this scheme has been based on another primerscheme. 
- `collections`: A collection of vocabulary to provide filtering/grouping of schemes.
    - `ARTIC`: Developed with the ARTIC network
    - `MODJADJI`: Developed with MODJADJI
    - `QUICK-LAB`: Developed with QUICK-LAB
    - `COMMUNITY`: Developed by the COMMUNITY
    - `WASTE-WATER`: Scheme capable of recovering genomes from high Ct samples (~30) samples, like wastewater. Typically 400bp schemes
    - `CLINICAL-ISOLATES`: Scheme capable of recovering genomes from medium Ct samples (~25) samples.  Typically ~1000bp schemes
    - `WHOLE-GENOME`: Scheme that can theoretically recover a full genome
    - `PANEL`: Scheme that can recover sections of a target genome
    - `MULTI-TARGET`: Scheme that contains more than one target

## PrimerNames Versions

Expected primernames (col3) in the primer.bed file

### `V1`

This is the first standard for primernames. It follows the general pattern of `{scheme-name|uuid}_{amplicon-number}_{LEFT|RIGHT}` and an optional `{_alt}` to denote spike in primers.

Regex:

 ```V1_PRIMERNAME = r"^[a-zA-Z0-9\-]+_[0-9]+_(LEFT|RIGHT)(_ALT[0-9]*|_alt[0-9]*)*$"```

Example:
```
SARS-CoV-2_10_LEFT
SARS-CoV-2_10_LEFT_alt1
```


### `V2`

This follows the pattern of `{scheme-name|uuid}_{amplicon-number}_{LEFT|RIGHT}_{primer-number}`. 
- This now enforces that splitting on `_` will produce a consistent length, allowing the safe indexing of attributes. 
- `primer-number` is not enforced to be continuous. Therefore, the `_0` numbered primer should not be thought of as the `original` and all other numbers as `alts`.


Example:
```
SARS-CoV-2_10_LEFT_0
SARS-CoV-2_10_LEFT_1
```

Regex:

```V2_PRIMERNAME = r"^[a-zA-Z0-9\-]+_[0-9]+_(LEFT|RIGHT)_[0-9]+$"```


## ARTIC-primer.bed Versions

These are rough guidelines for the format of the primer.bed file. The general format is based on the [.bed file](https://en.wikipedia.org/wiki/BED_(file_format)) and maintains compatibility with other tools.

colnames and indexes:
- `0 - chrom`: The name of the reference genome the primers are indexed to
- `1 - chromStart`: 0-based inclusive start coordinate
- `2 - chromEnd`: 0-based non-inclusive end coordinate
- `3 - primer-name`: Name of each primer
- `4 - pool`: The pool each primer should be added into. 1 based.
- `5 - strand`: Either `+` (forward) or `-` (reverse) primer
- `6 - primer-sequence`: The 5'-3' sequence of the primer



### V1
> Depreciated 

This was the original 6-col bedfile used very early in PrimalSchemes development, which excluded primer-sequence. 

### V2
> Legacy

This uses the 7 columns described above, alongside `V1:primernames`.
- Single chrom (reference) is expected
- No header lines

### V3
> Current

This uses the 7 columns described above, alongside `V2:primernames`.
- Multiple chrom (references)
- Circular primers are allowed. The start of x_LEFT can be greater than the end of x_RIGHT
- Header lines are used. Denoted with the `#` character 

 
## Scheme Version

In the form of `v{Major}.{Minor}.{Patch}`
- Major: New scheme being generated with different input params
- Minor: Change to primers. Either additional / removal of primers
- Patch: No change to primers. Often used for rebalancing or change in formatting


Regex:

`VERSION_PATTERN = r"^v\d+\.\d+\.\d+$"`


## Scheme Name

Must only contain `a-z`, `0-9`, and `-`. Cannot start or end with `-`

Regex:

`SCHEMENAME_PATTERN = r"^[a-z0-9][a-z0-9-]*[a-z0-9]$"`

# CLI

# `primal-page`

**Usage**:

```console
$ primal-page [OPTIONS] COMMAND [ARGS]...
```

**Options**:

* `--version`
* `--install-completion`: Install completion for the current shell.
* `--show-completion`: Show completion for the current shell, to copy it or customize the installation.
* `--help`: Show this message and exit.

**Commands**:

* `aliases`: Manage aliases
* `build-index`: Build an index.json file from all schemes...
* `create`: Create a new scheme in the required format
* `dev`: Development commands
* `download`: Download schemes from the index.json
* `modify`: Modify an existing scheme's metadata...
* `remove`: Remove a scheme's version from the repo,...
* `validate`: Validate a scheme

## `primal-page aliases`

Manage aliases

**Usage**:

```console
$ primal-page aliases [OPTIONS] COMMAND [ARGS]...
```

**Options**:

* `--help`: Show this message and exit.

**Commands**:

* `add`: Add an alias:schemename to the alias file
* `remove`: Remove an alias from the alias file.

### `primal-page aliases add`

Add an alias:schemename to the alias file

**Usage**:

```console
$ primal-page aliases add [OPTIONS] ALIASES_FILE ALIAS SCHEMENAME
```

**Arguments**:

* `ALIASES_FILE`: The path to the alias file to write to  [required]
* `ALIAS`: The alias to add  [required]
* `SCHEMENAME`: The schemename the alias refers to  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page aliases remove`

Remove an alias from the alias file. Does nothing if the alias does not exist

**Usage**:

```console
$ primal-page aliases remove [OPTIONS] ALIASES_FILE ALIAS
```

**Arguments**:

* `ALIASES_FILE`: The path to the alias file to write to  [required]
* `ALIAS`: The alias to add  [required]

**Options**:

* `--help`: Show this message and exit.

## `primal-page build-index`

Build an index.json file from all schemes in the directory

**Usage**:

```console
$ primal-page build-index [OPTIONS]
```

**Options**:

* `--gitaccount TEXT`: The name of the github account  [default: quick-lab]
* `--gitserver TEXT`: The name of the github server  [default: https://github.com/]
* `--parentdir PATH`: The parent directory  [default: .]
* `--git-commit-sha TEXT`: The git commit
* `--force / --no-force`: Force the creation of the index.json  [default: no-force]
* `--help`: Show this message and exit.

## `primal-page create`

Create a new scheme in the required format

**Usage**:

```console
$ primal-page create [OPTIONS]
```

**Options**:

* `--schemename TEXT`: The name of the scheme  [required]
* `--ampliconsize INTEGER RANGE`: Amplicon size  [x>=100; required]
* `--schemeversion TEXT`: Scheme version, default is parsed from config.json  [required]
* `--species INTEGER`: The species this scheme targets. Please use NCBI taxonomy ids  [required]
* `--authors TEXT`: Any authors. To provide multiple, use --authors '1' --authors '2'  [required]
* `--primerbed PATH`: The path to the primer.bed file  [required]
* `--reference PATH`: The path to the reference.fasta file  [required]
* `--schemestatus [withdrawn|deprecated|autogenerated|draft|tested|validated]`: Scheme status  [default: draft]
* `--citations TEXT`: Any associated citations. Please use DOI
* `--output PATH`: Where to output the scheme  [default: primerschemes]
* `--configpath PATH`: Where the config.json file is located
* `--algorithmversion TEXT`: The version of primalscheme or other
* `--description TEXT`: A description of the scheme
* `--derivedfrom TEXT`: Which scheme has this scheme been derived from
* `--primerclass [primerschemes]`: The primer class  [default: primerschemes]
* `--collection [ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP]`: The collection tags. To provide multiple, use --collection '1' --collection '2'
* `--link-protocol TEXT`: Optional link to protocol
* `--link-validation TEXT`: Optional link to validation data
* `--links-homepage TEXT`: Optional link to homepage
* `--link-vendor TEXT`: Optional link to vendors
* `--link-misc TEXT`: Optional miscellaneous link
* `--fix / --no-fix`: Attempt to fix the scheme  [default: no-fix]
* `--contact-info TEXT`: Contact information
* `--additional-files PATH`: Additional files to include in the ./work directory
* `--ref-select <TEXT PATH>...`: Reference selection file. In the form 'chromosome file'
* `--help`: Show this message and exit.


## `primal-page download`

Download schemes from the index.json

**Usage**:

```console
$ primal-page download [OPTIONS] COMMAND [ARGS]...
```

**Options**:

* `--help`: Show this message and exit.

**Commands**:

* `all`: Download all schemes from the index.json
* `scheme`: Download a scheme from the index.json

### `primal-page download all`

Download all schemes from the index.json

**Usage**:

```console
$ primal-page download all [OPTIONS]
```

**Options**:

* `--output PATH`: The directory the primerschemes dir will be created in  [required]
* `--index-url TEXT`: The URL to the index.json  [default: https://raw.githubusercontent.com/quick-lab/primerschemes/main/index.json]
* `--help`: Show this message and exit.

### `primal-page download scheme`

Download a scheme from the index.json

**Usage**:

```console
$ primal-page download scheme [OPTIONS] SCHEMENAME AMPLICONSIZE SCHEMEVERSION
```

**Arguments**:

* `SCHEMENAME`: The name of the scheme  [required]
* `AMPLICONSIZE`: Amplicon size  [required]
* `SCHEMEVERSION`: Scheme version  [required]

**Options**:

* `--output PATH`: The directory the primerschemes dir will be created in  [required]
* `--index-url TEXT`: The URL to the index.json  [default: https://raw.githubusercontent.com/quick-lab/primerschemes/main/index.json]
* `--help`: Show this message and exit.

## `primal-page modify`

Modify an existing scheme's metadata (info.json)

**Usage**:

```console
$ primal-page modify [OPTIONS] COMMAND [ARGS]...
```

**Options**:

* `--help`: Show this message and exit.

**Commands**:

* `author-add`: Append an author to the authors list in...
* `author-remove`: Remove an author from the authors list in...
* `author-reorders`: Reorder the authors in the info.json file
* `citation-add`: Append an citation to the authors list in...
* `citation-remove`: Remove an citation form the authors list...
* `collection-add`: Add a Collection to the Collection list in...
* `collection-remove`: Remove an Collection from the Collection...
* `contactinfo`: Change the contactinfo field in the info.json
* `derivedfrom`: Replaces the derivedfrom in the info.json...
* `description`: Replaces the description in the info.json...
* `license`: Replaces the license in the info.json file
* `link-add`: Add a link to the selected link field to...
* `link-remove`: Add a link to the selected link field to...
* `primerclass`: Change the primerclass field in the info.json
* `refselect-add`: Add a refselect file to the info.json
* `refselect-remove`: Removes a refselect file from the info.json
* `regenerate`: Validates the info.json and regenerate the...
* `status`: Change the status field in the info.json

### `primal-page modify author-add`

Append an author to the authors list in the info.json file

**Usage**:

```console
$ primal-page modify author-add [OPTIONS] SCHEMEINFO AUTHOR
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `AUTHOR`: The author to add  [required]

**Options**:

* `--author-index INTEGER`: The 0-based index to insert the author at. Default is the end  [required]
* `--help`: Show this message and exit.

### `primal-page modify author-remove`

Remove an author from the authors list in the info.json file

**Usage**:

```console
$ primal-page modify author-remove [OPTIONS] SCHEMEINFO AUTHOR
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `AUTHOR`: The author to remove  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify author-reorders`

Reorder the authors in the info.json file

**Usage**:

```console
$ primal-page modify author-reorders [OPTIONS] SCHEMEINFO [AUTHOR_INDEX]
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `[AUTHOR_INDEX]`: The indexes in the new order, separated by spaces. e.g. 1 0 2. Any indexes not provided will be appended to the end

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify citation-add`

Append an citation to the authors list in the info.json file

**Usage**:

```console
$ primal-page modify citation-add [OPTIONS] SCHEMEINFO CITATION
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `CITATION`: The citation to add  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify citation-remove`

Remove an citation form the authors list in the info.json file

**Usage**:

```console
$ primal-page modify citation-remove [OPTIONS] SCHEMEINFO CITATION
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `CITATION`: The citation to remove  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify collection-add`

Add a Collection to the Collection list in the info.json file

**Usage**:

```console
$ primal-page modify collection-add [OPTIONS] SCHEMEINFO COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}`: The Collection to add  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify collection-remove`

Remove an Collection from the Collection list in the info.json file

**Usage**:

```console
$ primal-page modify collection-remove [OPTIONS] SCHEMEINFO COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `COLLECTION:{ARTIC|MODJADJI|QUICK-LAB|COMMUNITY|WASTE-WATER|CLINICAL-ISOLATES|WHOLE-GENOME|PANEL|MULTI-TARGET|VARVAMP}`: The Collection to remove  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify contactinfo`

Change the contactinfo field in the info.json

**Usage**:

```console
$ primal-page modify contactinfo [OPTIONS] SCHEMEINFO CONTACTINFO
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `CONTACTINFO`: The contact information for this scheme. Use 'None' to remove the contact info  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify derivedfrom`

Replaces the derivedfrom in the info.json file

**Usage**:

```console
$ primal-page modify derivedfrom [OPTIONS] SCHEMEINFO DERIVEDFROM
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `DERIVEDFROM`: The new derivedfrom. Use 'None' to remove the derivedfrom  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify description`

Replaces the description in the info.json file

**Usage**:

```console
$ primal-page modify description [OPTIONS] SCHEMEINFO DESCRIPTION
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `DESCRIPTION`: The new description. Use 'None' to remove the description  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify license`

Replaces the license in the info.json file

**Usage**:

```console
$ primal-page modify license [OPTIONS] SCHEMEINFO LICENSE
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `LICENSE`: The new license. Use 'None' show the work is not licensed (Not recommended)  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify link-add`

Add a link to the selected link field to the info.json

**Usage**:

```console
$ primal-page modify link-add [OPTIONS] SCHEMEINFO LINKFIELD LINK
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `LINKFIELD`: The link field to add to. protocols, validation, homepage, vendors, misc  [required]
* `LINK`: The link to add.  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify link-remove`

Add a link to the selected link field to the info.json

**Usage**:

```console
$ primal-page modify link-remove [OPTIONS] SCHEMEINFO LINKFIELD LINK
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `LINKFIELD`: The link field to remove from. protocols, validation, homepage, vendors, misc  [required]
* `LINK`: The link to remove.  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify primerclass`

Change the primerclass field in the info.json

**Usage**:

```console
$ primal-page modify primerclass [OPTIONS] SCHEMEINFO PRIMERCLASS:{primerschemes}
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `PRIMERCLASS:{primerschemes}`: The primerclass to change to  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify refselect-add`

Add a refselect file to the info.json

**Usage**:

```console
$ primal-page modify refselect-add [OPTIONS] SCHEMEINFO CHROM REFSELECT
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `CHROM`: The chromosome to add  [required]
* `REFSELECT`: The reference selection file  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify refselect-remove`

Removes a refselect file from the info.json

**Usage**:

```console
$ primal-page modify refselect-remove [OPTIONS] SCHEMEINFO CHROM
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `CHROM`: The chromosome to remove  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify regenerate`

Validates the info.json and regenerate the README.md

**Usage**:

```console
$ primal-page modify regenerate [OPTIONS] SCHEMEINFO
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page modify status`

Change the status field in the info.json

**Usage**:

```console
$ primal-page modify status [OPTIONS] SCHEMEINFO [SCHEMESTATUS]:[withdrawn|deprecated|autogenerated|draft|tested|validated]
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]
* `[SCHEMESTATUS]:[withdrawn|deprecated|autogenerated|draft|tested|validated]`: The scheme class  [default: draft]

**Options**:

* `--help`: Show this message and exit.

## `primal-page remove`

Remove a scheme's version from the repo, will also remove size and schemename directories if empty

**Usage**:

```console
$ primal-page remove [OPTIONS] SCHEMEINFO
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]

**Options**:

* `--help`: Show this message and exit.

## `primal-page validate`

Validate a scheme

**Usage**:

```console
$ primal-page validate [OPTIONS] COMMAND [ARGS]...
```

**Options**:

* `--help`: Show this message and exit.

**Commands**:

* `all-schemes`: Validate all schemes in a directory.
* `scheme`: Validate a single scheme

### `primal-page validate all-schemes`

Validate all schemes in a directory. Calls the scheme command for each scheme and checks for final ref-select files.

**Usage**:

```console
$ primal-page validate all-schemes [OPTIONS] DIRECTORY
```

**Arguments**:

* `DIRECTORY`: The path to the directory  [required]

**Options**:

* `--help`: Show this message and exit.

### `primal-page validate scheme`

Validate a single scheme

**Usage**:

```console
$ primal-page validate scheme [OPTIONS] SCHEMEINFO
```

**Arguments**:

* `SCHEMEINFO`: The path to info.json  [required]

**Options**:

* `--help`: Show this message and exit.