README.md

---
license: mit
tasks:
  - protein-structure-prediction
frameworks:
  - pytorch
language:
  - en
  - zh
tags:
  - OneScience
  - Life Sciences
  - Protein Language Model
  - Protein Structure Prediction
  - Variant Effect Prediction
  - ESM
modelscope:
  model: OneScience/ESM
  data_path: data/
---

<p align="center">
  <strong>
    <span style="font-size: 30px;">ESM</span>
  </strong>
</p>

# Model Introduction

ESM (Evolutionary Scale Modeling) is a family of protein language models released by Meta AI / FAIR. It can be used for protein sequence representation, structure prediction, variant effect assessment, and fixed-backbone sequence design.

Paper: Evolutionary-scale prediction of atomic-level protein structure with a language model  
https://www.science.org/doi/10.1126/science.ade2574

# Model Description

This model package integrates PyTorch inference capabilities for ESM-1, ESM-2, MSA Transformer, ESMFold, ESM-1v, and ESM-IF1, and provides DCU runtime adaptation. The accompanying sample data is released with the ModelScope model package `OneScience/ESM`.

# Applicable Scenarios

| Scenario | Description |
| :---: | :--- |
| Protein sequence representation extraction | Input a FASTA file and output per-token, mean, BOS, or contact representations |
| Protein structure prediction | Input one or more amino acid sequences and output the corresponding PDB structure files |
| Variant effect scoring | Input a wild-type sequence and a DMS mutation table, and output mutation impact scores |
| Fixed-backbone sequence design | Input a PDB / CIF structure and chain ID, and sample candidate sequences satisfying backbone constraints |
| Structure-conditioned sequence scoring | Input a structure and candidate sequences, and compute conditional log-likelihood |
| ModelScope / OneCode runtime | After downloading the model project, quickly verify script connectivity in a biology-domain runtime environment |


# Usage Instructions

## 1. OneCode Usage

You can experience intelligent one-click AI4S programming through the OneCode online environment:

[Click to experience intelligent one-click AI4S programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)

## 2. Manual Installation and Usage

**Hardware Requirements**

- GPU or DCU runtime is recommended.
- CPU can be used for import checks and small-configuration connectivity verification, but full training and inference are slow.
- DCU users need to install DTK in advance. DTK 25.04.2 or later is recommended, or the OneScience-recommended version matching the current cluster.





**Environment Check**

- NVIDIA GPU:

```bash
nvidia-smi
```

- Hygon DCU:

```bash
hy-smi
```

### Download the Model Package

```bash
modelscope download --model OneScience/ESM --local_dir ./ESM
cd ESM
```

This model package already contains a small amount of sample data that can be used directly for default workflow verification.

### Install the Runtime Environment

**DCU Environment**

```bash
# First activate DTK and Conda
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# Supports uv installation
pip install onescience[bio-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
```

```bash
# If libraries cannot be found, activate CUDA as shown below
source ${ROCM_PATH}/cuda/env.sh
export LD_LIBRARY_PATH="$CONDA_PREFIX/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="$CONDA_PREFIX/lib/python3.11/site-packages/fastpt/torch/lib:$LD_LIBRARY_PATH"
```

After installation, return to the model package directory:

```bash
cd ./ESM
```

### Training and Inference Data Introduction

The FASTA, PDB / CIF, and DMS data used by the ESM examples have been released with the [ModelScope model package OneScience/ESM](https://modelscope.cn/models/OneScience/ESM). After downloading the complete model package, they can be used directly from the `data/` directory. This model package does not include a training entry point; the data is used for example inference and workflow verification. You can also download only the data directory:

```bash
modelscope download --model OneScience/ESM ESM/data --local_dir ./data
```
### Training Weights

The repository includes multiple ESM weights under `weight/`, which can be selected for inference as needed.

### Prepare Weights

Place the required ESM weights in the following directory:

```text
weight/
  checkpoints/
    esm2_t6_8M_UR50D.pt
    esmfold_3B_v1.pt
    esm1v_t33_650M_UR90S_1.pt
    esm_if1_gvp4_t16_142M_UR50.pt
    ...
```

If using a shared runtime directory, you can also specify the weight location through an environment variable:

```bash
export ESM_WEIGHT_DIR=/path/to/esm/weight
```

The default example reads:

- `weight/checkpoints/esm2_t6_8M_UR50D.pt`

The corresponding weights must be available for the ESMFold, ESM-1v, and ESM-IF1 examples.

### Default Example

```bash
bash scripts/infer.sh
```

The default example reads `data/fasta/few_proteins.fasta`, uses `esm2_t6_8M_UR50D.pt` to extract protein representations, and saves the results to `outputs/embeddings/`.

### Sequence Representation Extraction

```bash
python scripts/extract.py \
  weight/checkpoints/esm2_t6_8M_UR50D.pt \
  data/fasta/few_proteins.fasta \
  outputs/embeddings \
  --include mean per_tok \
  --repr_layers 6
```

### ESMFold Structure Prediction

```bash
python scripts/fold.py \
  -i data/fasta/few_proteins.fasta \
  -o outputs/pdb \
  --model-dir weight \
  --cpu-only
```

The output directory will generate one or more `.pdb` files. For formal GPU / DCU inference, you can remove `--cpu-only` and set `--chunk-size` or `--max-tokens-per-batch` according to available memory.

You can also explicitly enable ESMFold through the default script:

```bash
RUN_ESMFOLD=1 bash scripts/infer.sh
```

### Inverse Folding Sequence Sampling

```bash
python scripts/inverse_folding/sample_sequences.py \
  data/inverse_folding/5YH2.pdb \
  --chain A \
  --outpath outputs/sampled_seqs.fasta \
  --num-samples 1 \
  --nogpu
```

### Inverse Folding Sequence Scoring

```bash
python scripts/inverse_folding/score_log_likelihoods.py \
  data/inverse_folding/5YH2.pdb \
  data/inverse_folding/5YH2_mutated_seqs.fasta \
  --chain A \
  --outpath outputs/sequence_scores.csv \
  --nogpu
```

### Variant Effect Prediction

Variant effect prediction requires providing a wild-type sequence matching the DMS mutation column:

```bash
python scripts/variant_prediction/predict.py \
  --model-location esm1v_t33_650M_UR90S_1 \
  --sequence "${ESM_VARIANT_SEQUENCE}" \
  --dms-input data/variant_prediction/BLAT_ECOLX_Ranganathan2015.csv \
  --mutation-col mutant \
  --dms-output outputs/variant_prediction.csv \
  --offset-idx 24 \
  --scoring-strategy wt-marginals
```

# Data Format

Sample data is stored under `data/` by default:

```text
data/
  fasta/
    few_proteins.fasta
    some_proteins.fasta
  inverse_folding/
    5YH2.pdb
    5YH2.cif
    5YH2_mutated_seqs.fasta
    example.json
  variant_prediction/
    BLAT_ECOLX_Ranganathan2015.csv
    rho_pp.csv
    aggregated_rho.csv
    aggregated_rho_round3.csv
```

Where:

- FASTA files are used for sequence representation extraction and structure prediction.
- PDB / CIF files are used for inverse folding sampling and structure-conditioned sequence scoring.
- The variant effect prediction CSV must contain a mutation column. The default example column name is `mutant`, and the mutation format is like `A123B`.
- Custom DMS data must ensure that the wild-type sequence provided by `--sequence` is consistent with the wild-type amino acids in the mutation column.

# Verification

Static import check:

```bash
python scripts/check_import_boundaries.py
```

Syntax check:

```bash
python -B -c "import ast, pathlib; [ast.parse(p.read_text(encoding='utf-8'), filename=str(p)) for root in ['model', 'scripts', 'tests'] for p in pathlib.Path(root).rglob('*.py')]"
```

# Official OneScience Information

| Platform | OneScience Main Repository | Skills Repository |
| --- | --- | --- |
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |

# Citations and License

- This repository is based on the ESM open-source model and provides DCU adaptation.
- ESM-related source code uses the MIT License; see `LICENSE`. For the terms of use for model weights and data, refer to the corresponding publisher's instructions.
- For scientific research, cite the corresponding original ESM paper according to the specific submodel. For ESM-2 / ESMFold, cite: [Evolutionary-scale prediction of atomic-level protein structure with a language model](https://www.science.org/doi/10.1126/science.ade2574).