p-gupta/enzyme-model

Leak CURBER enzyme classification model starter

This is a starter model for the enzyme classification task. You train a model on your computer and upload it to a public Hugging Face (HF) model repository. Then you submit that repository to the Leak CURBER leaderboard. A free HF account is enough.

Task

The input is a Parquet file with the columns id and sequence (the enzyme amino acid sequence). Predict one EC number for each unique input. The output is JSON Lines with the keys id and prediction. The prediction is a list of distinct ECs, best first:

{"id":"v_0000001","prediction":["1.1.1.1"]}

The public data are train.parquet (with the ec_number label) and val_inputs.parquet (inputs only). Validation labels and test data are private. ec_vocabulary.json lists the permitted ECs.

An input can have several recorded ECs. train.parquet has one input–EC pair per row. We group identical inputs within each split, keep the first ID, and score each unique input once. Predict only the first ID of each unique input; do not predict repeat IDs. inputs.py applies the same grouping. Do not normalize sequences.

Only the first predicted EC counts. You can list up to 10, but the others do not change the score. We report top-1 accuracy at EC levels L4, L3, L2, and L1, and rank by L4. Equal scores favor the earlier submission.

If an input has n recorded ECs, the first prediction earns 1/n for each recorded EC whose first L components match it. We average this over unique inputs. With recorded ECs 1.1.1.1 and 1.1.1.2, the prediction 1.1.1.1 scores 0.5 at L4 and 1.0 at L3, L2, and L1. An empty prediction scores zero.

ec_vocabulary.json comes from ExPASy ENZYME, release 02-Sep-2026. SIB provides it under CC BY 4.0: https://enzyme.expasy.org/

Baseline

The baseline ignores the input. It predicts the most frequent training EC for every input. It uses only the training split.

Run the baseline

python -m pip install huggingface-hub==2.2.0
hf download starkAhmed43/leak-curber-enzyme-classification-starter --local-dir model
hf download starkAhmed43/leak-curber-enzyme-classification train.parquet val_inputs.parquet --repo-type dataset --local-dir data
cd model
python -m pip install -r requirements.txt
python predict.py --input ../data/val_inputs.parquet --output predictions.jsonl

To retrain the baseline:

python train.py --train ../data/train.parquet --output weights/baseline.json

To write the unique validation inputs to a file:

python inputs.py --input ../data/val_inputs.parquet --output unique_validation.parquet

Check with Docker

Run these commands in the model directory:

mkdir -p output && chmod 777 output
docker build --platform linux/amd64 -t my-enzyme-model .
docker run --rm --platform linux/amd64 --user 1000:1000 --network none --cpus 2 --memory 16g \
  -v "$PWD/../data:/input:ro" -v "$PWD/output:/output" \
  my-enzyme-model python /app/predict.py \
  --input /input/val_inputs.parquet --output /output/predictions.jsonl

output/predictions.jsonl must contain every unique input ID exactly once.

Replace the baseline

Use any model or library. Train wherever you like, but the model runs on CPU only. The evaluator gives it 2 vCPUs, 16 GB RAM, and 10 minutes for all inputs. It has no GPU and no network. Change Dockerfile, requirements.txt, and predict.py. Put the weights in your repository, with all other files that inference needs. Do not add the downloaded data or the output directory.

We build the image from your Dockerfile. The image must meet these limits:

• Linux x86-64, with Python at /usr/local/bin/python and working directory /app.
• The command is python /app/predict.py --input INPUT --output OUTPUT.
• The build must finish in 30 minutes. The repository must be 4 GiB or less.
• No file that predict.py writes may exceed 16 MiB.

We start Python directly. Docker entrypoints and shell startup hooks do not run. Use /tmp for temporary files.

An evaluation fails in these cases:

• A line is blank, or is not a JSON object with exactly the keys id and prediction.
• A unique input ID is missing, duplicated, or added.
• A prediction has a duplicate EC, more than 10 ECs, or an EC that is not in ec_vocabulary.json.
• Inference needs more than 10 minutes, or needs the network.

Submit

Choose the task assigned to you in this sheet.

1. Sign in: hf auth login.

2. Upload the model directory to a public model repository:

   hf upload YOUR_USERNAME/YOUR_MODEL . --repo-type model --no-private --exclude ".cache/**" "output/**" "predictions.jsonl"
   

   For an existing repository, make sure it is public. The --no-private flag does not change an existing repository. The repository must contain Dockerfile, predict.py, and README.md.

3. Open the leaderboard, select the Enzyme classification tab, and sign in with your registered account.

4. Under Submit model, enter the repository ID, the revision (default main), and a model name. Select Submit for validation.

5. Watch the progress under My submissions.

A submission records a fixed commit. Later changes to the repository do not change it. The validation score appears when the evaluation ends.

Limits and deadline

• Each team has 2 submissions per day and 20 in total for each task.
• Failed model runs count against these limits.
• Only one of your evaluations can be active at a time.
• Under My submissions, nominate one completed submission for the test evaluation. Each task has its own nomination. If you do not nominate, we use your best validation score.
• Your best and your nominated submissions stay protected. Other successful archives expire after 7 days.
• Submit and nominate before November 7, 2026, 23:59 IST. We will publish test scores after that.

Help

• If the leaderboard says your account is not approved, post your HF username in the Moodle course discussion forum.
• If the leaderboard says the evaluator is unavailable, refresh shortly. Post in the Moodle course discussion forum if the error persists.
• Post any other question in the Moodle course discussion forum.