SignSparK-BT
Back-translation models for evaluating sign language production. Each model takes SMPL+MANO pose features and produces spoken language text, so generated sign sequences can be scored by translating them back and comparing against the source sentence.
Here, we release the backtranslation models for SignSparK, which read from the same pose data as in LionelLow/SignSparK_data.
Code: JianHe0628/SignSparK_BT
Usage
pip install git+https://github.com/JianHe0628/SignSparK_BT
python tools/download_models.py --datasets PHOENIX-2014T CSL-Daily --dest ./checkpoints
Score a SignSparK .npy sample dump, reporting ground truth, generation and the drop between them:
signspark-bt score checkpoints/PHOENIX-2014T \
--body-npy .../body/<run>/seed102_clampstep0_PHOENIX-2014T.npy \
--hand-npy .../hand/<run>/seed102_clampstep0_PHOENIX-2014T.npy
Or from Python:
import torch
from signspark_bt import make_back_translation_model, back_translate
model = make_back_translation_model("checkpoints/PHOENIX-2014T")
# B x T x 240: body (60) + left hand (90) + right hand (90)
sentences = back_translate(model=model, poses=torch.rand(8, 64, 240))
Results
Ground-truth poses on the full test split. These are the upper-bound values: the best any generated sequence should theoretically score through our BT models.
| Model | clips | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 | chrF | ROUGE |
|---|---|---|---|---|---|---|---|
| PHOENIX-2014T | 642 | 38.08 | 25.43 | 18.80 | 15.00 | 37.34 | 38.18 |
| CSL-Daily | 1162 | 26.00 | 15.11 | 9.85 | 6.98 | 7.50 | 26.88 |
| How2Sign | 2183 | 23.07 | 10.33 | 5.73 | 3.53 | 19.81 | 23.58 |
CSL-Daily is scored per character, so its chrF is not comparable with the other datasets.
Models
| Model | dev BLEU-4 | selected at | language |
|---|---|---|---|
| PHOENIX-2014T | 15.26 | step 6200 | German |
| CSL-Daily | 7.05 | step 6500 | Chinese |
| How2Sign | 4.27 | step 9400 | English |
Each directory holds config.yaml, best.ckpt, gls.vocab and txt.vocab.
Input features
240 dimensions per frame, drawn from the SignSparK LMDBs:
| Stream | Source | Dims |
|---|---|---|
| Body | body_features[:, 66:] (last 10 joints, legs dropped) |
60 |
| Left hand | left_features[:, :90] |
90 |
| Right hand | right_features[:, :90] |
90 |
SignSparK .npy sample dumps store the left hand conjugated into right-hand convention; the loader undoes this. The face stream is not used.
Limitations
Note that backtranslation scores are heavily dependant on which backtranslation model and data representation you use.
This means that BLEU scores across papers are often hard to compare against, unless they have used the same model checkpoints.
This can be slightly remedied by reporting the relative percentage drops, as described in our paper. However, even so, backtranslation can still often be limited by the following:
- Since it is a language model, it may possibly produce a fluent sentence from bad poses.
- Sign Language Translation is still an open research problem; thus, the SLT model can be an unreliable proxy for evaluating production.
We recommend any use of backtranslation scores to be reported with its relative drop together with human evaluation and pose metrics.
Additionally, note that our released checkpoints here have all been trained on the re-optimized poses in LionelLow/SignSparK_data. Older checkpoints trained on the raw data are deprecated.
Citation
@inproceedings{low2026signspark,
title={SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning},
author={Low, Jianhe and Symeonidis-Herzig, Alexandre and Ivashechkin, Maksym and Sincan, Ozge Mercanoglu and Bowden, Richard},
booktitle={European Conference on Computer Vision},
pages={648--670},
year={2026},
organization={Springer}
}