YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
indic-transliterate (prototype)
Roman/Hinglish/code-mixed → native Indic script transliteration, built to sit in front of a TTS pipeline (pronunciation-preserving transliteration, not translation — e.g. "EMI" → "ईएमआई", not "किस्त").
⚠️ Status: early prototype, not production-ready
This checkpoint was trained on a deliberately small, capped slice of data (3,000 word pairs per language from AI4Bharat Aksharantar, ~13k examples total after language-temperature sampling and customer-support-data mixing) for 3 epochs, as an end-to-end pipeline validation step — not a production training run.
Measured on a held-out test set:
| Metric | Value |
|---|---|
| Character Error Rate | 91% |
| Exact match | 0% |
| Script validity | 99.9% |
Script validity (99.9%) shows the model reliably learned which script to emit per language tag — the architecture and data pipeline are wired correctly. Character accuracy (91% CER) is not usable yet — that needs a full-scale training run (full per-language data volume, more epochs) before this should be used for anything beyond pipeline testing.
What this is (and isn't)
- Is: a from-scratch, ~7.7M-parameter character-level Transformer
encoder-decoder (
torch.nn.Transformer-based), with<2xx>language-tag tokens (same convention as IndicXlit/NLLB/mBART), trained via Hugging Facetransformers. - Isn't: AI4Bharat's IndicXlit checkpoint or its fairseq architecture.
IndicXlit's fairseq codebase is not compatible with Python 3.12 (mutable
dataclass-default fields fairseq declares are hard-rejected by Python
3.11+'s
dataclassesmodule, across multiple files, not a single patchable spot) — see the training repository's README for the full writeup. This model uses the same task framing and tagging convention on a maintained stack instead of vendoring IndicXlit's literal weights.
Training data
AI4Bharat Aksharantar
— word-level Roman↔native pairs, 21 languages. Aksharantar's license is
mixed (CC-BY for manually-collected rows, CC0 for mined rows); this model
card carries cc-by-4.0 as the safe upper bound. See the training repo's
README for the full licensing note, including why Google Dakshina (CC BY-SA,
share-alike) was not used for this checkpoint.
Usage
from transliteration.inference import TransliterationEngine
engine = TransliterationEngine.from_checkpoint("path/to/this/checkpoint")
engine.transliterate("mera emi pending hai", language="hi")
Full pipeline code (data download/validation/training/evaluation/inference/ TTS integration): see the accompanying repository files.
Intended use
Pipeline validation and further fine-tuning. Not intended for production transliteration until retrained at full data scale — see Known Limitations in the training repo's README.
- Downloads last month
- 86