Initial commit

.gitattributes

@@ -25,3 +25,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zstandard filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+sp_m.model filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,58 @@
----
-license: mit
----
+---
+language: 
+- de
+- ca
+
+tags:
+- translation
+
+library_name: opennmt
+license: mit
+metrics:
+- bleu
+
+inference: false
+---
+
+### Introduction
+
+German - Catalan translation model for OpenNMT. These are the same models that we have in production at https://www.softcatala.org/traductor/.
+The models are quantified for low latency.
+
+### Usage
+
+Install the necessary dependencies:
+
+
+```bash
+pip3 install ctranslate2 pyonmttok
+```
+
+
+Simple tokenization & translation using Python:
+
+
+```python
+import ctranslate2
+import pyonmttok
+from huggingface_hub import snapshot_download
+model_dir = snapshot_download(repo_id="softcatala/opennmt-deu-cat", revision="main")
+
+tokenizer=pyonmttok.Tokenizer(mode="none", sp_model_path = model_dir + "/sp_m.model")
+tokenized=tokenizer.tokenize("Hallo Freunde")
+
+translator = ctranslate2.Translator(model_dir)
+translated = translator.translate_batch([tokenized[0]])
+print(tokenizer.detokenize(translated[0][0]['tokens']))
+```
+
+## Benchmarks
+
+| testset                               | BLEU  |
+|---------------------------------------|-------|
+| test dataset (from train/dev/test)   	| 37.2	|
+| Flores101 dataset                     | 24.8	|
+
+## Additional information
+* https://github.com/Softcatala/nmt-models
+* https://github.com/Softcatala/parallel-catalan-corpus

model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0768ae89bfcededf2baacd0aea05938554c84e8354ebe051cbffd309d5118564
+size 122582230

sp_m.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ab8c7972437bc43de3bdae9023ff05aca5c760fded45f0c88fa5250a963a9b3
+size 1193906