v2

README.md

@@ -0,0 +1,52 @@
+---
+license: apache-2.0
+datasets:
+- cerebras/SlimPajama-627B
+- bigcode/starcoderdata
+- sam-mosaic/orca-gpt4-chatml
+- alvations/globalvoices-en-es
+language:
+- en
+- es
+---
+<div align="center">
+
+  # TinyLlama-1.1B-translate-en-es
+
+</div>
+
+This is a finetuned version with a partial dataset from alvations/globalvoices-en-es to test performance on translation task. It has been trained to translate english to spanish and viceversa with only 20k rows from the dataset.
+
+The translation is not very accurate but it shows a lot of potential.
+
+In order to use it you have to follow the chatml standard like so:
+---
+english to spanish:
+```
+<|im_start|>user Translate this to spanish: ```A father and son, who have been living off grid for 20 years, encounter an outsider who threatens to destroy the utopia they've built.```
+<|im_start|>assistant 
+```
+This will provide the following result:
+```
+Un padre y hijo, que han vivido sin comida desde hace 20 años, encuentran un invitado quien amenaza con destruir la utopía que ellos han creado.
+```
+---
+spanish to english:
+```
+<|im_start|>user Traduce esto al ingles: ```España se queda sin Copilot para Windows 11: la regulación de la UE frena su despliegue en Europa.```
+<|im_start|>assistant 
+```
+Which will be completed as:
+```
+Spain is left without Copilot for Windows 11: the control of the UE has halted its deployment in Europe.
+```
+
+---
+The results are far from perfect but there are A LOT of room to improvement since it was finetuned with only 20k rows from the dataset (which has 355k rows) for 2 epoch. This training took only about 5 hours on a "M1 Pro" processor. 
+
+The base model used is a fine-tuned model with orca dataset [acalatrava/TinyLlama-1.1B-orca-gpt4](https://huggingface.co/acalatrava/TinyLlama-1.1B-orca-gpt4)
+
+### Training
+ - **Method**: QLORA
+ - **Time**: 10h on a M1 Pro 32GB
+ - **Based on**: [https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g](https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g) removing quantization since it's not supported on MPS

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:68aabef52d79ae78be22167ad4bc25c1cb5535aa7dba4d05190f95a60b718951
+oid sha256:45b41b0806b99333a580819febf0320f7313e20b0301a22d926aff95693c3a9e
 size 2200171846