---
tags:
- translation
- generated_from_trainer
datasets:
- cmu_hinglish_dog
metrics:
- bleu
model-index:
- name: t5-small_6_3-hi_en-to-en
  results:
  - task:
      name: Sequence-to-sequence Language Modeling
      type: text2text-generation
    dataset:
      name: cmu_hinglish_dog
      type: cmu_hinglish_dog
      args: hi_en-en
    metrics:
    - name: Bleu
      type: bleu
      value: 18.0863
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small_6_3-hi_en-to-en

This model was trained from scratch on the cmu_hinglish_dog dataset.
It achieves the following results on the evaluation set:
- Loss: 2.3662
- Bleu: 18.0863
- Gen Len: 15.2708

## Model description

Model generated using:<br />
```python make_student.py t5-small t5_small_6_3 6 3```<br />
Check this [link](https://discuss.huggingface.co/t/questions-on-distilling-from-t5/1193/9) for more information.

## Intended uses & limitations

More information needed

## Training and evaluation data

Used cmu_hinglish_dog dataset. Please check this [link](https://huggingface.co/datasets/cmu_hinglish_dog) for dataset description
 
## Translation:

* Source: hi_en: The text in Hinglish
* Target: en: The text in English



## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
| No log        | 1.0   | 126   | 3.0601          | 4.7146  | 11.9904 |
| No log        | 2.0   | 252   | 2.8885          | 5.9584  | 12.3418 |
| No log        | 3.0   | 378   | 2.7914          | 6.649   | 12.3758 |
| 3.4671        | 4.0   | 504   | 2.7347          | 7.3305  | 12.3854 |
| 3.4671        | 5.0   | 630   | 2.6832          | 8.3132  | 12.4268 |
| 3.4671        | 6.0   | 756   | 2.6485          | 8.339   | 12.3641 |
| 3.4671        | 7.0   | 882   | 2.6096          | 8.7269  | 12.414  |
| 3.0208        | 8.0   | 1008  | 2.5814          | 9.2163  | 12.2675 |
| 3.0208        | 9.0   | 1134  | 2.5542          | 9.448   | 12.3875 |
| 3.0208        | 10.0  | 1260  | 2.5339          | 9.9011  | 12.4321 |
| 3.0208        | 11.0  | 1386  | 2.5043          | 9.7529  | 12.5149 |
| 2.834         | 12.0  | 1512  | 2.4848          | 9.9606  | 12.4193 |
| 2.834         | 13.0  | 1638  | 2.4737          | 9.9368  | 12.3673 |
| 2.834         | 14.0  | 1764  | 2.4458          | 10.3182 | 12.4352 |
| 2.834         | 15.0  | 1890  | 2.4332          | 10.486  | 12.4671 |
| 2.7065        | 16.0  | 2016  | 2.4239          | 10.6921 | 12.414  |
| 2.7065        | 17.0  | 2142  | 2.4064          | 10.7426 | 12.4607 |
| 2.7065        | 18.0  | 2268  | 2.3941          | 11.0509 | 12.4087 |
| 2.7065        | 19.0  | 2394  | 2.3826          | 11.2407 | 12.3386 |
| 2.603         | 20.0  | 2520  | 2.3658          | 11.3711 | 12.3992 |
| 2.603         | 21.0  | 2646  | 2.3537          | 11.42   | 12.5032 |
| 2.603         | 22.0  | 2772  | 2.3475          | 12.0665 | 12.5074 |
| 2.603         | 23.0  | 2898  | 2.3398          | 12.0343 | 12.4342 |
| 2.5192        | 24.0  | 3024  | 2.3298          | 12.1011 | 12.5096 |
| 2.5192        | 25.0  | 3150  | 2.3216          | 12.2562 | 12.4809 |
| 2.5192        | 26.0  | 3276  | 2.3131          | 12.4585 | 12.4427 |
| 2.5192        | 27.0  | 3402  | 2.3052          | 12.7094 | 12.534  |
| 2.4445        | 28.0  | 3528  | 2.2984          | 12.7432 | 12.5053 |
| 2.4445        | 29.0  | 3654  | 2.2920          | 12.8409 | 12.4501 |
| 2.4445        | 30.0  | 3780  | 2.2869          | 12.6365 | 12.4936 |
| 2.4445        | 31.0  | 3906  | 2.2777          | 12.8523 | 12.5234 |
| 2.3844        | 32.0  | 4032  | 2.2788          | 12.9216 | 12.4204 |
| 2.3844        | 33.0  | 4158  | 2.2710          | 12.9568 | 12.5064 |
| 2.3844        | 34.0  | 4284  | 2.2643          | 12.9641 | 12.4299 |
| 2.3844        | 35.0  | 4410  | 2.2621          | 12.9787 | 12.448  |
| 2.3282        | 36.0  | 4536  | 2.2554          | 13.1264 | 12.4374 |
| 2.3282        | 37.0  | 4662  | 2.2481          | 13.1853 | 12.4416 |
| 2.3282        | 38.0  | 4788  | 2.2477          | 13.3259 | 12.4119 |
| 2.3282        | 39.0  | 4914  | 2.2448          | 13.2017 | 12.4278 |
| 2.2842        | 40.0  | 5040  | 2.2402          | 13.3772 | 12.4437 |
| 2.2842        | 41.0  | 5166  | 2.2373          | 13.2184 | 12.414  |
| 2.2842        | 42.0  | 5292  | 2.2357          | 13.5267 | 12.4342 |
| 2.2842        | 43.0  | 5418  | 2.2310          | 13.5754 | 12.4087 |
| 2.2388        | 44.0  | 5544  | 2.2244          | 13.653  | 12.4427 |
| 2.2388        | 45.0  | 5670  | 2.2243          | 13.6028 | 12.431  |
| 2.2388        | 46.0  | 5796  | 2.2216          | 13.7128 | 12.4151 |
| 2.2388        | 47.0  | 5922  | 2.2231          | 13.749  | 12.4172 |
| 2.2067        | 48.0  | 6048  | 2.2196          | 13.7256 | 12.4034 |
| 2.2067        | 49.0  | 6174  | 2.2125          | 13.8237 | 12.396  |
| 2.2067        | 50.0  | 6300  | 2.2131          | 13.6642 | 12.4416 |
| 2.2067        | 51.0  | 6426  | 2.2115          | 13.8876 | 12.4119 |
| 2.1688        | 52.0  | 6552  | 2.2091          | 14.0323 | 12.4639 |
| 2.1688        | 53.0  | 6678  | 2.2082          | 13.916  | 12.3843 |
| 2.1688        | 54.0  | 6804  | 2.2071          | 13.924  | 12.3758 |
| 2.1688        | 55.0  | 6930  | 2.2046          | 13.9563 | 12.4416 |
| 2.1401        | 56.0  | 7056  | 2.2020          | 14.0592 | 12.483  |
| 2.1401        | 57.0  | 7182  | 2.2047          | 13.8879 | 12.4076 |
| 2.1401        | 58.0  | 7308  | 2.2018          | 13.9267 | 12.3949 |
| 2.1401        | 59.0  | 7434  | 2.1964          | 14.0518 | 12.4363 |
| 2.1092        | 60.0  | 7560  | 2.1926          | 14.1518 | 12.4883 |
| 2.1092        | 61.0  | 7686  | 2.1972          | 14.132  | 12.4034 |
| 2.1092        | 62.0  | 7812  | 2.1939          | 14.2066 | 12.4151 |
| 2.1092        | 63.0  | 7938  | 2.1905          | 14.2923 | 12.4459 |
| 2.0932        | 64.0  | 8064  | 2.1932          | 14.2476 | 12.3418 |
| 2.0932        | 65.0  | 8190  | 2.1925          | 14.2057 | 12.3907 |
| 2.0932        | 66.0  | 8316  | 2.1906          | 14.2978 | 12.4055 |
| 2.0932        | 67.0  | 8442  | 2.1903          | 14.3276 | 12.4427 |
| 2.0706        | 68.0  | 8568  | 2.1918          | 14.4681 | 12.4034 |
| 2.0706        | 69.0  | 8694  | 2.1882          | 14.3751 | 12.4225 |
| 2.0706        | 70.0  | 8820  | 2.1870          | 14.5904 | 12.4204 |
| 2.0706        | 71.0  | 8946  | 2.1865          | 14.6409 | 12.4512 |
| 2.0517        | 72.0  | 9072  | 2.1831          | 14.6505 | 12.4352 |
| 2.0517        | 73.0  | 9198  | 2.1835          | 14.7485 | 12.4363 |
| 2.0517        | 74.0  | 9324  | 2.1824          | 14.7344 | 12.4586 |
| 2.0517        | 75.0  | 9450  | 2.1829          | 14.8097 | 12.4575 |
| 2.0388        | 76.0  | 9576  | 2.1822          | 14.6681 | 12.4108 |
| 2.0388        | 77.0  | 9702  | 2.1823          | 14.6421 | 12.4342 |
| 2.0388        | 78.0  | 9828  | 2.1816          | 14.7014 | 12.4459 |
| 2.0388        | 79.0  | 9954  | 2.1810          | 14.744  | 12.4565 |
| 2.0224        | 80.0  | 10080 | 2.1839          | 14.7889 | 12.4437 |
| 2.0224        | 81.0  | 10206 | 2.1793          | 14.802  | 12.4565 |
| 2.0224        | 82.0  | 10332 | 2.1776          | 14.7702 | 12.4214 |
| 2.0224        | 83.0  | 10458 | 2.1809          | 14.6772 | 12.4236 |
| 2.0115        | 84.0  | 10584 | 2.1786          | 14.709  | 12.4214 |
| 2.0115        | 85.0  | 10710 | 2.1805          | 14.7693 | 12.3981 |
| 2.0115        | 86.0  | 10836 | 2.1790          | 14.7628 | 12.4172 |
| 2.0115        | 87.0  | 10962 | 2.1785          | 14.7538 | 12.3992 |
| 2.0007        | 88.0  | 11088 | 2.1788          | 14.7493 | 12.3726 |
| 2.0007        | 89.0  | 11214 | 2.1788          | 14.8793 | 12.4045 |
| 2.0007        | 90.0  | 11340 | 2.1786          | 14.8318 | 12.3747 |
| 2.0007        | 91.0  | 11466 | 2.1769          | 14.8061 | 12.4013 |
| 1.9967        | 92.0  | 11592 | 2.1757          | 14.8108 | 12.3843 |
| 1.9967        | 93.0  | 11718 | 2.1747          | 14.8036 | 12.379  |
| 1.9967        | 94.0  | 11844 | 2.1764          | 14.7447 | 12.3737 |
| 1.9967        | 95.0  | 11970 | 2.1759          | 14.7759 | 12.3875 |
| 1.9924        | 96.0  | 12096 | 2.1760          | 14.7695 | 12.3875 |
| 1.9924        | 97.0  | 12222 | 2.1762          | 14.8022 | 12.3769 |
| 1.9924        | 98.0  | 12348 | 2.1763          | 14.7519 | 12.3822 |
| 1.9924        | 99.0  | 12474 | 2.1760          | 14.7756 | 12.3832 |
| 1.9903        | 100.0 | 12600 | 2.1761          | 14.7713 | 12.3822 |

### Evaluation results

| Data Split |   Bleu  |
|:----------:|:-------:|
| Validation | 17.8061 |
|    Test    | 18.0863 |

### Framework versions

- Transformers 4.20.0.dev0
- Pytorch 1.8.0
- Datasets 2.1.0
- Tokenizers 0.12.1