---
language:
- es
- zh

tags:
- translation

license: apache-2.0
---

# HelsinkiNLP-FineTuned-Legal-es-zh

This model is a fine-tuned version of [Helsinki-NLP/opus-tatoeba-es-zh](https://huggingface.co/Helsinki-NLP/opus-tatoeba-es-zh) on a dataset constructed by the author himself.
It achieves the following results on the evaluation set:
- Loss: 2.0905

## Model description

Transformer-based NMT model to translate from Spanish to Simplified Chinese, fine-tuned for legal domain.

## Intended uses & limitations

This model is the result of the master graduation thesis for the Tradumatics: Translation Technologies program at the Autonomous University of Barcelona.

The thesis intends to explain various theories and certain algorithm details about neural machine translation, thus this fine-tuned model only serves as a hands-on practice example for that objective, without any intention of productive usage.

## Training and evaluation data

The dataset is constructed from the Chinese translation of Spanish Civil Code, Spanish Constitution, and many other laws & regulations found in the database China Law Info (北大法宝 Beida Fabao), along with their source text found on Boletín Oficial del Estado and EUR-Lex.

There are 9972 sentence pairs constructed. 1000 are used for evaluation and the rest for training.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 10
- mixed_precision_training: Native AMP
- weight_decay: 0.01
- early_stopping_patience: 8

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.9584        | 0.36  | 400  | 2.6800          |
| 2.6402        | 0.71  | 800  | 2.5017          |
| 2.5038        | 1.07  | 1200 | 2.3907          |
| 2.3279        | 1.43  | 1600 | 2.2999          |
| 2.2258        | 1.78  | 2000 | 2.2343          |
| 2.1061        | 2.14  | 2400 | 2.1961          |
| 1.9279        | 2.5   | 2800 | 2.1569          |
| 1.9059        | 2.85  | 3200 | 2.1245          |
| 1.7491        | 3.21  | 3600 | 2.1227          |
| 1.6301        | 3.57  | 4000 | 2.1169          |
| 1.6871        | 3.92  | 4400 | 2.0979          |
| 1.5203        | 4.28  | 4800 | 2.1074          |
| 1.4646        | 4.63  | 5200 | 2.1024          |
| 1.4739        | 4.99  | 5600 | 2.0905          |
| 1.338         | 5.35  | 6000 | 2.0946          |
| 1.3152        | 5.7   | 6400 | 2.0974          |
| 1.306         | 6.06  | 6800 | 2.0985          |
| 1.1991        | 6.42  | 7200 | 2.0962          |
| 1.2113        | 6.77  | 7600 | 2.1092          |
| 1.1983        | 7.13  | 8000 | 2.1060          |
| 1.1238        | 7.49  | 8400 | 2.1102          |
| 1.1417        | 7.84  | 8800 | 2.1078          |


### Framework versions

- Transformers 4.7.0
- Pytorch 1.8.1+cu101
- Datasets 1.8.0
- Tokenizers 0.10.3