Mainak Manna commited on
Commit
e474943
1 Parent(s): e9e609c

First version of the model

Browse files
Files changed (1) hide show
  1. README.md +69 -0
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ language: Italian Spanish
4
+ tags:
5
+ - translation Italian Spanish model
6
+ datasets:
7
+ - dcep europarl jrc-acquis
8
+ widget:
9
+ - text: "che sempre il succitato direttore generale abbia inviato un messaggio e-mail a tutti i controllori finanziari della sua DG affinché non acconsentissero a erogare finanziamenti alle autorità palestinesi senza il suo previo consenso; —"
10
+
11
+ ---
12
+
13
+ # legal_t5_small_trans_it_es model
14
+
15
+ Model on translating legal text from Italian to Spanish. It was first released in
16
+ [this repository](https://github.com/agemagician/LegalTrans). This model is trained on three parallel corpus from jrc-acquis, europarl and dcep.
17
+
18
+
19
+ ## Model description
20
+
21
+ legal_t5_small_trans_it_es is based on the `t5-small` model and was trained on a large corpus of parallel text. This is a smaller model, which scales the baseline model of t5 down by using `dmodel = 512`, `dff = 2,048`, 8-headed attention, and only 6 layers each in the encoder and decoder. This variant has about 60 million parameters.
22
+
23
+ ## Intended uses & limitations
24
+
25
+ The model could be used for translation of legal texts from Italian to Spanish.
26
+
27
+ ### How to use
28
+
29
+ Here is how to use this model to translate legal text from Italian to Spanish in PyTorch:
30
+
31
+ ```python
32
+ from transformers import AutoTokenizer, AutoModelWithLMHead, TranslationPipeline
33
+
34
+ pipeline = TranslationPipeline(
35
+ model=AutoModelWithLMHead.from_pretrained("SEBIS/legal_t5_small_trans_it_es"),
36
+ tokenizer=AutoTokenizer.from_pretrained(pretrained_model_name_or_path = "SEBIS/legal_t5_small_trans_it_es", do_lower_case=False,
37
+ skip_special_tokens=True),
38
+ device=0
39
+ )
40
+
41
+ it_text = "che sempre il succitato direttore generale abbia inviato un messaggio e-mail a tutti i controllori finanziari della sua DG affinché non acconsentissero a erogare finanziamenti alle autorità palestinesi senza il suo previo consenso; —"
42
+
43
+ pipeline([it_text], max_length=512)
44
+ ```
45
+
46
+ ## Training data
47
+
48
+ The legal_t5_small_trans_it_es model was trained on [JRC-ACQUIS](https://wt-public.emm4u.eu/Acquis/index_2.2.html), [EUROPARL](https://www.statmt.org/europarl/), and [DCEP](https://ec.europa.eu/jrc/en/language-technologies/dcep) dataset consisting of 5 Million parallel texts.
49
+
50
+ ## Training procedure
51
+
52
+ ### Preprocessing
53
+
54
+ ### Pretraining
55
+ An unigram model with 88M parameters is trained over the complete parallel corpus to get the vocabulary (with byte pair encoding), which is used with this model.
56
+
57
+
58
+ ## Evaluation results
59
+
60
+ When the model is used for translation test dataset, achieves the following results:
61
+
62
+ Test results :
63
+
64
+ | Model | BLEU score |
65
+ |:-----:|:-----:|
66
+ | legal_t5_small_trans_it_es | 48.998|
67
+
68
+
69
+ ### BibTeX entry and citation info