Text2Text Generation
Transformers
Inference Endpoints
machineteacher commited on
Commit
6f21a96
1 Parent(s): afb9501

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +137 -1
README.md CHANGED
@@ -1,3 +1,139 @@
1
  ---
2
- license: cc-by-nc-4.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: cc-by-nc-sa-4.0
3
+ datasets:
4
+ - wi_locness
5
+ - matejklemen/falko_merlin
6
+ - paws
7
+ - paws-x
8
+ - asset
9
+ language:
10
+ - en
11
+ - de
12
+ - es
13
+ - ar
14
+ - ja
15
+ - ko
16
+ - zh
17
+ metrics:
18
+ - bleu
19
+ - rouge
20
+ - sari
21
+ - accuracy
22
+ library_name: transformers
23
  ---
24
+
25
+ # Model Card for mEdIT-xl
26
+
27
+ The `medit-xl` model was obtained by fine-tuning the `MBZUAI/bactrian-x-llama-7b-lora` model on the mEdIT dataset.
28
+
29
+ **Paper:** mEdIT: Multilingual Text Editing via Instruction Tuning
30
+
31
+ **Authors:** Vipul Raheja, Dimitris Alikaniotis, Vivek Kulkarni, Bashar Alhafni, Dhruv Kumar
32
+
33
+ ## Model Details
34
+
35
+ ### Model Description
36
+
37
+ - **Language(s) (NLP)**: Arabic, Chinese, English, German, Japanese, Korean, Spanish
38
+ - **Finetuned from model:** `MBZUAI/bactrian-x-llama-7b-lora`
39
+
40
+ ### Model Sources
41
+
42
+ - **Repository:** https://github.com/vipulraheja/medit
43
+ - **Paper:** https://arxiv.org/abs/2402.16472v1
44
+
45
+ ## How to use
46
+
47
+ Given an edit instruction and an original text, our model can generate the edited version of the text.<br>
48
+
49
+ ![task_specs](https://cdn-uploads.huggingface.co/production/uploads/60985a0547dc3dbf8a976607/816ZY2t0XPCpMMd6Z072K.png)
50
+
51
+ Specifically, our models support both multi-lingual and cross-lingual text revision. Note that the input and output texts are always in the same language. The monolingual
52
+ vs. cross-lingual setting is determined by comparing the language of the edit instruction in relation to the language of the input text.
53
+
54
+ ### Instruction format
55
+
56
+ Adherence to the following instruction format is essential; failure to do so may result in the model producing less-than-ideal results.
57
+
58
+ ```
59
+ instruction_tokens = [
60
+ "Instruction",
61
+ "Anweisung",
62
+ ...
63
+ ]
64
+
65
+ input_tokens = [
66
+ "Input",
67
+ "Aporte",
68
+ ...
69
+ ]
70
+
71
+ output_tokens = [
72
+ "Output",
73
+ "Produzione",
74
+ ...
75
+ ]
76
+
77
+ task_descriptions = [
78
+ "Fix grammatical errors in this sentence", # <-- GEC task
79
+ "Umschreiben Sie den Satz", # <-- Paraphrasing
80
+ ...
81
+ ]
82
+ ```
83
+
84
+ **The entire list of possible instructions, input/output tokens, and task descriptions can be found in the Appendix of our paper.**
85
+
86
+ ```
87
+ prompt_template = """### <instruction_token>:\n<task_description>\n### <input_token>:\n<input>\n### <output_token>:\n\n"""
88
+ ```
89
+
90
+ Note that the tokens and the task description need not be in the language of the input (in the case of cross-lingual revision).
91
+
92
+
93
+ ### Run the model
94
+
95
+ ```python
96
+ from transformers import AutoTokenizer, AutoModelForCausalLM
97
+
98
+ model_id = "grammarly/medit-xl"
99
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
100
+
101
+ model = AutoModelForCausalLM.from_pretrained(model_id)
102
+
103
+ # English GEC using Japanese instructions
104
+ prompt = '### 命令:\n文章を文法的にする\n### 入力:\nI has small cat ,\n### 出力:\n\n'
105
+
106
+ inputs = tokenizer(prompt, return_tensors='pt')
107
+
108
+ outputs = model.generate(**inputs, max_new_tokens=20)
109
+
110
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True)
111
+
112
+ # --> I have a small cat ,
113
+
114
+ # German GEC using Japanese instructions
115
+ prompt = '### 命令:\n文章を文法的にする\n### 入力:\nIch haben eines kleines Katze ,\n### 出力:\n\n'
116
+
117
+ # ...
118
+ # --> Ich habe eine kleine Katze ,
119
+ ```
120
+
121
+ #### Software
122
+ https://github.com/vipulraheja/medit
123
+
124
+ ## Citation
125
+
126
+ **BibTeX:**
127
+ ```
128
+ @article{raheja2023medit,
129
+ title={mEdIT: mEdIT: Multilingual Text Editing via Instruction Tuning},
130
+ author={Vipul Raheja and Dimitris Alikaniotis and Vivek Kulkarni and Bashar Alhafni and Dhruv Kumar},
131
+ year={2024},
132
+ eprint={2402.16472v1},
133
+ archivePrefix={arXiv},
134
+ primaryClass={cs.CL}
135
+ }
136
+ ```
137
+
138
+ **APA:**
139
+ Raheja, V., Alikaniotis, D., Kulkarni, V., Alhafni, B., & Kumar, D. (2024). MEdIT: Multilingual Text Editing via Instruction Tuning. ArXiv. /abs/2402.16472