lbourdois commited on
Commit
364f632
1 Parent(s): 23919f4

Add multilingual to the language tag

Browse files

Hi! A PR to add multilingual to the language tag to improve the referencing.

Files changed (1) hide show
  1. README.md +4 -8
README.md CHANGED
@@ -2,17 +2,13 @@
2
  language:
3
  - en
4
  - code
5
-
6
-
7
  tags:
8
  - pytorch
9
  - causal-lm
10
  - code-generation
11
  - The Pile
12
-
13
-
14
- license: apache-2.0
15
-
16
  ---
17
 
18
 
@@ -62,7 +58,7 @@ This model was trained for 47,000 steps at a batch size of 6,291,456 tokens per
62
 
63
  Following [Bavarian et al. 2022](https://arxiv.org/abs/2207.14255), we train the model to additionally perform infilling via a data transformation applied randomly to 90% of input contexts at train-time.
64
 
65
- Middle segments to infill were selected uniformly at random from contexts at the character level, and these contexts were then reformatted as
66
 
67
  \<SUF\> {last 1/3rd of the context} \<PRE\> {first 1/3rd of the context} \<MID\> {middle 1/3rd of the context} \<EOD\>
68
 
@@ -91,7 +87,7 @@ model = AutoModelForCausalLM.from_pretrained("CarperAI/FIM-NeoX-1.3B")
91
 
92
  ### Performing Infilling
93
 
94
- Suppose we have some text that we would like to perform infilling on at a certain cursor location”.
95
 
96
  This would have the form {some prelude text here} \<INFILLING LOCATION\> {some text following cursor}.
97
 
 
2
  language:
3
  - en
4
  - code
5
+ - multilingual
6
+ license: apache-2.0
7
  tags:
8
  - pytorch
9
  - causal-lm
10
  - code-generation
11
  - The Pile
 
 
 
 
12
  ---
13
 
14
 
 
58
 
59
  Following [Bavarian et al. 2022](https://arxiv.org/abs/2207.14255), we train the model to additionally perform infilling via a data transformation applied randomly to 90% of input contexts at train-time.
60
 
61
+ Middle segments to infill were selected uniformly at random from contexts at the character level, and these contexts were then reformatted as
62
 
63
  \<SUF\> {last 1/3rd of the context} \<PRE\> {first 1/3rd of the context} \<MID\> {middle 1/3rd of the context} \<EOD\>
64
 
 
87
 
88
  ### Performing Infilling
89
 
90
+ Suppose we have some text that we would like to perform infilling on at a certain cursor location�.
91
 
92
  This would have the form {some prelude text here} \<INFILLING LOCATION\> {some text following cursor}.
93