DanielHesslow commited on
Commit
36f9b2a
1 Parent(s): 6ae21fe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -15
README.md CHANGED
@@ -19,21 +19,33 @@ Model | #Params | d_model | layers | lm loss uniref-100
19
  [Large](https://huggingface.co/lightonai/RITA_l)| 680M | 1536 | 24 | 1.82
20
  [**XLarge**](https://huggingface.co/lightonai/RITA_xl)| 1.2B | 2048 | 24 | 1.70
21
 
 
22
 
23
-
24
- # Usage
25
-
26
  Instantiate a model like so:
27
-
28
- from transformers import AutoModel, AutoModelForCausalLM
29
- model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_xl, trust_remote_code=True")
30
- tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_xl")
31
-
32
- for generation use we support pipelines:
33
-
34
-
35
- rita_gen = pipeline('text-generation', model=model, tokenizer = tokenizer)
36
- sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, num_return_sequences=2, eos_token_id=2)
37
- for seq in sequences:
38
- print(f"seq: {seq['generated_text'].replace(' ', '')}")
 
 
 
 
 
 
 
 
 
 
 
 
 
39
 
 
19
  [Large](https://huggingface.co/lightonai/RITA_l)| 680M | 1536 | 24 | 1.82
20
  [**XLarge**](https://huggingface.co/lightonai/RITA_xl)| 1.2B | 2048 | 24 | 1.70
21
 
22
+ For full results see our preprint: https://arxiv.org/abs/2205.05789
23
 
24
+ ## Usage
 
 
25
  Instantiate a model like so:
26
+ ``` python
27
+ from transformers import AutoModel, AutoModelForCausalLM
28
+ model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_xl, trust_remote_code=True")
29
+ tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_xl")
30
+ ```
31
+ for generation we support pipelines:
32
+ ``` python
33
+ from transformers import pipeline
34
+ rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
35
+ sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2,
36
+ num_return_sequences=2, eos_token_id=2)
37
+ for seq in sequences:
38
+ print(f"seq: {seq['generated_text'].replace(' ', '')}")
39
+ ```
40
+
41
+ ## How to cite
42
+
43
+ @misc{RITA2022,
44
+ doi = {10.48550/ARXIV.2205.05789},
45
+ url = {https://arxiv.org/abs/2205.05789},
46
+ author = {Hesslow, Daniel and Zanichelli, Niccoló and Notin, Pascal and Poli, Iacopo and Marks, Debora},
47
+ title = {RITA: a Study on Scaling Up Generative Protein Sequence Models},
48
+ publisher = {arXiv},
49
+ year = {2022},
50
+ }
51