gojiteji commited on
Commit
8f7211d
1 Parent(s): 57f845f

AutoModelForConditionalGeneration -> T5ForConditionalGeneration

Browse files

It seems that `AutoModelForConditionalGeneration` doesn't exist in [HF API](https://huggingface.co/docs/transformers/model_doc/auto), so I changed to `T5ForConditionalGeneration` based on [config.json](https://huggingface.co/google/flan-ul2/blob/main/config.json) .

Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -100,9 +100,9 @@ For more efficient memory usage, we advise you to load the model in `8bit` using
100
 
101
  ```python
102
  # pip install accelerate transformers bitsandbytes
103
- from transformers import AutoModelForConditionalGeneration, AutoTokenizer
104
  import torch
105
- model = AutoModelForConditionalGeneration.from_pretrained("google/flan-ul2", device_map="auto", load_in_8bit=True)
106
  tokenizer = AutoTokenizer.from_pretrained("google/flan-ul2")
107
 
108
  input_string = "Answer the following question by reasoning step by step. The cafeteria had 23 apples. If they used 20 for lunch, and bought 6 more, how many apple do they have?"
@@ -118,9 +118,9 @@ Otherwise, you can load and run the model in `bfloat16` as follows:
118
 
119
  ```python
120
  # pip install accelerate transformers
121
- from transformers import AutoModelForConditionalGeneration, AutoTokenizer
122
  import torch
123
- model = AutoModelForConditionalGeneration.from_pretrained("google/flan-ul2", torch_dtype=torch.bfloat16, device_map="auto")
124
  tokenizer = AutoTokenizer.from_pretrained("google/flan-ul2")
125
 
126
  input_string = "Answer the following question by reasoning step by step. The cafeteria had 23 apples. If they used 20 for lunch, and bought 6 more, how many apple do they have?"
100
 
101
  ```python
102
  # pip install accelerate transformers bitsandbytes
103
+ from transformers import T5ForConditionalGeneration, AutoTokenizer
104
  import torch
105
+ model = T5ForConditionalGeneration.from_pretrained("google/flan-ul2", device_map="auto", load_in_8bit=True)
106
  tokenizer = AutoTokenizer.from_pretrained("google/flan-ul2")
107
 
108
  input_string = "Answer the following question by reasoning step by step. The cafeteria had 23 apples. If they used 20 for lunch, and bought 6 more, how many apple do they have?"
118
 
119
  ```python
120
  # pip install accelerate transformers
121
+ from transformers import T5ForConditionalGeneration, AutoTokenizer
122
  import torch
123
+ model = T5ForConditionalGeneration.from_pretrained("google/flan-ul2", torch_dtype=torch.bfloat16, device_map="auto")
124
  tokenizer = AutoTokenizer.from_pretrained("google/flan-ul2")
125
 
126
  input_string = "Answer the following question by reasoning step by step. The cafeteria had 23 apples. If they used 20 for lunch, and bought 6 more, how many apple do they have?"