llm-jp
/

llm-jp-13b-instruct-full-dolly-oasst-v1.0

Text Generation

text-generation-inference

Model card Files Files and versions Community

losyer8 commited on Oct 20, 2023

Commit

ff1fcd2

•

1 Parent(s): 6e89f8f

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -49,7 +49,7 @@ Checkpoints format: Hugging Face Transformers (Megatron-DeepSpeed format models
 - torch>=2.0.0
 - transformers>=4.34.0
 - tokenizers>=0.14.0
 ## Usage
@@ -57,7 +57,7 @@ Checkpoints format: Hugging Face Transformers (Megatron-DeepSpeed format models
 import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0")
-model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0", torch_dtype=torch.float16)
 text = "自然言語処理とは何か"
 text = text + "### 回答："
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)

 - torch>=2.0.0
 - transformers>=4.34.0
 - tokenizers>=0.14.0
+- accelerate==0.23.0
 ## Usage
 import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0")
+model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0", device_map="auto", torch_dtype=torch.float16)
 text = "自然言語処理とは何か"
 text = text + "### 回答："
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)