File size: 1,484 Bytes
709c543 6686b78 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** [More Information Needed]
### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## Uses
```
gen_kwargs = {
"max_new_tokens": 100,
"top_k": 70,
"top_p": 0.8,
"do_sample": True,
"no_repeat_ngram_size": 2,
"bos_token_id": tokenizer.bos_token_id,
"eos_token_id": tokenizer.eos_token_id,
"pad_token_id": tokenizer.pad_token_id,
"temperature": 0.8,
"use_cache": True,
"repetition_penalty": 1.2,
"num_return_sequences": 1
}
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
ft = 'gpt-j-onlyk_v2'
tokenizer = AutoTokenizer.from_pretrained(ft)
model = AutoModelForCausalLM.from_pretrained(ft, torch_dtype=torch.float16, low_cpu_mem_usage=True)
model.to(device)
prepared = tokenizer.encode(inp, return_tensors='pt').to(model.device)
out = model.generate(input_ids=prepared, **gen_kwargs)
generated = tokenizer.decode(out[0])
```
|