metadata
language:
- en
tags:
- text-generation
- finetuned
datasets:
- neulab/tldr
license: apache-2.0
pipeline_tag: text-generation
Commonsense-QA-Mistral-7B
This is a finetuned model of mistralai/Mistral-7B-Instruct-v0.1 with neulab/tldr dataset.
The model is loaded in 4-bit and fine-tuned with LoRA.
Usage
Loading of model:
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"rvv-karma/BASH-Coder-Mistral-7B",
low_cpu_mem_usage=True,
return_dict=True,
torch_dtype=torch.bfloat16,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("rvv-karma/BASH-Coder-Mistral-7B", trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"
Sample:
pipe = pipeline(
task="text-generation",
model=model,
tokenizer=tokenizer,
return_full_text=False,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=13,
max_new_tokens=8
)
prompt = """QUESTION: fix a given ntfs partition
ANSWER: """
result = pipe(prompt)
generated = result[0]['generated_text']
print(generated)
# Output: sudo ntfsfix {{/dev/sdXN}}