Mistral-7B-v0.1 for Instruction Following

This is the Mistral-7B-v0.1 model, fine-tuned using the Finance Alpaca dataset.

Usage in Transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftConfig

config = PeftConfig.from_pretrained("deepset/mistral-7b-bnb-4bit-finance-alpaca")
tokenizer = AutoTokenizer.from_pretrained(
    config.base_model_name_or_path,
    model_max_length=2048,
    padding_side="right",
)
model = AutoModelForCausalLM.from_pretrained(
    "deepset/mistral-7b-bnb-4bit-finance-alpaca",
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt_template = """### Instruction:
{}

### Response:
{}"""

inputs = tokenizer(
    [
        prompt_template.format(
            "Answer concisely, no more than one sentence. Also respond in a journalistic tone. What are capital gains taxes?",  # instruction
            "",  # output - leave this blank for generation!
        )
    ],
    return_tensors="pt"
).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=400, use_cache=True)
print(tokenizer.batch_decode(outputs)[0])

gives

<s> ### Instruction:
Answer concisely, no more than one sentence. Also respond in a journalistic tone. What are capital gains taxes?

### Response:
Capital gains taxes are taxes on profits from the sale of investments, such as stocks, bonds, and real estate.</s>