RichardErkhov/silvercoder67_-_Mistral-7b-instruct-v0.2-summ-sft-e2m-8bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Mistral-7b-instruct-v0.2-summ-sft-e2m - bnb 8bits

• Model creator: https://huggingface.co/silvercoder67/
• Original model: https://huggingface.co/silvercoder67/Mistral-7b-instruct-v0.2-summ-sft-e2m/

Original model description:

license: cc-by-nc-4.0

Description to load and test will be added soon. More details on training and data will be added aswell.

Loading the Model

Use the following Python code to load the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

TBD

Generating Text

To generate text, use the following Python code:

text = "Hi, my name is "
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))