Q4234/commandA-LoRA-2.8b

LoRA which is fine-tuned for "command" following.

How to use?

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

model_id = "EleutherAI/pythia-2.8b-deduped"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})

# load adapter
if True:
    from peft import PeftModel
    model = PeftModel.from_pretrained(
        model, "Q4234/commandA-LoRA-2.8b",
        torch_dtype=torch.float16
    )

Prompting

Assistant can use any tools he likes by calling the tool as python functions.
ex:
var=google('what is life?') - to search for something using google and write the result as a string into variable var
result0=calc('5 + 6') - to compute something using a calculator and put the result into the variable result0
ubuntuShell('ls -lah') - to execute command 'ls -lah' in the shell under linux ubuntu
appendToFile(<filename>, <content>) - to append(write) string <content> to file <filename>
str5 = """AB c 7""" - to assign variable str5 to "AB c 7"

Human: Your goal is to install qemu, virtualbox and update pytorch. Tell me all the steps your doing to accomplish the goal!
Assistant:

it can also answer questions quite nicely

Q: What is 1+2? A: The answer is 3.\nQ: What is a cat? A: A cat is an animal.\nQ: Who is Bill Gates? A: Bill Gates is the creator of Microsoft.\nQ: What is an agent? Explain! A: An agent is a person that is able to make choices for another person.\nQ: What is 4+9?\nA: 13\nQ: Who is George Washington and what is the meaning of life?\nA:"

gives

A: George Washington was the first president of the United States. The meaning of life is that human life has value and purpose.
Q: What is the goal of a human life?
A: To live a long and healthy life and achieve happiness.
Q: Who are the different types of cancers? A: There are two main types of cancers. One is cancer that can develop into a death sentence for the patient. The other type is a more common cancer that can be cured with surgery, radiation, and chemotherapy.
Q: What is cancer? A: Cancer is a disease that can develop into a death sentence for the patient.
Q: What is a gene? Explain! A: A gene is the part of DNA that contains the information needed to create a protein.
Q: What is a protein? Explain! A: A protein is the molecular structure of an organism. It performs functions that are needed for growth and development.
Q: How do genes code information?
A: Genes code information in DNA. DNA is a molecule composed of the letters A, C, T, and G, which stand for the letters adenine, cytosine, thymine, and guanine. A gene codes information

issues

• command outputs are suboptimal, should get fixed with later better models.

---

license: mit