gathnex/gathllama-2 · Hugging Face

Model Summary

Base Pre-trained Model: llama-2-7b-hf Fine-Tuning Dataset: vicgalle/alpaca-gpt4 In this model, we leveraged the power of the open-source pre-trained model known as llama-2-7b-hf as the foundation. However, to make this model excel in a particular natural language processing task, we embarked on a fine-tuning journey.

Our fine-tuning dataset consisted of a substantial 50,000 samples sourced from the Alpaca Dataset, carefully curated and preprocessed to ensure data quality. Through the fine-tuning process, we guided the model through five training epochs.

During these epochs, the model underwent iterative refinement, learning task-specific nuances and adapting its knowledge to the intricacies of the Alpaca Dataset. The goal was to enhance the model's performance and make it a proficient tool for a specific application. This comprehensive fine-tuning approach ensures that the model is well-suited to handle the intricacies of real-world data and deliver accurate results.

QA format:

system_prompt = 'Below is an instruction that describes a task. Write a response that appropriately completes the request\n'
B_INST, E_INST = "### Instruction:\n", "### Response:\n"

prompt = f"{system_prompt}{B_INST}{user_prompt.strip()}\n\n{E_INST}"

<s>Below is an instruction that describes a task. Write a response that appropriately completes the request
### Instruction:
explain newtoen's 3rd law with example
### Response:

where the model generates the text after "### Response:".

Chat format:

Here's a simple example to illustrate this law:

Imagine you're on a skateboard, and you push against a wall with your foot. When you apply force (the action) to the wall by pushing, the wall pushes back on your foot with an equal force in the opposite direction (the reaction). This is why you can feel the wall pushing against your foot.

Another everyday example is when you walk. As your foot pushes backward against the ground (action), the ground pushes your foot forward with an equal force (reaction), propelling you forward.

Newton's Third Law helps explain how objects move and interact with each other, emphasizing that forces always come in pairs and act in opposite directions.

Disclimer : The model output will be very based on the hyperparameter you're giving.

Training

Model

Architecture: a Transformer-based model with next-word prediction objective
Dataset size: 50k samples
Epochs: 5
Precision: fp16
GPUs: 2xV100-16G
Training time: 2 days

Software

Sample Code

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer, pipeline
model = AutoModelForCausalLM.from_pretrained("gathnex/gathllama-2",  load_in_4bit = True)
tokenizer = AutoTokenizer.from_pretrained("gathnex/gathllama-2")
# for better inference
model.config.use_cache = True
model.eval()
#testing
def stream(user_prompt):
    runtimeFlag = "cuda:0"
    system_prompt = 'Below is an instruction that describes a task. Write a response that appropriately completes the request'
    B_INST, E_INST = "### Instruction:\n", "### Response:\n"

    prompt = f"{system_prompt}{B_INST}{user_prompt.strip()}\n\n{E_INST}"

    inputs = tokenizer([prompt], return_tensors="pt",add_special_tokens=False).to(runtimeFlag)

    streamer = TextStreamer(tokenizer, skip_prompt=True,
                                    skip_special_tokens=True)

    # Despite returning the usual output, the streamer will also print the generated text to stdout.
    _ = model.generate(**inputs, streamer=streamer, max_new_tokens=500)

stream("explain how to impress our CEO, i'm joined 5 month ago in the company. give tips and tricks")