Andyrasika/Gemma-ChatML

Model Card for Model ID

Model Details

Google has released Gemma, a new family of state-of-the-art open Large Language Models (LLMs), available in 7B and 2B parameter versions. While Gemma offers versatility, fine-tuning it with ChatML has posed challenges. This model aims to demonstrate fine-tuning Gemma 7B using Hugging Face TRL.

Steps:

• Setting Up the Development Environment
• Dataset Creation and Preparation
• Fine-Tuning Gemma with TRL and SFTTrainer
• Testing and Evaluating the Fine-Tuned LLM

Model Description

• Developed by: AndyRasika
• Finetuned from model [optional]: philschmid/gemma-tokenizer-chatml

Uses

Fine-tuning Gemma, Google's advanced open Large Language Model (LLM), using ChatML and Hugging Face TRL opens up a multitude of potential applications across various domains:

1. Natural Language Understanding (NLU):

• Enhancing Gemma's ability to comprehend and interpret natural language queries, commands, and conversations. Improving sentiment analysis, named entity recognition, and text classification tasks in applications such as chatbots, virtual assistants, and customer support systems. Text Generation and Summarization:

• Generating coherent and contextually relevant text for content generation tasks such as article writing, story generation, and summarization. Creating personalized content recommendations and summaries based on user preferences and input.

2. Language Translation and Localization:

• Fine-tuning Gemma for translation tasks to improve the accuracy and fluency of translated text across multiple languages.
• Facilitating localization efforts by adapting Gemma to specific dialects, slang, or cultural nuances.

3. Information Retrieval and Question Answering:

• Improving Gemma's ability to retrieve relevant information from vast datasets or knowledge bases in response to user queries.
• Enhancing question answering systems for providing accurate and concise responses to user questions across various domains.

4. Content Moderation and Filtering:

• Training Gemma to identify and moderate inappropriate or harmful content in online forums, social media platforms, and other user-generated content.
• Implementing content filtering mechanisms to ensure user safety and compliance with community guidelines.

5. Personalization and Recommendation Systems:

• Tailoring Gemma to individual user preferences and behavior patterns for personalized product recommendations, content suggestions, and targeted advertisements.
• Enhancing recommendation systems in e-commerce, streaming platforms, and social media networks to improve user engagement and satisfaction.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from peft import AutoPeftModelForCausalLM
from transformers import  AutoTokenizer, pipeline

peft_model_id = "Andyrasika/Gemma-ChatML"

# Load Model with PEFT adapter
tokenizer = AutoTokenizer.from_pretrained(peft_model_id)
model = AutoPeftModelForCausalLM.from_pretrained(peft_model_id, device_map="auto", torch_dtype=torch.float16)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
# get token id for end of conversation
eos_token = tokenizer("<|im_end|>",add_special_tokens=False)["input_ids"][0]

prompts = [
    "What is the capital of Germany? Explain why thats the case and if it was different in the past?",
    "Write a Python function to calculate the factorial of a number.",
    "A rectangular garden has a length of 25 feet and a width of 15 feet. If you want to build a fence around the entire garden, how many feet of fencing will you need?",
    "What is the difference between a fruit and a vegetable? Give examples of each.",
]

def test_inference(prompt):
    prompt = pipe.tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
    outputs = pipe(prompt, max_new_tokens=1024, do_sample=True, temperature=0.7, top_k=50, top_p=0.95, eos_token_id=eos_token)
    return outputs[0]['generated_text'][len(prompt):].strip()


for prompt in prompts:
    print(f"    prompt:\n{prompt}")
    print(f"    response:\n{test_inference(prompt)}")
    print("-"*50)

Output

 prompt:
What is the capital of Germany? Explain why thats the case and if it was different in the past?
/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/pipelines/base.py:1157: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
  warnings.warn(
    response:
The capital of Germany is Berlin. Berlin is the capital of Germany because it is the largest city in Germany, and it is the political center of the country. Berlin is also home to the German parliament, the Bundestag, and the federal government. In the past, the capital of Germany was Berlin, but it was divided into two parts during the Cold War. The western part of Berlin was controlled by the United States, France, and the United Kingdom, while the eastern part was controlled by the Soviet Union. Berlin was also the site of the Berlin Wall, which separated the two sides of the city from 1961 to 1989. The fall of the Berlin Wall marked the end of the Cold War, and Germany was reunified in 1990. Berlin has since become the capital of a reunified Germany.
--------------------------------------------------
    prompt:
Write a Python function to calculate the factorial of a number.
    response:
def factorial(n):
    fact = 1
    for i in range(1, n+1):
        fact = fact * i
    return fact
--------------------------------------------------
    prompt:
A rectangular garden has a length of 25 feet and a width of 15 feet. If you want to build a fence around the entire garden, how many feet of fencing will you need?
    response:
The total amount of fencing needed is 70 feet. This is determined by multiplying the perimeter of the garden (which is 70 feet) by the cost per foot of the fencing material.
--------------------------------------------------
    prompt:
What is the difference between a fruit and a vegetable? Give examples of each.
    response:
A fruit is the part of a plant that contains the seeds. Examples of fruits are apples, oranges, strawberries, and grapes. A vegetable is the part of a plant that is used for food but does not contain seeds. Examples of vegetables are carrots, potatoes, broccoli, and lettuce.
--------------------------------------------------

• PEFT 0.8.2