RichardErkhov/TechxGenus_-_CodeGemma-2b-8bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

CodeGemma-2b - bnb 8bits

• Model creator: https://huggingface.co/TechxGenus/
• Original model: https://huggingface.co/TechxGenus/CodeGemma-2b/

Original model description:

tags: - code - gemma library_name: transformers pipeline_tag: text-generation license: other license_name: gemma-terms-of-use license_link: https://ai.google.dev/gemma/terms

CodeGemma

We've fine-tuned Gemma-2b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves 54.9 pass@1 on HumanEval-Python. This model operates using the Alpaca instruction format (excluding the system prompt).

Usage

Here give some examples of how to use our model:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
tokenizer = AutoTokenizer.from_pretrained("TechxGenus/CodeGemma-2b")
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/CodeGemma-2b",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=2048)
print(tokenizer.decode(outputs[0]))

With text-generation pipeline:

from transformers import pipeline
import torch
PROMPT = """<bos>### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
generator = pipeline(
    model="TechxGenus/CodeGemma-2b",
    task="text-generation",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
result = generator(prompt, max_length=2048)
print(result[0]["generated_text"])

Note

Model may sometimes make errors, produce misleading contents, or struggle to manage tasks that are not related to coding. It has undergone very limited testing. Additional safety testing should be performed before any real-world deployments.