README.md

metadata

library_name: peft
license: mit
language:
  - en
pipeline_tag: text-generation

AlpaGo: GPT-NeoX-20B Model Trained with Qlora Technique

AlpaGo is an adapter model trained using the Qlora technique on top of the GPT-NeoX-20B model. This repository contains the code and resources for AlpaGo, which can be used for natural language processing tasks. AlpaGo is built on the GPT-NeoX-20B architecture and developed by Math And AI Institute.

Features

• AlpaGo adapter model trained with the Qlora technique
• Based on the GPT-NeoX-20B model, providing high-quality natural language processing capabilities on Engilish Language

Installation

1. Clone the AlpaGo repository:

!git clone https://huggingface.co/myzens/AlpaGo

2. Install the latest version of Python 3 if you haven't already.

3. Install the required dependencies:

!pip install -r requirements.txt

Usage

You can utilize AlpaGo to perform natural language processing tasks. Here's an example of how to use it:

from peft import PeftModel
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, GenerationConfig
model_id = "EleutherAI/gpt-neox-20b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(model, "myzens/AlpaGo")

#You can change Here.
PROMPT = """Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
Write a short story about a lost key that unlocks a mysterious door.
### Response:"""

inputs = tokenizer(PROMPT, return_tensors="pt")
input_ids = inputs["input_ids"].cuda()

generation_config = GenerationConfig(
    temperature=0.6,
    top_p=0.95,
    repetition_penalty=1.15,

)

print("Generating...")
generation_output = model.generate(
    input_ids=input_ids,
    generation_config=generation_config,
    return_dict_in_generate=True,
    output_scores=True,
    max_new_tokens=256,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
)

for s in generation_output.sequences:
    print(tokenizer.decode(s))