Model Card: Neural-phi2
Model Details
- Model Name: Neural-phi2
- Model Type: Large Language Model (LLM)
- Model Architecture: A finetuned version of the Phi2 model from Microsoft, utilizing Direct Preference Optimization (DPO) on the
distilabel-intel-orca-dpo-pairs
dataset. - Model Size: Approximately 2B parameters
- Training Data: The model was finetuned on the
distilabel-intel-orca-dpo-pairs
dataset, which consists of chat-like prompts and responses. - Training Procedure: The Phi2 model was finetuned using the DPO technique. The training process involved:
- Loading and formatting the
distilabel-intel-orca-dpo-pairs
dataset - Defining the training configuration, including batch size, learning rate, and number of epochs
- Initializing the DPO Trainer and training the model
- Saving the finetuned model and tokenizer
- Loading and formatting the
Training Parameters
This section outlines the key training parameters used to finetune the Phi2 model from Microsoft using the Direct Preference Optimization (DPO) technique on the distilabel-intel-orca-dpo-pairs
dataset, resulting in the Neural-phi2 model.
- SFT Model Name:
phi2-sft-alpaca_loraemb-right-pad
- New Model Name:
Neural-phi2-v2
- Dataset:
argilla/distilabel-intel-orca-dpo-pairs
- Tokenizer: Custom tokenizer created from the
phi2-sft-alpaca_loraemb-right-pad
model - Quantization Config:
load_in_4bit=True
bnb_4bit_quant_type="nf4"
bnb_4bit_compute_dtype=torch.float16
- LoRA Config:
r=16
lora_alpha=64
lora_dropout=0.05
bias="none"
task_type="CAUSAL_LM"
target_modules=["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"]
- Training Arguments:
per_device_train_batch_size=1
gradient_accumulation_steps=8
gradient_checkpointing=True
learning_rate=5e-7
lr_scheduler_type="linear"
max_steps=500
optim="paged_adamw_32bit"
warmup_steps=100
bf16=True
report_to="wandb"
- DPO Trainer:
loss_type="sigmoid"
beta=0.1
max_prompt_length=768
max_length=1024
Intended Use
The Neural-phi2 model is intended to be used as a general-purpose language model for a variety of natural language processing tasks, such as text generation, summarization, and question answering. It may be particularly useful in applications where the model needs to generate coherent and contextually appropriate responses, such as in chatbots or virtual assistants.
Sample Inference Code
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load the Neural-phi2 model and tokenizer
model = AutoModelForCausalLM.from_pretrained("Neural-phi2")
tokenizer = AutoTokenizer.from_pretrained("Neural-phi2")
# Define a sample prompt
messages = [
{"role": "system", "content": "You are a helpful chatbot assistant."},
{"role": "user", "content": "Hello, how are you today?"}
]
# Format the prompt in ChatML format
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
# Create a pipeline and generate a response
pipeline = transformers.pipeline("text-generation", model=model, tokenizer=tokenizer)
output = pipeline(
prompt,
do_sample=True,
temperature=0.7,
top_p=0.9,
num_return_sequences=1,
max_new_tokens=100,
)
# Print the generated response
print(output[0]["generated_text"])
Limitations and Biases
As with any large language model, the Neural-phi2 model may exhibit biases present in its training data, such as societal biases or factual inaccuracies. Additionally, the model's performance may degrade for tasks or inputs that are significantly different from its training data. Users should carefully evaluate the model's outputs and make appropriate adjustments for their specific use cases.
Performance
The performance of the Neural-phi2 model has not been extensively evaluated or benchmarked as part of this project. Users should conduct their own evaluations to assess the model's suitability for their specific tasks and use cases.
Ethical Considerations
The use of large language models like Neural-phi2 raises several ethical considerations, such as the potential for generating harmful or biased content, the risk of misuse, and the importance of transparency and accountability. Users should carefully consider these ethical implications and take appropriate measures to mitigate potential harms.
- Downloads last month
- 14