metadata
library_name: transformers
license: apache-2.0
datasets:
- Locutusque/hercules-v4.5
language:
- en
inference:
parameters:
do_sample: true
temperature: 1
top_p: 0.7
top_k: 4
max_new_tokens: 250
repetition_penalty: 1.1
Hercules-phi-2
We fine-tuned phi2 on Locutusque's Hercules-v4.5.
Model Details
Model Description
This model has capabilities in math, coding, function calling, roleplay, and more. We fine-tuned it using all examples of Hercules-v4.5.
- Developed by: M4-ai
- Language(s) (NLP): English
- License: apache-2.0
Uses
General purpose assistant, question answering, chain-of-thought, etc..
Evaluation
Coming soon
Training Details
Training Data
https://huggingface.co/datasets/Locutusque/hercules-v4.5
Training Hyperparameters
- Training regime: bf16 non-mixed precision
Technical Specifications
Hardware
We used 8 Kaggle TPUs, and we trained at a global batch size of 1152.