metadata
license: mit
datasets:
- databricks/databricks-dolly-15k
language:
- en
library_name: adapter-transformers
pipeline_tag: text-generation
Model Card for TarsDolly
Model Details
TarsDolly is FT on HuggingFaceH4/zephyr-7b-alpha using Dolly-15k dataset for general chat.
Preprocessing
Dolly-15k dataset was converted into chatml style for futher FT on zephyr-7b-beta model
Training Hyperparameters
epochs 4 batch_size 1 block_size 2096 lr 2e-4