| license: mit | |
| datasets: | |
| - databricks/databricks-dolly-15k | |
| language: | |
| - en | |
| library_name: adapter-transformers | |
| pipeline_tag: text-generation | |
| # Model Card for TarsDolly | |
| ## Model Details | |
| TarsDolly is FT on HuggingFaceH4/zephyr-7b-alpha using Dolly-15k dataset for general chat. | |
| #### Preprocessing | |
| Dolly-15k dataset was converted into chatml style for futher FT on zephyr-7b-beta model | |
| #### Training Hyperparameters | |
| epochs 4 | |
| batch_size 1 | |
| block_size 2096 | |
| lr 2e-4 |