Commit
•
75d20ad
1
Parent(s):
602f82f
adding model card
Browse files
README.md
CHANGED
|
@@ -1,8 +1,6 @@
|
|
| 1 |
---
|
| 2 |
-
model_type: Fine-tuned 7B model for chat.
|
| 3 |
license: {apache-2.0}
|
| 4 |
base_model: {openchat/openchat_3.5}
|
| 5 |
-
demo: [Hugging Face Spaces](https://huggingface.co/spaces/tenyx/TenyxChat-7B-v1)
|
| 6 |
---
|
| 7 |
|
| 8 |
# TenyxChat: Language Model Alignment using Tenyx Fine-tuning
|
|
@@ -11,6 +9,12 @@ Introducing TenyxChat, a series of ChatGPT-like models trained to function as us
|
|
| 11 |
|
| 12 |
We fine-tune [Openchat-3.5](https://arxiv.org/pdf/2309.11235.pdf) with our proprietary approach ([blog](https://www.tenyx.com/post/forgetting-and-toxicity-in-llms-a-deep-dive-on-fine-tuning-methods), [service](https://www.tenyx.com/fine-tuning)), which shows an increase in [MT-Bench](https://arxiv.org/abs/2306.05685), without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 ([GitHub](https://github.com/huggingface/alignment-handbook)).
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
## Usage
|
| 16 |
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
license: {apache-2.0}
|
| 3 |
base_model: {openchat/openchat_3.5}
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
# TenyxChat: Language Model Alignment using Tenyx Fine-tuning
|
|
|
|
| 9 |
|
| 10 |
We fine-tune [Openchat-3.5](https://arxiv.org/pdf/2309.11235.pdf) with our proprietary approach ([blog](https://www.tenyx.com/post/forgetting-and-toxicity-in-llms-a-deep-dive-on-fine-tuning-methods), [service](https://www.tenyx.com/fine-tuning)), which shows an increase in [MT-Bench](https://arxiv.org/abs/2306.05685), without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 ([GitHub](https://github.com/huggingface/alignment-handbook)).
|
| 11 |
|
| 12 |
+
# Model details
|
| 13 |
+
|
| 14 |
+
- Model type: Fine-tuned 7B model for chat.
|
| 15 |
+
- License: Apache 2.0
|
| 16 |
+
- Base model: OpenChat 3.5 ([https://huggingface.co/openchat/openchat_3.5](https://huggingface.co/openchat/openchat_3.5))
|
| 17 |
+
- Demo: Hugging face space
|
| 18 |
|
| 19 |
## Usage
|
| 20 |
|