File size: 3,733 Bytes
04f5373 0157d20 ffff1f8 4743f67 ffff1f8 c87cebd ffff1f8 c87cebd ffff1f8 4743f67 ffff1f8 4743f67 ffff1f8 4743f67 ffff1f8 ead8392 ffff1f8 b4fb33d ffff1f8 b4fb33d ffff1f8 b4fb33d ffff1f8 4743f67 ffff1f8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
license: apache-2.0
language:
- en
base_model:
- meta-llama/Llama-3.1-8B-Instruct
pipeline_tag: text-generation
---
# Cat1.0
## Overview
Cat1.0 is a fine-tuned version of **Llama-3-1-8b base model**, optimized for roleplaying, logic, and reasoning tasks. Utilizing iterative fine-tuning and human-AI chat logs, this model works well for numerous chat scenarios.
## Model Specifications
- **Parameters**: 8 Billion (8B)
- **Precision**: bf16 (Brain Floating Point 16-bit)
- **Fine-Tuning Method**: LoRa (Low-Rank Adaptation)
- **Lora Rank**: 32
- **Lora Alpha**: 64
- **Learning Rate**: 0.0008
- **Training Epochs**: 4
- **Datasets Used**:
- cat1.0 Roleplay Dataset
- cat1.0 Reasoning and Logic Dataset
- **Fine-Tuning Approach**: Iterative Fine-Tuning using self-chat logs
## Recommended Settings
To achieve optimal performance with this model, I recommend the following settings:
- **Minimum Probability (`min_p`)**: `0.05`
- **Temperature**: `1.1` or higher
> **Note**: Due to the nature of the fine-tuning, setting the temperature to `1.1` or higher helps prevent the model from repeating itself and encourages more creative and coherent responses.
## Usage Instructions
I recommend using the [oobabooga text-generation-webui](https://github.com/oobabooga/text-generation-webui) for an optimal experience. Load the model in `bf16` precision and enable `flash-attention2` for improved performance.
### Installation Steps
1. **Clone the WebUI Repository**:
```bash
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
```
2. **Install Dependencies**:
```bash
pip install -r requirements.txt
```
3. **Download the Model**:
Download the fine-tuned model from [Hugging Face](#) and place it in the `models` directory.
4. **Launch the WebUI**:
```bash
python server.py --bf16 --flash-attention
```
### Sample Prompt Formats
You can interact with the model using either **chat format** or **chat-instruct format**. Here's an example:
```plaintext
Ryan is a computer engineer who works at Intel.
Ryan: Hey, how's it going Natalie?
Natalie: Good, how are things going with you, Ryan?
Ryan: Great, I'm just doing just great.
```
### Text Generation Example
## Model Capabilities
Below are some examples showcasing the model's performance in various roleplay scenarios:
### Roleplay Examples
## Limitations and Tips
While this model excels in chat and roleplaying scenarios, it isn't perfect. If you notice the model repeating itself or providing less coherent responses:
- **Increase the Temperature**: Setting the temperature higher (≥ `1.1`) can help generate more diverse and creative outputs.
- **Adjust `min_p` Setting**: Ensuring `min_p` is at least `0.05` can prevent low-probability tokens from being excluded, enhancing the response quality.
## Acknowledgments
- **oobabooga text-generation-webui**: A powerful interface for running and interacting with language models. [GitHub Repository](https://github.com/oobabooga/text-generation-webui)
- **Hugging Face**: For hosting the model and providing a platform for collaboration. [Website](https://huggingface.co/)
- **Meta** For pre-training the Llama-3.1-8B Base Model that was used for fine-tuning. [Model Card](https://huggingface.co/meta-llama/Llama-3.1-8B)
*For any issues or questions, please open an issue in this repository.* |