Update README.md

README.md

@@ -13,7 +13,7 @@ datasets:
 - SkunkworksAI/reasoning-0.01
 ---
 
-# Uploaded  model
+# Uploaded model
 
 - **Developed by:** colesmcintosh
 - **License:** apache-2.0
@@ -21,4 +21,124 @@ datasets:
 
 This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+
+---
+
+## Model Details:
+- **Base Model**: unsloth/llama-3.2-1b-instruct-bnb-4bit
+- **Training Dataset**: [SkunkworksAI/reasoning-0.01](https://huggingface.co/datasets/SkunkworksAI/reasoning-0.01) – Chain-of-thought reasoning dataset with 29.9k examples to improve the model's ability to solve reasoning problems step-by-step.
+- **Techniques**:
+  - **LoRA (Low-Rank Adaptation)**: Fine-tuning to enhance conversational abilities without overfitting.
+  - **QLoRA (4-bit Quantization)**: Used for reducing model size and speeding up inference times without sacrificing too much accuracy.
+  - **RoPE Scaling**: To handle long-sequence token inputs effectively (up to 64k tokens).
+
+---
+
+## Training Details:
+
+The fine-tuning process was conducted using the `SFTTrainer` class from the `trl` library, which is optimized for training transformer models using reinforcement learning techniques. The training process was structured as follows:
+
+### Training Configuration:
+```python
+from trl import SFTTrainer
+from transformers import TrainingArguments, DataCollatorForSeq2Seq
+from unsloth import is_bfloat16_supported
+
+trainer = SFTTrainer(
+    model = model,
+    tokenizer = tokenizer,
+    train_dataset = dataset,
+    dataset_text_field = "text",
+    max_seq_length = max_seq_length,
+    data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),
+    dataset_num_proc = 2,
+    packing = False, # Can make training 5x faster for short sequences.
+    args = TrainingArguments(
+        per_device_train_batch_size = 2,
+        gradient_accumulation_steps = 4,
+        warmup_steps = 5,
+        max_steps = 60,
+        learning_rate = 2e-4,
+        fp16 = not is_bfloat16_supported(),
+        bf16 = is_bfloat16_supported(),
+        logging_steps = 1,
+        optim = "adamw_8bit",
+        weight_decay = 0.01,
+        lr_scheduler_type = "linear",
+        seed = 3407,
+        output_dir = "outputs",
+    ),
+)
+```
+
+### Key Training Parameters:
+- **Batch Size**: `2` per device
+- **Gradient Accumulation Steps**: `4` to accumulate gradients over multiple forward passes, allowing for effective training with smaller batch sizes.
+- **Learning Rate**: `2e-4` with a linear decay schedule.
+- **Warmup Steps**: `5` steps used to gradually increase the learning rate at the start of training.
+- **Max Training Steps**: `60` total training steps.
+- **FP16/BF16 Precision**: The model uses FP16 unless BF16 is supported, in which case it switches to BF16 precision for faster training on GPUs that support it.
+- **Optimizer**: `adamw_8bit` – Adam optimizer with 8-bit memory-efficient operations, which reduces GPU memory usage during training.
+- **Weight Decay**: `0.01` for regularization, preventing the model from overfitting.
+
+### Dataset:
+- **Dataset Used for Training**: [SkunkworksAI/reasoning-0.01](https://huggingface.co/datasets/SkunkworksAI/reasoning-0.01) – The dataset contains **29.9k examples** of chain-of-thought reasoning instruction/output pairs.
+  
+### Collation Strategy:
+- **Data Collator**: `DataCollatorForSeq2Seq` is used to handle padding and tokenization efficiently, ensuring sequences are of the correct length during training.
+
+---
+
+## Inference Example:
+
+To run inference using the fine-tuned model, follow this code snippet:
+
+```python
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="colesmcintosh/Llama-3.1-8B-Instruct-Mango",
+    max_seq_length=64000,
+    dtype=None,  # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
+    load_in_4bit=True
+)
+
+# Enable native 2x faster inference
+FastLanguageModel.for_inference(model)
+
+# Prepare the input message
+messages = [
+    {"role": "user", "content": "Describe a tall tower in the capital of France."},
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,  # Must add for generation
+    return_tensors="pt",
+).to("cuda")
+
+from transformers import TextStreamer
+text_streamer = TextStreamer(tokenizer, skip_prompt=True)
+
+# Generate response from the model
+_ = model.generate(input_ids=inputs, streamer=text_streamer, max_new_tokens=128, use_cache=True, temperature=1.5, min_p=0.1)
+```
+
+This snippet will generate a response based on the input message and run inference using the `FastLanguageModel` class with the optimizations included in the `Unsloth` framework.
+
+You can also use Hugging Face's AutoModelForPeftCausalLM. Only use this if you do not have unsloth installed. It can be hopelessly slow, since 4bit model downloading is not supported, and Unsloth's inference is 2x faster.
+```python
+from peft import AutoPeftModelForCausalLM
+from transformers import AutoTokenizer
+model = AutoPeftModelForCausalLM.from_pretrained(
+    "colesmcintosh/Llama-3.1-8B-Instruct-Mango", # YOUR MODEL YOU USED FOR TRAINING
+    load_in_4bit = load_in_4bit,
+)
+tokenizer = AutoTokenizer.from_pretrained("colesmcintosh/Llama-3.1-8B-Instruct-Mango")
+```
+I highly do NOT suggest - use Unsloth if possible!
+
+## Additional Information
+If you have any questions, feedback, or would like to collaborate on projects using this model, feel free to reach out to me on [LinkedIn](https://www.linkedin.com/in/cole-mcintosh) or visit my website at [colemcintosh.io](https://colemcintosh.io/). I’m always open to discussing AI, model development, and innovative solutions!