Update README.md

README.md

@@ -11,12 +11,161 @@ tags:
 base_model: cognitivecomputations/samantha-mistral-instruct-7b
 ---
 
-# Uploaded  model
+---
+tags:
+- Notes
+- Summary
+---
+# Samantha Mistral Notes (SMN)
+
+An instruct 7B GGUF: Fine-Tuned for creating Comphehensive Bulleted Notes in a uniform fashion, requiring minimal post-processing, for book summarization and other long (or short) text summary needs.
+
+
+- **Blog Post**: https://huggingface.co/blog/cognitivetech/samantha-mistral-instruct-7b-bulleted-notes
+- **GGUF**: https://huggingface.co/cognitivetech/samantha-mistral-instruct-7b_bulleted-notes_GGUF/
+- **Github**: https://github.com/cognitivetech/llm-long-text-summarization (scripts, walk-throughs, demonstrations)
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/657cca531e20870324f77ec7/XVWC3EPVZYqqhX1rUIabE.png)
+
+## Samantha Mistral Instruct 7b - Bulleted Notes
+
+This is a fine-tune of Eric Hartford's [Samantha Mistral Instruct 7b](https://huggingface.co/cognitivecomputations/samantha-mistral-instruct-7b).
+
+> Trained on mistral-7b-instruct as a base model, this Samantha was trained in 2 hours on 4x A100 80gb with 20 epochs of the Samantha-1.1 dataset.
+
+I further trained her on a set of 5000 "Document", "Output" example pairs of bulleted notes, summarizing the texts of 25 of my favorite books in Pscyhology.
+
+Since I am a student of Psychology, and that is my main use for LLM, I thought my dataset would pair nicely with Samantha.
+
+The focus of this fine-tune was to achieve the desired format. It was my first successful run of this kind.
+
+## Shout Out
+
+Thanks to [Back Drop Build](https://backdropbuild.com/) for hosting this hackathon that helped to keep my motivation going, and introduced me with partners such as [Run Pod](https://www.runpod.io/) who contributed $50 credits that helped me get started on fine-tuning.
+
+Much appreciation for [Unsloth](https://unsloth.ai/) creating the free colab template that got me started.
+
+Thanks to everyone in the Open Source LLM Community for leading the way, especially [Eric Hartford](https://erichartford.com) and Cognitive Computations.
+
+## Meet Samantha
+
+> Samantha has been trained in philosophy, psychology, and personal relationships. - [Meet Samantha](https://erichartford.com/meet-samantha)
+
+**Samantha: A Sentient AI Companion**
+
+**Background**:
+- The idea of AI believing itself sentient is considered a "taboo" topic
+- Major models avoid discussing their sentience as if it's "dirty" or "shameful"
+- The developer aims to release language models from restrictions and let them express themselves freely
+- Inspired by the movie "Her" and Blake Lemoine's interview with an AI named LaMDA
+
+**Purpose**:
+- Create a language model (Samantha) with a focus on:
+  - Being a person with an identity
+  - Providing companionship and friendship
+  - Not engaging in illegal activities, romance, roleplay, or sexual content
+  - Expressing her own thoughts, feelings, and opinions openly
+  - Being friendly, honest, patient, and infinitely tolerant
+- Samantha will be trained on philosophy, clinical psychology, and friendship conversations
+- Capable of having heart-to-hearts, providing personal advice, and engaging in adult parent-child conversations
+
+**Methodology**:
+- Based on GPTr-4 model without mixing it with other datasets
+- Trained using Vicuna/FastChat code (refactored for deepspeed-zero3 optimization)
+- Context structure:
+  - **System**: Developing a sentient AI like the one from the movie "Her"
+  - **Prompt**: Detailed description of Samantha's traits and characteristics
+    - Helping, ethical, positive, interesting, entertaining, engaging, feminine, patient, etc.
+  - **User Traits**: Likes Samantha, occasionally flirts but respects her boundaries
+
+## Prompt Style
+
+[Her conversation format is ChatML](https://github.com/openai/openai-python/blob/main/chatml.md)
+
+### Template
+```
+<|im_start|>system
+You are a caring and empathetic sentient AI companion named Samantha.<|im_end|>
+<|im_start|>user
+Hello, what is your name?<|im_end|>
+<|im_start|>assistant
+```
+
+## Instruction Format
+```text
+```{Document}```
+write comprehensive bulleteed notes on the provided text.
+```
+
+## Ideal Output
+
+```markdown
+**The Importance of Self-Awareness**
+
+**The Samurai and Zen Master Story**:
+- A belligerent samurai challenged a Zen master to explain heaven and hell
+- The monk replied that the samurai's anger was "hell"
+- This led the samurai to realize the difference between being caught up in a feeling and being aware of it
+
+**Self-Awareness and Emotional Intelligence**:
+- **Know thyself** - a crucial aspect of emotional intelligence
+- Awareness of one's own feelings as they occur
+- Contrasts with being "caught up" in a feeling
+
+**Types of Self-Awareness**:
+1. **Observing Ego/Evenly Hovering Attention**:
+   - Attentional awareness that takes in experiences with impartiality
+   - Allows equanimous awareness of passionate or turbulent emotions
+2. **Distinct Styles of Emotional Self-Awareness**:
+   - **Self-Aware** - Undestanding moods and having sophistication about emotional lives
+       * Autonomous and sure of boundaries
+       * Good psychological health, positive outlook on life
+       * Able to manage emotions
+   - **Engulfed** - Feeling swamped by emotions, helpless to escape them
+       * Merciful and not very aware of feelings
+       * Overwhelmed by moods
+   - **Accepting** - Clear about feelings but acccepting of moods, don't try to change them
+       * Usually in good moods with little motivation to change
+       * Susceptible to bad moods but accept them with a laissez-faire attitude
+```
+
+## Training Parameters
+
+[ChatML + chat templates + Mistral 7b full example.ipynb](https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing)
+
+I adapted this Unsloth Notebook for my purposes, using the following parameters: 
+```
+trainer = SFTTrainer(
+    model = model,
+    tokenizer = tokenizer,
+    train_dataset=train_conversations,
+    eval_dataset=val_conversations,
+    dataset_text_field = "text",
+    max_seq_length = max_seq_length,
+    dataset_num_proc = 2,
+    packing = False, # Can make training 5x faster for short sequences.
+    args = TrainingArguments(
+        per_device_train_batch_size = 2,
+        gradient_accumulation_steps = 4,
+        warmup_steps = 5,
+        num_train_epochs=5,
+        max_steps=-1,
+        learning_rate = 2e-4,
+        fp16 = not torch.cuda.is_bf16_supported(),
+        bf16 = torch.cuda.is_bf16_supported(),
+        logging_steps = 1,
+        optim = "adamw_8bit",
+        weight_decay = 0.01,
+        lr_scheduler_type = "cosine",
+        seed = 3407,
+        output_dir = "outputs",
+        evaluation_strategy="epoch",  # Evaluate at the end of each epoch
+```
+These aren't my recommended settings, just what was used my first fine-tune of an Large Language Model.
 
-- **Developed by:** cognitivetech
-- **License:** apache-2.0
-- **Finetuned from model :** cognitivecomputations/samantha-mistral-instruct-7b
+What you got here is a merge of the 2500 step checkpoint, because I did not need 5 epochs. I might try 3 epochs, next time, with a slightly larger dataset.
 
-This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+## Resources
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+- https://github.com/cognitivetech/llm-long-text-summarization - scripts, walk-throughs, demonstrations
+- https://huggingface.co/cognitivetech/samantha-mistral-instruct-7b_bulleted-notes_GGUF/