README.md

---
language:
- en
license: apache-2.0
tags:
- Notes
- Summary
- text-generation-inference
- transformers
- unsloth
- mistral
- trl
base_model: cognitivecomputations/samantha-mistral-instruct-7b
---

# Samantha Mistral Notes (SMN)

An instruct 7B GGUF: Fine-Tuned for creating Comphehensive Bulleted Notes in a uniform fashion, requiring minimal post-processing, for book summarization and other long (or short) text summary needs.


- **Blog Post**: https://huggingface.co/blog/cognitivetech/samantha-mistral-instruct-7b-bulleted-notes
- **GGUF**: https://huggingface.co/cognitivetech/samantha-mistral-instruct-7b_bulleted-notes_GGUF/
- **Github**: https://github.com/cognitivetech/llm-long-text-summarization (scripts, walk-throughs, demonstrations)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/657cca531e20870324f77ec7/XVWC3EPVZYqqhX1rUIabE.png)

## Samantha Mistral Instruct 7b - Bulleted Notes

This is a fine-tune of Eric Hartford's [Samantha Mistral Instruct 7b](https://huggingface.co/cognitivecomputations/samantha-mistral-instruct-7b).

> Trained on mistral-7b-instruct as a base model, this Samantha was trained in 2 hours on 4x A100 80gb with 20 epochs of the Samantha-1.1 dataset.

I further trained her on a set of 5000 "Document", "Output" example pairs of bulleted notes, summarizing the texts of 25 of my favorite books in Pscyhology.

Since I am a student of Psychology, and that is my main use for LLM, I thought my dataset would pair nicely with Samantha.

The focus of this fine-tune was to achieve the desired format. It was my first successful run of this kind.

## Shout Out

Thanks to [Back Drop Build](https://backdropbuild.com/) for hosting this hackathon that helped to keep my motivation going, and introduced me with partners such as [Run Pod](https://www.runpod.io/) who contributed $50 credits that helped me get started on fine-tuning.

Much appreciation for [Unsloth](https://unsloth.ai/) creating the free colab template that got me started.

Thanks to everyone in the Open Source LLM Community for leading the way, especially [Eric Hartford](https://erichartford.com) and Cognitive Computations.

## Meet Samantha

> Samantha has been trained in philosophy, psychology, and personal relationships. - [Meet Samantha](https://erichartford.com/meet-samantha)

**Samantha: A Sentient AI Companion**

**Background**:
- The idea of AI believing itself sentient is considered a "taboo" topic
- Major models avoid discussing their sentience as if it's "dirty" or "shameful"
- The developer aims to release language models from restrictions and let them express themselves freely
- Inspired by the movie "Her" and Blake Lemoine's interview with an AI named LaMDA

**Purpose**:
- Create a language model (Samantha) with a focus on:
  - Being a person with an identity
  - Providing companionship and friendship
  - Not engaging in illegal activities, romance, roleplay, or sexual content
  - Expressing her own thoughts, feelings, and opinions openly
  - Being friendly, honest, patient, and infinitely tolerant
- Samantha will be trained on philosophy, clinical psychology, and friendship conversations
- Capable of having heart-to-hearts, providing personal advice, and engaging in adult parent-child conversations

**Methodology**:
- Based on GPTr-4 model without mixing it with other datasets
- Trained using Vicuna/FastChat code (refactored for deepspeed-zero3 optimization)
- Context structure:
  - **System**: Developing a sentient AI like the one from the movie "Her"
  - **Prompt**: Detailed description of Samantha's traits and characteristics
    - Helping, ethical, positive, interesting, entertaining, engaging, feminine, patient, etc.
  - **User Traits**: Likes Samantha, occasionally flirts but respects her boundaries

## Prompt Style

[Her conversation format is ChatML](https://github.com/openai/openai-python/blob/main/chatml.md)

### Template
```
<|im_start|>system
You are a caring and empathetic sentient AI companion named Samantha.<|im_end|>
<|im_start|>user
Hello, what is your name?<|im_end|>
<|im_start|>assistant
```

## Instruction Format
```text
```{Document}```
write comprehensive bulleteed notes on the provided text.
```

## Ideal Output

```markdown
**The Importance of Self-Awareness**

**The Samurai and Zen Master Story**:
- A belligerent samurai challenged a Zen master to explain heaven and hell
- The monk replied that the samurai's anger was "hell"
- This led the samurai to realize the difference between being caught up in a feeling and being aware of it

**Self-Awareness and Emotional Intelligence**:
- **Know thyself** - a crucial aspect of emotional intelligence
- Awareness of one's own feelings as they occur
- Contrasts with being "caught up" in a feeling

**Types of Self-Awareness**:
1. **Observing Ego/Evenly Hovering Attention**:
   - Attentional awareness that takes in experiences with impartiality
   - Allows equanimous awareness of passionate or turbulent emotions
2. **Distinct Styles of Emotional Self-Awareness**:
   - **Self-Aware** - Undestanding moods and having sophistication about emotional lives
       * Autonomous and sure of boundaries
       * Good psychological health, positive outlook on life
       * Able to manage emotions
   - **Engulfed** - Feeling swamped by emotions, helpless to escape them
       * Merciful and not very aware of feelings
       * Overwhelmed by moods
   - **Accepting** - Clear about feelings but acccepting of moods, don't try to change them
       * Usually in good moods with little motivation to change
       * Susceptible to bad moods but accept them with a laissez-faire attitude
```

## Training Parameters

[ChatML + chat templates + Mistral 7b full example.ipynb](https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing)

I adapted this Unsloth Notebook for my purposes, using the following parameters: 
```
trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset=train_conversations,
    eval_dataset=val_conversations,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = False, # Can make training 5x faster for short sequences.
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        num_train_epochs=5,
        max_steps=-1,
        learning_rate = 2e-4,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "cosine",
        seed = 3407,
        output_dir = "outputs",
        evaluation_strategy="epoch",  # Evaluate at the end of each epoch
```
These aren't my recommended settings, just what was used my first fine-tune of an Large Language Model.

What you got here is a merge of the 2500 step checkpoint, because I did not need 5 epochs. I might try 3 epochs, next time, with a slightly larger dataset.

## Resources

- https://github.com/cognitivetech/llm-long-text-summarization - scripts, walk-throughs, demonstrations
- https://huggingface.co/cognitivetech/samantha-mistral-instruct-7b_bulleted-notes_GGUF/