README.md

---
language:
- en
license: other
tags:
- AI
- ConversationalAI
pipeline_tag: conversational
inference: false
model-index:
- name: LLmRa-1.3B_V2
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 30.46
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=L-R/LLmRa-1.3B_V2
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 53.03
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=L-R/LLmRa-1.3B_V2
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 26.06
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=L-R/LLmRa-1.3B_V2
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 36.46
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=L-R/LLmRa-1.3B_V2
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 59.27
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=L-R/LLmRa-1.3B_V2
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 0.0
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=L-R/LLmRa-1.3B_V2
      name: Open LLM Leaderboard
---

<h1 style="text-align: center">LLmRa-1.3B-V2</h1>
<h2 style="text-align: center">A conversational Open Pre-trained Transformer Language Model fine-tune.</h2>

**LLmRa 1.3B-V2**, as a proof-of-concept fine-tune of [facebook/opt-1.3b](https://huggingface.co/facebook/opt-1.3b) optimized for dialogue.

**Disclaimer:** NSFW data was included in the fine-tuning of this model. Although SFW inputs will usually result in SFW outputs, you are advised to **chat at your own risk. This model is not suitable for use by minors.**

**Warning:** This model is **NOT** suitable for use by minors. **It will output X-rated content under certain circumstances.**

**Model Fine-Tuned on LLmRa-100K conversational dataset - small version**

---

## Usage Format

To effectively utilize the model, follow this structured format for engaging text-based conversations:

**1. Initialization**

Here is how you can define the personality of the language model:

```
<|system|>[Persona]
```

- **Persona**: You can define a specific persona or context for the AI, but it's optional. It can be a character, a role, or just a style of interaction.

**2. AI Introduction**

```
<|user|>[User input]<|model|>
```
- Users can start the conversation by entering their message within `<|user|>` and closing with `<|model|>`.

---

### Example Usage:

Here's an example of how to start a conversation with the AI:

```
<|system|>I'm here to provide information and assistance on a wide range of topics.
<|model|>Hello! Welcome to our AI-powered assistant. How can I assist you today?
<|user|>Tell me about the history of artificial intelligence.
<|model|>
```

Continue the conversation as needed. This structured format helps maintain a smooth and engaging interaction with the AI.

You are not required to include `User`, you can change it to your prefered name or leave it blank You may also add the AI name, example:

```
<|user|>YourNameHere: Hello.<|model|>CharacterName:
```

You can also use this instruct prompt example:

```
<|system|>What is one plus one?<|model|>
```

## Loading The Model

To use the model and interact with it, use the Python code below:

```Python
from transformers import (AutoModelForCausalLM,
                          AutoTokenizer,
                          pipeline,
                          )

model = AutoModelForCausalLM.from_pretrained('L-R/LLmRa-1.3B-V2')
tokenizer = AutoTokenizer.from_pretrained('L-R/LLmRa-1.3B-V2')

pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=100)

input_question = 'QUESTION HERE'

question_formatted = f'<|system|>{input_question}<|model|>'

result = pipe(question_formatted)

print(f"[model]: {result[0]['generated_text'][len(question_formatted):]}")
```

## Known issues

Model doesn't some of the times follow instructions.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_L-R__LLmRa-1.3B_V2)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |34.21|
|AI2 Reasoning Challenge (25-Shot)|30.46|
|HellaSwag (10-Shot)              |53.03|
|MMLU (5-Shot)                    |26.06|
|TruthfulQA (0-shot)              |36.46|
|Winogrande (5-shot)              |59.27|
|GSM8k (5-shot)                   | 0.00|