leaderboard-pr-bot's picture
Adding Evaluation Results
bf317f3 verified
|
raw
history blame
9.56 kB
metadata
license: apache-2.0
library_name: Bunkatopics
datasets:
  - bunkalab/topic_based_chatml_dpo_pairs
base_model: teknium/OpenHermes-2.5-Mistral-7B
widget:
  - text: Tell a danish joke in french
pipeline_tag: text-generation
model-index:
  - name: TopicNeuralHermes-2.5-Mistral-7B
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AI2 Reasoning Challenge (25-Shot)
          type: ai2_arc
          config: ARC-Challenge
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: acc_norm
            value: 67.06
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: HellaSwag (10-Shot)
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: acc_norm
            value: 85.44
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU (5-Shot)
          type: cais/mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 63.66
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: TruthfulQA (0-shot)
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: mc2
            value: 55.47
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Winogrande (5-shot)
          type: winogrande
          config: winogrande_xl
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 78.3
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GSM8k (5-shot)
          type: gsm8k
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 54.21
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B
          name: Open LLM Leaderboard

image/png

Model description

TopicNeuralHermes 2.5 Mistral 7B is a refined model developed through fine-tuning with a specific subset of data, selected via Topic Modeling Techniques using Bunkatopics, as a continuing from OpenHermes 2.5.

The model was trained on a refined DPO dataset. The objective was to train the model on a small portion of the DPO data. To achieve this, we compared two datasets used to train the reward model: the rejected Llama answers and the accepted ChatGPT answers from the DPO dataset. We then conducted topic modeling on both datasets, keeping only the topics that existed in the accepted dataset but not in the rejected one. Our hypothesis is that these topics encapsulate the main differences between the two answering styles.

This method allows for quicker convergence with significantly less data (around 1/6 of the initial dataset). The Dataset can be found at bunkalab/topic_based_chatml_dpo_pairs

Special thanks to mlabonne for creating the colab notebook that facilitated the DPO Strategy.

Results of the model can be found here: We do as well as similar models with way less data and computing power :)

image/png

Topic Analysis

We applied the topic modeling method to both datasets, extracting 30 topics from each. These topics were characterized using the 10 most specific unigrams or bigrams. We then compared the two sets of topics (30 from each dataset) and retained those in the accepted dataset that shared fewer than 2 terms with any topic in the rejected dataset

We found the 13 distinctive following topics described by 10 terms each:

Emotional Dynamics: feelings, Quinn, Austin, minority women, teaching, schools, individual, personality, backgrounds, triggers.

Global Knowledge Queries: question, information, geography, news articles, Step, answer, capital city, pipeline system, country, analogy.

Digital Interactions and Queries: questions, question, PersonX, modem, answers, effect relationship, Quora, browser, answer, e-commerce.

Business and Cybersecurity: email, businesses, initiatives, innovation, advertising papers, spam, breaches, antivirus, payments, prospects.

Lifestyle and Wellness: sleep, exercise, gifts, shopping, Casey, stores, stress, headaches, options, mood.

Wildlife Ecology: birds, prey, animals, species, infection, nest, eggs, bacteria, insects, kitty condo.

Environmental Science and Climate: temperature, gases, greenhouse, emissions, perturbation, sulfur, dioxide, climate change, water, heat.

Maritime and Mechanical Engineering: ship, bowling, propulsion, beam width, Filing cabinet, LED, lane, containment area, lawnmower, rotors.

Cultural and Social Dynamics: Lindsey, museum, Kate, Rachel, Jason, Alex, Erin, conversation, Laura, exhibits.

Political Media Analysis: media platforms, election, politics, teenagers, elections, White House, Barack Obama, nation, Confederate, depression.

International Relations and Policy: cooperation, EU, nations, alliance, NATO, European Union, member states, policy, monarch, Brexit.

Astrophysics and Physical Sciences: electrons, km, Moon, acceleration, orbit, friction, current, asteroid, electron, collector emitter.

Film Critique and Analysis: movie review, film, reviewer, sentiment, critic, flaws, DVD, plot, opinion, originality.

While those topics are not domain-specific, they did not appear right away in the rejected dataset. Further research need to undersand the reason behind the prominence of those topics in the accepted dataset.

Usage

You can run this model using LM Studio or any other frontend.

You can also run this model using the following code:

import transformers
from transformers import AutoTokenizer

# Format prompt
message = [
    {"role": "system", "content": "You are a helpful assistant chatbot."},
    {"role": "user", "content": "What is Topic Modeling?"}
]
tokenizer = AutoTokenizer.from_pretrained('charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B')
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model='charlesdedampierre/TopicNeuralHermes-2.5-Mistral-7B',
    tokenizer=tokenizer
)

# Generate text
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    num_return_sequences=1,
    max_length=200,
)
print(sequences[0]['generated_text'])

Training hyperparameters

LoRA:

  • r=16
  • lora_alpha=16
  • lora_dropout=0.05
  • bias="none"
  • task_type="CAUSAL_LM"
  • target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj']

Training arguments:

  • per_device_train_batch_size=4
  • gradient_accumulation_steps=4
  • gradient_checkpointing=True
  • learning_rate=5e-5
  • lr_scheduler_type="cosine"
  • max_steps=200
  • optim="paged_adamw_32bit"
  • warmup_steps=100

DPOTrainer:

  • beta=0.1
  • max_prompt_length=1024
  • max_length=1536

You can find the results of the running on Weights & Biases: https://wandb.ai/bunka/huggingface/runs/xq59p47g?workspace=user-charlesdedampierre

Model Family Tree

image/png

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 67.36
AI2 Reasoning Challenge (25-Shot) 67.06
HellaSwag (10-Shot) 85.44
MMLU (5-Shot) 63.66
TruthfulQA (0-shot) 55.47
Winogrande (5-shot) 78.30
GSM8k (5-shot) 54.21