eurecom-ds/Phi-3-mini-4k-socratic

Socratic LLM

Using Large Language Models (LLMs) in education presents unique challenges. Typically, LLMs are designed to provide direct answers to questions, which can hinder students' critical thinking and self-discovery skills. To address this, we focus on fine-tuning LLMs to facilitate Socratic interactions. Instead of giving straightforward answers, these models guide students to explore and find the answers themselves. We achieve this through Direct Preference Optimization (DPO). We test our approach with diverse datasets, including various educational materials and Socratic dialogues. Using advanced models like GPT-4o for evaluation, our results show that DPO successfully fine-tunes LLMs for Socratic dialogue, enhancing their educational value.

This repository contains the source material for the paper "Fine Tuning a Large Language Model for Socratic Interactions" (accepted at KDD-2024).

Check out training pipeline at GitHub - socratic-llm.

And you can also run it with Ollama: eurecom-ds/phi-3-mini-4k-socratic!

Prompt Format

See Inference template.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import urllib.request

with urllib.request.urlopen("https://raw.githubusercontent.com/GiovanniGatti/socratic-llm/kdd-2024/templates/inference.txt") as f:
    inference_prompt_template = f.read().decode('utf-8')

model = AutoModelForCausalLM.from_pretrained(
  "eurecom-ds/Phi-3-mini-4k-socratic",
  torch_dtype=torch.bfloat16,
  trust_remote_code=True,
  device_map="cuda",
)

tokenizer = AutoTokenizer.from_pretrained("eurecom-ds/Phi-3-mini-4k-socratic", trust_remote_code=True)

_input = "Student: Professor, why did Einstein say that God does not play dice?"

content = inference_prompt_template.format(input=_input)
formatted = tokenizer.apply_chat_template(
   [{"role": "user", "content": content}, ], tokenize=False, add_generation_prompt=True
)
encoded_inputs = tokenizer([formatted, ], return_tensors="pt").to("cuda")

generate_kwargs = dict(encoded_inputs)

output = model.generate(**generate_kwargs)
response = tokenizer.decode(output[0], skip_special_tokens=True)[len(_prompt) + 1:]