README.md

---
language: en
license: apache-2.0
datasets:
- derek-thomas/ScienceQA
- allenai/ai2_arc
tags:
- education
- stem
- computer science
- data science
- engineering
- biology
- chemistry
---


# STEMerald-2b

**Model name:** STEMerald-2b

**Model description:** 
STEMerald-2b is a fine-tuned version of the Gemma-2b model, designed specifically for answering university-level STEM multiple-choice questions. This model leverages advanced fine-tuning techniques, including Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), to enhance its accuracy and reliability in providing educational support.

<p align="center">
    <img src="STEMerald_pic.jpeg" alt="STEMerald picture" width="400"/>
</p>

## Model Details

**Base Model:** [Gemma-2b](https://arxiv.org/abs/2403.08295)

**Architecture:** Decoder-only Language Model (Causal)

**Parameters:** 2.51 billion

**Quantized Version:** STEMerald-2b-4bit (with 4-bit NormalFloat)

**Training Framework:** PyTorch with Hugging Face Transformers

## Datasets

The model was fine-tuned on a variety of datasets tailored for STEM education, including:

- **EPFL Preference Pairs Dataset:** 1522 university-level STEM questions with 26k preference pairs, annotated by students using ChatGPT-3.5 with Chain-of-Thought (CoT).
- **Stack Exchange Dataset:** Questions and answers from various topics such as math, computer science, and engineering.
- **Orca-Math:** 200k grade-school math word problems to enhance reasoning capabilities.
- **EPFL MCQA Dataset**: Dataset of multiple-choice questions with explanation (for CoT) extracted from the winning pairs of EPFL preference pairs.
- **ScienceQA:** Multiple-choice questions on biology, physics, chemistry, economics, earth science, and engineering practices.
- **AI2 Reasoning Challenge (ARC):** Grade-school level multiple-choice science questions.

## Training Process

The training process for STEMerald-2b involved multiple steps:

1. **Supervised Fine-Tuning (SFT):** Initial training on datasets like Orca-Math to improve reasoning abilities.
2. **Direct Preference Optimization (DPO):** Training on preference pairs from EPFL and Stack Exchange datasets to align model outputs with preferred answers.
3. **MCQA Fine-Tuning:** Specialization for multiple-choice question answering using datasets like ScienceQA and ARC.

## Performance

The performance of STEMerald-2b was evaluated using various metrics:

- **Accuracy:** The model achieved high accuracy across multiple test sets, demonstrating its effectiveness in answering STEM questions.
- **Qualitative Evaluation:** The model's answers were evaluated for logical consistency, truthfulness, clarity, and coherence with the final answer.

### Results

| Model Version                     | Accuracy (Non-Quantized) | Accuracy (Quantized) |
|-----------------------------------|--------------------------|----------------------|
| it-ORCA-DPO-MCQA _(STEMerald-2b)_ | 0.750                    | 0.720                |
| it-DPO-MCQA                       | 0.744                    | 0.720                |
| it-MCQA                           | 0.736                    | 0.700                |
| it-ORCA-MCQA                      | 0.722                    | 0.714                |
| MCQA                              | 0.702                    | 0.654                |
| DPO-MCQA                          | 0.694                    | 0.674                |
| Gemma-it-OneShot                  | 0.546                    | 0.520                |
| Gemma-it                          | 0.518                    | 0.518                |

Micro-averaged accuracy over three MCQA test sets(EPFL MCQA, ScienceQA and ARC).

## Use Cases

STEMerald-2b can be utilized as a STEM course assistant, providing support in areas such as:

- Answering university-level multiple-choice STEM questions.
- Offering detailed explanations and reasoning for answers.
- Enhancing student engagement and learning efficiency during independent studies.

## Ethical Considerations

While STEMerald-2b aims to provide accurate and helpful responses, it is important to consider potential ethical implications:

- **Over-Reliance:** Students might become overly dependent on the model for answers, potentially affecting their independent learning and problem-solving skills.
- **Accuracy:** Although efforts were made to ensure the truthfulness of responses, there is still a possibility of incorrect answers. Teacher supervision is crucial.

## Limitations

- The model's performance may vary based on the specific context and nature of the questions.
- Quantization reduces memory footprint but may slightly affect accuracy.

## Conclusion

STEMerald-2b offers a promising solution for enhancing STEM education through advanced language model capabilities. By leveraging fine-tuning techniques and comprehensive datasets, it aims to provide accurate and accessible learning support for students.

## How to Use

You can use the model directly with the `transformers` library:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("matsant01/STEMerald-2b")
model = AutoModelForCausalLM.from_pretrained("matsant01/STEMerald-2b")

input_text = "Question: What is the derivative of x^2? \nOptions: A. 4x B. 2*x^2 C. 2x D. 2\nAnswer:"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

For the quantized version, use:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4"
)

tokenizer = AutoTokenizer.from_pretrained("matsant01/STEMerald-2b-4bit")
model = AutoModelForCausalLM.from_pretrained("matsant01/STEMerald-2b-4bit", quantization_config=quantization_config)
```


## Acknowledgements

We acknowledge the contributions of the EPFL and Stack Exchange communities for their invaluable datasets, and the Hugging Face team for their support and tools that made this project possible.

## Contact

For any questions or feedback, please contact:
- [Antonio Mari](https://github.com/antoniomari) (antonio.mari@epfl.ch)
- [Matteo Santelmo](https://github.com/matsant01) (matteo.santelmo@epfl.ch)
- [Stefano Viel](https://github.com/stefanoviel) (stefano.viel@epfl.ch)