Zephyr-Summarizer-3B

---

1. Introduction

Zephyr-Summarizer-3B produces faithful, abstractive summaries across documents, dialogues and transcripts.

2. Evaluation Results

Comprehensive Benchmark Results

	Benchmark	BriefLM	Zephyr-1B	Condenser	Zephyr-Summarizer-3B
Core Reasoning Tasks	Math Reasoning	0.613	0.638	0.643	0.660
	Logical Reasoning	0.808	0.827	0.819	0.850
	Common Sense	0.770	0.774	0.744	0.796
Language Understanding	Reading Comprehension	0.736	0.770	0.744	0.776
	Question Answering	0.604	0.639	0.632	0.664
	Text Classification	0.806	0.797	0.793	0.850
	Sentiment Analysis	0.774	0.792	0.809	0.825
Generation Tasks	Code Generation	0.706	0.726	0.744	0.752
	Creative Writing	0.699	0.703	0.677	0.726
	Dialogue Generation	0.676	0.690	0.671	0.720
	Summarization	0.762	0.794	0.796	0.814
Specialized Capabilities	Translation	0.798	0.803	0.797	0.831
	Knowledge Retrieval	0.695	0.699	0.673	0.732
	Instruction Following	0.792	0.803	0.771	0.808
	Safety Evaluation	0.777	0.777	0.751	0.790

Overall Performance Summary

The Zephyr-Summarizer-3B demonstrates strong performance across all evaluated benchmark categories, with particularly notable results in reasoning and generation tasks.

3. Chat Website & API Platform

We offer a chat interface and API for you to interact with Zephyr-Summarizer-3B. Please check our official website for more details.

4. How to Run Locally

Please refer to our code repository for more information about running Zephyr-Summarizer-3B locally.

Temperature

We recommend setting the temperature parameter to 0.6.

5. License

This repository is released under the apache-2.0 license. The model supports commercial use.

6. Contact

If you have any questions, please contact us at info@zephyr.works.