Mandoline

Mandoline helps developers evaluate and improve LLM applications in ways that matter to users.

Create custom metrics that align with your specific use case, evaluate LLM performance in real situations, and track improvements over time.

Documentation

• Getting Started
• Core Concepts
• API Reference

Tutorials

• Multimodal Evaluation: Evaluate LLMs Across Text and Vision Tasks
• Model Selection: Comparing LLMs for Creative Tasks
• Prompt Engineering: Reduce Unwanted LLM Behaviors

Analysis & Insights

• Multimodal Language Model Evaluation: A Creative Coding Challenge
• Refusal Rates in Open-Source vs. Proprietary Language Models
• Comparing Refusal Behavior Across Top Language Models

Leaderboards

• Refusals

SDKs

• Node.js
• Python

Support

• GitHub Issues
• Email