Repository: confident-ai/deepeval
Issue: add more agentic metric
Root cause: Unimplemented Feature Gap

**❗BEFORE YOU BEGIN❗**  
Are you on discord? 🤗 We'd love to have you asking questions on discord instead: https://discord.com/invite/a3K9c8GRGt  

**Is your feature request related to a problem? Please describe.**  
The deepeval framework is excellent and already includes some Agent-related evaluation metrics, such as task completion and tool correctness.  
However, compared to other evaluation frameworks, the **Agentic metrics** available in deepeval are relatively limited.  
In my work, I often need more fine-grained Agent behavior metrics, such as plan quality, execution efficiency, and multi-dimensional evaluation of tool usage. These are not fully supported by deepeval at present and require integrating other frameworks manually.  

**Describe the solution you'd like**  
I would love to see deepeval natively support more **Agentic evaluation metrics**, borrowing from other frameworks such as **TruLens** and **ragas**. Examples include:  

From **[TruLens](https://github.com/truera/trulens)**:  
- **LogicalConsistency**: Evaluates the logical consistency of the agent's planning and execution.  
- **ExecutionEfficiency**: Evaluates the efficiency of the agent's execution.  
- **PlanAdherence**: Evaluates whether the execution follows the plan.  
- **PlanQuality**: Evaluates the quality of the agent's plan.  
- **ToolSelection**: Evaluates the appropriateness of the agent's tool selection.  
- **ToolCalling**: Evaluates the quality of the agent's tool calling.  
- **ToolQuality**: Evaluates the quality and reliability of the tools themselves.  

From **[ragas](https://github.com/explodinggradients/ragas)**:  
- **Topic Adherence**: Evaluates whether the AI system can adhere to predefined domain scopes and avoid out-of-scope answers.  
- **Tool Call Accuracy**: Evaluates the performance of the LLM in selecting and calling the required tools for a task.  
- **Agent Goal Accuracy**: Evaluates whether the LLM correctly identifies and achieves user goals (binary metric).  

**Describe alternatives you've considered**  
Currently, I have to combine deepeval, TruLens, and ragas to achieve full coverage of the Agentic metrics I need.  
While this approach works, it increases the complexity of the evaluation pipeline and makes it harder to reproduce results.  

**Additional context**  
Adding these metrics into deepeval would greatly increase its appeal to developers building and evaluating complex LLM agents, making it a more complete one-stop evaluation framework.  


@danerlt thanks for sharing, super happy to add it. Something I'd like to understand really quick is do you use all these metrics at once per evaluation?

I don't evaluate all metrics, but I generally pay more attention to such related metrics as whether the tool calls in the Agent's multi-turn conversations are correct, whether the tool parameters are correct, and whether the Agent has completed the user's questions.

Hey @danerlt, we've added new agent metrics for both single and multi-turn. A new release will be out by the end of today, look forward to it 👀 :) 

@danerlt its released!