Repository: truera/trulens
Issue: [BUG]Metric calculation and corresponding interpretation
Root cause: Documentation Deficiency

**Bug Description**
What happened?
1）I tried to use other frameworks such as ragas and trulens to calculate context_relevance for my data sets, but the two frameworks gave different results.Is it because of different calculation methods?
2）My understanding is that trulens uses llm to participate in the evaluation. May I ask whether the evaluation of trulens only requires the designation of an llm and does not require the calculation of indicators using the embedding model?
3）Are there any relationships and differences between context_relevance and context_relevance_with_cot_reasons for metrics in trulens？
**To Reproduce**
Which steps should someone take to run into the same error? A small, reproducible code example is useful here.

**Expected behavior**
A clear and concise description of what you expected to happen.

**Relevant Logs/Tracebacks**
Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. If the issue is related to the TruLens dashboard, please also include a screenshot.

**Environment:**
 - OS: [e.g. MacOS, Windows]
 - Python Version
 - TruLens version
 - Versions of other relevant installed libraries

**Additional context**
Add any other context about the problem here.


<!-- Answer -->
1. Ragas and TruLens frameworks provide different results for `context_relevance` because they use different models and methodologies for evaluating context relevance. TruLens uses various models like GPT-3.5, GPT-4, and Claude-2, each with its own implementation of the `context_relevance` feedback function, which can lead to different results due to differences in model architecture, training data, and evaluation techniques <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/base.py)</sup>.

2. TruLens does indeed use LLMs (Large Language Models) to participate in the evaluation. The evaluation in TruLens primarily involves the designation of an LLM and does not require the calculation of indicators using an embedding model <sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/base.py)</sup>.

3. The `context_relevance` metric in TruLens is a straightforward measure of how relevant a given context is to a query. In contrast, `context_relevance_with_cot_reasons` includes additional reasoning steps (chain-of-thought) to justify the relevance score. This additional reasoning can provide more nuanced and potentially more accurate relevance assessments, but it also introduces variability depending on how the reasoning is generated and interpreted by the model <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb)</sup><sup>[[3]](https://github.com/truera/trulens/blob/main/docs/trulens_eval/getting_started/core_concepts/rag_triad.md)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


@dosu is right here. We will be adding new context relevance benchmarks to our [benchmarks page](https://www.trulens.org/trulens_eval/evaluation_benchmarks/) soon.

<!-- Answer -->
1. Ragas and TruLens frameworks provide different results for `context_relevance` because they use different models and methodologies for evaluating context relevance. TruLens uses various models like GPT-3.5, GPT-4, and Claude-2, each with its own implementation of the `context_relevance` feedback function, which can lead to different results due to differences in model architecture, training data, and evaluation techniques [[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb)[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/base.py).

2. TruLens does indeed use LLMs (Large Language Models) to participate in the evaluation. The evaluation in TruLens primarily involves the designation of an LLM and does not require the calculation of indicators using an embedding model [[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/base.py).

3. The `context_relevance` metric in TruLens is a straightforward measure of how relevant a given context is to a query. In contrast, `context_relevance_with_cot_reasons` includes additional reasoning steps (chain-of-thought) to justify the relevance score. This additional reasoning can provide more nuanced and potentially more accurate relevance assessments, but it also introduces variability depending on how the reasoning is generated and interpreted by the model [[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb)[[3]](https://github.com/truera/trulens/blob/main/docs/trulens_eval/getting_started/core_concepts/rag_triad.md).

We will be adding new context relevance benchmarks to our [benchmarks page](https://www.trulens.org/trulens_eval/evaluation_benchmarks/) soon.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*