Repository: explodinggradients/ragas
Issue: why is answer_relevancy always nan？
Root cause: Validation Gap

[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.

**Your Question**
why is answer_relevancy always nan？
   
** output**
 {'answer_relevancy': nan, 'faithfulness': 0.2308}


**Code Examples**

    def eval_run()
      answer_relevancy = AnswerRelevancy()
      faithfulness = Faithfulness()

      dataset = [{
            "query": query,  # 问题
            "retrieved_contexts": [context],  # 上下文
            "response": answer  # 答案
        }]

        # 将 evalsets 列表转换为 Dataset 对象
        ds_case = Dataset.from_list(dataset)
        # 获取结果  获取上下文
        # 开始评测
        column_map = {"user_input": "query", "retrieved_contexts": "retrieved_contexts", "response": "response"}
        metrics = [answer_relevancy, faithfulness]
        return self.eval(dataset, column_map, metrics)


    def eval(self, dataset, column_map, metrics):
        result = evaluate(
            dataset=dataset,
            metrics=metrics,
            llm=self.llm,
            embeddings=self.embeddings,
            column_map=column_map,
        )
        print(result)
        print(result.scores)
        df = result.to_pandas()
        print(df.head())
        return result


hey @LillyChen which model are you using?

![image](https://github.com/user-attachments/assets/0235ad9c-a530-4c53-bfcd-40434f779d98)

Do you have any tracing setup? what you can do is easily setup tracing and check if these are the reasons for errors

> hey @LillyChen which model are you using?
> 
> ![image](https://private-user-images.githubusercontent.com/5261489/400653854-0235ad9c-a530-4c53-bfcd-40434f779d98.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzYyMzA0NjQsIm5iZiI6MTczNjIzMDE2NCwicGF0aCI6Ii81MjYxNDg5LzQwMDY1Mzg1NC0wMjM1YWQ5Yy1hNTMwLTRjNTMtYmZjZC00MDQzNGY3NzlkOTgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDEwNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAxMDdUMDYwOTI0WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NzhhNzE4M2Q4MmQ3MzViNDRmMzk2NDhiNzQ3ZDAwNjI5N2NmNGNhMWZjZDllYjgxNjlkYWY2NjA3ZDRlZWJmNyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.0G6rxPyV_SLx4o7uzPxRCdexVYN_drnDFt2rUUG1OP0)
> 
> Do you have any tracing setup? what you can do is easily setup tracing and check if these are the reasons for errors

         os.environ["OPENAI_API_KEY"] = "sk-"
        self.llm = LangchainLLMWrapper(ChatOpenAI(
            openai_api_key="sk-",
            openai_api_base="https://dashscope.aliyuncs.com/compatible-mode/v1",
            model_name="qwen-plus",
            temperature=0.7,
            top_p=0.8,
        ))
        os.environ['DASHSCOPE_API_KEY'] = 'sk-'
        self.embeddings = LangchainEmbeddingsWrapper(DashScopeEmbeddings())

got it - i could be the model you are using, can you check the traces? 

have you setup any tracing tools?

> got it - i could be the model you are using, can you check the traces?
> 
> have you setup any tracing tools?

What  tracing tools should be used for tracking? Please recommend it. Thank you

https://docs.arize.com/phoenix is an OSS one available that you could try