Repository: truera/trulens
Issue: Getting RecursionError: maximum recursion depth exceeded while calling a Python object while creating tru_recorder.
Root cause: External Dependency Breakage

from trulens_eval import Feedback, LiteLLM, TruLlama
from trulens_eval.feedback import Groundedness
from trulens_eval.feedback.provider.hugs import Huggingface


import numpy as np
os.environ["HUGGINGFACE_API_KEY"] = "MY_HUGGINGFACE_API_KEY"

LiteLLM.set_verbose = True
provider = LiteLLM(model_engine="huggingface/mistralai/Mistral-7B-Instruct-v0.1")


f_qa_relevance = Feedback(
    provider.relevance_with_cot_reasons,
    name="Answer Relevance"
).on_input_output()
context_selection = TruLlama.select_source_nodes().node.text

import numpy as np

f_qs_relevance = (
    Feedback(provider.qs_relevance,
             name="Context Relevance")
    .on_input()
    .on(context_selection)
    .aggregate(np.mean)
)
GROUNDED = Groundedness(groundedness_provider=provider)


f_groundedness = (
    Feedback(GROUNDED.groundedness_measure_with_cot_reasons, name="Groundedness")
    .on(context_selection.collect())
    .on_output()
    .aggregate(GROUNDED.grounded_statements_aggregator)
)

from trulens_eval import TruLlama
from trulens_eval import FeedbackMode

tru_recorder = TruLlama(
    sentence_window_engine,
    app_id="App_1",
    feedbacks=[
        f_qa_relevance,
        f_qs_relevance,
        f_groundedness
    ]
)

This is my code, i am unable to create a tru_recorder with it. I am using a huggingface pipeline for RAG.

<!-- Greeting -->
Hey there, @sadaf0714! 👋 I'm [Dosu](https://dosu.dev), your friendly neighborhood bot, here to lend a hand with bugs, answer your questions, and guide you through contributing while we wait for a human maintainer to swing by. I'm diving into your issue and will get back to you with a more detailed response soon!




![image](https://github.com/truera/trulens/assets/123365493/3aacc70f-abfd-4319-9d27-49edec535cfe)
This is the error snapshot.

<!-- Answer -->
The `RecursionError` you're encountering, especially with the traceback pointing to `matplotlib`, suggests a complex issue possibly related to infinite recursion in your code or interactions with dependencies. Here's a focused approach to troubleshoot and potentially resolve the problem:

1. **Check for Infinite Recursion**: Ensure there's no logical error in your custom code that could cause infinite recursion, particularly in parts of the code that involve text processing or feedback aggregation.

2. **Review Matplotlib Integration**: Since `matplotlib` is involved in the error, review any plotting or visualization logic in your project. An inadvertent recursive call in this part of your code could be the culprit.

3. **Examine Huggingface Pipeline Interactions**: Investigate how the Huggingface pipeline's output is used, especially if it's part of a complex workflow that includes data transformation or visualization with `matplotlib`. There might be unexpected recursive calls triggered by these interactions.

4. **Environment and Dependency Check**: Look into your project's dependencies for any known issues or incompatibilities between `matplotlib`, the Huggingface pipeline, and other libraries you're using. Updating or adjusting these dependencies might resolve the issue.

5. **Isolate and Diagnose**: Try isolating the problematic section by systematically commenting out parts of the code related to `matplotlib` and feedback processing. This can help identify the exact source of the recursion error.

6. **Temporary Increase in Recursion Limit**: As a diagnostic step, you can temporarily increase the Python recursion limit with `sys.setrecursionlimit()`, but be cautious as this doesn't solve the underlying problem and could lead to a stack overflow.

By focusing on these areas, you should be able to narrow down the cause of the `RecursionError` and find a path towards resolving it.


<!-- Dosu Comment Footer -->
<details>
<summary>

#### Details
</summary>

**Sources**
- [trulens_eval/examples/expositional/use_cases/iterate_on_rag/3_harmless_eval.ipynb](https://github.com/truera/trulens/blob/main/trulens_eval/examples/expositional/use_cases/iterate_on_rag/3_harmless_eval.ipynb)
- [trulens_eval/trulens_eval/tru_llama.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tru_llama.py)
- [trulens_eval/trulens_eval/feedback/feedback.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/feedback.py)
- [trulens_explain/trulens/utils/tru_logger.py](https://github.com/truera/trulens/blob/main/trulens_explain/trulens/utils/tru_logger.py)
- [trulens_explain/trulens/visualizations.py](https://github.com/truera/trulens/blob/main/trulens_explain/trulens/visualizations.py)
- [trulens_eval/trulens_eval/feedback/provider/hugs.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/hugs.py)
- [trulens_eval/trulens_eval/feedback/provider/endpoint/hugs.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/hugs.py)
- [trulens_explain/trulens/nn/models/_model_base.py](https://github.com/truera/trulens/blob/main/trulens_explain/trulens/nn/models/_model_base.py)
- [trulens_explain/trulens/nn/backend/__init__.py](https://github.com/truera/trulens/blob/main/trulens_explain/trulens/nn/backend/__init__.py)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*


@sadaf0714 can you share your huggingface rag pipeline code so we can reproduce/help? Assuming this is the same application as your other open issue?

<img width="1276" alt="Screenshot 2024-05-24 at 15 10 58" src="https://github.com/truera/trulens/assets/73607255/42f9cfaf-11ff-4534-a992-866758d5c836">
<img width="1276" alt="Screenshot 2024-05-24 at 15 11 21" src="https://github.com/truera/trulens/assets/73607255/ab620985-6774-4555-9573-9111e32e8840">


<img width="693" alt="Screenshot 2024-05-24 at 15 12 30" src="https://github.com/truera/trulens/assets/73607255/403c78f7-88c5-4341-96c2-5fee3677fddc">


Same error here.


![KakaoTalk_20240603_174729181](https://github.com/truera/trulens/assets/127589724/7c21af66-8a05-40fb-9046-cebbdbb5ff6f)



this is how i created feedback metrices 
f_qa_relevance = Feedback(
    provider.relevance_with_cot_reasons,
    name="Answer Relevance"
  ).on_input_output()
  query = Select.Record.app.retriever._get_relevant_documents.args.query
  context = Select.Record.app.retriever.get_relevant_documents.rets[:].page_content

  f_context_relevance = (
      Feedback(
          provider.qs_relevance_with_cot_reasons,
          name="Context Relevance",
      )
      .on(query)
      .on(context)
      .aggregate(np.mean)
  )
  grounded = Groundedness(groundedness_provider=provider)

  f_groundedness = (
      Feedback(
          grounded.groundedness_measure_with_cot_reasons,
          name="Groundedness",
      )
      .on(context.collect())
      .on_output()
      .aggregate(grounded.grounded_statements_aggregator)
  )
and tru_recorder:
tru_recorder = TruChain(
      qa,
      app_id="App_1",
      feedbacks=[
          f_qa_relevance,
          f_context_relevance,
          f_groundedness,
          f_lang_match
      ]
  )
@wahidur028 @guangyao-quan 
let me know if this helps?

@sadaf0714 could you please tell me how can I fix it.
![KakaoTalk_20240603_181240684](https://github.com/truera/trulens/assets/127589724/38b13e4b-b9ae-4b05-9005-2d376c5f791e)

also plz check
![image](https://github.com/truera/trulens/assets/127589724/5523ad01-8b90-447d-95db-eff26aeab51c)



> @sadaf0714 could you please tell me how can I fix it. ![KakaoTalk_20240603_181240684](https://private-user-images.githubusercontent.com/127589724/336011266-38b13e4b-b9ae-4b05-9005-2d376c5f791e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc0MDY0MzIsIm5iZiI6MTcxNzQwNjEzMiwicGF0aCI6Ii8xMjc1ODk3MjQvMzM2MDExMjY2LTM4YjEzZTRiLWI5YWUtNGIwNS05MDA1LTJkMzc2YzVmNzkxZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjAzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYwM1QwOTE1MzJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0xMGVhMTA5ZmQ1YzUyMzg1NDdmZWI4NmYwNWNiYzllM2IyOTdkMzI2ZmI2ZGRhZTY5MTRmZGU5NWE5NjM2YmM2JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.L9CFlZhmJidz-s83o_eGp1nT0XQpVjfiJ-wgdUJoOYg)
> 
> also plz check ![image](https://private-user-images.githubusercontent.com/127589724/336012023-5523ad01-8b90-447d-95db-eff26aeab51c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc0MDY0MzIsIm5iZiI6MTcxNzQwNjEzMiwicGF0aCI6Ii8xMjc1ODk3MjQvMzM2MDEyMDIzLTU1MjNhZDAxLThiOTAtNDQ3ZC05NWRiLWVmZjI2YWVhYjUxYy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjAzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYwM1QwOTE1MzJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yYzU0Mzk1OGZkNjU0ZjlhNzUyNTczMTg1NGI5ZTFmMzQyOWEzNWViMDk4YzY0NWY4MWYyY2JkZjQ1MDUzNmJmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.OE4mIvANPNzYEnaW_sX-APL64zLW_xmtpcwQSziof-M)

import it like this:
from trulens_eval import Feedback, LiteLLM, TruLlama, Select, TruChain, FeedbackMode
from trulens_eval.feedback import Groundedness

![image](https://github.com/truera/trulens/assets/127589724/7c519fb1-d9b7-4997-ad95-48b330c6736e)



tru_recorder = TruLlama()
whenever I call this function, I get the "maximum recursion depth exceeded" error. But I use custom RAG, following [link](https://colab.research.google.com/github/truera/trulens/blob/main/trulens_eval/examples/quickstart/quickstart.ipynb)
the code works fine.
Is there any issue in TruLlama()?

Here is my code. Please suggest how to handle this error

```
import os
from llama_index.core import  VectorStoreIndex, get_response_synthesizer, Settings, StorageContext, PromptTemplate, load_index_from_storage, SimpleDirectoryReader
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from trulens_eval import Feedback, Select

# HuggingFace API key
os.environ["HUGGINGFACE_TOKEN"] = "hf_RyUYYSrF----------VStqeKYsj"

#Openai API key
os.environ["OPENAI_API_KEY"] = "sk-7N1V337g-----7jQGT3BlbkFJVo-----enMlKbnHhW"

# Configure custom embedding model
embed_model = HuggingFaceEmbedding(model_name="wahid028/GIST-Law-Embed")
Settings.embed_model = embed_model

hf_token = "hf_YCZNvLuVr-----ZQcdbNVQhWdvb"


from transformers import AutoTokenizer

HF_model = "meta-llama/Meta-Llama-3-8B"

tokenizer = AutoTokenizer.from_pretrained(
    HF_model,
    token=hf_token,
)

stopping_ids = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("
"),
]

llm = HuggingFaceLLM(
    model_name=HF_model,
    model_kwargs={
        "token": hf_token,
        "torch_dtype": torch.bfloat16,  # comment this line and uncomment below to use 4bit
    },
    generate_kwargs={
        "do_sample": True,
        "temperature": 0.7,
        "top_p": 0.9,
    },

    tokenizer_name= HF_model,
    tokenizer_kwargs={"token": hf_token},
    stopping_ids=stopping_ids,
)

Settings.llm = llm

documents = SimpleDirectoryReader("../RAG/data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=3)

response = query_engine.query("What are the .....?")
print(str(response))

eval_questions = []
with open('eval_questions.txt', 'r') as file:
    for line in file:
        # Remove newline character and convert to integer
        item = line.strip()
        print(item)
        eval_questions.append(item)
        
        # You can try your own question:
new_question = "What are the primary objectives ..?"
eval_questions.append(new_question)

from trulens_eval import Tru
tru = Tru()

tru.reset_database()

from trulens_eval.feedback.provider.openai import OpenAI
import numpy as np

# Initialize provider class
provider = OpenAI()

# select context to be used in feedback. the location of context is app specific.
from trulens_eval.app import App
context = App.select_context(query_engine)

# Define a groundedness feedback function
f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons)
    .on(context.collect()) # collect context chunks into a list
    .on_output()
)

# Question/answer relevance between overall question and answer.
f_answer_relevance = (
    Feedback(provider.relevance)
    .on_input_output()
)
# Question/statement relevance between question and each context chunk.
f_context_relevance = (
    Feedback(provider.context_relevance_with_cot_reasons)
    .on_input()
    .on(context)
    .aggregate(np.mean)
)

from trulens_eval import TruLlama
tru_query_engine_recorder = TruLlama(query_engine,
    app_id='LlamaIndex_App1',
    feedbacks=[f_groundedness, f_answer_relevance, f_context_relevance])

RecursionError: maximum recursion depth exceeded
```

I'm getting the exact same error. Is there a fix yet?

Hi folks - there is a fix here: https://github.com/truera/trulens/pull/1176

You can try it out by installing from the branch:

```
pip install git+https://github.com/piotrm0/trulens.git@piotrm/jsonify-stack-size-protect#subdirectory=trulens_eval
```

@joshreini1 Now it's working fine for me. Thanks for your support.