Repository: truera/trulens
Issue: [BUG] Groundedness and Context Relevance Error due to Debug Timeout
Root cause: Configuration Error

**Bug Description**
I have been following Langchain [quickstart](https://github.com/truera/trulens/blob/main/trulens_eval/examples/quickstart/langchain_quickstart.ipynb) and getting the below error:

**Relevant Logs/Tracebacks**
```
python3.11/site-packages/trulens_eval/database/sqlalchemy_db.py - 273 - INFO - 🛑 feedback result context_relevance_with_cot_reasons FAILED feedback_result_hash_18d5d9a0beaa219c773969aea44bbfc8 - TP.submit with debug timeout_4
``` 
and,

```
python3.11/site-packages/trulens_eval/database/sqlalchemy_db.py - 273 - INFO - 🛑 feedback result groundedness_measure_with_cot_reasons FAILED feedback_result_hash_f74b317d2063c75b6028016f1dbe1a97 - TP.submit with debug timeout_1
```
**Environment:**
 - OS: MacOS
 - Python Version: 3.11
 - TruLens version: 0.27.2
 - langchain : 0.1.14
 - langchain-community: 0.0.31
 - langchain-core: 0.1.40
 - langchain-openai: 0.1.1 

**Additional context**
I am able to run the context relevance and groundedness directly.
```
provider.context_relevance_with_cot_reasons(question="abc", context="abcdef")
```



<!-- Greeting -->
Hey @bhardwaj-vipul, nice to see you diving into the depths of TruLens again! Running into a bit of a debug timeout snag, huh? Let's see what we can do about that.

<!-- Answer -->
Based on the error messages you've provided, it seems like the `context_relevance_with_cot_reasons` and `groundedness_measure_with_cot_reasons` methods are taking longer than the specified debug timeout value to complete. This is likely causing the `TP.submit` method to raise a `TimeoutError`, as it's designed to do when a task doesn't finish within the set timeout period.

The debug timeout value is set to 600.0 seconds (10 minutes) in the `TP` (thread processing) class. If a task submitted through the `TP` class's `submit` method does not complete within this timeframe, a `TimeoutError` is raised, and an error message is logged indicating that the run of the function timed out. This behavior is implemented in the `_run_with_timeout` method of the `TP` class. You can find more details about this in the [threading.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/utils/threading.py) file.

Given this, there are a few potential solutions you could consider:

1. **Increasing the Timeout Value**: If the timeout is too short for the task to complete, you could consider increasing the `DEBUG_TIMEOUT` value. However, this would require modifying the `TP` class in the TruLens codebase, which might not be ideal.

2. **Optimizing the Task**: If the task consistently takes too long, you might need to optimize the function's logic or break the task into smaller, more manageable pieces that can complete within the timeout period.

3. **Concurrency and Threading Adjustments**: Review the usage of threads and concurrency in your application. Ensure that the number of concurrent tasks does not exceed the thread pool's capacity, which could lead to deadlocks or long waiting times. The `TP` class sets a maximum of 128 threads.

4. **Debugging and Monitoring**: Utilize logging to monitor and debug which tasks are timing out and why. The error logging implemented in the `_run_with_timeout` method provides valuable information for this purpose.

5. **Review External Dependencies**: If the task involves network calls or dependencies on external services, ensure those services are responsive and meet your timing requirements. External service delays could contribute to timeouts.

6. **Handling Exceptions**: Implement robust exception handling around tasks that might time out. This could involve retry logic or fallback mechanisms to ensure the application remains resilient in the face of timeouts.

Please note that these are general suggestions and the exact solution might depend on the specifics of your use case and the tasks that are timing out. If you continue to experience issues, please provide more details about the tasks that are timing out and any additional information that might help diagnose the issue.

## Sources
- [trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb)
- [trulens_eval/trulens_eval/feedback/provider/base.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/base.py)
- [trulens_eval/trulens_eval/utils/threading.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/utils/threading.py)

<!-- Dosu Comment Footer -->
<details>
<summary>

#### About Dosu
</summary>

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.
</details>


Hi @bhardwaj-vipul , I'm having trouble replicating. Are the other outputs in the notebook as expected? Specifically, the `display(rec)` in the `Retrieve records and feedback` section? If so, what does it show as the output there?

Hey @bhardwaj-vipul - can you give us an update to @piotrm0 's questions so we can help debug? Thanks!

Closing due to no response. @bhardwaj-vipul if this issue is ongoing and you're able to share more information to help reproduce, please re-open.