Repository: truera/trulens
Issue: Error Executing 'llama_index_quickstart.py' and Integrating with RAG: Common Issue Across Examples.
Root cause: External Dependency Breakage

I encountered an execution error while running the "llama_index_tru_test.py" from the examples directory. The same issue arose during an earlier attempt to integrate trueval_lens with my RAG (Retrieval-Augmented Generation) code. Below is the traceback of the error encountered
```
Traceback (most recent call last):
  File "/home/devuser/RAG_chatbot/llama_index_tru_test.py", line 108, in <module>
    tru_query_engine_recorder = TruLlama(
                                ^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/tru_llama.py", line 245, in __init__
    kwargs['instrument'] = LlamaInstrument(app=self)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/tru_llama.py", line 178, in __init__
    include_classes=LlamaInstrument.Default.CLASSES(),
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/tru_llama.py", line 86, in <lambda>
    BaseComponent,
    ^^^^^^^^^^^^^
NameError: name 'BaseComponent' is not defined
```

**Environment**
Python version: 3.11
Operating System: CentOS7

Would appreciate any guidance on resolving this issue or if there's a workaround available.

Can you verify whether your environment has [`llama_index`](https://pypi.org/project/llama-index/) version `0.9.15post2` or newer installed?

I need to check on that.

If [llama_index](https://pypi.org/project/llama-index/) is installed, it's possible that the issue you're encountering is connected to the one described here: https://github.com/truera/trulens/issues/898

Hey, @aaronvarghese. 
I encountered the `NameError: name 'BaseComponent' is not defined` error while working with version 0.9.13 of llama-index.
After that, I have tested the code with two llama-index versions. Let me share that with you.

1. For `llama-index==0.9.15post2` version I'm facing the below error:
```
Traceback (most recent call last):
  File "/home/devuser/RAG_chatbot/llama_index_tru_test.py", line 130, in <module>
    display(rec)
    ^^^^^^^
NameError: name 'display' is not defined
Feedback Function exception caught: Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 112, in sync
    loop = asyncio.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/feedback.py", line 550, in run
    result_and_meta, part_cost = sync(
                                 ^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 117, in sync
    return loop.run_until_complete(awaitable)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/tasks.py", line 277, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 458, in atrack_all_costs_tally
    result, cbs = await Endpoint.atrack_all_costs(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 439, in atrack_all_costs
    return await Endpoint._atrack_costs(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 536, in _atrack_costs
    result: T = await desync(__func, *args, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 88, in desync
    res = await asyncio.to_thread(func, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/base_events.py", line 829, in run_in_executor
    executor.submit(func, *args), loop=self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/threading.py", line 75, in submit
    return super().submit(
           ^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/concurrent/futures/thread.py", line 169, in submit
    raise RuntimeError('cannot schedule new futures after '
RuntimeError: cannot schedule new futures after interpreter shutdown

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/feedback.py", line 555, in run
    raise RuntimeError(
RuntimeError: Evaluation of groundedness_measure_with_cot_reasons failed on inputs: 
{'source': ['![](https://s.turbifycdn.com/aah/paulgraham/essays-5.gif)|\n'
            '![](https://sep.turbifycdn.com/ca/Img/tr
cannot schedule new futures after interpreter shutdown.

Feedback Function exception caught: Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 112, in sync
    loop = asyncio.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/feedback.py", line 550, in run
    result_and_meta, part_cost = sync(
                                 ^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 117, in sync
    return loop.run_until_complete(awaitable)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/tasks.py", line 277, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 458, in atrack_all_costs_tally
    result, cbs = await Endpoint.atrack_all_costs(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 439, in atrack_all_costs
    return await Endpoint._atrack_costs(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 536, in _atrack_costs
    result: T = await desync(__func, *args, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 88, in desync
    res = await asyncio.to_thread(func, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/base_events.py", line 829, in run_in_executor
    executor.submit(func, *args), loop=self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/threading.py", line 75, in submit
    return super().submit(
           ^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/concurrent/futures/thread.py", line 169, in submit
    raise RuntimeError('cannot schedule new futures after '
RuntimeError: cannot schedule new futures after interpreter shutdown

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/feedback.py", line 555, in run
    raise RuntimeError(
RuntimeError: Evaluation of relevance failed on inputs: 
{'prompt': 'What did the author do growing up?',
 'response': 'The author mentioned that before college, they worked on two '
  
cannot schedule new futures after interpreter shutdown.

Feedback Function exception caught: Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 112, in sync
    loop = asyncio.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/feedback.py", line 550, in run
    result_and_meta, part_cost = sync(
                                 ^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 117, in sync
    return loop.run_until_complete(awaitable)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/tasks.py", line 277, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 458, in atrack_all_costs_tally
    result, cbs = await Endpoint.atrack_all_costs(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 439, in atrack_all_costs
    return await Endpoint._atrack_costs(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 536, in _atrack_costs
    result: T = await desync(__func, *args, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/asynchro.py", line 88, in desync
    res = await asyncio.to_thread(func, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/asyncio/base_events.py", line 829, in run_in_executor
    executor.submit(func, *args), loop=self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/utils/threading.py", line 75, in submit
    return super().submit(
           ^^^^^^^^^^^^^^^
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/concurrent/futures/thread.py", line 169, in submit
    raise RuntimeError('cannot schedule new futures after '
RuntimeError: cannot schedule new futures after interpreter shutdown

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devuser/anaconda3/envs/py311/lib/python3.11/site-packages/trulens_eval/feedback/feedback.py", line 555, in run
    raise RuntimeError(
RuntimeError: Evaluation of qs_relevance failed on inputs: 
{'question': 'What did the author do growing up?',
 'statement': '![](https://s.turbifycdn.com/aah/paulgraham/essays-5.gif)|\n'

cannot schedule new futures after interpreter shutdown.
```

2. For latest version `llama-index==0.10.5` version I'm facing the below error:
```
Traceback (most recent call last):
  File "/home/devuser/RAG_chatbot/llama_index_tru_test.py", line 47, in <module>
    from llama_index.readers.web import SimpleWebPageReader
ModuleNotFoundError: No module named 'llama_index.readers.web'
(py311_1) [root@co7-50-7-133 RAG_chatbot]# python3 llama_index_tru_test.py
🦑 Tru initialized with db url sqlite:///default.sqlite .
🛑 Secret keys may be written to the database. See the `database_redact_keys` option of `Tru` to prevent this.
Traceback (most recent call last):
  File "/home/devuser/RAG_chatbot/llama_index_tru_test.py", line 47, in <module>
    from llama_index.readers.web import SimpleWebPageReader
ModuleNotFoundError: No module named 'llama_index.readers.web'
```
I checked llama-index documentation also for how to use `SimpleWebPageReader`. It seems correct to me.

Hey @shreyashpatel-crest - support for Llama-Index versions > 0.10 coming in #891 , can you install from that branch and give it a try? Thanks!

Hey @joshreini1, Cool. Will try that solution and let you know.

Support for Llama-Index > 0.10 is merged to main and will be released in 0.24 today or tomorrow.

@shreyashpatel-crest please let me know if you have any persistent issues. Thanks!

Support for Llama-Index > 0.10 now released in 0.24.0!