Repository: truera/trulens
Issue: Could not locate app.retrieve.args.query in app/record
Root cause: Documentation Deficiency

I'm trying to run trulens to evaluate the RAG triad metrics on my langchain RAG

I'm doing:

```  
truchain = TruChain(rag, app_id="RAG_app", feedbacks=[f_groundedness, f_qa_relevance, f_context_relevance])
for question in df_qa['query']:
    truchain(question)
```  


which throws this error:

`Could not locate app.retrieve.args.query in app/record.
Run of <lambda> in <Thread(TP.submit with debug timeout_0, started 11156877312)> failed with: Could not locate app.retrieve.args.query in app/record.
exception calling callback for <Future at 0x1698bf750 state=finished raised RuntimeError>`

The RAG successfully runs the query but the metrics do not get computed. Any suggestions on how to fix?

Hi; can you share the structure of your rag app? The "app.retrieve.args.query" indicates the location of a component inside the app to focus on and may vary from app to app depending on their structure.

> Hi; can you share the structure of your rag app? The "app.retrieve.args.query" indicates the location of a component inside the app to focus on and may vary from app to app depending on their structure.

Hi, I'm running a very simple model just to try out this package. I'm creating a RetrievalQA chain using `langchain.chains.RetrievalQA`, the RAG works fine. I then pass it to a `TruChain` object as shown above which is what isn't working.

The structure of the repo is very simple right now. I just have a `main.py` where the majority of the code is located. I put the feedback functions in a `eval.py` that I import but everything else is in that file

Is there anything else I can explain about the structure that would be helpful? Thanks for helping out!

Hey @hsjok !

Added a new example of a Langchain RAG with LCEL that I think is what you're looking for. Can you try this out and see if it helps? https://www.trulens.org/trulens_eval/langchain_quickstart/

Hi @joshreini1 
I'm experiencing the same problem.
In the colab script, everything works fine, but when I run the code on my own retriever, I get the following error:

`✅ In groundedness_measure_with_cot_reasons, input source will be set to __record__.app.first.steps.context.first.get_relevant_documents.rets.collect() .
✅ In groundedness_measure_with_cot_reasons, input statement will be set to __record__.main_output or `Select.RecordOutput` .
✅ In relevance, input prompt will be set to __record__.main_input or `Select.RecordInput` .
✅ In relevance, input response will be set to __record__.main_output or `Select.RecordOutput` .
✅ In qs_relevance, input question will be set to __record__.main_input or `Select.RecordInput` .
✅ In qs_relevance, input statement will be set to __record__.app.first.steps.context.first.get_relevant_documents.rets .
`


`.../site-packages/trulens_eval/utils/serial.py", line 362, in get
    raise KeyError(
KeyError: 'Key not in dictionary: first'
`


`Could not locate app.first.steps.context.first.get_relevant_documents.rets in app/record.
Could not locate app.first.steps.context.first.get_relevant_documents.rets.collect() in app/record.
`


Hey @yardenfren1996 - looks like you may be on an earlier dev version of TruLens. Can you re-install from main?

@joshreini1 Thanks for responding so quickly.
Unfortunately there is still an error, 

Same error or different? @yardenfren1996 

same @joshreini1 

See this in email but not sure where you moved the thread to. Can you point
me?


On Sun, Dec 31, 2023 at 1:58 PM yardenfren1996 ***@***.***>
wrote:

> I think it might be better if we will talk in here.
>
> I deleted my entire env.
> Than clone the trulens repository and installing locally from main as you
> said.
>
> ￼
>
> This is the entire error.
> There is somthing i can change to make it work?
>
> Thank you very much
> > On 31 Dec 2023, at 18:19, Josh Reini ***@***.***> wrote:
> >
> >
> > Hey @yardenfren1996 <https://github.com/yardenfren1996> - looks like
> you may be on an earlier dev version of TruLens. Can you re-install from
> main?
> >
> > —
> > Reply to this email directly, view it on GitHub <
> https://github.com/truera/trulens/issues/685#issuecomment-1872986827>, or
> unsubscribe <
> https://github.com/notifications/unsubscribe-auth/ALLC6L54KH44AROI5F3VLU3YMGGARAVCNFSM6AAAAABAXDFYDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSHE4DMOBSG4>.
>
> > You are receiving this because you were mentioned.
> >
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/truera/trulens/issues/685#issuecomment-1873017286>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AORAKDSKAN7443VEZU3ZF2LYMGYVLAVCNFSM6AAAAABAXDFYDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZTGAYTOMRYGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


@joshreini1 
Is there a way I can fix/avoid this problem?

@yardenfren1996 can you share the code where you define the context selection?

This might have been fixed by https://github.com/truera/trulens/pull/755 , Can you test with newest release?

Closing due to no response. @yardenfren1996 please re-open if you still have this issue with the latest release.