Repository: truera/trulens
Issue: LlamaIndex: Embeddings are included in the input JSON when using a QueryBundle instead of a string
Root cause: Algorithmic Error

When using `TruLlama` recorder, and specifying a `QueryBundle` instead of a string upon calling the `query()` method, the whole `QueryBundle` with the embeddings list will be serialized into `Record.main_input`:

```python
with tru_recorder as recording:
    bundle = QueryBundle(
        query_str='Who wrote the article?\n\nWho co-authored the article?',
        custom_embedding_strs=['authors']
    )
    response = query_engine.query(bundle)
    print(recording.records[-1].main_input['embedding']) # [0.009615207090973854, -0.02261897176504135, ...
```

This doesn't seem intentional. Now all those embeddings are included to the feedback functions when using the `.on_input()` shortcut, adding irrelevant context, as well as needlessly consuming a lot of tokens. And it also clutters the dashboard (which doesn't render JSON correctly when `query_str` has more than one line):

![image](https://github.com/truera/trulens/assets/20743993/c776dbc6-c235-4fa0-9a5b-9f5ec6327a64)

The first problem can be remedied by specifying custom selectors, but I couldn't override `Record.main_input` itself, to remove the dashboard clutter.

Are there any workarounds for this? I think just setting `main_input['embedding']` to `None` will be sufficient. I tried it manually like this, right after calling `query()`:

```python
bundle.embedding = None
recording.records[-1].main_input['embedding'] = None
```

But it has no effect.

I think it would be great if this case was handled out of the box! I imagine I'm not the only one using QueryBundles to specify a custom embedding string.

Hi; if you'd like to define your feedback function on only part of the bundle, you can use `Record.main_input['query_str']` for example. This will not get rid of the UI clutter though. Will look at better detecting the main input in this case.

Next release will pull the `query_str` out of the bundle as the main input (see https://github.com/truera/trulens/pull/690 if you'd like to try it out early). With that change, the embeddings and other parts of the bundle will not be accessible through `Record.main_input` or `RecordInput` but can still be accessed through `Record.app.query[0].args.str_or_query_bundle`. By access here I mean for defining feedback functions. If you'd like to get these components from records yourself, you can do this:
```python
with tru_recorder as recording:
   # run your .query on a bundle
rec = recording.get().layout_calls_as_app() # important
from trulens_eval.utils.serial import Lens
next(Lens().app.query[0].args.str_or_query_bundle.get(rec))
```
 

> if you'd like to define your feedback function on only part of the bundle, you can use `Record.main_input['query_str']`

Yes, that's what I already did. But like you said, it doesn't get rid of the UI clutter.

> Next release will pull the `query_str` out of the bundle as the main input (see #690 if you'd like to try it out early).

Awesome! This will resolve the issue.

> With that change, the embeddings and other parts of the bundle will not be accessible through `Record.main_input` or `RecordInput` but can still be accessed through `Record.app.query[0].args.str_or_query_bundle`. By access here I mean for defining feedback functions.

That's great, because I do still need to use `custom_embedding_strs`! With this change, could you please provide an example of how I can pass `custom_embedding_strs` as an input to a feedback function (instead of `.on_input()`)? This is how I'm doing it now:

```python
f_qs_relevance = (
	Feedback(tru_provider.qs_relevance_with_cot_reasons,
			name="Context Relevance")
	.on(Select.RecordInput.custom_embedding_strs[:]) # How would I replace this line?
	.on(context_selection)
	.aggregate(np.mean)
)
```



Hey @theoratkin ! The query_str extraction was just released in trulens_eval==0.19.2.

On your second question:

`.on(Select.Record.app.query[0].args.str_or_query_bundle.custom_embedding_strs[:])`

should do the trick.

> Hey @theoratkin ! The query_str extraction was just released in trulens_eval==0.19.2.
> 
> On your second question:
> 
> `.on(Select.Record.app.query[0].args.str_or_query_bundle.custom_embedding_strs[:])`
> 
> should do the trick.

Fantastic! Just upgraded, it works great. Thank you both!


The alternative for the Selector there is `Select.Record.calls[-1].args.str_or_query_bundle.custom_embedding_strs[:]`. We are also adding an alias `Select.RecordArgs` for `Select.Record.calls[-1].args` to simplify this a bit. The advantage in these is that they will work regardless of what the main app method is, whether `query` or something else. `calls[-1]` refers to the last method to be added to the call list which is the last method to return from an instrumented call, which is the main app method.