Repository: truera/trulens
Issue: [BUG] Bedrock provider always gives: "Groundedness per statement in source:   0%"
Root cause: Interface Contract Mismatch

**Bug Description**
I'm following quickstart with [langChain](https://www.trulens.org/trulens_eval/getting_started/quickstarts/langchain_quickstart/#send-your-first-request)
However, I want to use Bedrock as an LLM and as an evaluation provider. I'm able to get answer relevance and context relevance but groundedness doesn't appear on a dashboard and I'm getting "Groundedness per statement in source:   0%| "

**To Reproduce**
Follow [quickstart](https://www.trulens.org/trulens_eval/getting_started/quickstarts/langchain_quickstart/) with few tweaks:
1. instead of `llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)` use `from langchain_aws import ChatBedrock`
2. instead of `embeddings = OpenAIEmbeddings()` use `from langchain_community.embeddings import BedrockEmbeddings`
3. instead of `provider = OpenAI()` use `from trulens_eval.feedback.provider import Bedrock`

**Expected behaviour**
Groundedness should be present on a dashboard and should be higher than 0 for the question mentioned in the [quickstart](https://www.trulens.org/trulens_eval/getting_started/quickstarts/langchain_quickstart/)


**Relevant Logs/Tracebacks**
```
✅ In Groundedness, input source will be set to __record__.app.first.steps__.context.first.get_relevant_documents.rets.collect() .
✅ In Groundedness, input statement will be set to __record__.main_output or `Select.RecordOutput` .
✅ In relevance, input prompt will be set to __record__.main_input or `Select.RecordInput` .
✅ In relevance, input response will be set to __record__.main_output or `Select.RecordOutput` .
✅ In context_relevance_with_cot_reasons, input question will be set to __record__.main_input or `Select.RecordInput` .
✅ In context_relevance_with_cot_reasons, input context will be set to __record__.app.first.steps__.context.first.get_relevant_documents.rets .
Starting dashboard ...
[nltk_data] Downloading package punkt to
[nltk_data]     /Users/Arseniy_Ivanov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
Groundedness per statement in source:   0%|                                        | 0/3 [00:00<?, ?it/s]
```

**Environment:**
 - OS: MacOS
 - Python Version: Python 3.12.2
 - TruLens version: trulens_eval==0.29.0


<!-- Answer -->
The issue with the Bedrock provider always giving "Groundedness per statement in source: 0%" is likely due to the feedback function for groundedness not being correctly set up or the context not being properly collected and utilized.

### Steps to Resolve

1. **Ensure Context Collection**: Verify that the context is correctly collected and passed to the feedback function. Ensure that the `context.collect()` method is working as expected and that the context chunks are being properly utilized in the feedback function.

2. **Check Response Handling**: Ensure that the response handling logic is correctly implemented and that the necessary metrics are being extracted and utilized.

3. **Verify Supporting Evidence**: Ensure that the response from the model includes the required fields and that the parsing logic correctly extracts and processes this information.

### Example Code Snippet

Here is an example of how you might modify the code to ensure that the context is correctly collected and passed to the feedback function:

```python
from trulens_eval.feedback.provider import Bedrock
from trulens_eval.feedback import Feedback

# Initialize the Bedrock provider
provider = Bedrock()

# Define the groundedness feedback function
f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons)
    .on(context.collect())  # Ensure context chunks are collected into a list
    .on_output()
)

# Ensure that the context is being correctly collected
context_chunks = context.collect()
if not context_chunks:
    raise ValueError("Context chunks are not being collected properly.")

# Pass the collected context to the feedback function
f_groundedness = f_groundedness.on(context_chunks).on_output()
```

### Additional Debugging

To further debug the issue, you can add print statements or logging to verify that the context is being collected and passed correctly:

```python
# Collect context chunks
context_chunks = context.collect()
print("Collected context chunks:", context_chunks)

# Check if context chunks are empty
if not context_chunks:
    raise ValueError("Context chunks are not being collected properly.")

# Pass the collected context to the feedback function
f_groundedness = f_groundedness.on(context_chunks).on_output()
```

By ensuring that the context is correctly collected and passed to the feedback function, you should be able to resolve the issue with the groundedness evaluation and obtain a score higher than 0%.


<!-- Dosu Comment Footer -->
<details>
<summary>

#### Details
</summary>

**Sources**
- [trulens_eval/examples/quickstart/langchain_quickstart.ipynb](https://github.com/truera/trulens/blob/main/trulens_eval/examples/quickstart/langchain_quickstart.ipynb)
- [trulens_eval/trulens_eval/feedback/provider/endpoint/bedrock.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/bedrock.py)
- [trulens_eval/trulens_eval/feedback/provider/bedrock.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/bedrock.py)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*


I have collected context exactly like that and looks like it is collected properly

So far I figured out that `generate_score_and_reasons` methods fail silently. It would be very beneficial to have some error provided back if methods fail

in bedrock.py in method _create_chat_completion

for amazon.titan-text-express-v1 
json.loads(response.get('body').read()) returns empty string

for 'anthropic.claude-3-sonnet-20240229-v1:0'
json.loads(response.get('body').read()) returns empty string

I believe it is because the body is formatted in the wrong way
It looks like this:
```
body = json.dumps(
    {
        "prompt": f"\n\nHuman:{messages_str}\n\nAssistant:",
        "temperature": 0,
        "top_p": 1,
        "max_tokens_to_sample": 4095
    }
```
But it should be like this:
```
bodyNew = json.dumps(
    {
        "system": "Respond as helpfull assistant", 
        "messages": [{"role": "user", "content": f"\n\nHuman:{messages_str}\n\nAssistant:"}], 
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 4095,
    }
)
```

Thanks @Arseniy-II - will take a look