Repository: truera/trulens
Issue: [BUG] Langchain Provider _create_chat_completion crashing
Root cause: Validation Gap

**Bug Description**
What happened?
Hi Trulens team, I was trying to use the trulens langchain provider object with a custom langchain LLM object but when I tried to calculate the relevance score it crashed. I get the error that 'str' object has no attribute 'content'.

**To Reproduce**
Run the following script:

```
from typing import Any, List, Optional
from langchain_core.callbacks.manager import CallbackManagerForLLMRun
from langchain_core.language_models.llms import LLM
from trulens.providers.langchain import Langchain


class CustomLLM(LLM):
    n: int

    def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        if stop is not None:
            raise ValueError("stop kwargs are not permitted.")
        return prompt[: self.n]

    @property
    def _llm_type(self) -> str:
        return "custom"


llm = CustomLLM(n=5)
langchain_provider = Langchain(chain=llm)
rel = langchain_provider.relevance(
    prompt="Where is Germany?",
    response="Germany is in Earth."
)
print(rel)
```

**Expected behavior**
I expect the above code to print the relevance score without crashing.

**Relevant Logs/Tracebacks**

langchain request failed <class 'AttributeError'>='str' object has no attribute 'content'. Retries remaining=3.
langchain request failed <class 'AttributeError'>='str' object has no attribute 'content'. Retries remaining=2.
langchain request failed <class 'AttributeError'>='str' object has no attribute 'content'. Retries remaining=1.
langchain request failed <class 'AttributeError'>='str' object has no attribute 'content'. Retries remaining=0.
Traceback (most recent call last):
  File "C:\Users\alonl\PycharmProjects\pythonProject\test_provider.py", line 65, in <module>
    rel = langchain_provider.relevance(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alonl\PycharmProjects\pythonProject\.venv\Lib\site-packages\trulens\feedback\llm_provider.py", line 525, in relevance
    return self.generate_score(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alonl\PycharmProjects\pythonProject\.venv\Lib\site-packages\trulens\feedback\llm_provider.py", line 105, in generate_score
    response = self.endpoint.run_in_pace(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alonl\PycharmProjects\pythonProject\.venv\Lib\site-packages\trulens\core\feedback\endpoint.py", line 323, in run_in_pace
    raise RuntimeError(
RuntimeError: Endpoint langchain request failed 4 time(s): 
	'str' object has no attribute 'content'
	'str' object has no attribute 'content'
	'str' object has no attribute 'content'
	'str' object has no attribute 'content'

**Environment:**

- OS: both MacOS and Windows
- Python Version: 3.12.0
- TruLens version: 1.0.1
- trulens-providers-langchain version: 1.0.1

**Additional context**
I think I have an idea of what is happening which I hope helps:

When you go into the Langchain object you can see that it takes both BaseLLM and BaseChatModel classes as its "chain" attribute. Although both have the invoke method each of their invoke methods returns a different value, the BaseLLM's invoke returns a string while the BaseChatModel's invoke returns a BaseMessage object. This creates an issue in the Langchain object's _create_chat_completion method, specifically if the following statement goes through: `predict = self.endpoint.chain.invoke(messages, **kwargs).content`. It then calls invoke() and tries to extract the content attribute assuming its a BaseMessage which works if you are using a BaseChatModel but if you are using a BaseLLM model it will crash given that invoke() returns a string. 

<!-- Greeting -->
Hello, @alonlahav123! I'm here to assist you with any bugs, questions, or contribute to the discussion. Let's tackle this challenge together!

<!-- Answer -->
To fix the `'str' object has no attribute 'content'` error, you need to ensure that the `invoke` method of your custom LangChain LLM object returns an object that has a `content` attribute when processing messages. The error occurs because the code expects the result of `self.endpoint.chain.invoke(messages, **kwargs)` to have a `content` attribute, but it is receiving a string instead.

Here's the relevant part of the code that needs attention:

```python
def _create_chat_completion(
    self,
    prompt: Optional[str] = None,
    messages: Optional[Sequence[Dict]] = None,
    **kwargs,
) -> str:
    if prompt is not None:
        predict = self.endpoint.chain.invoke(prompt, **kwargs)

    elif messages is not None:
        messages = [_convert_message(message) for message in messages]
        predict = self.endpoint.chain.invoke(messages, **kwargs).content

    else:
        raise ValueError("`prompt` or `messages` must be specified.")

    return predict
```

To fix this, you can modify the `_create_chat_completion` method to handle the case where `invoke` returns a string. Here is an updated version of the method:

```python
def _create_chat_completion(
    self,
    prompt: Optional[str] = None,
    messages: Optional[Sequence[Dict]] = None,
    **kwargs,
) -> str:
    if prompt is not None:
        predict = self.endpoint.chain.invoke(prompt, **kwargs)

    elif messages is not None:
        messages = [_convert_message(message) for message in messages]
        result = self.endpoint.chain.invoke(messages, **kwargs)
        predict = result.content if hasattr(result, 'content') else result

    else:
        raise ValueError("`prompt` or `messages` must be specified.")

    return predict
```

This modification checks if the result of `invoke` has a `content` attribute and uses it if available; otherwise, it uses the result directly.

Additionally, based on the provided `src/providers/litellm/trulens/providers/litellm/provider.py` file, it seems the correct method to call on the `chain` object is `completion` instead of `invoke`. Here is the updated method reflecting this:

```python
def _create_chat_completion(
    self,
    prompt: Optional[str] = None,
    messages: Optional[Sequence[Dict]] = None,
    **kwargs,
) -> str:
    completion_args = kwargs
    completion_args["model"] = self.model_engine
    completion_args.update(self.completion_args)

    if messages is not None:
        completion_args["messages"] = messages

    elif prompt is not None:
        completion_args["messages"] = [
            {"role": "system", "content": prompt}
        ]

    else:
        raise ValueError("`prompt` or `messages` must be specified.")

    comp = completion(**completion_args)

    assert isinstance(comp, object)

    return comp["choices"][0]["message"]["content"]
```

This ensures that the correct method is used and the result is handled appropriately <sup>[[1]](https://github.com/truera/trulens/blob/main/src/providers/langchain/trulens/providers/langchain/provider.py)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/langchain.py)</sup><sup>[[3]](https://github.com/truera/trulens/blob/main/src/providers/litellm/trulens/providers/litellm/provider.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


Hey @alonlahav123 - we just released support for `BaseLLM` use in the Langchain provider in TruLens 1.0.2.

Give it a try and re-open if the issue persists. Thanks!

`pip install -U trulens-providers-langchain`