Problems with using LangChain Sagemaker Embedding for Llama 2.0 Inference Endpoint in Sagemaker
Labels: name: area: embeddings, name: auto:bug

### Issue you'd like to raise.

Hi all😊 ,

I work in a network-isolated SageMaker environment. There I have hosted a llama 2.0 -7b chat Inference Endpoint (from [HF ](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf))

For a project I would like to work with LangChain.

Currently I am failing with the embeddings.
I can't use the HF embedding engine because of the network isolation.
hf_embedding = HuggingFaceInstructEmbeddings()

Alternatively I found a SageMaker embedding:
https://python.langchain.com/docs/integrations/text_embedding/sagemaker-endpoint

I used the same code from the docu:

```python
from typing import Dict, List
from langchain.embeddings import SagemakerEndpointEmbeddings
from langchain.embeddings.sagemaker_endpoint import EmbeddingsContentHandler
import json

class ContentHandler(EmbeddingsContentHandler):
    content_type = "application/json"
    accepts = "application/json"

    def transform_input(self, inputs: list[str], model_kwargs: Dict) -> bytes:
        """
        Transforms the input into bytes that can be consumed by SageMaker endpoint.
        Args:
            inputs: List of input strings.
            model_kwargs: Additional keyword arguments to be passed to the endpoint.
        Returns:
            The transformed bytes input.
        """
        # Example: inference.py expects a JSON string with a "inputs" key:
        input_str = json.dumps({"inputs": inputs, **model_kwargs})  
        return input_str.encode("utf-8")

    def transform_output(self, output: bytes) -> List[List[float]]:
        """
        Transforms the bytes output from the endpoint into a list of embeddings.
        Args:
            output: The bytes output from SageMaker endpoint.
        Returns:
            The transformed output - list of embeddings
        Note:
            The length of the outer list is the number of input strings.
            The length of the inner lists is the embedding dimension.
        """
        # Example: inference.py returns a JSON string with the list of
        # embeddings in a "vectors" key:
        response_json = json.loads(output.read().decode("utf-8"))
        return response_json["vectors"]`


content_handler = ContentHandler()

embeddings = SagemakerEndpointEmbeddings(
    endpoint_name=ENDPOINT_NAME,
    region_name=REGION_NAME,
    content_handler=content_handler ,
)

query_result = embeddings.embed_query("foo")
```

But i get the following error:
`---------------------------------------------------------------------------
ModelError                                Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/langchain/embeddings/sagemaker_endpoint.py:153, in SagemakerEndpointEmbeddings._embedding_func(self, texts)
[...]

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (422) from primary with message "Failed to deserialize the JSON body into the target type: inputs: invalid type: sequence, expected a string at line 1 column 11". See https://eu-central-1.console.aws.amazon.com/cloudwatch/home?xxxxx in account xxxxxx for more information.`

### Suggestion:

_No response_