mistralai

/

Mistral-7B-v0.1

like 3.22k

I've built sagemaker endpoint and wanna generate embedding vector but, it seems that I can't do it with this model.

I'm showing contenthandler.

class ContentHandler(EmbeddingsContentHandler):
content_type = "application/json"
accepts = "application/json"

def transform_input(self, inputs: list[str], model_kwargs: Dict) -> bytes:
    """
    Transforms the input into bytes that can be consumed by SageMaker endpoint.
    Args:
        inputs: List of input strings.
        model_kwargs: Additional keyword arguments to be passed to the endpoint.
    Returns:
        The transformed bytes input.
    """
    # Example: inference.py expects a JSON string with a "inputs" key:
    input_str = ' '.join(inputs)
    input_str = json.dumps({"inputs": input_str, **model_kwargs})
    return input_str.encode("utf-8")

def transform_output(self, output: bytes) -> List[List[float]]:
    """
    Transforms the bytes output from the endpoint into a list of embeddings.
    Args:
        output: The bytes output from SageMaker endpoint.
    Returns:
        The transformed output - list of embeddings
    Note:
        The length of the outer list is the number of input strings.
        The length of the inner lists is the embedding dimension.
    """
    # Example: inference.py returns a JSON string with the list of
    # embeddings in a "vectors" key:
    response_json = json.loads(output.read().decode("utf-8"))
    return response_json