Dict validation error
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models, name: integration: aws

### System Info

Trying to execute the chatbot script with sagemaker endpoint of LLAMA2 llm model getting dict validation error for RetrievalQA

Request:
def retreiveFromLL(userQuery: str) -> QueryResponse:
    pre_prompt = """
 


    You are a helpful, respectful and honest assistant. Answer exactly in detail from the context

    

    Answer the question below from context below :

    """
    prompt = pre_prompt + "CONTEXT:\n\n{context}\n" +"Question : {question}" + "[\INST]"
    llama_prompt = PromptTemplate(template=prompt, input_variables=["context", "question"])
    chain_type_kwargs = {"prompt": llama_prompt}

    embeddings = SentenceTransformerEmbeddings(model_name=EMBEDDING_MODEL)

    # Initialize PGVector index
    vector_db = PGVector(
        embedding_function=embeddings,
        collection_name='CSE_runbooks',
        connection_string=CONNECTION_STRING,
    )
    print("**Invoking PGVector")

        # Custom ContentHandler to handle input and output to the SageMaker Endpoint
    class LlamaChatContentHandler(LLMContentHandler):
        content_type = "application/json"
        accepts = "application/json"

        def transform_input(self, inputs: str, model_kwargs: Dict = {}) -> bytes:
            payload = {
        "inputs": pre_prompt,
                "parameters": {"max_new_tokens":2000, "top_p":0.9, "temperature":0.1}}
            input_str = ' '.join(inputs)
            input_str = json.dumps(payload)
            print(payload)
            return input_str.encode("utf-8")

        def transform_output(self, output: bytes) -> str:
            response_json = json.loads(output.read().decode("utf-8"))
            content = response_json[0]["generated_text"]
            return content
    # Initialize SagemakerEndpoint
    print("Invoking LLM SageMaker Endpoint")
    llm = SagemakerEndpoint(
        endpoint_name=LLAMA2_ENDPOINT,
        region_name=AWS_REGION,
        content_handler=LlamaChatContentHandler(),
        callbacks=[StreamingStdOutCallbackHandler()],
        endpoint_kwargs={"CustomAttributes": "accept_eula=true"},
    )
    print(llm)    

    # Create a RetrievalQA instance with Pinecone as the retriever
    query = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=vector_db, return_source_documents=True, chain_type_kwargs=chain_type_kwargs)

    print("**Invoking query")
    result = query({"query": userQuery})
    response = result["result"]

Error:
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
    exec(code, module.__dict__)
  File "/home/ec2-user/milvus/qa_UI.py", line 26, in <module>
    userResponse = getLLMResponse(user_input)
  File "/home/ec2-user/milvus/getLLMResponse1.py", line 37, in getLLMResponse
    userResponse = retreiveFromLL(userQuery)
  File "/home/ec2-user/milvus/getLLMResponse1.py", line 97, in retreiveFromLL
    query = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=vector_db, return_source_documents=True, chain_type_kwargs=chain_type_kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/langchain/chains/retrieval_qa/base.py", line 103, in from_chain_type
    return cls(combine_documents_chain=combine_documents_chain, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for RetrievalQA
retriever
  value is not a valid dict (type=type_error.dict)

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Execute the code

### Expected behavior

response from LLM