Self Query Retriever with Google Flan T5 models issue
Labels: name: area: memory, name: area: vector store, name: auto:bug

### System Info

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/output_parsers/json.py:32  │
│ in parse_and_check_json_markdown                                                                 │
│                                                                                                  │
│   29                                                                                             │
│   30 def parse_and_check_json_markdown(text: str, expected_keys: List[str]) -> dict:             │
│   31 │   try:                                                                                    │
│ ❱ 32 │   │   json_obj = parse_json_markdown(text)                                                │
│   33 │   except json.JSONDecodeError as e:                                                       │
│   34 │   │   raise OutputParserException(f"Got invalid JSON object. Error: {e}")                 │
│   35 │   for key in expected_keys:                                                               │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/output_parsers/json.py:25  │
│ in parse_json_markdown                                                                           │
│                                                                                                  │
│   22 │   json_str = json_str.strip()                                                             │
│   23 │                                                                                           │
│   24 │   # Parse the JSON string into a Python dictionary                                        │
│ ❱ 25 │   parsed = json.loads(json_str)                                                           │
│   26 │                                                                                           │
│   27 │   return parsed                                                                           │
│   28                                                                                             │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/json/__init__.py:346 in loads                      │
│                                                                                                  │
│   343 │   if (cls is None and object_hook is None and                                            │
│   344 │   │   │   parse_int is None and parse_float is None and                                  │
│   345 │   │   │   parse_constant is None and object_pairs_hook is None and not kw):              │
│ ❱ 346 │   │   return _default_decoder.decode(s)                                                  │
│   347 │   if cls is None:                                                                        │
│   348 │   │   cls = JSONDecoder                                                                  │
│   349 │   if object_hook is not None:                                                            │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/json/decoder.py:337 in decode                      │
│                                                                                                  │
│   334 │   │   containing a JSON document).                                                       │
│   335 │   │                                                                                      │
│   336 │   │   """                                                                                │
│ ❱ 337 │   │   obj, end = self.raw_decode(s, idx=_w(s, 0).end())                                  │
│   338 │   │   end = _w(s, end).end()                                                             │
│   339 │   │   if end != len(s):                                                                  │
│   340 │   │   │   raise JSONDecodeError("Extra data", s, end)                                    │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/json/decoder.py:355 in raw_decode                  │
│                                                                                                  │
│   352 │   │   try:                                                                               │
│   353 │   │   │   obj, end = self.scan_once(s, idx)                                              │
│   354 │   │   except StopIteration as err:                                                       │
│ ❱ 355 │   │   │   raise JSONDecodeError("Expecting value", s, err.value) from None               │
│   356 │   │   return obj, end                                                                    │
│   357                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/query_constructor/b │
│ ase.py:37 in parse                                                                               │
│                                                                                                  │
│    34 │   │   try:                                                                               │
│    35 │   │   │   expected_keys = ["query", "filter"]                                            │
│    36 │   │   │   allowed_keys = ["query", "filter", "limit"]                                    │
│ ❱  37 │   │   │   parsed = parse_and_check_json_markdown(text, expected_keys)                    │
│    38 │   │   │   if len(parsed["query"]) == 0:                                                  │
│    39 │   │   │   │   parsed["query"] = " "                                                      │
│    40 │   │   │   if parsed["filter"] == "NO_FILTER" or not parsed["filter"]:                    │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/output_parsers/json.py:34  │
│ in parse_and_check_json_markdown                                                                 │
│                                                                                                  │
│   31 │   try:                                                                                    │
│   32 │   │   json_obj = parse_json_markdown(text)                                                │
│   33 │   except json.JSONDecodeError as e:                                                       │
│ ❱ 34 │   │   raise OutputParserException(f"Got invalid JSON object. Error: {e}")                 │
│   35 │   for key in expected_keys:                                                               │
│   36 │   │   if key not in json_obj:                                                             │
│   37 │   │   │   raise OutputParserException(                                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OutputParserException: Got invalid JSON object. Error: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/IPython/core/magics/execution.py:132 │
│ 5 in time                                                                                        │
│                                                                                                  │
│   1322 │   │   else:                                                                             │
│   1323 │   │   │   st = clock2()                                                                 │
│   1324 │   │   │   try:                                                                          │
│ ❱ 1325 │   │   │   │   exec(code, glob, local_ns)                                                │
│   1326 │   │   │   │   out=None                                                                  │
│   1327 │   │   │   │   # multi-line %%time case                                                  │
│   1328 │   │   │   │   if expr_val is not None:                                                  │
│ in <module>:1                                                                                    │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/base.py:149 in      │
│ __call__                                                                                         │
│                                                                                                  │
│   146 │   │   │   )                                                                              │
│   147 │   │   except (KeyboardInterrupt, Exception) as e:                                        │
│   148 │   │   │   run_manager.on_chain_error(e)                                                  │
│ ❱ 149 │   │   │   raise e                                                                        │
│   150 │   │   run_manager.on_chain_end(outputs)                                                  │
│   151 │   │   final_outputs: Dict[str, Any] = self.prep_outputs(                                 │
│   152 │   │   │   inputs, outputs, return_only_outputs                                           │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/base.py:143 in      │
│ __call__                                                                                         │
│                                                                                                  │
│   140 │   │   )                                                                                  │
│   141 │   │   try:                                                                               │
│   142 │   │   │   outputs = (                                                                    │
│ ❱ 143 │   │   │   │   self._call(inputs, run_manager=run_manager)                                │
│   144 │   │   │   │   if new_arg_supported                                                       │
│   145 │   │   │   │   else self._call(inputs)                                                    │
│   146 │   │   │   )                                                                              │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/conversational_retr │
│ ieval/base.py:110 in _call                                                                       │
│                                                                                                  │
│   107 │   │   │   )                                                                              │
│   108 │   │   else:                                                                              │
│   109 │   │   │   new_question = question                                                        │
│ ❱ 110 │   │   docs = self._get_docs(new_question, inputs)                                        │
│   111 │   │   new_inputs = inputs.copy()                                                         │
│   112 │   │   new_inputs["question"] = new_question                                              │
│   113 │   │   new_inputs["chat_history"] = chat_history_str                                      │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/conversational_retr │
│ ieval/base.py:191 in _get_docs                                                                   │
│                                                                                                  │
│   188 │   │   return docs[:num_docs]                                                             │
│   189 │                                                                                          │
│   190 │   def _get_docs(self, question: str, inputs: Dict[str, Any]) -> List[Document]:          │
│ ❱ 191 │   │   docs = self.retriever.get_relevant_documents(question)                             │
│   192 │   │   return self._reduce_tokens_below_limit(docs)                                       │
│   193 │                                                                                          │
│   194 │   async def _aget_docs(self, question: str, inputs: Dict[str, Any]) -> List[Document]:   │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/retrievers/self_query/base │
│ .py:96 in get_relevant_documents                                                                 │
│                                                                                                  │
│    93 │   │   inputs = self.llm_chain.prep_inputs({"query": query})                              │
│    94 │   │   structured_query = cast(                                                           │
│    95 │   │   │   StructuredQuery,                                                               │
│ ❱  96 │   │   │   self.llm_chain.predict_and_parse(callbacks=callbacks, **inputs),               │
│    97 │   │   )                                                                                  │
│    98 │   │   if self.verbose:                                                                   │
│    99 │   │   │   print(structured_query)                                                        │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/llm.py:281 in       │
│ predict_and_parse                                                                                │
│                                                                                                  │
│   278 │   │   )                                                                                  │
│   279 │   │   result = self.predict(callbacks=callbacks, **kwargs)                               │
│   280 │   │   if self.prompt.output_parser is not None:                                          │
│ ❱ 281 │   │   │   return self.prompt.output_parser.parse(result)                                 │
│   282 │   │   else:                                                                              │
│   283 │   │   │   return result                                                                  │
│   284                                                                                            │
│                                                                                                  │
│ /home/cloud/anaconda3/envs/mir/lib/python3.10/site-packages/langchain/chains/query_constructor/b │
│ ase.py:50 in parse                                                                               │
│                                                                                                  │
│    47 │   │   │   │   **{k: v for k, v in parsed.items() if k in allowed_keys}                   │
│    48 │   │   │   )                                                                              │
│    49 │   │   except Exception as e:                                                             │
│ ❱  50 │   │   │   raise OutputParserException(                                                   │
│    51 │   │   │   │   f"Parsing text\n{text}\n raised following error:\n{e}"                     │
│    52 │   │   │   )                                                                              │
│    53                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OutputParserException: Parsing text
json  "query": "patient medical notes", "ID": "11542052"  "old": ""
 raised following error:
Got invalid JSON object. Error: Expecting value: line 1 column 1 (char 0)

### Who can help?

@hwchase17 
@ag

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

model_id = 'google/flan-t5-xl'
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline(
  model=model_id,
  tokenizer=tokenizer,
  max_length=2048,
  temperature=0.1,
  top_p=0.95,
  repetition_penalty=1.
)

llm = HuggingFacePipeline(pipeline=pipe)
document_content_description = "Patient medical notes"
metadata_field_info = [
    AttributeInfo(
        name="ID",
        description="The unique identifier 'ID' of the patient",
        type="string",
    ),
    AttributeInfo(
        name="source",
        description="source of the document",
        type="string",
    ),
]
retriever = SelfQueryRetriever.from_llm(
    llm,
    db,
    document_content_description,
    metadata_field_info,
    verbose = True
)
memory=ConversationBufferMemory(memory_key="chat_history",output_key='answer')
chain = ConversationalRetrievalChain.from_llm(
llm = llm,
retriever=retriever,
memory=memory, 
get_chat_history=lambda h :h)

### Expected behavior

Expected behavior should be something as attached
![outputofvicuna](https://github.com/hwchase17/langchain/assets/95228400/17ef861a-43c1-4159-9ac1-59292b8dd9e1)