delayedkarma/langchain-issues huggingface dataset:
PydanticOutputParser failing to parse basic string into JSON
Labels: name: auto:bug, name: area: models

### System Info

Given a string resembling a JSON-string array, such as I'm often getting from LLM completions, I'm trying to convert the string to a Python list of strings. I'm finding the the `PydanticOutputParser` is throwing an error when plain old `json.loads` with `strict=False` is doing fine.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

For example this code:
```python
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
import json

class Lines(BaseModel):
    lines: list[str] = Field(description="array of strings")

line_parser = PydanticOutputParser(pydantic_object=Lines)
lines = '[\n  "line 1",\n  "line 2"\n]'

print("Just JSON:")
print(json.loads(lines, strict=False))
print("LangChain Pydantic:")
print(line_parser.parse(lines))
```

produces the following output:
```sh
Just JSON:
['line 1', 'line 2']
LangChain Pydantic:
Traceback (most recent call last):
  File "/opt/miniconda3/envs/gt/lib/python3.10/site-packages/langchain/output_parsers/pydantic.py", line 27, in parse
    json_object = json.loads(json_str, strict=False)
  File "/opt/miniconda3/envs/gt/lib/python3.10/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
  File "/opt/miniconda3/envs/gt/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/miniconda3/envs/gt/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/benjaminbasseri/Dropbox/Work/GT Schools/scratch work/meme-generation/parse_test.py", line 14, in <module>
    print(line_parser.parse(lines))
  File "/opt/miniconda3/envs/gt/lib/python3.10/site-packages/langchain/output_parsers/pydantic.py", line 33, in parse
    raise OutputParserException(msg, llm_output=text)
langchain.schema.output_parser.OutputParserException: Failed to parse Lines from completion [
  "line 1",
  "line 2"
]. Got: Expecting value: line 1 column 1 (char 0)
```

### Expected behavior

I would expect the parser to be able to handle a simple case like this
Fixing _InactiveRpcError in MatchingEngine vectorstore
Labels: name: area: vector store, name: auto:bug

  - Description: There was an issue with the MatchingEngine VectorStore, preventing from using it with a public endpoint. In the Google Cloud library there are two similar methods for private or public endpoints : `match()` and `find_neighbors()`.
  - Issue: Fixes #8378 
  - This uses the `google.cloud.aiplatform` library : https://github.com/googleapis/python-aiplatform/blob/main/google/cloud/aiplatform/matching_engine/matching_engine_index_endpoint.py
Update chroma.py
Labels: name: area: vector store, name: auto:nit

- Description: add minor fix codes in  langchain.vectorstors.chroma  to return distances value .
  so 
    qa=RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=retriever, #a chroma db 
        return_source_documents=True,
        chain_type_kwargs={"prompt": prompt, "memory": memory},
    )
  res = qa(query)
  Now res can return distances value in the metadata dict. 
 
  answer, docs = res["result"], res["source_documents"]
  for document in docs:
       print("\n> [source file path]: " + document.metadata["source"] )
       print("> [cosine  similarity]:" + str(1.0-document.metadata["distances"]) )   # Add This  and it will print 1-distance_value
       print(">[text slice]:" + document.page_content)

    It is a very minor change.  I tested on the [https://github.com/PromtEngineer/localGPT/blob/main/run_localGPT.py#L237]
  
- Issue: the issue #4710 #5416 Maybe this fix is not directly with the issues , but I guess the intuition is alike.
- Dependencies: same as the main branch
- Tag maintainer: @baskaryan, @eyurtsev, @hwchase17, @rlancemartin
StreamingStdOutCallbackHandler().on_llm_start isn't called since version 0.0.272 (0.0.271 still works)
Labels: name: auto:bug, name: area: models

### System Info

0.0.272, linux, python 3.11.2 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

I have a pretty standard RetievalQA chain like this. The `on_llm_start` callback isn't executed since version 0.0.272 (I verified that 0.0.271 is working)

```
    loader = TextLoader("data.txt")
    documents = loader.load()
    openai = ChatOpenAI(**llm_params, callbacks= [StreamingStdOutCallbackHandler()])

    text_splitter = RecursiveCharacterTextSplitter(**chunking_params)
    texts = text_splitter.split_documents(documents)

    embeddings = OpenAIEmbeddings()
    docsearch = Chroma.from_documents(texts, embeddings)

    qa = RetrievalQA.from_chain_type(
        llm=openai,
        retriever=docsearch.as_retriever(),
    )
```

### Expected behavior

`on_llm_start` callback should be called
Add XMLOutputParser
Labels: name: auto:enhancement, name: area: models

**Description**
Adds new output parser, this time enabling the output of LLM to be of an XML format. Seems to be particularly useful together with Claude model. 
Addresses [issue 9820](https://github.com/langchain-ai/langchain/issues/9820).

**Twitter handle**
@deepsense_ai @matt_wosinski
Add sqlite-vss as a vector database
Labels: name: area: vector store, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
This adds sqlite-vss as an option for a vector database. Contains the code and a few tests. Tests are passing and the library sqlite-vss is added as optional as explained in the contributing guidelines. I adjusted the code for lint/black/ and mypy. It looks that everything is currently passing.

Adding sqlite-vss was mentioned in this issue: https://github.com/langchain-ai/langchain/issues/1019. 
Also mentioned here in the sqlite-vss repo for the curious: https://github.com/asg017/sqlite-vss/issues/66

Maintainer tag: @baskaryan
Feature/add sqlite vss
Labels: name: area: vector store, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
`create_pandas_dataframe_agent` does not pass the `handle_parsing_error` variable into the underlying Agent.
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

Python Version: 3.11.4 (main, Jul  5 2023, 08:40:20) [Clang 14.0.6 ]
Langchain Version: 0.0.273
Jupyter Notebook Version: 5.3.0

### Who can help?

@hwchase17 
@agola

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce the behaviour:
1. Create a function to be used 
```
def _handle_error(error) -> str:  
    return str(error)[:50]
```
2. Run `create_pandas_dataframe_agent`, pass in `handle_parsing_errors=_handle_error`


### Expected behavior

I expect an output similar to the last section of https://python.langchain.com/docs/modules/agents/how_to/handle_parsing_errors, but instead I still get the normal OutputParserError
Getting invalid URL post after updating langchain from 0.0.200 to 0.0.267 
Labels: name: auto:bug, name: area: models

### System Info

`langchain` -  0.0.267 
`openai` - 0.27.7
MacOS 13.4.1 (22F82)

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

This code was working when I used `langchain 0.0.200` (The call is to `AzureOpenAI` endpoint)
```python
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name=model_config.model_engine,
                 deployment_id=model_config.model_engine,
                 temperature=model_config.temperature,
                 max_tokens=model_config.max_tokens_for_request,
                 top_p=model_config.top_p,
                 openai_api_key=endpoint_config.api_key.secret,
                 api_base=endpoint_config.api_base,
                 api_type=endpoint_config.api_type,
                 api_version=endpoint_config.api_version)

chat_prompt = [SystemMessage(...), HumanMessage(...)]
response = llm(chat_prompt)
```

Once I updated `langchain` to `0.0.267`, I'm getting this error
```shell 
openai.error.InvalidRequestError: Invalid URL (POST /v1/openai/deployments/gpt-35-turbo/chat/completions)
```

The endpoint itself is working (once I reverted back to `0.0.200` all started to work again)

### Expected behavior

I except to get an answer from the LLM
Function calling use the wrong context
Labels: name: area: memory, name: auto:bug, name: auto:question

### Issue you'd like to raise.

I'm using function calls to obtain and process data. I have two classes designed as follows:

```python
class GetDataInput:
    user_input: str = Field(description="User input")

class ProcessDataInput:
    data_url: str = Field(description="Link to the data")
```

Moreover, I'm employing the `ConversationBufferWindowMemory`.

While the initial question posed to the bot yields the expected response, I encounter an issue when asking a subsequent question. Occasionally, instead of fetching new data, the bot utilizes the previous `data_url`.

How can I effectively manage this kind of situation?

Thank you for your assistance.

### Suggestion:

_No response_
Impl chat message prompt type/dict (#10033)
Labels: name: auto:improvement, name: area: models

## Description
Dumping BaseMessagePromptTemplate to json and loading to original Template is difficult, as retrieving exact type of message type from json is not possible.
Therefore, I suggest adding _msg_type / _type property on BaseMessagePromptTemplate and utilized it on _dict and added load_message_prompt,
like In [BasePromptTemplate dict method](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/schema/prompt_template.py#L108-L116).

## Issue
#10033 

## Dependencies
None
Conversational Retrieval Chain.from_llm integration with router chains
Labels: name: area: vector store, name: auto:question, name: area: models

### Issue you'd like to raise.

The idea is, that I have a vector store with a conversational retrieval chain.from_llm, and I want to create other functions such as send an email, etc. so that when a user queries for something, it determines if it should use the Conv retrieval chain or the other functions such as sending an email function, and it seems I need to use the router to achieve this, but when I do, I get a lot of errors such as
 
```
"default_chain -> prompt
  field required (type=value_error.missing)
default_chain -> llm
  field required (type=value_error.missing)
default_chain -> combine_docs_chain
  extra fields not permitted (type=value_error.extra)
```
How do i integrate with conv retrieval chain with routers to achieve my purpose, the only examples i have seen do not use conv retrieval chain


### Suggestion:

_No response_
context to pydantic_object
Labels: name: auto:improvement, name: area: models

### Feature request

while parsing LLM output with `pydantic_object` it would be nice to send to the `parse` function `context` object , like `pydantic` docs does here: https://docs.pydantic.dev/latest/usage/validators/#validation-context

### Motivation

In order to do validation on LLM output rather than just parse it, in some cases to do the validation we need some context.

### Your contribution

This change may require to support `pydantic` v2 not sure about backword capabilities. 

in `PydanticOutputParser.parse` function insted of `parse_obj` we should use `model_validate` then we could send context object(it can be optional) to `model_validate`

from pydantic:

```python
    @typing_extensions.deprecated(
        'The `parse_obj` method is deprecated; use `model_validate` instead.', category=PydanticDeprecatedSince20
    )
    def parse_obj(cls: type[Model], obj: Any) -> Model:  # noqa: D102
        warnings.warn('The `parse_obj` method is deprecated; use `model_validate` instead.', DeprecationWarning)
        return cls.model_validate(obj)
```
Add message type property method on BaseMessagePromptTemplate
Labels: name: auto:improvement, name: area: models

### Feature request

I'm utilizing BaseMessagePromptTemplate with external DBs, for version control.
However, dumping BaseMessagePromptTemplate to json and loading to original Template is difficult, as retrieving exact type of message type from json is not possible.
Therefore, it would be useful to add _msg_type property on BaseMessagePromptTemplate, 
like In [BasePromptTemplate dict method](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/schema/prompt_template.py#L108-L116).


### Motivation

Dumping ChatPromptTemplate to json and loading to original Template is difficult,
as retrieving exact type of MessageLike (Union[BaseMessagePromptTemplate, BaseMessage, BaseChatPromptTemplate]) property is difficult.

### Your contribution

If you allow me, I'd like to make a pull request regarding to this.
Add AliCloudPaiEas class in langchain.llms to access alicloud pai-eas service.
Labels: name: auto:enhancement, name: area: models

### Description
  This PR aims at adding new llm type  AliCloudPaiEAS to support service accessing for alicloud product named pai-eas service. PAI-EAS can help users to deploy llm models on alicloud.  

### Usage
The user needs to start an eas service on alicloud and then set the service's access url and token as a params for AliCloudPaiEAS. Then AliCloudPaiEAS  object as a llm object is passed to the Chain to run an output. 
The usage is like:
`llm = AliCloudPaiEAS(eas_service_url="your_access_url",eas_service_token="your_access_token"`
`llm_chain = LLMChain(prompt=prompt, llm=llm)
`
`llm_chain.run("Hello there.")`

### Test and example
The integration test case is located at libs/langchain/tests/integration_tests/llms/test_alicloud_eas.py
The example notebook is located at docs/extras/integrations/lls/alicloud_api_eas.ipynb
Does anyone created agent with tool which takes "dataframe" and "user_input" as input variables in langchain?
Labels: name: area: agent, name: auto:question

### Issue you'd like to raise.

Does anyone created agent with tool which takes "dataframe" and "user_input" as input variables in langchain?
I do not want use dataframe agent which is there already in langchain. As I need pass further instruction in the prompt template.

### Suggestion:

_No response_
Issue: AttributeError: 'list' object has no attribute 'embedding'
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### Issue you'd like to raise.

Hello, 
I am facing some issues with the code I am trying to run as it did run perfectly up until recently when an Error started being thrown out as AttributeError: 'list' object has no attribute 'embedding'. Below is the traceback if the error. Please do let me know if some code snippet excerpts will also be needed to facilitate debugging the code.

Traceback:
File "/home/ataliba/llm/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
    exec(code, module.__dict__)
File "/home/ataliba/LLM_Workshop/Experimental_Lama_QA_Retrieval/Andro_GPT_Llama2.py", line 268, in <module>
    response = qa_chain.run(user_query, callbacks=[cb])
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/chains/base.py", line 481, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/chains/base.py", line 288, in __call__
    raise e
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/chains/base.py", line 282, in __call__
    self._call(inputs, run_manager=run_manager)
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/chains/conversational_retrieval/base.py", line 134, in _call
    docs = self._get_docs(new_question, inputs, run_manager=_run_manager)
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/chains/conversational_retrieval/base.py", line 286, in _get_docs
    docs = self.retriever.get_relevant_documents(
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/schema/retriever.py", line 208, in get_relevant_documents
    raise e
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/schema/retriever.py", line 201, in get_relevant_documents
    result = self._get_relevant_documents(
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/vectorstores/base.py", line 571, in _get_relevant_documents
    docs = self.vectorstore.max_marginal_relevance_search(
File "/home/ataliba/llm/lib/python3.10/site-packages/langchain/vectorstores/docarray/base.py", line 197, in max_marginal_relevance_search
    np.array(query_embedding), docs.embedding, k=k

### Suggestion:

_No response_
add bloomz_7b, llama-2-7b, llama-2-13b, llama-2-70b to ErnieBotChat
Labels: name: auto:improvement, name: area: models

  - Description: Add bloomz_7b, llama-2-7b, llama-2-13b, llama-2-70b to ErnieBotChat, which only supported ERNIE-Bot-turbo and ERNIE-Bot.
  - Issue: #10022,
  - Dependencies: no extra dependencies
Support more models for ErnieBotChat
Labels: name: auto:improvement, name: area: models

### Feature request

The class ErnieBotChat defined in libs/langchain/langchain/chat_models/ernie.py only supports 2 models, which are ERNIE-Bot-turbo and ERNIE-Bot. While a bunch of new models are supported by BCE (Baidu Cloud Engine), such as llama-2-7b chat (https://cloud.baidu.com/doc/WENXINWORKSHOP/s/Rlki1zlai). It would be better that this class supported more models. 

### Motivation

When I was using ErnieBotChat, I've found that ErnieBotChat cannot recognize models other than  ERNIE-Bot-turbo and ERNIE-Bot. Instead, it raised an error "Got unknown model_name {self.model_name}".

### Your contribution

If possible, I would be happy to help resolve this issue. I plan to add all models BCE (Baidu Cloud Engine) supports at this time(2023-8-31). The fixes will be simple, just add more cases around line 106 in the file  libs/langchain/langchain/chat_models/ernie.py  from the master branch.
fix: Loading documents from a Youtube Url
Labels: name: area: doc loader, name: auto:bug, name: auto:improvement

### System Info

MacOS M2 13.4.1 (22F82)

### Who can help?

@eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce behaviour:
1. Run the [tutorial](https://python.langchain.com/docs/integrations/document_loaders/youtube_audio) with the default parameters `save_dir = "~/Downloads/YouTube"`
2. After calling `docs = loader.load()` the docs will be empty



I have implemented a dummy fix for the interim. 

The error is here in this file: from langchain.document_loaders.blob_loaders.youtube_audio import YoutubeAudioLoader

`YouTubeAudioLoader.yield_blobs` method
loader = FileSystemBlobLoader(self.save_dir, glob="*.m4a")

```
       # This doesn't always work (MacOS)
        loader = FileSystemBlobLoader(self.save_dir, glob="*.m4a")
```

The reason it doesn't work is that it's trying to use ~/Downloads/YouTube.

The fix I propose is either:
- Use the FULL file path in `save_dir` in the tutorial.
- Replace the problematic line with this, so that it finds the actual directory, even if you prefer to use `~` for specifying file paths.

```
        loader = FileSystemBlobLoader(os.path.expanduser(self.save_dir), glob="*.m4a")
```

### Expected behavior

There should be documents in the loader.load() variable.


### My Fix
```
   # Yield the written blobs
        """
        you could fix save_dir like this...

        (old)
        save_dir = "~/Downloads/YouTube"
        (new)
        "/Users/shawnesquivel/Downloads/YouTube"

        """


        # This doesn't always work (MacOS)
        loader = FileSystemBlobLoader(self.save_dir, glob="*.m4a")

        # This works
        loader = FileSystemBlobLoader(os.path.expanduser(self.save_dir), glob="*.m4a")

```