delayedkarma/langchain-issues huggingface dataset:
ValidationError: 2 validation errors for PydanticOutputFunctionsParser
Labels: name: auto:bug, name: area: models

### System Info

I was just trying to run the Tagging tutorial (no code modification on colab). 
https://python.langchain.com/docs/use_cases/tagging

And on the below code part,
```chain = create_tagging_chain_pydantic(Tags, llm)```

I got this error.
```
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
[<ipython-input-8-4724aee0c891>](https://localhost:8080/#) in <cell line: 1>()
----> 1 chain = create_tagging_chain_pydantic(Tags, llm)

2 frames
[/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py](https://localhost:8080/#) in __init__(__pydantic_self__, **data)
    339         values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340         if validation_error:
--> 341             raise validation_error
    342         try:
    343             object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 2 validation errors for PydanticOutputFunctionsParser
pydantic_schema
  subclass of BaseModel expected (type=type_error.subclass; expected_class=BaseModel)
pydantic_schema
  value is not a valid dict (type=type_error.dict)
```

Is this a bug? 

langchain version
```
!pip show langchain

Name: langchain
Version: 0.0.274
Summary: Building applications with LLMs through composability
Home-page: https://github.com/langchain-ai/langchain
Author: 
Author-email: 
License: MIT
Location: /usr/local/lib/python3.10/dist-packages
Requires: aiohttp, async-timeout, dataclasses-json, langsmith, numexpr, numpy, pydantic, PyYAML, requests, SQLAlchemy, tenacity
Required-by: 
```



### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just by running the colab notebook on the tagging tutorial.
No modification applied.
https://python.langchain.com/docs/use_cases/tagging

### Expected behavior

Finishing running the notebook without any issues.
Update SeleniumURLLoader to use webdriver Service in favor of deprecated executable_path parameter
Labels: name: area: doc loader, name: auto:improvement

Description: This commit uses the new Service object in Selenium webdriver as executable_path has been [deprecated and removed in selenium version 4.11.2](https://github.com/SeleniumHQ/selenium/commit/9f5801c82fb3be3d5850707c46c3f8176e3ccd8e)
Issue: https://github.com/langchain-ai/langchain/issues/9808
Tag Maintainer: @eyurtsev

Note -
First attempt was to create a Service object and pass empty if executable_path not provided as such:
`def _get_driver(self) -> Union["Chrome", "Firefox"]:
"""Create and return a WebDriver instance based on the specified browser.

    Raises:
        ValueError: If an invalid browser is specified.

    Returns:
        Union[Chrome, Firefox]: A WebDriver instance for the specified browser.
    """
    if self.browser.lower() == "chrome":
        from selenium.webdriver import Chrome
        from selenium.webdriver.chrome.options import Options as ChromeOptions
        from selenium.webdriver.chrome.service import Service

        chrome_options = ChromeOptions()
        chrome_service = Service()

        for arg in self.arguments:
            chrome_options.add_argument(arg)

        if self.headless:
            chrome_options.add_argument("--headless")
            chrome_options.add_argument("--no-sandbox")
        if self.binary_location is not None:
            chrome_options.binary_location = self.binary_location
        if self.executable_path is None:
            chrome_service.path = self.executable_path
        return Chrome(options=chrome_options, service=chrome_service)
    elif self.browser.lower() == "firefox":
        from selenium.webdriver import Firefox
        from selenium.webdriver.firefox.options import Options as FirefoxOptions
        from selenium.webdriver.firefox.service import Service

        firefox_options = FirefoxOptions()
        firefox_service = Service()

        for arg in self.arguments:
            firefox_options.add_argument(arg)

        if self.headless:
            firefox_options.add_argument("--headless")
        if self.binary_location is not None:
            firefox_options.binary_location = self.binary_location
        if self.executable_path is None:
            firefox_service.path = self.executable_path
        return Firefox(options=firefox_options, service=firefox_service)
    else:
        raise ValueError("Invalid browser specified. Use 'chrome' or 'firefox'.")`
However in testing I found this crashes when executable_path is not passed in with the follow Exception: NoSuchDriverException(f"Unable to locate or obtain driver for {options.capabilities['browserName']}"). This updated pull request revises to only pass Service object when executable_path is provided
Update SeleniumURLLoader to use webdriver Service in favor of deprecated executable_path parameter
Labels: name: area: doc loader, name: auto:improvement

Description: This commit uses the new Service object in Selenium webdriver as executable_path has been [deprecated and removed in selenium version 4.11.2](https://github.com/SeleniumHQ/selenium/commit/9f5801c82fb3be3d5850707c46c3f8176e3ccd8e)
Issue: https://github.com/langchain-ai/langchain/issues/9808
Tag Maintainer: @eyurtsev 


Note - 
First attempt was to create a Service object and pass empty if executable_path not provided as such: 
`def _get_driver(self) -> Union["Chrome", "Firefox"]:
        """Create and return a WebDriver instance based on the specified browser.

        Raises:
            ValueError: If an invalid browser is specified.

        Returns:
            Union[Chrome, Firefox]: A WebDriver instance for the specified browser.
        """
        if self.browser.lower() == "chrome":
            from selenium.webdriver import Chrome
            from selenium.webdriver.chrome.options import Options as ChromeOptions
            from selenium.webdriver.chrome.service import Service

            chrome_options = ChromeOptions()
            chrome_service = Service()

            for arg in self.arguments:
                chrome_options.add_argument(arg)

            if self.headless:
                chrome_options.add_argument("--headless")
                chrome_options.add_argument("--no-sandbox")
            if self.binary_location is not None:
                chrome_options.binary_location = self.binary_location
            if self.executable_path is None:
                chrome_service.path = self.executable_path
            return Chrome(options=chrome_options, service=chrome_service)
        elif self.browser.lower() == "firefox":
            from selenium.webdriver import Firefox
            from selenium.webdriver.firefox.options import Options as FirefoxOptions
            from selenium.webdriver.firefox.service import Service

            firefox_options = FirefoxOptions()
            firefox_service = Service()

            for arg in self.arguments:
                firefox_options.add_argument(arg)

            if self.headless:
                firefox_options.add_argument("--headless")
            if self.binary_location is not None:
                firefox_options.binary_location = self.binary_location
            if self.executable_path is None:
                firefox_service.path = self.executable_path
            return Firefox(options=firefox_options, service=firefox_service)
        else:
            raise ValueError("Invalid browser specified. Use 'chrome' or 'firefox'.")`

However in testing I found this crashes when executable_path is not passed in with the follow Exception: NoSuchDriverException(f"Unable to locate or obtain driver for {options.capabilities['browserName']}"). This updated pull request revises to only pass Service object when executable_path is provided
Renaming RetrievalQAWithSources to RetrievalQAWithCitations
Labels: name: auto:improvement, name: area: models

### Feature request

Hey, I'm trying to gather some empirical evidence that RetrievalQAWithSources chain often hallucinates to returns all sources rather than cite them.
Current Issue:
Assuming you have a retriever that returns 4 sources, RetrievalQAWithSources gets confused with the below prompt:
Given the following extracted parts of a long document and a question, create a final answer with references ("SOURCES").
and usually returns all the 4 sources that were fetched by the retriever. 
Possible Solution:
Given the following extracted parts of a long document and a question, create a final answer and also include all references to cite your answer in CITATIONS.

### Motivation

Having it named as "SOURCES" often leads to confusion in the prompt.
"cite" involves referring to, acknowledging, or using a source or example to support a claim, idea, or statement.

### Your contribution

I can create a PR to carry forward this change, across the various places we use Citations. Please let me know the kind of evidence or information I'll have to present to make my case for it.
ImportError: cannot import name 'StreamlitCallbackHandler' from 'langchain.callbacks' 
Labels: name: auto:bug, name: area: models

### System Info

**Background**
I'm trying to run the Streamlit Callbacks example

https://python.langchain.com/docs/integrations/callbacks/streamlit

My code is copy + paste but I get an error from the LangChain library import.

**Error**
```
ImportError: cannot import name 'StreamlitCallbackHandler' from 'langchain.callbacks' (/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/callbacks/__init__.py)
```

**What I tried**
- Upgrading LangChain `pip install langchain --upgrade`
- Reinstall `pip uninstall langchain` then `pip install langchain`


**Version**
langchain 0.0.274                 
streamlit 1.24.1 

**System**
Mac M2 Ventura –13.4.1 


### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Run the code snippet `streamlit run app.py`
2. Error message

**Run the code with `streamlit run app.py`**
```
import streamlit as st
from langchain.llms import OpenAI
from langchain.agents import AgentType, initialize_agent, load_tools
from langchain.callbacks import StreamlitCallbackHandler

from dotenv import find_dotenv, load_dotenv
load_dotenv(find_dotenv())

st_callback = StreamlitCallbackHandler(st.container())

llm = OpenAI(temperature=0, streaming=True)
tools = load_tools(["ddg-search"])
agent = initialize_agent(
    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)

if prompt := st.chat_input():
    st.chat_message("user").write(prompt)
    with st.chat_message("assistant"):
        st_callback = StreamlitCallbackHandler(st.container())
        response = agent.run(prompt, callbacks=[st_callback])
        st.write(response)
```

### Expected behavior

Should look like the tutorial

<img width="844" alt="image" src="https://github.com/langchain-ai/langchain/assets/94336773/f5a5f166-4013-4c2b-9776-b268156c41f8">
Update SeleniumURLLoader to use webdriver Service in favor of deprecated executable_path parameter
Labels: name: area: doc loader, name: auto:improvement

Description: This commit uses the new Service object in Selenium webdriver as executable_path has been [deprecated and removed in selenium version 4.11.2](https://github.com/SeleniumHQ/selenium/commit/9f5801c82fb3be3d5850707c46c3f8176e3ccd8e)
Issue: [9808](https://github.com/langchain-ai/langchain/issues/9808)
Tag Maintainer: @eyurtsev
Issue: Selenium Webdriver parameter executable_path deprecated
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain version 0.0.273
selenium version 4.11.2

In version 4.11.2, [Selenium fully deprecated and removed executable_path parameter from webdriver](https://github.com/SeleniumHQ/selenium/commit/9f5801c82fb3be3d5850707c46c3f8176e3ccd8e) in favor of using Service class object to pass in path/executable_path parameter. This results in a crash when using SeleniumURLLoader with executable_path upon upgrading selenium to 4.11.2. 

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Upgrade selenium to version 4.11.2. Load SeleniumURLLoader with executable_path parameter. 

### Expected behavior

Expected Result: SeleniumURLLoader is instantiated 
Actual Result: ERROR: WebDriver.__init__() got an unexpected keyword argument 'executable_path'
It should be possible to pass client to SagemakerEndpointEmbeddings & SagemakerEndpoint class
Labels: name: area: embeddings, name: auto:bug, name: auto:improvement, name: area: models

### System Info

LangChain 0.0.274, Python



### Who can help?

@hwchase17 @agola

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When initializing the the SagemakerEndpointEmbeddings & SagemakerEndpoint class, I pass the client but still get the error:  Could not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid. (type=value_error)
    content_handler = ContentHandler_LLM()
    embeddings = SagemakerEndpoint(
        client= get_sagemaker_client(),
        endpoint_name= endpointName,
        model_kwargs  = model_kwargs ,
        content_handler=content_handler,
    )
content_handler = ContentHandler_Embeddings()
embeddings = SagemakerEndpointEmbeddings(
    client= get_sagemaker_client(),
    endpoint_name= endpointName,
    content_handler=content_handler,
)
As you can see from the documentation below, the validate_environment function does not use the client that I am passing and instead tries creating its own client which causes the issue:
SOURCE: https://api.python.langchain.com/en/latest/_modules/langchain/embeddings/sagemaker_endpoint.html#SagemakerEndpointEmbeddings
SOURCE: https://api.python.langchain.com/en/latest/_modules/langchain/llms/sagemaker_endpoint.html#SagemakerEndpoint



### Expected behavior

Passing the client works with BedrockEmbeddings class. The validate_environment function checks if there is value in client, then it just returns the existing values. You can see from below snippet in the class:
SOURCE: https://api.python.langchain.com/en/latest/_modules/langchain/embeddings/bedrock.html
 @root_validator()
    def validate_environment(cls, values: Dict) -> Dict:
        """Validate that AWS credentials to and python package exists in environment."""

        if values["client"] is not None:
            return values

Please fix this in both the SagemakerEndpointEmbeddings and  SagemakerEndpoint classes to check if value already has client then just return the values as done in BedrockEmbeddings class above.

Meanwhile, is there a work around for this?
DOC: How to Connect Database Agent with SqlServer
Labels: name: area: agent, name: auto:question, name: auto:documentation, name: stale

### Issue with current documentation:

from langchain.agents import create_sql_agent
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.sql_database import SQLDatabase
from langchain.llms.openai import OpenAI
from langchain.agents import AgentExecutor
from langchain.agents.agent_types import AgentType
from langchain.chat_models import ChatOpenAI


db = SQLDatabase.from_uri("sqlite:///../../../../../notebooks/Chinook.db")
toolkit = SQLDatabaseToolkit(db=db, llm=OpenAI(temperature=0))

# how to connect to sql server instead of sqlite???


### Idea or request for content:

I need to connect with my sqlserver database, but the documention does not explain how to connect langchain database agent to sqlserver.
GoogleDriveLoader - AttributeError: 'Credentials' object has no attribute 'with_scopes'
Labels: name: area: doc loader, name: auto:bug

### System Info

Google Auth version 2.22.0
Python version 3.8.8
langchain version: 0.0.273

### Who can help?

@eyurtsev the GoogleDriveLoader does not seem to be working as expected. I get a error of AttributeError: 'Credentials' object has no attribute 'with_scopes'.

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I run the code below (and many other versions of the Google Drive code) to import data from Google Drive. However, no matter what I try I get an error of "AttributeError: 'Credentials' object has no attribute 'with_scopes'". The troubleshooting seems to indicate incompatibility with version of Google Auth and Python, but the versions are compatible. I have gone through all the documentation and tutorials online about connecting Google Drive to Langchain, and I have complete and verified every step multiple times. I am at a loss of what else to try to get langchain to connect to Google Drive. Thanks in advance for your help!!!



from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.document_loaders import GoogleDriveLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma

folder_id = "1ExuF7GUaeDzJpuDn8ThH6t8LBcmjAKE_"
loader = GoogleDriveLoader(
    folder_id=folder_id,
    recursive=False
)
docs = loader.load()

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=4000, chunk_overlap=0, separators=[" ", ",", "\n"]
    )

texts = text_splitter.split_documents(docs)
embeddings = OpenAIEmbeddings()
db = Chroma.from_documents(texts, embeddings)
retriever = db.as_retriever()

llm = ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo")
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever)

while True:
    query = input("> ")
    answer = qa.run(query)
    print(answer)

### Expected behavior

I expect langchain to be able to connect to me Google Drive files and folders.
Issue: vector store supports custom database connections?
Labels: name: area: vector store, name: auto:question, name: auto:enhancement

### Issue you'd like to raise.

May I inquire whether the vector stroe accommodates the provision for customized database connection pooling?

Typically, opening a database connection is an expensive operation, especially if the database is remote.  Pooling keeps the connections active so that, when a connection is later requested, one of the active ones is used in preference to having to create another one.



### Suggestion:

Maybe it could be like this?

```python
engine = sqlalchemy.create_engine(connection_string, pool_size=10)
conn = engine.connect()

store = PGVector(
    custom_connection=conn,
    embedding_function=embeddings
)
```
Add support tags and dataview fields to ObsidianLoader
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

Obsidian markdown documents frequently have additional metadata beyond what is in the frontmatter: tags within the document, and (for many users) dataview plugin values.

Add support for this.

### Motivation

Surfacing tags and dataview fields would unlock more abilities for self-querying obsidian data (https://python.langchain.com/docs/modules/data_connection/retrievers/self_query/chroma_self_query)

### Your contribution

I plan to make a PR for this.
Issue: RuntimeWarning: coroutine 'AsyncCallbackHandler.on_chat_model_start' was never awaited
Labels: name: area: memory, name: area: agent, name: auto:bug, name: area: models

### Issue you'd like to raise.

Hello, I tried to use AsyncFinalIteratorCallbackHandler (which inherits from AsyncCallbackHandler) to implement async streaming, then encountered following issue:

libs/langchain/langchain/callbacks/manager.py:301: RuntimeWarning: coroutine 'AsyncCallbackHandler.on_chat_model_start' was never awaited
  getattr(handler, event_name)(*args, **kwargs)

following is the demo:

`
import os
import asyncio
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory
from langchain.agents import AgentType
from langchain.agents import initialize_agent
from langchain.callbacks.streaming_aiter_final_only import AsyncFinalIteratorCallbackHandler

os.environ["OPENAI_API_KEY"] = "<your openai key>"

async_streaming_handler = AsyncFinalIteratorCallbackHandler()

memory = ConversationBufferMemory(
    memory_key="chat_history", return_messages=True)
llm = ChatOpenAI(temperature=0, streaming=True, model="gpt-3.5-turbo-16k-0613")
agent_chain = initialize_agent(
    [], llm, agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, verbose=True, memory=memory)


async def streaming():
    result = agent_chain.run(
        'how to gain good reviews from customer?', callbacks=[async_streaming_handler])
    print(f"result: {result}")
    while True:
        token = await async_streaming_handler.queue.get()
        print(f"async token: {token}")
        await asyncio.sleep(0.1)

asyncio.run(streaming())

`
how can I deal with it, could anyone help me?


### Suggestion:

_No response_
Add ChatGLM for llm and chat_model by using ChatGLM API
Labels: name: lgtm, name: auto:enhancement, name: area: models

using sample:
```
endpoint_url = API URL
ChatGLM_llm = ChatGLM(
    endpoint_url=endpoint_url,
    api_key=Your API Key by ChatGLM
)
print(ChatGLM_llm("hello"))
```

```
model = ChatChatGLM(
    chatglm_api_key="api_key",
    chatglm_api_base="api_base_url",
    model_name="model_name"
)
chain = LLMChain(llm=model)
```
Description: The call of ChatGLM has been adapted.
Issue: The call of ChatGLM has been adapted.
Dependencies: Need python package `zhipuai` and `aiostream`
Tag maintainer: @baskaryan
Twitter handle: None

I remove the compatibility test for pydantic version 2, because pydantic v2 can't not pickle classmethod,but BaseModel use @root_validator is a classmethod decorator.