delayedkarma/langchain-issues huggingface dataset:
Add datadog-langchain integration doc
Labels: name: lgtm, name: auto:documentation

## Description
Added a doc about the [Datadog APM integration for LangChain](https://github.com/DataDog/dd-trace-py/pull/6137). 
Note that the integration is on `ddtrace`'s end and so no code is introduced/required by this integration into the langchain library. For that reason I've refrained from adding an example notebook (although I've added setup instructions for enabling the integration in the doc) as no code is technically required to enable the integration.

Tagging @baskaryan as reviewer on this PR, thank you very much!

## Dependencies
Datadog APM users will need to have `ddtrace` installed, but the integration is on `ddtrace` end and so does not introduce any external dependencies to the LangChain project. 

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
TypeError("aiohttp.client.ClientSession.request() got multiple values for keyword argument 'auth'" with arun and requests_get tool on an agent
Labels: name: area: agent, name: auto:bug

### System Info

langchain 0.0.234
Python 3.10.9
aiohttp 3.8.4

Getting the following error when running an agent in async with agent.arun('Get time in Salta of Argentina') with a requests_get tool.
[tool/start] [1:chain:AgentExecutor > 7:tool:requests_get] Entering Tool run with input:
"http://worldtimeapi.org/api/timezone/America/Argentina/Salta"
[tool/error] [1:chain:AgentExecutor > 7:tool:requests_get] [78.06s] Tool run errored with error:
TypeError("aiohttp.client.ClientSession.request() got multiple values for keyword argument 'auth'"

The issue lies in https://github.com/hwchase17/langchain/commit/663b0933e488383e6a9bc2a04b4b1cf866a8ea94 which was done to fix issue #7542

### Who can help?

@agola11 @EricSpeidel 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create an agent utilizing the requests_get tool
2. Run the agent in async using arun with a prompt making it generate a GET request using the tool


### Expected behavior

The auth should not be pass explicitly to sent to session.request in the code below. It will pass through automatically as part of kwargs. 

```
@asynccontextmanager
    async def _arequest(
        self, method: str, url: str, **kwargs: Any
    ) -> AsyncGenerator[aiohttp.ClientResponse, None]:
        """Make an async request."""
        if not self.aiosession:
            async with aiohttp.ClientSession() as session:
                async with session.request(
                    method, url, headers=self.headers, auth=self.auth, **kwargs
                ) as response:
                    yield response
        else:
            async with self.aiosession.request(
                method, url, headers=self.headers, auth=self.auth, **kwargs
            ) as response:
                yield response

```
docs: fix a couple of small indentation errors in the strings
Labels: name: auto:documentation

Fixed a few indentations I came across in the docs @baskaryan 
<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Ability to translate/interpret LLM tool requests
Labels: name: area: agent, name: auto:enhancement

### Feature request

I'm getting this back from my LLM:

Observation: Use Search is not a valid tool, try another one.

Obviously the LLM would like to use the search tool, but didn't request it just right. It seems LangChain has been thoroughly tested with OpenAI, but not so much other models. Which is totally understandable considering there are now thousands of models and variations.

Although I'm digging through the code to find a way to do this myself, it would be awesome if LangChain had a well documented module we could enable to interpret or translate response from the LLM.

Obviously in this case I could use the tool list, and string find, and if any of the tool names are found to go with that tool. 


### Motivation

Increased compatibility with open source LLM

### Your contribution

Generic example:

tool_names = # python list

for tool_name in tool_names:
    if ai_reply.find(tool_name) >= 0:
        # use tool
fix
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
OpenAI functions Agent  is not working with SQLDatabaseChain.
Labels: name: area: agent, name: auto:bug

### Issue with current documentation:

Hello every one! 

 I am following the tutorial of [OpenAI function Agent ](https://python.langchain.com/docs/modules/agents/agent_types/openai_functions_agent). 


I downloaded  the chinook database in MySQL , I made the connection successfully and got good answers from this database using the [SQLDataBase agent  tutorial.](https://python.langchain.com/docs/modules/agents/toolkits/sql_database). However, when I try to do sql query using the OpenAI function agent it fails. 

I got the following error: 

```InvalidRequestError: 'MySQL DB' does not match '^[a-zA-Z0-9_-]{1,64}$' - 'functions.1.name```

When I change to other kind of agents such as CHAT_ZERO_SHOT_REACT_DESCRIPTION is works good. 



### Idea or request for content:

_No response_
Refactored `sql_database`
Labels: name: lgtm, name: auto:refactor

The `sql_database.py` is unnecessarily placed in the root code folder.
A similar code is usually placed in the `utilities/`.
As a byproduct of this placement, the sql_database is [placed on the top level of classes in the API Reference](https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.sql_database) which is confusing and not correct.


- moved the `sql_database.py` from the root code folder to the `utilities/`

@baskaryan
Error when loading JSON file using S3FileLoader
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain: 0.0.236

### Who can help?

@eyurtsev 
When attempting to load a JSON file from S3, I encounter the following error:
`An error occurred: Json schema does not match the Unstructured schema`

Don't know if it is related to https://github.com/hwchase17/langchain/issues/2222

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Use this code:
```
from langchain.document_loaders import S3FileLoader
loader = S3FileLoader(bucket, file_key)
documents = loader.load()
```
Getting this error:
`An error occurred: Json schema does not match the Unstructured schema`

### Expected behavior

I anticipate that it will load the file into the documents, just as it does for other file types I use.
Add etherscan document loader
Labels: name: lgtm, name: area: doc loader, name: auto:enhancement

@rlancemartin 
The modification includes:
* etherscanLoader
* test_etherscan
* document ipynb

I have run the test, lint, format, and spell check. I do encounter a linting error on ipynb, I am not sure how to address that.
```
docs/extras/modules/data_connection/document_loaders/integrations/Etherscan.ipynb:55: error: Name "null" is not defined  [name-defined]
docs/extras/modules/data_connection/document_loaders/integrations/Etherscan.ipynb:76: error: Name "null" is not defined  [name-defined]
Found 2 errors in 1 file (checked 1 source file)
```
- Description: The Etherscan loader uses etherscan api to load transaction histories under specific accounts on Ethereum Mainnet.
- No dependency is introduced by this PR.
- Twitter handle: glazecl
VectorStoreRetrieverMemory exclude additional input keys feature
Labels: name: lgtm, name: area: vector store, name: auto:improvement


  - Description: Added a parameter in VectorStoreRetrieverMemory which filters the input given by the key when constructing the buffering the document for Vector. This feature is helpful if you have certain inputs apart from the VectorMemory's own memory_key that needs to be ignored e.g when using combined memory, we might need to filter the memory_key of the other memory, Please see the issue.
  - Issue: #7695
  - Tag maintainer: @rlancemartin, @eyurtsev
Add regex control over separators in character text splitter
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
#7854

Added the ability to use the `separator` ase a regex or a simple character.
Fixed a bug where `start_index` was incorrectly counting from -1.

Who can review?
@eyurtsev
@hwchase17 
@mmz-001
Add Golden Query Tool
Labels: name: area: agent, name: lgtm, name: auto:enhancement

**Description:**  Golden Query is a wrapper on top of the [Golden Query API](https://docs.golden.com/reference/query-api) which enables programmatic access to query results on entities across Golden's Knowledge Base. For more information about Golden API, please see the [Golden API Getting Started](https://docs.golden.com/reference/getting-started) page.
**Issue:** None
**Dependencies:** requests(already present in project)
**Tag maintainer:** @hinthornw
SET search_path TO {self._schema} is executing by SQLDatabase for all databases except 'snowflake' and 'bigquery'.
Labels: name: area: agent, name: auto:bug

### System Info

LangChain Version- 0.0.235, Windows, Python Version-3.9.16

As per below source code of SQLDatabase, before executing any sql query, connection.exec_driver_sql(f"SET search_path TO {self._schema}") is executed for all database except 'snowflake' and 'bigquery'.

if self._schema is not None:
                if self.dialect == "snowflake":
                    connection.exec_driver_sql(
                        f"ALTER SESSION SET search_path='{self._schema}'"
                    )
                elif self.dialect == "bigquery":
                    connection.exec_driver_sql(f"SET @@dataset_id='{self._schema}'")
                else:
                    connection.exec_driver_sql(f"SET search_path TO {self._schema}")
            cursor = connection.execute(text(command))

As per my knowledge, The SET search_path command is specific to PostgreSQL, not Oracle. This is why I am getting the following error. 
sqlalchemy.exc.DatabaseError: (oracledb.exceptions.DatabaseError) ORA-00922: missing or invalid option
[SQL: SET search_path TO evr1]
(Background on this error at: https://sqlalche.me/e/20/4xp6)

Oracle does not recognize this command.

I think it is better to use SCHEMA.TABLE in all sql queries.



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

oracle_connection_str = f"oracle+oracledb://{username}:{password}@{hostname}:{port}/?service_name={service_name}"
db = SQLDatabase.from_uri(
    oracle_connection_str,
    schema="evr1",
    include_tables=[ ],
    sample_rows_in_table_info=3,
)

llm = ChatOpenAI(model_name=GPT_MODEL, temperature=0, openai_api_key=OpenAI_API_KEY)

toolkit = SQLDatabaseToolkit(
    db=db,
    llm=llm,
)

agent_executor = create_sql_agent(
    llm=llm,
    toolkit=toolkit,
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    return_intermediate_steps=True,
    handle_parsing_errors=_handle_error,
    verbose=True,
)

response = agent_executor(user_input)

### Expected behavior

Should not execute SET search_path TO {self._schema} for Oracle
Bedrock Embeddings: Add support for endpoint_url
Labels: name: area: embeddings, name: auto:improvement

### System Info

Bedrock Embeddings doesn't have support for modifying the endpoint_url, the LLMs one have.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Not able to provide a custom endpoint_url

### Expected behavior

Able to provide a custom endpoint_url
Issue: Cannot replicate search function on Langchain Milvus 
Labels: name: area: vector store, name: auto:bug, name: auto:question

### Issue you'd like to raise.

Hi I've tried doing the search functionalities using pymilvus and everything works perfectly fine doing similarity search, i am able to get the most relevant documents. However, when i try using the same credentials and parameters on Langchain.vectorstore.Milvus i am unable to replicate the results. Not sure if the connection is correct but my similarity search is returning a blank list. This is the exact same credentials i'm using currently for my on-prem Milvus

`vdb = Milvus(
    collection_name = 'bbc_news',
    embedding_function = embedding_model,
    connection_args = {'host': 'server_host','port':'19530','user':'admin','password':'password','database':'db_dsg'},
    consistency_level = "Session",
    index_params = index_params,
    search_params = search_params
)`

`vdb.similarity_search("What is the price of crude oil")` 

Returns a blank list. Am i doing anything wrong here? 

### Suggestion:

_No response_
Issue: Not getting good chat  results on enabling Coversation Memory in langchain
Labels: name: area: memory, name: auto:bug, name: auto:question

### Issue you'd like to raise.

Hi,
Here is how i have initilized conversation memory 
memory = ConversationSummaryBufferMemory(llm=llm_model,memory_key="chat_history",return_messages=True,max_token_limit=500)
Here is how I have used ConversationalRetrievalChain
chain=ConversationalRetrievalChain.from_llm(llm_model,retriever=vector.as_retriever(search_kwargs={"k": 10}), memory=memory,verbose=True)
I could see that answer for first question will be good, on asking further questions with 
result = chain({"question": <question>}) answer from bot is not good for few questions. With verbose enabled I have observed that apart from the actual history langchain is adding  few more conversations by itself. Is there a way to supress this(adding additional conversation)
I also tried with ConversationBufferWindowMemory aswell. There also Iam seeing the performance drop. I tried with new langchain version as well "0.0.235", there also Iam seeing the same issue. Could you help me to undersatnd if this is an existing issue or any mistake am I doing while configuring



### Suggestion:

_No response_
Embeddings Callbacks
Labels: name: area: embeddings, name: auto:improvement

Similar to https://github.com/hwchase17/langchainjs/pull/1859 from the JS lib (though haven't made serializable yet). 

Example run: https://dev.smith.langchain.com/public/674633d8-9c38-4cdc-bd11-53202879fad1/r

Would like alignment on 
- New base class vs update existing
- Whether OK to stop supporting 'chunk_size' at call time for a handful

And this still doesn't address our retriever -> vector store -> embeddings flows. Will need to update a lot of those similarity score methods to accept callbacks since they're all called under the hood.
improve insert performance for hologres
Labels: name: needs test, name: area: vector store, name: auto:improvement

Hi,
I made some code changes on Hologres vector store to improve the data insertion performance.
The code has passed the format/lint/spell check. And I have run the unit test for Hologres connecting to my own database.
Please check this PR, thanks!
@rlancemartin, @eyurtsev

Best,
Changgeng
Equal Sized Chunks | Adds BalancedRecursiveCharacterTextSplitter to text_splitter.py
Labels: name: auto:improvement

  - Description: the RecursiveCharacterTextSplitter often leaves final chunks that are too small too be useful. I added this class to ensure that all chunk sizes conform to the desired chunk size. Included docs and a Juypter notebook.
  - Issue: None,
  - Dependencies: None,
  - Tag maintainer: @rlancemartin, @eyurtsev @baskaryan,
  - Twitter handle: @J_Shelby_J
add serializer methods
Labels: name: lgtm, name: area: vector store, name: auto:improvement

Description: I have added two methods serializer and deserializer methods. There was method called save local but it saves the to the local disk. I wanted the vectorstore in the format using which i can push it to the sql database's blob field. I have used this while i was working on something

@rlancemartin, @eyurtsev
Qdrant Vectorstore
Labels: name: area: vector store, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

@baskaryan @rlancemartin, @eyurtsev