delayedkarma/langchain-issues huggingface dataset:
Cannot create "n" responses
Labels: name: area: agent, name: auto:bug

### System Info

Verified in dockerimage python:3.9

### Who can help?

@hwchase17
@agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

how to reproduce?

1. docker run -it python:3.9 bash 
2. pip install langchain
3. pip install openai
4. run script
```
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import SystemMessage
    
chat_model = ChatOpenAI(n=3)
prompt = "What is the capital of France?"
message = SystemMessage(content=prompt)
responses = chat_model.predict_messages([message], n=3) 
print(responses)
```

output:
content='The capital of France is Paris.' additional_kwargs={} example=False

### Expected behavior

Hello, 
When running the script I expect to get 3 responses, because I set the parameter `n=3` during initialization of ChatOpenAI and in the call of predict_messages. 

still the response is a single answer!

please let me know how to correctly use the parameter n or fix the current behavior!

best regards,
LW
Need support for async memory, especially for ConversationSummaryMemory and ConversationSummaryBufferMemory
Labels: name: area: memory, name: auto:improvement

### Feature request

`ConversationSummaryMemory` and `ConversationSummaryBufferMemory` can be used as memory within async conversational chains, but they themselves are fully blocking. In particular, `Chain.acall()` is async, but it calls `Chain.prep_outputs()`, which calls `self.memory.save_context()`. In the case of `ConversationSummaryBufferMemory`, `save_context()` calls `prune()`, which calls `SummarizerMixin.predict_new_summary()`, which creates an `LLMChain` and then calls `LLMChain.predict()`. We really need to be able to specify an async handler when we create the memory object, pass that through to the `LLMChain`, and ultimately `await LLMChain.apredict()`.

### Motivation

The blocking nature of the `LLMChain.predict()` calls means that otherwise-async chains end up blocking regularly. If you use an ASGI server like `uvicorn`, this means that lots of requests can end up waiting. You can wrap `uvicorn` in a WSGI server like `gunicorn` so that there are multiple processes available, but blocking one whole process every time the memory object needs to summarize can still block any websocket users connected to the blocked process. It's just not good.

### Your contribution

We're going to have to solve this problem for ourselves, and we'd be happy to try to contribute a solution back to the community. However, we'd be first-time contributors and we're not sure whether there are others already making similar improvements.
Issue: Agent with memory and intermediate steps
Labels: name: area: memory, name: area: agent, name: auto:bug, name: auto:question

### Issue you'd like to raise.

Hi everyone,

I'm having trouble working with the agent structure in Langchain.
I would like to have an agent that has memory and can return intermediate steps.
However It doesn't work, when I make an agent executor, the return_intermediate_steps=True makes it crash as the memory cannot read the output and when I use the agent directly, the return_intermediate_steps=True does nothing.

Do you know if there is a way to have an agent with memory and that can return intermediate steps at the same time ?

Thanks for your help

### Suggestion:

_No response_
langchain_touchmed
Labels: name: area: doc loader, name: auto:documentation

langchain_touchmed，增加 prompt

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
fix apparent spelling inconsistencies
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: fix apparent spelling inconsistencies in raising ImportError
  - Issue: None
  - Dependencies: None
  - Tag maintainer: @baskaryan
  - Twitter handle: @MLOpsj

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Add Support for Loading Documents from Huawei OBS
Labels: name: area: doc loader, name: auto:enhancement

Description:
This PR adds support for loading documents from Huawei OBS (Object Storage Service) in Langchain. OBS is a cloud-based object storage service provided by Huawei Cloud. With this enhancement, Langchain users can now easily access and load documents stored in Huawei OBS directly into the system.

Key Changes:
- Added a new document loader module specifically for Huawei OBS integration.
- Implemented the necessary logic to authenticate and connect to Huawei OBS using access credentials.
- Enabled the loading of individual documents from a specified bucket and object key in Huawei OBS.
- Provided the option to specify custom authentication information or obtain security tokens from Huawei Cloud ECS for easy access.

How to Test:
1. Ensure the required package "esdk-obs-python" is installed.
2. Configure the endpoint, access key, secret key, and bucket details for Huawei OBS in the Langchain settings.
3. Load documents from Huawei OBS using the updated document loader module.
4. Verify that documents are successfully retrieved and loaded into Langchain for further processing.

Please review this PR and let us know if any further improvements are needed. Your feedback is highly appreciated!

@rlancemartin, @eyurtsev
Issue: Structure of this repo is confusing
Labels: name: auto:improvement

### Issue you'd like to raise.

This repo was split up between the core package and some experimental package. While this is a good idea, it's kinda odd to have both packages in the same repo. Even further you're releasing versions for both of these packages within the same repo. So 5 days ago there was a 0.0.5 release while later on you had 0.0.249.

I've actually never seen such a structure before and it's messing up scripted automation to follow releases.

### Suggestion:

Create a new repo for `LangChain Experimental` and have those releases there.
Enhance retrieval callback hooks to capture retrieved document IDs and embeddings
Labels: name: area: embeddings, name: area: vector store, name: auto:enhancement

### Feature request

Enhance the retrieval callback hooks to include:
- document IDs from the underlying vector store
- document embeddings used during retrieval

### Motivation

I want to build a callback handler that enables LangChain users to visualize their data in [Phoenix](https://github.com/Arize-ai/phoenix), an open-source tool that provides debugging workflows for retrieval-augmented generation. At the moment, I am only able to get retrieved document text from the callback system, not the IDs or embeddings of the retrieved documents.

### Your contribution

I am willing to implement, test, and document this feature with guidance from the LangChain team. I am also happy to provide feedback on an implementation by the LangChain team by building an example callback handler using the enhancement retrieval hook functionality.
improve insert performance for hologres
Labels: name: area: vector store, name: auto:improvement

Hi,
I made some code changes on the Hologres vector store to improve the data insertion performance.
The code has passed the format/lint/spell check. And I have run the unit test for Hologres connecting to my own database.
There is already an integration test `test_hologres.py` that covers the content of this change.
Please check this PR, thanks!

@baskaryan 
@rlancemartin, @eyurtsev

Best,
Changgeng,
Developer @ Alibaba Cloud
fix bug in AzureML Endpoint
Labels: name: auto:bug, name: size:M

Description:
fix bug in azureml_endpoint.py

Issue:
when using the AzureMLOnlineEndpoint class without passing the content_formatter parameter, an AttributeError will occur. similarly, with classAzureMLChatOnlineEndpoint.

Details:
1. update ContentFormatterBase to inherit from BaseModel for Pydantic validation support.
2. add validators to AzureMLOnlineEndpoint and AzureMLChatOnlineEndpoint with informative error messages for enhanced usability.

Dependencies:
no dependencies required
Issue: How to use chain_type args in LLMchain?
Labels: name: auto:question

### Issue you'd like to raise.

I am currently working on the task of **article rewriting**, and both the QA chain and Summarize chain are not quite suitable for my task. 
Since I need to rewrite **multiple "lengthy"** articles, I would like to know how to use the chain_type in LLMchain. Alternatively, are there any other methods to achieve segment-level rewriting? Thank you.

### Suggestion:

_No response_
Petals API Support
Labels: name: auto:enhancement

### Feature request

Currently, langchain supports petals, but I think we should also support using a petals API endpoint. https://github.com/petals-infra/chat.petals.dev

### Motivation

The idea here is that users don't need to run the base application on their system, and can just use the API directly.

I think this is useful as a developer for speed, and just testing things quickly. I also think it would be the easiest and most reliabile way for any langchain user to get access to a high quality LLM on low-spec hardware.

### Your contribution

I am happy to add this myself if I can be guided on what parts of the code need changed.
implement async exec for base.py
Labels: name: auto:improvement

Can we such a method for async? The return type is specified as `asyncio.Task[Dict[str, str]]` to represent the dictionary return type wrapped in an `asyncio.Task`. `Since _acall()` is an asynchronous method, any coroutine calls within it should be awaited. So, we use await when calling `run_manager.on_text()` to properly await the completion of the asynchronous operation.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Add metadata parameter(s) to SQLDatabase class
Labels: name: auto:enhancement

### Feature request

Most databases include comment metadata on both tables and columns in a table. It would be nice to be able to pass this additional context to the LLM to get a better response. 

The implementation could include adding two parameters to the [SQLDatabase](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/utilities/sql_database.py) object. For example:

- `table_metadata_sql` (perhaps `table_comment_sql`) runs a query and expects a table with two columns `table_name` and `comment`
- `column_metadata_sql` (perhaps `column_comment_sql` runs a query and expects a table with three columns `table_name`, `col_name`, and `comment`

Perhaps these two params could be combined into a single `metadata_sql` which returns for column table with `table_name`, `table_comment`, `column_name`, and `column_comment` 

### Motivation

The main motivation behind this is to provide the LLM additional context beyond the CREATE TABLE and sample rows. Although this will be a more costly request (more tokens), I believe in many instances will lead to better SQL being generated. I also want to encourage better documentation of database objects in the data warehouse or data lake. 

### Your contribution

Happy to submit a PR for this. Please weigh in on any design decisions: 

- how many parameters to use 1 or 2?
- where to include the comments in the prompt, part of the CREATE TABLE as SQL comments?
ValueError: Expected metadata value to be a str, int, or float, got [{'text': 'Git', 'url': '#git'}] which is a <class 'list'> when storing into Chroma vector stores using using element mode of UnstructuredMarkdownLoader
Labels: name: area: doc loader, name: area: vector store, name: auto:bug

### System Info

LangChain: 0.0.248
Python: 3.9.17
OS version: Linux 6.1.27-43.48.amzn2023.x86_64

### Who can help?

I will submit a PR for a solution to this problem

### Information

- [ ] The official example notebooks/scripts
- [x] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Code:
```python
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.document_loaders import UnstructuredMarkdownLoader
from langchain.text_splitter import CharacterTextSplitter

def testElement():
    loader = UnstructuredMarkdownLoader(
        "filepath", mode="elements")
    documents = loader.load()
    text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=0)
    split_docs = text_splitter.split_documents(documents)

    embeddings = OpenAIEmbeddings()
    docsearch = Chroma.from_documents(split_docs, embeddings)
```
Also need to have a link format in the markdown file to be load, for example:
```
- [Google Developer Documentation Style Guide](https://developers.google.com/style)
```
Error Message:
```
    138         # isinstance(True, int) evaluates to True, so we need to check for bools separately
    139         if not isinstance(value, (str, int, float)) or isinstance(value, bool):
--> 140             raise ValueError(
    141                 f"Expected metadata value to be a str, int, or float, got {value} which is a {type(value)}"
    142             )
ValueError: Expected metadata value to be a str, int, or float, got [{'text': 'Git', 'url': '#git'}] which is a <class 'list'>
```

### Expected behavior

I expect to see the split documents loaded into Chroma, however, this raise error for not passing type check for metadata.
Update openai_functions.py to handle KeyError: 'function_call' error
Labels: name: auto:bug

Description: Sometime when trying to custom use Output parsers for function call we get KeyError: 'function_call' when function_call key is not present in the response for a particular request. This exception handles that case.

<img width="1275" alt="Screenshot 2023-08-01 at 5 42 52 AM" src="https://github.com/langchain-ai/langchain/assets/108733252/a1031a80-9541-4fed-81e8-a3e60e288538">

Issue: N/A
Tag maintainer: @rlancemartin 
Dependencies: None
AzureSearch Vector Store: Moving the usage of additional_fields into context of it's definition (bug fix from python error)
Labels: name: lgtm, name: area: vector store, name: auto:bug

Description: Using Azure Cognitive Search as a VectorStore.    Calling the `add_texts` method throws an error if there is no metadata property specified.   The `additional_fields` field is set in an `if` statement and then is used later outside the if statement.  This PR just moves the declaration of `additional_fields` below and puts the usage of it in context.

Issue: https://github.com/langchain-ai/langchain/issues/8544

Tagging @rlancemartin, @eyurtsev as this is related to Vector stores.

`make format`, `make lint`, `make spellcheck`, and `make test` have been run
Llama formatter
Labels: name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Vector Store: Azure Cognitive Search :: add_texts throws an error if called with no metadata
Labels: name: area: vector store, name: auto:bug

### System Info

Python: v3.11
Langchain: v0.0.248

### Who can help?

I'll submit a PR for it tonight, just wanted to get the Issue in before.  

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [x] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


Background

I'm using Azure Cognitive Search as my vector store.  In `azuresearch`, if you call `addtexts` with no metadata, you get an exception 

Steps to reproduce behavior:

1. create AzureSearch object 
2. call `add_texts` without specifying the metadata parameter
3. You'll get an error

> UnboundLocalError: cannot access local variable 'additional_fields' where it is not associated with a value


### Expected behavior

No error occurs it adds the text to the `texts` parameter to the Vector store.
rfc: fallbacks on model
Labels: name: auto:improvement

![Screenshot 2023-07-31 at 1 11 31 PM](https://github.com/langchain-ai/langchain/assets/22008038/6f8284c2-fbef-4b8f-bea6-899ad5296e13)

https://dev.smith.langchain.com/public/dadc238d-0737-4211-a1da-1e4aa363548b/r