delayedkarma/langchain-issues huggingface dataset:
sync problem
Labels: name: area: memory, name: auto:bug

https://github.com/langchain-ai/langchain/blob/991b448dfce7fa326e70774b5f38c9576f1f304c/libs/langchain/langchain/chains/base.py#L327C25-L327C25

IMPORTANT: chain.acall is a async function.


Chain.prep_inputs will invoke memory.load_memory_variables

in ConversationBufferWindowMemory.load_memory_variables.
```python
 buffer: Any = self.buffer[-self.k * 2 :] if self.k > 0 else []
 ```
 ```python
    @property
    def buffer(self) -> List[BaseMessage]:
        """String buffer of memory."""
        return self.chat_memory.messages

 ```
 
 if chat_memory is type of RedisChatMessageHistory.
 ```python
    @property
    def messages(self) -> List[BaseMessage]:  # type: ignore
        """Retrieve the messages from Redis"""
        _items = self.redis_client.lrange(self.key, 0, -1)
        items = [json.loads(m.decode("utf-8")) for m in _items[::-1]]
        messages = messages_from_dict(items)
        return messages
 ```
 
the lrange is a sync invoking...



IMPORTANT: chain.acall is a async function.

in fastapi or other scene.

a chain.acall will blocking by this invoke. can you change the invoke of Chain.prep_inputs to:
```python
inputs = await asyncify(self.prep_inputs)(inputs)
```
Lazy import in vectorstore
Labels: name: area: memory, name: area: vector store, name: auto:improvement

### Feature request

To import a big list of documents in a vectorstore, you must load all the documents in memory, split the documents, and import the list of documents.

They consume a lot of memory just to import. Sometime, the number of documents is too big to feet in memory.

I propose to generalize the `lazy_...()` with some generator. Then, only one document at a time can be in memory.

### Motivation

Reduce the memory foot print.
Accept to import, in one loop, a big list of documents.

### Your contribution

- Small contribution. I use  lazy approach in my pull request to load google documents.
Update pydantic format instruction prompt
Labels: name: auto:nit

- remove unopened bracket

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Enums not used with OpenAI Multi Function Agent
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

langchain==0.0.261

### Who can help?

@hwchase17

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When running the OpenAI Multi Functions Agent, **enums** in custom structured tools are not being used. Using the enums for data validation works fine, but the LLM does not use enums to produce enumerated parameter values.  For reference, OpenAI's Function Calling takes the JSON schema for a function, which can pass the JSON schema enums to the LLM to be used when producing parameter values.

1. Create a custom structured tool with an enum input
2. Give tool to agent
3. Run a query that uses the tool
4. Agent may hallucinate an invalid enum

### Expected behavior

The expected behavior is two-fold:

1. Provide Python type annotations in BaseModel arg_schemas to the LLM instead of just the Field() object
2. Correct the error similar to the `StructuredChatOutputParserWithRetries`
add support AsyncIteratorCallbackHandler with buffer
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description:Add new class [ChunkAsyncIteratorCallbackHandler], like AsyncIteratorCallbackHandler with buffer (chunk)
  - Issue: the issue None
  - Dependencies: None
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Can the atomicity of multiple write operations be guaranteed through the Agent
Labels: name: area: agent, name: auto:question, name: auto:enhancement

### Feature request

Can the atomicity of multiple write operations be guaranteed through the Agent



### Motivation

If two actions need to be executed within an agent and each action modifies the status of external tools or services, then if the first action writing operation is successful and the second action writing fails. How to ensure the revocation of the first write operation through an agent to ensure atomic: "all or nothing".

### Your contribution

Does the default agent runtime of langchain support this operation? If not, can we support it through the following ideas:
1: using langchain callback ?
2: inspired by BabyAGI feature: generate and pretend to execute tasks based on a given objective. I think the pretend operation can meet this requirement. Is this correct？
I would like GPT to quickly and accurately embody any characters or tools.
Labels: name: auto:enhancement, name: area: models

### Feature request

Sometimes, GPT might not accurately portray characters or tools, so before I ask GPT a question, I need to come up with a set of prompts. 
However, in most cases, these prompts may not capture the characteristics of the characters or tools effectively. To address this, I've developed a pip package that collects various prompts. How can I make Chain automatically use my package when it needs to embody characters (as a tool)? Or how can I optimize the pip package to better align with LangChain's requirements? 
This is the address of my repository:[https://github.com/limaoyi1/GPT-prompt](https://github.com/limaoyi1/GPT-prompt)

### Motivation

Sometimes, GPT might not accurately portray characters or tools, so before I ask GPT a question, I need to come up with a set of prompts. 

### Your contribution

Modify my pip package or submit pr
ConversationSummaryBufferMemory broken due to missing token usage
Labels: name: area: memory, name: auto:bug, name: area: models

### System Info

"langchain": "^0.0.125"
Windows 11 x64

### Who can help?

@nfcampos 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

When trying to retrieve token usage from LangChain, it returns a empty object. There's already an issue open about that. 
However, it gets slightly worse: Apparently, this breaks other components too, such as the `ConversationSummaryBufferMemory` which relies on Token count.

Longer explanation and code on this other issue comment: https://github.com/langchain-ai/langchain/issues/2359#issuecomment-1673926888

### Expected behavior

Completion tokens among others are expected to be returned. Maybe fall back to tiktoken ([py](https://pypi.org/project/tiktoken/)/[js](https://www.npmjs.com/package/@dqbd/tiktoken)) or similar when token count is not available? Just an idea.
Issue: StructuredOutputParser failures due to invalid JSON
Labels: name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

I think the `ResponseScheam`  and `StructuredOutputParser` functionality is really convenient but unfortunately, it often fails in my use case due to the model outputting a slightly wrongly formatted JSON. It's usually due to extra or missing quotes.

Any suggestions on how to solve this?

### Suggestion:

Thinking about writing some extra logic to fix the missing/extra quotes issue, but not sure if anyone has a better solution
Agents final thought as parameter
Labels: name: area: agent, name: auto:improvement

### System Info

I am using langchain in a different language, but I cannot change the following sentence as it is hard-coded into the Agent class

```
            thoughts += (
                "\n\nI now need to return a final answer based on the previous steps:"
            )
```

https://github.com/langchain-ai/langchain/blob/a5a4c53280b4dae8ea2e09430fed88e0cd4e03d2/libs/langchain/langchain/agents/agent.py#L588

### Who can help?

@hwchase17 
@agola11 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

-

### Expected behavior

-
Referencing experimental version of SQLDatabaseChain from the production project
Labels: name: auto:bug

https://github.com/langchain-ai/langchain/blob/a5a4c53280b4dae8ea2e09430fed88e0cd4e03d2/libs/langchain/langchain/chains/loading.py#L376

Work was done earlier this year to move all experimental and vulnerable code over to the experimental project. SQLDatabaseChain had (and still has) an open CVE against it so it was moved to the experimental project. However the experimental version of SQLDatabaseChain is being referenced in the production project on the line above. Our organization would like to start using LangChain but we cannot due to the open High severity CVE. Is it possible to either patch the vulnerability in code or complete the move of SQLDatabaseChain from the production project to the experimental?
Issue: I want to attach metadata in my PGVector vector store used as retriever for my ConversationChain memory
Labels: name: area: memory, name: area: vector store, name: auto:question, name: stale

### Issue you'd like to raise.

I am creating a chatbot that records the history of the conversation through vectors in my vector store per user that uses the chatbot, and for this, I identify who is the owner of the vectors through metadata.
How can I add this metadata to the vectors if I'm using the vector store as a retriever for the memory in my ConversationChain?

This is my code:
```
import os
from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.memory import VectorStoreRetrieverMemory
from langchain.vectorstores.pgvector import PGVector
from langchain.embeddings.openai import OpenAIEmbeddings

exit_conditions = ("q", "quit", "exit")
metadata={"id": "John Doe", "key": 123}
llm = OpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"), temperature=0)
store = PGVector(
            collection_name="chatbot_embeddings",
            connection_string=os.getenv("POSTGRES_CONNECTION_STRING"),
            embedding_function=OpenAIEmbeddings(),
            collection_metadata=metadata
        )

while True:
    query = input("> ")
    if query in exit_conditions:
        break

    conversation_with_summary = ConversationChain(
        llm=llm,
        memory=VectorStoreRetrieverMemory(retriever=store.as_retriever()),
        verbose=True,
        metadata=metadata,
    )
    print(conversation_with_summary.predict(input=query))
```

### Suggestion:

_No response_
updated default of n_sentence_context to 2
Labels: name: area: vector store, name: auto:improvement

n_sentences_context determines the amount of sentences (chunks) that are added before/after the matching chunk when Vectara returns results. This additional context usually results in better context in the Grounded Generation (aka retrieval-augmented generation) use-cases. 

Replace this comment with:
  - Description: updated n_sentences_context to default at 2.
  - Tag maintainer: @rlancemartin, @eyurtsev
  - Twitter handle: @ofermend
TypeError: expected string or buffer
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

When I used to connect and azure openai Emabeddings and Milvus used , Below Exception happende
Traceback (most recent call last):
  File "D:\Corent\AI\LangChain\azure\azure_connection.py", line 29, in <module>
    VectorStore = Milvus.from_texts(
                  ^^^^^^^^^^^^^^^^^^
  File "D:\Corent\AI\LangChain\azure\venv\Lib\site-packages\langchain\vectorstores\milvus.py", line 822, in from_texts   
    vector_db.add_texts(texts=texts, metadatas=metadatas)
  File "D:\Corent\AI\LangChain\azure\venv\Lib\site-packages\langchain\vectorstores\milvus.py", line 422, in add_texts    
    embeddings = self.embedding_func.embed_documents(texts)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Corent\AI\LangChain\azure\venv\Lib\site-packages\langchain\embeddings\openai.py", line 478, in embed_documents
    return self._get_len_safe_embeddings(texts, engine=self.deployment)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Corent\AI\LangChain\azure\venv\Lib\site-packages\langchain\embeddings\openai.py", line 341, in _get_len_safe_embeddings
    token = encoding.encode(
            ^^^^^^^^^^^^^^^^
  File "D:\Corent\AI\LangChain\azure\venv\Lib\site-packages\tiktoken\core.py", line 116, in encode
    if match := _special_token_regex(disallowed_special).search(text):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or buffer
get rid of google-api-python-client as it is has a huge filesystem footprint
Labels: name: auto:improvement

**Description:** 
Refactored the `GoogleSearchAPIWrapper` to utilize the `requests` library for direct interactions with the Google Custom Search API. This eliminates the dependency on the `google-api-python-client` library, significantly reducing the filesystem footprint.

**Reasoning:** 
Upon closer inspection, I discovered that the `googleapiclient` has a filesystem footprint of 84.8 MiB. This is particularly heavy for applications, especially when the primary use-case is just a single call to Google Search. I use langchain in a AWS lambda environment and getting rid of `google-api-python-client` greatly helped me to be able to run my agent serverless. Further reasoning:

1. **Microservice Architecture:** In a microservices ecosystem, each service ideally should be lightweight and singular in function. Including bulky libraries for minimal features increases the overhead without proportionate benefits.

2. **AWS Lambdas:** AWS Lambda has packaging size limits. For languages like Python, the deployment package size (zipped) should be under 50 MB for direct upload and under 250 MB when uploaded from S3. Reducing unnecessary bulk is crucial for efficient AWS Lambda deployments.

3. **Optimized Docker Images:** Large libraries increase the size of Docker images, which subsequently affects deployment times, scaling efficiency, and even costs (especially in serverless environments where you pay per execution time).

4. **Faster Start Times:** In cloud functions or similar serverless architectures, a lighter package means faster cold starts.

Making this change is a step towards optimized, efficient, and cost-effective deployments, especially in serverless and microservice environments. 

**Dependencies:** 
- `requests`

**Tag maintainer:** 
@hinthornw

Please help me to test this, I hope some GitHub actions will test this PR, the change is rather small and inside a private function, so I hope all tests will pass.
remove the last ";" if the SQL is end with ";"
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: remote the useless ; in the end of the sql
  - Issue: If the sql is end with ";", it will have some problem for hive with sqlalchemy (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Update faiss.py    delete bug fixed
Labels: name: area: vector store, name: auto:bug

# After invoking the 'remove_ids' function, the indices of vectors in the Faiss Index will remain consecutive. 

# To ensure consistency between the keys of 'index_to_docstore_id' and the indices of vectors in the Faiss Index, modify the keys of 'index_to_docstore_id' to consecutive numbers, like 0, 3, 4 - > 0, 1, 2.
Metaphor Search throws key error when `publishedDate` is missing
Labels: name: auto:bug, name: area: models

### System Info

LangChain: v0.0.260
Python: v3.11

### Who can help?

@hwchase17 @eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Sometime `Metaphor` tool returns results without publishedDate which makes the Tool wrapper break due not guarding against empty values:

```
langchain/utilities/metaphor_search.py", line 169, in _clean_results

"published_date": result["publishedDate"],

KeyError: 'publishedDate'
```

### Expected behavior

Preferably the Tool should guard agains missing keys in API response.
ArxivLoader support searching by arxiv id_list
Labels: name: area: doc loader, name: auto:improvement

### Feature request

If I know exactly what paper(s) I'd like to load, it would be nice to be able to retrieve them by their arxiv IDs. The arxiv library seems to support this:

```
import arxiv

paper = next(arxiv.Search(id_list=["1605.08386v1"]).results())
```

### Motivation

Sometimes searching by exact paper title still returns the incorrect paper. Searching by ID should be more fool-proof.

### Your contribution

🤷🏻‍♂️
ArxivLoader incorrect results
Labels: name: area: doc loader, name: auto:bug

### System Info

Latest pip versions

### Who can help?

@eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I tried searching by exact title in the following way:

```python
docs = ArxivLoader(query="MetaGPT: Meta Programming for Multi-Agent Collaborative Framework", load_max_docs=1).load()
```
But the result is incorrect. The search works properly on the arxiv website. 

### Expected behavior

Correct paper returned
Takeoff integration
Labels: name: auto:enhancement, name: area: models

## Description:
This PR adds the Titan Takeoff Server to the available LLMs in LangChain.

Titan Takeoff is an inference server created by [TitanML](https://www.titanml.co/) that allows you to deploy large language models locally on your hardware in a single command. Most generative model architectures are included, such as Falcon, Llama 2, GPT2, T5 and many more.

Read more about Titan Takeoff here:
- [Blog](https://medium.com/@TitanML/introducing-titan-takeoff-6c30e55a8e1e)
- [Docs](https://docs.titanml.co/docs/titan-takeoff/getting-started)

#### Testing
As Titan Takeoff runs locally on port 8000 by default, no network access is needed. Responses are mocked for testing.

- [x] Make Lint
- [x] Make Format
- [x] Make Test

#### Dependencies
No new dependencies are introduced. However, users will need to install the titan-iris package in their local environment and start the Titan Takeoff inferencing server in order to use the Titan Takeoff integration.

Thanks for your help and please let me know if you have any questions.

cc: @hwchase17 @baskaryan
Update `ArgillaCallbackHandler` as of latest `argilla` release
Labels: name: auto:improvement

Hi @agola11, or whoever is reviewing this PR 😄 

## What's in this PR?

As of the latest Argilla release, we'll change and refactor some things to make some workflows easier, one of those is how everything's pushed to Argilla, so that now there's no need to call `push_to_argilla` over a `FeedbackDataset` when either `push_to_argilla` is called for the first time, or `from_argilla` is called; among others.

We also add some class variables to make sure those are easy to update in case we update those internally in the future, also to make the `warnings.warn` message lighter from the code view.

P.S. Regarding the Twitter/X mention feel free to do so at either https://twitter.com/argilla_io or https://twitter.com/alvarobartt, or both if applicable, otherwise, just the first Twitter/X handle.