delayedkarma/langchain-issues huggingface dataset:
Qdrant: Ability to add vectors to already existing collection & set vector size
Labels: name: auto:improvement

Problem:
Whenever we add vectors to an existing qdrant collection that collection gets deleted and re-created from scratch if we use for example `VectorstoreIndexCreator()`.

This PR solves this issue and adds further automatisation abilities without breaking the already existing behavior.

In addition to that this PR adds the ability to configure a fixed vector size, eliminating the need to make a single embedding just to figure out the vector size and further reducing api usage resulting in reduced costs for end-users.

So this PR introduces the following two flags:

`recreate_collection = Optional[bool]`

`vector_size = Optional[int]`

How this solution works:

If recreate_collection=False, it checks if collection exists, if not, continue like before this PR ((re)create collection)
if collection exists, and recreate_collection=False, and init_from=None then do not execute recreate_collection()

VectorstoreIndexCreator code example (after PR):

```python
        VectorstoreIndexCreator(
            vectorstore_cls=Qdrant,
            text_splitter=RecursiveCharacterTextSplitter(
                chunk_size=int(os.environ.get('CHUNK_SIZE_TOKENS', 400)),
                separators=sepList,
                chunk_overlap=0,
                length_function=get_tokens,
            ),
            vectorstore_kwargs=dict(
                host=os.environ.get('QDRANT_HOST', 'qdrant'),
                port=int(os.environ.get('QDRANT_PORT', '6333')),
                grpc_port=int(os.environ.get('QDRANT_GRPC_PORT', '6334')),
                prefer_grpc=bool(os.environ.get('QDRANT_PREFER_GRPC', 'True')),
                collection_name=self.request.collection_name,
                recreate_collection=False, # <-- this is new, default value is True
                vector_size=int(os.environ.get('VECTOR_DIMENSIONS', '1536')), # <-- this is new, default value is None
            ),
        ).from_documents(self.documents)
```

--

Maintainer responsibilities:
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
Add upserting and deletion capability to PGVector
Labels: name: auto:improvement, name: auto:documentation

Currently, the PGVector vectorstore only supports adding embeddings to the store. This PR adds upserting and deletion capability to the store, so you can update existing embeddings if the underlying text have changed. This is useful if you have indexed a set of documents but some of the documents are being changed and you want to keep the index up-to-date.

To use this feature, simply provided ids for all texts/or documents. The pgvector notebook has been updated accordingly.

The fixes issue #6866. 

👀  @rlancemartin, @eyurtsev

In case you want to announce on Twitter, my Twitter handle is @angelol
Bug fix related to base vectorstore's metadata.
Labels: name: auto:bug, name: auto:improvement

If I use Pinecone to create a vectorbase, by calling from_documents:
```python
    vectorstore = Pinecone.from_documents(
        documents=source_chunks, embedding=embedding, index_name='apple')
```
 the vectors of the same source, have the same content. 
![image](https://github.com/hwchase17/langchain/assets/44468894/28c9576c-c3d1-493f-b594-54f5adb43309)

Issue also described here: https://stackoverflow.com/questions/76401057/why-does-pinecone-repeatedly-return-the-same-result-from-my-series-of-documents

This happens because when I create documents like this:
```python
    source_chunks = []
    for source in docs:
        for chunk in splitter.split_text(source.page_content):
            source_chunks.append(
                Document(page_content=chunk, metadata=source.metadata))
```
the source.metadata is not copied by default. To enforce that, I need to use `Document(page_content=chunk, metadata=source.metadata.copy()))`.

Although this is a user implementation issue, such a thing can happen quite often. To prevent this, we can make sure that the metadata dict is copied by default. 

After wasting 2 hours of my time trying to find out why am I getting dublicate responses on similarity search, I realized that this should be addressed. 

Let me know what you think.
Thanks!
Add local support for audio models
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Conversation Summary Buffer Memory does not accept a system message 
Labels: name: auto:bug

### System Info

I intend to use the conversation summary buffer memory with ChatOpenAI in a conversation chain. For the chat, there's a need to set the system message to instruct and give appropriate personality to the chat assistant. However, system message is not supported to be inserted in the memory either via the save_context (the documented way) or the memory.chat_memory.message.insert()

The summary of the chat itself seems to use the system message to send the summary. This makes Conversation Summary Buffer Memory incompatible with ChatOpenAI.



### Who can help?

@hwchase17  @agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Using either of the two methods to use the system message

1. memory.chat_memory.messages.insert(0, system_message)
2. memory.save_context({"input": SystemMessage(content=system_message)}, {"output": ""})

### Expected behavior

We should be able to use any memory with ChatOpenAI as these need to be modular but ConversationSummaryBufferMemory seems incompatible with it due to system message.
Marqo Vector Store Examples & Type Hints
Labels: name: auto:improvement, name: auto:documentation

This PR improves the example notebook for the Marqo vectorstore implementation by adding a new RetrievalQAWithSourcesChain example. The `embedding` parameter in `from_documents` has its type updated to `Union[Embeddings, None]` and a default parameter of None because this is ignored in Marqo.

This PR also upgrades the Marqo version to 0.11.0 to remove the device parameter after a breaking change to the API.

Related to #7068 @tomhamer @hwchase17
Bagatur/clarifai update
Labels: name: auto:enhancement, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
What is the best size for a chunk
Labels: name: auto:question

### Issue you'd like to raise.

I used RecursiveCharacterTextSplitter.from_tiktoken_encoder to split a document, and if I set chunk_size to 2000, OpenAI cannot answer my question by the documents, if I set chunk_size to 500, OpenAI can work very well. I want to know, As a rule of thumb, what is the best size for a chunk

### Suggestion:

_No response_
[RFC] add interface for evaluating messages
Labels: name: auto:improvement

still a wip
<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
docs: `ecosystem/integration` update
Labels: name: auto:documentation

The example notebooks with `CallbackHandlers` examples were in the  `ecosystem/integration` folder. 
They should be in the `modules/callbacks/integrations` folder.

- Moved notebooks. See ^.
- Created correspondent  `ecosystem/integration` pages in place of examples. Now no Jupyter Notebooks are here, but only .md and .mdx .
- Added descriptions to the Integration Cards.

@baskaryan
Add Xorbits Dataframe as a Document Loader
Labels: name: auto:enhancement, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
- [Xorbits](https://doc.xorbits.io/en/latest/) is an open-source computing framework that makes it easy to scale data science and machine learning workloads in parallel. Xorbits can leverage multi cores or GPUs to accelerate computation on a single machine, or scale out up to thousands of machines to support processing terabytes of data.

- This PR added support for the Xorbits document loader, which allows langchain to leverage Xorbits to parallelize and distribute the loading of data. 
- Dependencies: This change requires the Xorbits library to be installed in order to be used.
`pip install xorbits`
- Request for review: @rlancemartin, @eyurtsev
- Twitter handle: https://twitter.com/Xorbitsio
Fix: OpenAIFunctionsAgent often returns python as a tool in planning, when argument parsing fails
Labels: name: auto:bug

  - Description: With OpenAIFunctionsAgent, often a plan includes `python` as a tool, whose `arguments` is a `str` of the code to execute. In such a case, `_parse_ai_message` fails due to `JSONDecodeError`. This PR does a hack to fix the problem by creating `{"__arg1": arguments}` following the existing hack. I'm unsure when the arguments become string, so I put a condition to do that only for `python` as a tool.
  - Tag maintainer: @hinthornw
  - Twitter handle: mapped
DOC: Broken Python Information Link in Langchain Documentation
Labels: name: auto:bug, name: auto:documentation

### Issue with current documentation:

Python Documentation for Output Parser Unavailable

URL: https://python.langchain.com/docs/modules/prompts/output_parsers.html

### Idea or request for content:

I am currently taking the "Langchain/lesson/2/models, prompts, and parsers" course from deeplearning.ai. While working on the course material, I encountered difficulties with the output parser in Python. To seek assistance and better understand the usage of the output parser, I attempted to access the documentation for the Python implementation. However, I received a "page not found" error when trying to access the Python documentation.
adding max_marginal_relevance_search method to MongoDBAtlasVectorSearch
Labels: name: auto:improvement

Adding a maximal_marginal_relevance method to the MongoDBAtlasVectorSearch vectorstore enhances the user experience by providing more diverse search results

Issue: #7304

@rlancemartin, @eyurtsev --> Please make sure that I'm using correctly the maximal_marginal_relevance (from utils).
Update audio.py added local execution version
Labels: name: auto:improvement

To allow users to easily execute models for audio conversion locally I am adding a class on document_loaders.parsers.audio called OpenAIWhisperParserLocal() Takes two optional arguments: device and lang_model. 

It checks if there is a GPU available and automatically select the best model based on the memory available.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
wip: add caching option for telegram
Labels: name: auto:improvement

todo: write proper description and test this out.

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Solving anthropic packaging version issue
Labels: name: auto:bug

  - Description: Solving, anthropic packaging version issue by clearing the mixup from package.version that is being confused with version from - importlib.metadata.version. 

  - Issue: it fixes the issue #7283 
  - Maintainer: @hwchase17 

The following change has been explained in the comment - https://github.com/hwchase17/langchain/issues/7283#issuecomment-1624328978
Evaluation returns 'INCORRECT' when ground truth is empty and prediction is empty
Labels: name: auto:bug

### System Info

Langchain version: 0.0.225
Python version: 3.8.5

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When ground truth and model prediction are both empty strings, the evaluation model returns 'INCORRECT'. I expect the evalution to return 'CORRECT'

I ran the below piece of code.

```
llm_bit = AzureChatOpenAI(deployment_name='gpt-4-32k', temperature=0)
test_gt = [{'question': 'What is the name of the company?', 'gt': 'Company A'}, {'question': 'What is the name of the building', 'gt': ''}]
test_output = [{'question': 'What is the name of the company?', 'prediction': 'Company A'}, {'question': 'What is the name of the building', 'prediction': ''}]
eval_chain = QAEvalChain.from_llm(llm_bit)
temp = eval_chain.evaluate(
        test_gt, test_output, question_key="question", answer_key="gt", prediction_key="prediction"
    )
temp
```

### Expected behavior

Received output: [{'text': 'CORRECT'}, {'text': 'INCORRECT'}]
Expected output: [{'text': 'CORRECT'}, {'text': 'CORRECT'}]
Add support for filters and namespaces in similarity search in Pinecone similarity_score_threshold
Labels: name: lgtm, name: auto:improvement

At the moment, pinecone vectorStore does not support filters and namespaces when using similarity_score_threshold search type. 
In this PR, I've implemented that. It passes all the kwargs except "score_threshold" as that is not a supported argument for method "similarity_search_with_score".

Tag maintainers/contributors who might be interested:

@dev2049 @baskaryan  @rlancemartin @eyurtsev
Working cpal
Labels: name: auto:enhancement, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Wrapper function for OpenAI Function in LLMChain
Labels: name: auto:improvement

  - Description: This pull request does two things: 1) Add wrapper function for pydantic model, JSON Schema, and dictionary to function calls in LLM Chain. 2) Add an additional class for the output parser class so pydantic model returns a string that could be added to AIMessage for memory storage.
  - Issue: #6933 
  - Dependencies: N/A
  - Tag maintainer: @baskaryan
  - Twitter handle: #7t_eric
OpenAI Chat agents don't make use of OpenAI API `messages` parameter.
Labels: name: auto:bug

### System Info

Langchain: 0.0.225
OS: Arch Linux
Python: 3.11

### Who can help?

@hwchase17 
@agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Use any of langchain's openai chat agents with the memory/chat history feature results in the chat history/memory being sent to the openai API in the SYSTEM message, and with incorrect roles specified.

### Expected behavior

While that might be appropriate for certain types of message (Maybe compressed or summarized from older conversations), I expected the chat history memory to be utilizing openai's [messages](https://platform.openai.com/docs/api-reference/chat#chat/create-messages) parameter.

It's much easier to parse (If ever needed) since it's an array of messages. This is related to https://github.com/hwchase17/langchain/issues/7285 which is an even bigger issue that addressing this one could resolve.