delayedkarma/langchain-issues huggingface dataset:
Release 0.0.337 breaks MongoDBAtlasVectorSearch.from_connection_string?
Labels: name: area: vector store, name: auto:bug

### System Info

Hi there,

I have a LangChain app at https://huggingface.co/spaces/bstraehle/openai-llm-rag/blob/main/app.py.  Using the latest release 0.0.337 produces the error below.  Pinning the library to release 0.0.336 works as expected. 

:blue_heart: LangChain, thanks!

Bernd

---

Traceback (most recent call last):
  File "/home/user/app/app.py", line 129, in invoke
    db = document_retrieval_mongodb(llm, prompt)
  File "/home/user/app/app.py", line 91, in document_retrieval_mongodb
    db = MongoDBAtlasVectorSearch.from_connection_string(MONGODB_URI,
  File "/home/user/.local/lib/python3.10/site-packages/langchain/vectorstores/mongodb_atlas.py", line 109, in from_connection_string
    raise ImportError(
ImportError: Could not import pymongo, please install it with `pip install pymongo`.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce the behavior:

1. In file https://huggingface.co/spaces/bstraehle/openai-llm-rag/blob/main/requirements.txt, unpin the langchain library (or pin it to release 0.0.337).
2. Use the app at https://huggingface.co/spaces/bstraehle/openai-llm-rag with MongoDB selected to invoke `MongoDBAtlasVectorSearch.from_connection_string`, which produces the error.

### Expected behavior

When using release 0.0.337 `MongoDBAtlasVectorSearch.from_connection_string`, error "ImportError: Could not import pymongo, please install it with `pip install pymongo`." should not happen.
migrate openai audio api
Labels: name: lgtm, name: area: doc loader, name: auto:improvement, name: size:S

for issue https://github.com/langchain-ai/langchain/issues/13162
migrate openai audio api, as [openai v1.0.0 Migration Guide](https://github.com/openai/openai-python/discussions/742) 

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
improve sparql-qa-chain
Labels: name: auto:improvement, name: size:XL

**Description:**
* fixed the SPARQL queries for schema extraction - in some cases not all classes were loaded
* improved the prompt for creating natural language answers from the query results
* fixed the RDF example - the content was not retrieved from the graph DB due to the name not matching the one in the graph
* added an OWL example with a local file - simplified, but inspired by @LorenzBuehmann's example in #8907

**Issue:**
loosely related to issue #8907 and the discussion within PR #11240. the issues mentioned there were resolved in #9136, but made me notice another issue where classes were missing in the extracted schema

**Dependencies:**
none

**Tag maintainer:**
@baskaryan
Add embedchain retriever
Labels: name: lgtm, name: area: embeddings, name: auto:enhancement

**Description:**

This commit adds embedchain retriever along with tests and docs. Embedchain is a RAG framework to create data pipelines.

**Twitter handle:**
- [Taranjeet's twitter](https://twitter.com/taranjeetio) and [Embedchain's twitter](https://twitter.com/embedchain)

**Reviewer**
@hwchase17
DOC: add info about how to update the template and the packages of an app created from a template
Labels: name: auto:documentation

### Issue with current documentation:

How to update the template and packages of an app created from a template?
I checked:
https://github.com/langchain-ai/langchain/tree/master/templates
and a couple of templates' README.mds, but this info is missing and it's not obvious for us citizen devs.
I supposed it should be done via langchain-cli, but there's no such option.
So pls. provide a solution and add it to docs.

### Idea or request for content:

How to update the template and packages of an app created from a template?
feat: load all namespaces
Labels: name: lgtm, name: area: doc loader, name: auto:improvement

  - **Description:** This change allows for the `MWDumpLoader` to load all namespaces including custom by default instead of only loading the [default namespaces](https://www.mediawiki.org/wiki/Help:Namespaces#Localisation).
  - **Tag maintainer:** @hwchase17
Implemented the option to include output from map chain in the final result of ConversationalRetrievalChain
Labels: name: auto:bug

**Description:** 
MapReduceDocumentChain contains a flag named “return_intermediate_steps” that allows intermediate steps to be outputted along with the final result. Normally, when this flag is set to True, the final result dictionary should contain a list of intermediate results produced during the map reduce process. However, when a MapReduceDocumentChain object with “return_intermediate_steps” set to True is used as a part of a ConversationalRetrievalChain object, an error will arise, making it impossible for users to obtain intermediate results of the execution of a ConversationalRetrievalChain object with map reduce.

The main cause of the issue seems to be inside  _call() of class BaseConversationalRetrievalChain, where method run() is invoked on self.combine_docs_chain. 
<img width="502" alt="image" src="https://github.com/langchain-ai/langchain/assets/88512829/c4f71b0d-489f-4b7f-badc-1635c03d76f0">

When a MapReduceDocumentChain object with “return_intermediate_steps” set to True is evaluated, there will be two output keys, named “output_text” and “intermediate_steps”, respectively. Therefore, _ _call_ _() should be used to evaluate such an object. However, inside the method _call() of class BaseConversationalRetrievalChain, it is hardcoded that run() is to be used to evaluate the object, resulting in this error
<img width="697" alt="image" src="https://github.com/langchain-ai/langchain/assets/88512829/4264ade0-381c-459c-86ee-7340dc201a60">

This PR modifies the existing code of the ConversationalRetrievalChain to verify if the map reduce document chain passed into combine_docs_chain has the return_intermediate_result flag set to True, and add the intermediate result of map reduce to the output of ConversationalRetrievalChain accordingly.

 **Issue:** 
 #11209

**Dependencies:**
No new dependencies have been added.

I worked closely with @PitcairnHotdog on this PR
Issue: When using Qianfan chat model and enabling streaming, get ValueError
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Here is my code:

"""For basic init and call"""
import os
import qianfan
from langchain.chat_models import QianfanChatEndpoint
from langchain.chat_models.base import HumanMessage
os.environ["QIANFAN_AK"] = "myak"
os.environ["QIANFAN_SK"] = "mysk"

chat = QianfanChatEndpoint(
    streaming=True,
)
res = chat.stream([HumanMessage(content="给我一篇100字的睡前故事")], streaming=True)
for r in res:
    print("chat resp:", r)

And after it prints two sentences, returns an error. The full error message is:
Traceback (most recent call last):
  File "d:\work\qianfan_test.py", line 13, in <module>
    for r in res:
  File "C:\Users\a1383\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\chat_models\base.py", line 220, in stream
    raise e
  File "C:\Users\a1383\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\chat_models\base.py", line 216, in stream
    generation += chunk
  File "C:\Users\a1383\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\schema\output.py", line 94, in __add__
    message=self.message + other.message,
  File "C:\Users\a1383\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\schema\messages.py", line 225, in __add__
    additional_kwargs=self._merge_kwargs_dict(
  File "C:\Users\a1383\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\schema\messages.py", line 138, in _merge_kwargs_dict
    raise ValueError(
ValueError: Additional kwargs key created already exists in this message.

I am only following the official langchain documentation:https://python.langchain.com/docs/integrations/chat/baidu_qianfan_endpoint 
And it is not working. What have I done wrong?

### Suggestion:

_No response_
Fix for oracle schema parsing stated on the issue #7928
Labels: name: lgtm, name: auto:bug

  - **Description:** In this pull request, we address an issue related to assigning a schema to the SQLDatabase class when utilizing an Oracle database. The current implementation encounters a bug where, upon attempting to execute a query, the alter session parse is not appropriately defined for Oracle, leading to an error, 
  - **Issue:** #7928,
  - **Dependencies:** No dependencies,
  - **Tag maintainer:** @baskaryan,
fixed `openai_assistant` namespace
Labels: name: area: agent, name: auto:bug

BUG: langchain.agents.openai_assistant has a reference as
`from langchain_experimental.openai_assistant.base import OpenAIAssistantRunnable`
should be 
`from langchain.agents.openai_assistant.base import OpenAIAssistantRunnable`

This prevents building of the API Reference docs
implement the summary buffer fix without insantiating new BaseMessage…
Labels: name: area: memory, name: auto:bug

**Description:** Despite this class intending to summarize just enough messages to fit within a set context size, the product of this class always contains ALL messages PLUS a summary system message at the front, ultimately making token count longer rather than shorter. This commit addresses the issue by fixing the single bug where the buffer is constantly reset to the internal message store even after messages are summarized and removed. 

- The error does not occur with in-memory History buffers, but instead fails when trying to access and modify an immutable database (such as a Postgres DB)
- [My organization's project repo](https://github.com/UGA-Innovation-Factory/AlfredBot) contains the code that triggered this error to be discovered, as well as output Verbose=True files for the current langchain master, my initial fix, and this revised solution that doesn't replicate any extra Lists.

**Issues:** Couldn't find one, misread the issue I thought was similar with my last pull request.

@efriis 

Reason for changes in diff order:
- An index is stored with the moving summary buffer to represent up to which messages it represents
- The buffer property only returns memory outside of the summary buffer, without a duplicate List this time
- This satisfy's the need for the required member `type` for any BaseMessage constructor
- Update the prune index
- Clear index when clearing other members
New update broke embeddings models
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: integration: chroma

### System Info

LangChain version: 0.0.337
Python version: 3.10.13

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

db = Chroma.from_documents(docs, AzureOpenAIEmbeddings())

### Expected behavior

This worked on previous versions of LangChain using OpenAIEmbeddings(), but now I get this error

BadRequestError: Error code: 400 - {'error': {'message': 'Too many inputs. The max number of inputs is 16.  We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.', 'type': 'invalid_request_error', 'param': None, 'code': None}}
update OpenAI function agents' llm validation
Labels: name: lgtm, name: auto:improvement, name: area: models, name: size:S

  - **Description:** This PR modifies the LLM validation in OpenAI function agents to check whether the LLM supports OpenAI functions based on a property (`supports_oia_functions`) instead of whether the LLM passed to the agent `isinstance` of `ChatOpenAI`. This allows classes that extend `BaseChatModel` to be passed to these agents as long as they've been integrated with the OpenAI APIs and have this property set, even if they don't extend `ChatOpenAI`. 
  - **Issue:** N/A
  - **Dependencies:** none
Issue: Missing some input keys in langchain even when it's present - unclear how prompt args are treated
Labels: name: area: vector store, name: auto:bug, name: area: models, name: integration: chroma

### Issue you'd like to raise.

Why does the below code complain that extra_instructions is a missing key, even though it's learning included in input_variables=["context", "question", "extra_instructions"]?

Any help is greatly appreciated.


    vectorstore = Chroma(
        collection_name=collection_name,
        persist_directory=chroma_db_directory,
        embedding_function=embedding,
    )
    
    prompt_template = """
    
    {extra_instructions}
    

    {context}
    

    {question}
    
    Continuation:
    """
    
    PROMPT = PromptTemplate(
        template=prompt_template,
        input_variables=["context", "question", "extra_instructions"],
    )

    qa = RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=vectorstore.as_retriever(
            search_kwargs={"k": 1}
        ),
        chain_type_kwargs={"verbose": True, "prompt": PROMPT},
    )



### Suggestion:

_No response_
OpenAI Functions Extraction Chain not returning a list
Labels: name: auto:bug, name: area: models, name: extraction

### System Info

python==3.10.13
langchain==0.0.336
pydantic==1.10.13

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Noticed an error with Pydantic validation when the schema contains optional lists. Here are the steps to reproduce the issue and the error that I am getting.

1. A basic extraction scheme is defined using Pydantic.
```python
from pydantic import BaseModel, Field
class Properties(BaseModel):
        person_names: Optional[List[str]] = Field([], description="The names of the people")
        person_heights: Optional[List[int]] = Field([], description="The heights of the people")
        person_hair_colors: Optional[List[str]] = Field([], description="The hair colors of the people")
```

2. Extraction chain is created to extract the defined fields from a document.
```python
from langchain.chat_models import ChatOpenAI
from langchain.chains import create_extraction_chain_pydantic
llm = ChatOpenAI(
                temperature=0, model="gpt-3.5-turbo", request_timeout=20, max_retries=1
            )
chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)
```

3. When we run the extraction on a document, sometimes the OpenAI function call does not return the `info` field as a `list` but instead as a `dict`. That creates a validation error with Pydantic, even if the extracted fields are perfectly given in the returned dictionary.
```python
inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""
response = chain.run(inp)
```
4. The error and the traceback are as follows
```python
File "pydantic/main.py", line 549, in pydantic.main.BaseModel.parse_raw
File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for PydanticSchema
info
  value is not a valid list (type=type_error.list)
```

### Expected behavior

We can make the Pydantic validation pass by maybe simply casting the `info` field into a list if it is somehow returned as a dictionary by the OpenAI function call.
Output parser fails with index out of range error but doesn't give actual fail reason in case of GooglePalm
Labels: name: auto:bug, name: area: models

### System Info

- Python 3.11.5
- google-ai-generativelanguage==0.3.3
- langchain==0.0.336

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce the issue:

1. Create a simple GooglePalm llm
2. Run simple chain with some medical related prompt e.g: `Tell me reasons why I am having symptoms of cold`

Error:
```
    return self.parse(result[0].text)
                      ~~~~~~^^^
IndexError: list index out of range
```

### Expected behavior

There is a proper error thrown by GooglePalm in completion object

```
Completion(candidates=[],
           result=None,
           filters=[{'reason': <BlockedReason.SAFETY: 1>}],
           safety_feedback=[{'rating': {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                                        'probability': <HarmProbability.HIGH: 4>},
                             'setting': {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                                         'threshold': <HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE: 2>}}])
```
If same is relayed to user it will be more useful.
Add VertexAI Chuck Norris template
Labels: name: lgtm, name: auto:enhancement, name: template

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
fix: filtering in opensearch was not working
Labels: name: area: vector store, name: auto:bug, name: integration: elasticsearch

- changed the filtering inside the query (boolean filtering)

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:**: I simply adjusted the boolean filtering with the proper format. 
  - **Issue:** the issue #
  - **Dependencies:** None
  - **Tag maintainer:** @baskaryan, @eyurtsev, @hwchase17
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
RAG but for fact checking
Labels: name: area: vector store, name: auto:enhancement, name: area: models

### Feature request

I would be very nice to be able to build something similar to RAG, but for checking an assumption quality against a knowledge base.

### Motivation

Currently, RAG allows a semantic search but does not help when it comes to evaluating the quality of a user input, based on your vector db.

The point is not to fact-check the news in a newspaper (must be very hard actually...), but to evaluate truth in a random prompt.


### Your contribution

- prompting
- PR
- doc
- discussions
Introduce search_kwargs for Custom Parameters in BingSearchAPIWrapper
Labels: name: lgtm, name: auto:improvement

Added a `search_kwargs` field to BingSearchAPIWrapper in `bing_search.py,` enabling users to include extra keyword arguments in Bing search queries. This update, like specifying language preferences, adds more customization to searches. The `search_kwargs` seamlessly merge with standard parameters in `_bing_search_results` method.