delayedkarma/langchain-issues huggingface dataset:
Pinecone MultiqueryRetriver with multiple OR metadata filters
Labels: name: area: vector store, name: auto:question, name: integration: pinecone

### Issue you'd like to raise.

HI there, I am trying to use Multiquery retiever on pinecone vectordb with multiple filters. I specifically need to use an OR operator. 

For example, I would like to retrieve documents with metadata having the metadata "category" : 'value1' OR 'value2' OR 'value3'

this is my current implementation:


```
import pinecone
from langchain.llms import Cohere
from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain.vectorstores import Pinecone

pinecone.init(api_key=pine_api_key, environment=pinecone_env)

index_name = "index_name"

index = pinecone.Index(index_name)

vectorstore = Pinecone(index, embeddings, "text")

llm = Cohere(cohere_api_key=cohere_api_key)

retriever_from_llm = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 10, 
                                                    'filter': {'user_id': '42',
                                                               'category': "c1"}}), llm=llm
)
```
At this point I am able to sucessfully filter by user_id and category when it is only one value, but when I add more values as such 

'category': ["c1", "c2", "c3"]. 

It does not work, when retrieving documents as such:

```
question = "what is foo?"

unique_docs = retriever_from_llm.get_relevant_documents(query=question)
```

Error:

```
ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'date': 'Thu, 23 Nov 2023 00:24:58 GMT', 'x-envoy-upstream-service-time': '0', 'content-length': '108', 'server': 'envoy'})
HTTP response body: {"code":3,"message":"illegal condition for field category, got ["c1","c2"]","details":[]}.
```

How can I add a filters that returns documents with category c1 Or c2 Or c3, using pincone and MultiqueryRetriver?

### Suggestion:

_No response_
refactor: Reduce duplicate code of import openai
Labels: name: auto:refactor, name: area: models, name: size:L

<!-- Thank you for contributing to LangChain!

use `import_openai` to import openai, remove duplicate code of import

and There are subtle differences in the import code in different places

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
MessageHistory type field serialized twice twice
Labels: name: area: memory, name: auto:bug, name: integration: redis

```python
from langchain.memory import RedisChatMessageHistory

history = RedisChatMessageHistory("foo")

history.add_user_message("hi!")

history.add_ai_message("whats up?")
```
{**<ins>"type": "human"**</ins>, "data": {"content": "hi!", "additional_kwargs": {}, <ins>**"type": "human"**</ins>, "example": false}}

I suspect it's a bug in the serialization or does it serve any purpose being in that structure with a duplicate k/v?

https://github.com/langchain-ai/langchain/blob/163bf165ed2d6ae453aad1bca4ed56814d81bf5b/libs/core/langchain_core/messages/base.py#L113C1-L115C1
rm master condition
Labels: name: auto:nit, name: size:XS

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
GenericLoader.from_filesystem "exclude" not working
Labels: name: area: doc loader, name: auto:bug

### System Info

Python 3.9.6, Langchain 0.0.334

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I'm experimenting with some simple code to load a local repository to test CodeLlama, but the "exclude" in GenericLoader.from_filesystem seems not working:

`from langchain.document_loaders.generic import GenericLoader
from langchain.document_loaders.parsers import LanguageParser
from langchain.text_splitter import Language

repo_path = "../../my/laravel/project/"

# Load
loader = GenericLoader.from_filesystem(
    repo_path,
    glob="**/*",
    suffixes=[".php"],
    parser=LanguageParser(
        parser_threshold=2000,
    ),
    exclude=["../../my/laravel/project/vendor/", "../../my/laravel/project/node_modules/", "../../my/laravel/project/storage/", "../../my/laravel/project/public/", "../../my/laravel/project/tests/", "../../my/laravel/project/resources/"]
)

documents = loader.load()
len(documents)
`

Am I missing something obvious? I cannot find any example...with or without the exclude, the length of docs is the same (and if I just print "documents" I see files in the folders I excluded).

### Expected behavior

I would expect that listing subpaths from the main path then these would be excluded.
Random Application Lockdown on ECS Fargate with Langchain and FastAPI
Labels: name: area: memory, name: auto:bug, name: area: models, name: integration: aws

### System Info

**Operating system/architecture:**
Linux/X86_64
**CPU | Memory**
8 vCPU | 16 GB
**Platform version**
1.4.0
**Launch type**
FARGATE

**Project libraries:**
snowflake-sqlalchemy==1.4.6
python-dotenv==0.21.0
openai==0.27.2
langchain==0.0.336
pandas==2.0.2
boto3==1.26.144
colorama==0.4.6
fastapi==0.100.1
pydantic~=1.10.8
pytest~=7.1.2
uvicorn~=0.17.6
cassio==0.1.3
sentry-sdk==1.29.2
langsmith==0.0.66
numpy==1.24.3
SQLAlchemy==1.4.46
psycopg2-binary==2.9.7
tiktoken==0.4.0
httpx==0.24.1
unidecode==1.3.7
transformers==4.28.0
transformers[torch]
tensorflow==2.12.1
keras==2.12.0

**Python version of the project:**
python:3.10-slim-bullseye

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

It's quite challenging to replicate the error as it appears to be rather random. After a few requests, FastAPI stops responding following the OPTIONS query of the endpoint. This issue seems to be attributable to one of the libraries in use. I observed this error after a code refactoring in the project, moving from legacy chains to chains with LCEL. Since this refactoring, the ECS system has exhibited peculiar behavior. Extensive debugging has been conducted throughout the codebase, yet there are no indications of the error's origin. It's worth noting that everything functions flawlessly in local emulation, with no occurrence of any unusual errors. The problem arises when the code is deployed to the ECS Fargate instance, and I want to emphasize that this issue did not exist before the aforementioned changes were made.
<img width="996" alt="Captura de pantalla 2023-11-22 a la(s) 5 51 24 p  m" src="https://github.com/langchain-ai/langchain/assets/122487744/15c12aeb-6016-468d-81de-b31a4204ca78">


### Expected behavior

I need someone to help me with new ways to debug this extremely rare bug, to give me ideas on what to do, what to show from my machine, what can be done, or if it's some incompatibility between the libraries. I haven't been able to pinpoint the specific point where the program stops, and it's proving to be very challenging.
UTF-8 content is not loaded correctly with RecursiveUrlLoader 
Labels: name: area: doc loader, name: auto:bug

### System Info

Langchain version: 0.0.339
Python version: 3.11.5
Running on Ubuntu 22.04.3 LTS via WSL 2

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Running the steps described in the guide for RecursiveUrlLoader:  [https://python.langchain.com/docs/integrations/document_loaders/recursive_url](url)

2. Loading some content with UTF-8 encoding, the Python docs for example: [https://docs.python.org/3/](url)
Exact code used: 
```python
from pprint import pprint

from langchain.document_loaders import RecursiveUrlLoader
from bs4 import BeautifulSoup


def load_python_docs():
    url = "https://docs.python.org/3/"
    loader = RecursiveUrlLoader(
        url=url, max_depth=2, extractor=lambda x: BeautifulSoup(x, "html.parser").text
    )
    return loader.load()


pprint([doc.metadata for doc in load_python_docs()])
```

3. If you print the loaded documents, you should be able to see these kind of encoding issues:
```
{'description': 'Editor, Adam Turner,. This article explains the new features '
                 'in Python 3.12, compared to 3.11. Python 3.12 was released '
                 'on October 2, 2023. For full details, see the changelog. '
                 'Summary â\x80\x93 Release hi...',
  'language': None,
  'source': 'https://docs.python.org/3/whatsnew/3.12.html',
  'title': 'Whatâ\x80\x99s New In Python 3.12 — Python 3.12.0 documentation'}
```

### Expected behavior

Should load content correctly, using the right encoding to parse the document. I suppose the issue is due to the fact that the `_get_child_links_recursive` method is calling `requests.get` and not specifying the encoding for the response. A quick fix that worked for me was to include the following line, just after the GET request:  `response.encoding = response.apparent_encoding`
docs: top menu
Labels: name: lgtm, name: auto:documentation, name: size:S

Addressed this issue with the top menu: It allocates too much space. If the screen is small, then the top menu items are split into two lines and look unreadable.
Another issue is with several top menu items: "Chat our docs" and "Also by LangChain". They are compound of several words which also hurts readability. The top menu items should be 1-word size.
Updates:
- "Chat our docs" -> "Chat" (the meaning is clean after clicking/opening the item)
- "Also by LangChain" -> "🦜️🔗"
- "🦜️🔗" moved before "Chat" item. This new item is partially copied from the first left item, the "🦜️🔗 LangChain". This design (with two 🦜️🔗 elements, visually splits the top menu into two parts. The first item in each part holds the 🦜️🔗 symbols and, when we click the second 🦜️🔗 item, it opens the drop-down menu. So, we've got two visually similar parts, which visually split the top menu on the right side: the LangChain Docs (and Doc-related items) and the lift side: other LangChain.ai (company) products/docs.
CC @efriis
Langchain-cli Does not install a usable binary.
Labels: name: auto:bug, name: area: models

### System Info

Langchain version: 0.0.339
Laangserve version: 0.0.30
Langchain-cli version: 0.0.19

Running on apple silicone: MacBook Pro M3 max using a zsh shell.



### Who can help?

@erick

### Information

- [x] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [x] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Running the installation steps in the guide with
`pip3 install -U langchain-cli`

And then trying to run `langchain app` just results in zsh responding with `zsh: command not found: langchain`.

### Expected behavior

Running the installation steps in the guide with
`pip3 install -U langchain-cli`

Should allow me to run

`langchain app new my-app`
[wip] BUGFIX: backwards compatible core namespacing
Labels: name: auto:bug, name: size:XL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
DOC: missing imports in "Types of MessagePromptTemplate"
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

Some langchain objects are used but not imported in [Types of MessagePromptTemplate](https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/msg_prompt_templates) documentation.

Specifically, it lacks the following:
```python
from langchain.prompts import HumanMessagePromptTemplate
from langchain.prompts import ChatPromptTemplate
```
and

```python
from langchain.schema.messages import HumanMessage, AIMessage
```



### Idea or request for content:

_No response_
Issue: <Handling Python Package Conflicts in Langchain When Integrating External APIs for Agent Tools>
Labels: name: area: agent, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

**Description:**
Hi, sir. I am currently developing an Agent for scientific tasks using Langchain. The first tool I intend to integrate is a function for generating a mesh, implemented through the external API `dolfinx`. However, I've encountered a compatibility issue between my Langchain packages and the `dolfinx` package. This seems to be a broader problem related to designing an Agent that leverages diverse external APIs.

The concern here is that as I continue to expand the Agent's capabilities with more tools utilizing various external APIs, I might face additional conflicts. This issue could be a common challenge in the development process of creating Agents with multiple, diverse external integrations. Here's a snippet of the code for reference:
```
python
from mpi4py import MPI
from dolfinx import mesh, io

@tool("create_mesh")
def create_mesh_tool(text: str):
    """Returns a dolfinx.mesh.Mesh object, useful for generating meshes for PDEs.
    The input should always be a string."""
    domain = mesh.create_unit_square(MPI.COMM_WORLD, 8, 8, mesh.CellType.quadrilateral)
    return domain

tools = [make_random_num, create_mesh_tool]
```

I am seeking advice or solutions on how to effectively manage and resolve these package conflicts within the Langchain framework, particularly when integrating tools that depend on external APIs like dolfinx.


### Suggestion:

_No response_
Add Nasa tool
Labels: name: area: agent, name: auto:enhancement, name: size:L

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
 
 **Description:**
 This PR adds a new NASA toolkit that provides users access to the NASA's Image and Video Library API, with potential to expand and include other available NASA APIs in the future. 

Integration tests and an example notebook showing the toolkit's use have been added. 

Who can review?
@baskaryan , @eyurtsev , @hwchase17

**We also ran the following checks:**
make format passed ✅
make lint passed ✅
Files touched by this PR passed the linter ✅
passed integration tests via local jupyter notebook testing on NasaAPIWrapper ✅
make spell_check passed ✅

The original issue that this ticket addresses can be found here: https://github.com/langchain-ai/langchain/issues/11787
DB-bound (or, "Convertor-based") prompt templates + Cassandra and Feast implementation
Labels: name: auto:enhancement, name: area: models, name: size:XXL

This PR adds a new (sub)type of Prompt Template to reduce boilerplate around "retrieving from a DB to fill a prompt".

A generic subclass of `StringPromptTemplate` provides the common behaviour, which is then further subclassed as a Cassandra-specific and a Feast-specific ready-to-use classes. The accompanying notebook/doc well shows rationale and usage and points to the way for a developer to expand with other DB-specific implementations.

A picture to summarize the decoupling of inputs / template variables with a DB-lookup acting as a bridge:

![Cassandra-bound prompt template](https://user-images.githubusercontent.com/14221764/284892387-d8dcfd8f-828a-4ae8-9874-b4961747eaa5.png)

#### Remarks

- I could not test building the docs due to a `No docs found in "templates"` from Docusaurus - might be related to the recent langchain/langchain_core refactor. Sorry about that.
- The notebook includes the same image I left here; however it is linked from `user-images.githubusercontent.com` - I am not sure of the right place for those. I'd ask for advice on this point.

I'd like to draw @eyurtsev 's attention to this PR - Thank you very much!
chain input not pass into Python_REPL tool input by '\n'
Labels: name: area: agent, name: auto:bug

### System Info

langchain==0.0.339
windows11
python==3.10

### Who can help?

simple chain with tool code ,but not pass full input to tool cause '\n'
`
model = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0, verbose=False, streaming=True)

custom_tool_list = [PythonREPLTool(),#CustomPythonExec()
                        ]
memory = ConversationBufferMemory(memory_key="chat_history")

agent_executor = initialize_agent(
        custom_tool_list,
        llm=model,
        agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
        verbose=True,
        max_iterations=1,
        memory=memory,
        early_stopping_method="generate",
        agent_kwargs={"prefix": custom_prefix},
        handle_parsing_errors="Check your output and make sure it conforms",

agent_executor.run("some prompt") 
`
logs:
[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [4.94s] Exiting Chain run with output:
{
  "text": "Thought: Do I need to use a tool? Yes\nAction: Python_REPL\nAction Input: import pandas as pd\ndf = pd.read_csv('./statics/20231123_000614_test.csv')\nline_count = len(df
)\nline_count"
}
[tool/start] [1:chain:AgentExecutor > 4:tool:Python_REPL] Entering Tool run with input:
"import pandas as pd"
2023-11-23 00:06:19 - Python REPL can execute arbitrary code. Use with caution.
[tool/end] [1:chain:AgentExecutor > 4:tool:Python_REPL] [4ms] Exiting Tool run with output:
""


### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [4.94s] Exiting Chain run with output:
{
  "text": "Thought: Do I need to use a tool? Yes\nAction: Python_REPL\nAction Input: import pandas as pd\ndf = pd.read_csv('./statics/20231123_000614_test.csv')\nline_count = len(df
)\nline_count"
}
[tool/start] [1:chain:AgentExecutor > 4:tool:Python_REPL] Entering Tool run with input:
"import pandas as pd"
2023-11-23 00:06:19 - Python REPL can execute arbitrary code. Use with caution.
[tool/end] [1:chain:AgentExecutor > 4:tool:Python_REPL] [4ms] Exiting Tool run with output:
""


### Expected behavior

pass full input string to tool
Issue: Error `Got invalid JSON object. Error: Extra data:`when using `SelfQueryRetriever`
Labels: name: area: vector store, name: auto:bug, name: area: models

### Issue you'd like to raise.

I followed the lecture by Harrison on langchain on deeplearning.ai and when i follow it I am getting error `Got invalid JSON object. Error: Extra data:`when using `SelfQueryRetriever` to `get_relevant_documents`. 

Using the following code:
```python
document_content_description = "Lecture notes"

retriever = SelfQueryRetriever.from_llm(
    llm=model,
    vectorstore=vectordb,
    metadata_field_info=metadata_field_info,
    document_contents=document_content_description,
    verbose=True,
    handle_parsing_errors=True,
)

docs = retriever.get_relevant_documents(question)
```

### Suggestion:

_No response_