delayedkarma/langchain-issues huggingface dataset:
Issue: Problems getting filtering to work with langchain, OpenSearchVectorSearch, similarity_search_with_score
Labels: name: area: vector store, name: auto:question, name: integration: aws

### Issue you'd like to raise.

Using langchain on AWS lambda function to call an AWS OpenSearch Serverless vector index.  I did not file as a bug, as I suspect that this is a usage problem, but it might be a bug.

I have built an OpenSearch vector index, using AWS OpenSearch Serverless, and are using it to perform vector queries against document chunks.  This is all working well. I would now like to add the ability to filter these searches by document (there may be hundreds or thousands of vector entries per document).  There is metadata included with each entry, which includes a DocumentId field, which is a string-ized guid.  I thought it would be simple to filter against this metadata attribute, but cannot get it working.  

The call I am making is:
```
    docs = vsearch.similarity_search_with_score(
        theQuery,
        index_name="vector-index2",
        vector_field = "vector_field",
        search_type="script_scoring",
        pre_filter=filter_dict,
        space_type="cosinesimil",
        engine="faiss",
        k = 3
    )
```

Typical unfiltered output:
{"doc": {"page_content": "seat is  installed ... facing", "metadata": {"DocumentId": "{8B5D5890-0000-C1E8-B0B6-678C8509665D}", "DocumentTitle": "2023-Toyota-RAV4-OM.pdf", "DocClassName": "Document"}}, "score": 1.8776696}]

For filter_dict, I have tried:
    filter_dict = {'DocumentId':theDocId}                                    <-- Gets an error 
    filter_dict = {'bool': {'must': {'term': {'DocumentId': theDocId} } } }  <-- doesn’t find anything
    filter_dict = {'bool': {'must': {'match': {'DocumentId': theDocId} } } } <-- doesn’t find anything
    filter_dict = {'must': {'match': {'DocumentId': theDocId} } }            <-- Gets an error
    filter_dict = {'match': {'DocumentId': theDocId} }                         <-- doesn’t find anything

theDocId is a string set to an input parameter and looks like this:  {8B5D5890-0000-C1E8-B0B6-678C8509665D}.  Results are the same if I use a string constant (i.e. '{8B5D5890-0000-C1E8-B0B6-678C8509665D}')

Same results if I use search_type=”approximate_search” in conjunction with boolean_filter=filter_dict

Any suggestions on how to get this to work?


### Suggestion:

_No response_
`StructuredOutputParser.get_format_instructions` missing commas
Labels: name: auto:bug, name: area: models

### System Info

n/a

### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

https://github.com/langchain-ai/langchain/blob/v0.0.322/libs/langchain/langchain/output_parsers/structured.py#L86

This line doesn't append commas in the expected JSON schema.

```none
# What it does
{
    "foo": List[string]  // a list of strings
    "bar": string  // a string
}

# What it should do I think
{
    "foo": List[string],  // a list of strings
    "bar": string  // a string
}
```

### Expected behavior

I think the suggestion to a LLM to be close to valid JSON (even though it has JSON comments).
ModuleNotFoundError: No module named 'rapidfuzz'
Labels: name: auto:bug

### System Info

https://github.com/langchain-ai/langchain/tree/d2cb95c39d5569019ab3c6aa368aa937d8dcc465
(just above `v0.0.322`)

MacBook Pro, M1 chip, macOS Ventura 13.5.2

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

With Python 3.10.11:

```bash
python -m venv venv
source venv/bin/activate
pip install poetry
poetry install --with test
make test
```

This gets:

```
poetry run pytest --disable-socket --allow-unix-socket tests/unit_tests/
Traceback (most recent call last):
  File "/Users/james.braza/code/langchain/venv/bin/poetry", line 5, in <module>
    from poetry.console.application import main
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/poetry/console/application.py", line 11, in <module>
    from cleo.application import Application as BaseApplication
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/cleo/application.py", line 12, in <module>
    from cleo.commands.completions_command import CompletionsCommand
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/cleo/commands/completions_command.py", line 10, in <module>
    from cleo import helpers
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/cleo/helpers.py", line 5, in <module>
    from cleo.io.inputs.argument import Argument
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/cleo/io/inputs/argument.py", line 5, in <module>
    from cleo.exceptions import CleoLogicError
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/cleo/exceptions/__init__.py", line 3, in <module>
    from cleo._utils import find_similar_names
  File "/Users/james.braza/code/langchain/venv/lib/python3.10/site-packages/cleo/_utils.py", line 8, in <module>
    from rapidfuzz.distance import Levenshtein
ModuleNotFoundError: No module named 'rapidfuzz'
```


### Expected behavior

I expected installation to install everything necessary to run tests
demonstrate use of RetrievalQAWithSourcesChain.from_chain
Labels: name: auto:improvement, name: area: models

 **Description:** 
Documents further   usage of  RetrievalQAWithSourcesChain in an existing test. I'd not found much documented usage of RetrievalQAWithSourcesChain and how to get the sources out.  This additional code will hopefully  be useful to other potential users of this retriever.

 **Issue:** No raised issue
 
**Dependencies:** No new dependencies needed to run the test (it already needs `open-ai`, `faiss-cpu` and  `unstructured`).

Note - `make lint` showed 8 linting errors  in unrelated files
Improve Redis add_documents efficiency
Labels: name: lgtm, name: area: vector store, name: auto:improvement, name: integration: redis

  - **Description:** Small fix to make the `add_documents` functionality a bit more performant regarding embedding creation
  - **Issue:** https://github.com/langchain-ai/langchain/issues/11496
  - **Dependencies:** nothing new
  - **Tag maintainer:** @baskaryan @Spartee 
  - **Twitter handle:** N/A

Addresses the inefficiency noted in the tagged issue when `add_documents()` is used on the Redis vectorstore.
Issue: can I only use ConversationSummaryMemory in a chain as context without renew it
Labels: name: area: memory, name: auto:question

### Issue you'd like to raise.

Hi, I was wondering if there is a mechanism that I can only use the ConversationSummaryMemory in a chain as the context of a conversation. After the chain is completed, I can retain the old memory without adding the new conversation to it. 

### Suggestion:

_No response_
Fix exec command in langserve Dockerfile template
Labels: name: auto:bug

@eyurtsev -- This is a super quick and easy fix.  This is my first contribution, so please let me know if I have missed doing anything that I should have! 


Description:
Add `poetry run` to the the Dockerfile template file final exec command
```
CMD exec poetry run uvicorn test_serve.server:app --host 0.0.0.0 --port $PORT
```
instead of just
```
CMD exec uvicorn test_serve.server:app --host 0.0.0.0 --port $PORT
```

Issue:
Without the `poetry run` added, `uvicorn` is not found at the exec line when running `docker compose up`
Fix redis vectorfield schema defaults
Labels: name: area: vector store, name: auto:bug, name: integration: redis

  - **Description:** refactors the redis vector field schema to properly handle default values, includes a new unit test suite. 
  - **Issue:** N/A
  - **Dependencies:** nothing new.
  - **Tag maintainer:** @baskaryan @Spartee 
  - **Twitter handle:** this is a tiny fix/improvement :) 

This issue was causing some clients/cuatomers issues when building a vector index on Redis on smaller db instances (due to fault default values in index configuration). It would raise an error like:

```redis.exceptions.ResponseError: Vector index initial capacity 20000 exceeded server limit (852 with the given parameters)```

This PR will address this moving forward.
Bagatur/js ts nav
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
OPENAI_API_KEY_PATH is not supported
Labels: name: auto:bug, name: area: models

### System Info

Langchain HEAD

### Who can help?

@hwchase17 @ag

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

OpenAI uses a standard `OPENAI_API_KEY_PATH` environment variable that can be used to provide a key located in a file. This is useful for instance for generating short-lived, automatically renewed keys. Currently, Langchain complains that an API key must be provided even when this environment variable is set.

### Expected behavior

Langchain should use the value of OPENAPI_API_KEY_PATH with a higher priority than OPENAI_API_KEY when set.
Allowing openapi agent to communicate with multiple APIs
Labels: name: area: agent, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Changed planner and API_PLANNER_PROMPT in planner_prompt to enable create_openapi_agent communication with multiple APIs (list of ReducedOpenAPISpec). This is useful when more than one API is required to complete a task, for example in a software that is built on micro services. In addition, I changed the README in the devcontainer folder to clarify how to use the link of VS Code Dev Container (before that the following were not shown: /<yourusername>/<yourclonedreponame>).
  - **Issue:**
  - **Dependencies:**
  - **Tag maintainer:** @baskaryan , @eyurtsev , @hwchase17
  - **Twitter handle:** @Ido_Nitsan

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Update epsilla.py
Labels: name: area: vector store, name: auto:improvement

Description: Including collection_name as an input variable.
Issue: Previously, wasn't accepting collection_name as an input variable.
Tag maintainer: @hwchase17

Was unable to perform `make test` locally. Ran 'poetry install --with test', but there's no group called "test" in 'pyproject.toml'. Sorry, new to PRs...
Router + BaseCombineDocumentsChain output_key
Labels: name: auto:bug, name: auto:question, name: area: models

### System Info

Hi!

I have been trying to create a router chain which will route to StuffDocumentsChains.

It seems that default output_key for BaseCombineDocumentsChain is output_text instead of 'text' like in typical chains. 
```py
class BaseCombineDocumentsChain(Chain, ABC):
    """Base interface for chains combining documents.

    Subclasses of this chain deal with combining documents in a variety of
    ways. This base class exists to add some uniformity in the interface these types
    of chains should expose. Namely, they expect an input key related to the documents
    to use (default `input_documents`), and then also expose a method to calculate
    the length of a prompt from documents (useful for outside callers to use to
    determine whether it's safe to pass a list of documents into this chain or whether
    that will longer than the context length).
    """

    input_key: str = "input_documents"  #: :meta private:
    output_key: str = "output_text"  #: :meta private:
```
### Question: Is it done on purpose or is just inconsistency?


---
On the other hand
MultiPromptChain has hardcoded output_keys as 'text'.
```py
class MultiPromptChain(MultiRouteChain):
    """A multi-route chain that uses an LLM router chain to choose amongst prompts."""

    @property
    def output_keys(self) -> List[str]:
        return ["text"]
```
So it is necessary to always change output_key in any CombineDocuments like chains. 



### Who can help?

@hwchase17
@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Create a router chain that routes to StuffDocumentsChain accordingly with documentation. 

### Expected behavior

BaseCombineDocumentsChain should have conssistent default outputs keys as other chains. 

Or

MultiPromptChain/MultiRouteChain should not have hardcoded output keys
RetryWithErrorOutputParser for chains and summary memory
Labels: name: area: memory, name: auto:enhancement, name: area: models

### Feature request

Is it possible to support scenarios for using `RetryWithErrorOutputParser` when prompt is injected in a chain or a summary memory so this prompt is not directly formatted?

### Motivation

I am trying to support complex prompts, multi-chain and combined memory scenarios causing errors regularly when the LLM generate content response in a different structure as stablished in the format instructions.

### Your contribution

I could make the PR with some basic implementation guidelines.
Issue: RSSFeedLoader unreliable publish date
Labels: name: area: doc loader, name: auto:improvement

### Issue you'd like to raise.

The `publish_date` field returned in the metadata seems unreliable and is very often empty for many news sites. The publish date can easily be parsed directly from the RSS feed from the `pubDate` tag which, although optional, is pretty standard and is much more reliable than trying to pull it from the article itself which is currently being done using the `NewsURLLoader`.

### Suggestion:

We can override the `publish_date` coming from the `NewsURLLoader` metadata with the `published` field from the feed entry in the `RSSFeedLoader`
Update README.md
Labels: name: auto:documentation

Improved the documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Improved the documentation , 

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Error reshaping array in _extract_images_from_page() in pdf.py
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain==0.0.320
Ubuntu 20.04
Python 3.11.5

### Who can help?

@SuperJokerayo  @baskaryan 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When I try to parse a PDF and extract an image I get the following error:

```
  File "<.....>/lib/python3.11/site-packages/langchain/document_loaders/parsers/pdf.py", line 106, in _extract_images_from_page
    np.frombuffer(xObject[obj].get_data(), dtype=np.uint8).reshape(
ValueError: cannot reshape array of size 45532 into shape (4644,3282,newaxis)
```

The code I have to run the parser is:
```
loader = PyPDFLoader(doc["file"], extract_images=True)
pages = loader.load_and_split()
```

I can't give you a copy of the document since it's confidential. It's a scanned 11-page document.

### Expected behavior

I expected `load_and_split()` to return a list of 11 pages with the text parsed from the image in each page.

If I change to `extract_images=False` then it returns an empty list of 0 pages as expected.
Issue: Prioritze sources in the vector store to consider certain sources first
Labels: name: area: vector store, name: auto:improvement, name: integration: pinecone

### Issue you'd like to raise.

Let's say you have a PDF file, an Excel file, and a PowerPoint file all indexed into a pinecone database, they are similar in that, a query can have information relevant in all sources, but now we want the user to indicate which source should be considered first when they ask a question, is there a way to do that?
For the conversational retrieval chain, you have this retriever there
retriever = vectorstore.as_retriever( search_kwargs = {"filter": {
                                                        "file_types": {"$in": file_types},
                                                        "genre": {"$in": genre},
                                                        "keys": {"$in": keys},
                                                        },
                                                        "k": 3
                                                    }),
how do i make sure i can prioritize sources such that that source comes first, if its not there, we check the others

### Suggestion:

_No response_
Finetuned OpenAI models cost calculation #11715
Labels: name: auto:improvement, name: area: models

**Description:**
Add cost calculation for fine tuned models (new and legacy), this is required after OpenAI added new models for fine tuning and separated the costs of I/O for fine tuned models. 
Also I updated the relevant unit tests
see https://platform.openai.com/docs/guides/fine-tuning for more information.
issue: https://github.com/langchain-ai/langchain/issues/11715

  - **Issue:** 11715
  - **Twitter handle:** @nirkopler
Add a config chain before the ConversationalRetrievalChain : search Kwargs
Labels: name: auto:question, name: auto:enhancement, name: area: models

### Feature request

I want to add a config chain prior to the ConversationalRetrievalChain that is going to process the query and set the retriever's search kwargs ( k, fetch_k, lambda_mult ) depending on the question, how can I do that and pass the parameters from the config chain output

### Motivation

The motivation is to have a dynamic ConversationalRetrievalChain that retrieves the documents the right way for each question.

### Your contribution

I have none for now, I don't know where to start
Fix flexible dimension and doc for DingoDB
Labels: name: area: vector store, name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->