delayedkarma/langchain-issues huggingface dataset:
'ValueError: Missing some input keys: {'query'}' but i added it?
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

long story short, i use streamlit to make a demo where i can upload a pdf, click a button and the content is extracted automatically based on the prompt template's questions.
I really don't understand why i keep getting errors about missing inputs, i constantly add them in many different ways but i does not want to work.
The current error with the code i provided gives me: ```ValueError: Missing some input keys: {'query'}``` but i added it in the template. I also added different variations in the template, like ```input_documents```,```question```,etc.. nothing seems to work.

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

code:
```
prompt_template="""... {context} ... {query}"""
questions = """ ...big chunk of questions ..."""
prompt = PromptTemplate(template=prompt_template, input_variables=["context", "query"])

            rawText = get_pdf_text(pdf)
            textChunks = get_text_chunks(rawText)
            vectorstore = get_vectorstore(textChunks, option)
            docs = vectorstore.similarity_search(questions)
            llm = ChatOpenAI(model_name='gpt-3.5-turbo', temperature=0.1)
            chain_type_kwargs = {"prompt": prompt}
            chain = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=vectorstore.as_retriever(),
                                                chain_type_kwargs=chain_type_kwargs)
            st.write(chain.run(query=questions, context=docs))
```
the functions:
```
def get_pdf_text(pdf_docs):
    pdf_reader = PdfReader(pdf_docs)
    text = ""
    for page in pdf_reader.pages:
        text += page.extract_text()
    return text


def get_text_chunks(text):
    text_splitter = CharacterTextSplitter(
        separator="\n",
        chunk_size=1000,
        chunk_overlap=200,
        length_function=len
    )
    chunks = text_splitter.create_documents(text)
    return chunks


def get_vectorstore(text_chunks, freeEmbedding):
    embeddings = OpenAIEmbeddings()
    if freeEmbedding == "Gratis (dar incet)":
        embeddings = HuggingFaceInstructEmbeddings(model_name="hkunlp/instructor-xl")
    vectorstoreDB = FAISS.from_documents(documents=text_chunks, embedding=embeddings)
    return vectorstoreDB
```



### Expected behavior

For me to get an output based on the prompt
fix (parsers/json): do not escape double quotes if already escaped
Labels: name: auto:bug

This PR fixes an issues I found when upgrading to a more recent version of Langchain. I was using 0.0.142 before, and this issue popped up already when the `_custom_parser` was added to `output_parsers/json`.

Anyway, the issue is that the parser tries to escape quotes when they are double-escaped (e.g. `\\"`), leading to OutputParserException. 
This is particularly undesired in my app, because I have an Agent that uses a single input Tool, which expects as input a JSON string with the structure: 
```python
{
    "foo": string,
    "bar": string
}
```
The LLM (GPT3.5) response is (almost) always something like `"action_input": "{\\"foo\\": \\"bar\\", \\"bar\\": \\"foo\\"}"` and since the upgrade this is not correctly parsed.
fix (Html2TextTransformer): allow configuration of html2text
Labels: name: lgtm, name: area: doc loader, name: auto:improvement

Hi, this PR enables configuring the html2text package, instead of being bound to use the hardcoded values. While simply passing `ignore_links` and `ignore_images` to the `transform_documents` method was possible, I preferred passing them to the `__init__` method for 2 reasons:

1. It is more efficient in case of subsequent calls to `transform_documents`.
2. It allows to move the "complexity" to the instantiation, keeping the actual execution simple and general enough. IMO the transformers should all follow this pattern, allowing something like this:
```python
# Instantiate transformers
transformers = [
    TransformerA(foo='bar'),
    TransformerB(bar='foo'),
    # others
]

# During execution, call them sequentially
documents = ...
for tr in transformers:
    documents = tr.transform_documents(documents)
```

Thanks for the reviews!
The sources field is blank in case of passing custom prompt to RetrievalQAWithSourcesChain.from_chain_type
Labels: name: area: vector store, name: auto:bug, name: area: models

### System Info

python: 3.11

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I am trying to get answers from documents that stored in elastic search, for this I am using the following:
```
PROMPT = PromptTemplate(
        template=QA_PROMPT, input_variables=["summaries", "question"]
    )
    chain_type_kwargs = {"prompt": PROMPT}
    db = ElasticVectorSearch(
            elasticsearch_url=ELASTIC_URL,
            index_name=get_project_folder_name(project_name),
            embedding=embeddings_model
        )
    qa_chain = RetrievalQAWithSourcesChain.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=db.as_retriever(),
        chain_type_kwargs=chain_type_kwargs,
        return_source_documents=True,
        verbose=True
        )

  answer = qa_chain({"question": question})
```

I noticed that sources appear in the response normally in case of not passing chain_type_kwargs=chain_type_kwargs, once I started using chain_type_kwargs=chain_type_kwargs, to apply the custom prompt I get the sources field as blank.

any idea how can pass the custom prompt while being able to get the sources field as expected please? 

### Expected behavior

The source filed should have the name of the files that the answer retrieved from.
Issue: how to merge two vector dbs?
Labels: name: area: embeddings, name: area: vector store, name: auto:question

### Issue you'd like to raise.

hi,

how do I merge 2 vector dbs? 
I am trying to update an existing db with new information


    vectorstore = FAISS.from_documents(docs_chunked, Embeddings())
    vectorstore.save_local("faiss_index_table_string")
    vector_db = FAISS.load_local("faiss_index_table_string", Embeddings())


I want to do something like 


    vectorstore2 = FAISS.from_documents(docs_chunked2, Embeddings())
    vectorstore2.update_local("faiss_index_table_string")
    vector_db_updated = FAISS.load_local("faiss_index_table_string", Embeddings())


### Suggestion:

_No response_
I tried running a simple Langchain code from the docs. This is my error : Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details..             I tried using different Openai API Keys...still not working. Does anyone know how to fix it?
Labels: name: auto:question, name: area: models

### System Info

I tried running it in replit

### Who can help?

_No response_

### Information

- [x] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

import openai
import os
from langchain.llms import OpenAI
llm = OpenAI(temperature=0.3)
print(llm.predict("What is the capital of India?"))


### Expected behavior

I followed a tutorial and the expected output is a prediction of the given text
Added runtime argument to Deep Lake Vector Store initialization
Labels: name: area: vector store, name: auto:improvement

In this PR I am adding optional runtime argument that is responsible for transferring data to Deep Lake's Managed Service, and also change docs accordingly so that users know how to use that argument.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
time_weighted_retriever: use a timestamp if needed
Labels: name: lgtm, name: area: vector store, name: auto:improvement

If last_accessed_at metadata is a float use it as a timestamp. This allows to support vector stores that do not store datetime objects like ChromaDb.

Fixes: https://github.com/langchain-ai/langchain/issues/3685

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
docs: Fix the syntax error, replace "dotenv.load_env()" with "dotenv.…
Labels: name: lgtm, name: auto:documentation

Description:  The documents incorrectly mentions "dotenv.load_env()", but it should actually be "dotenv.load_dotenv()". You can see the screenshot below for reference:

python-dotenv: 1.0.0

![image](https://github.com/langchain-ai/langchain/assets/2959046/94dc4b51-cc2f-412d-92e9-16b8ff0d513e)
Fix typo in code_understanding.ipynb
Labels: name: auto:documentation

seperate -> separate

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Fixed regex bug in RetrievalQAWithSources in previous update
Labels: name: auto:bug

  - Description: In my previous PR, I had modified the code to catch all kinds of [SOURCES, sources, Source, Sources]. However, this change included checking for a colon or a white space which should actually have been only checking for a colon.
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: @baskaryan 
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.
Extend DynamoDBChatMessageHistory to support composite keys
Labels: name: area: memory, name: auto:improvement

  - Description: Adds two optional parameters to the DynamoDBChatMessageHistory class to enable users to pass in a name for their PrimaryKey, or a Key object itself to enable the use of composite keys, a common DynamoDB paradigm. 
  
  [AWS DynamoDB Key docs](https://aws.amazon.com/blogs/database/choosing-the-right-dynamodb-partition-key/)
  
  - Issue: N/A
  - Dependencies: N/A
  - Twitter handle: N/A
Azure Search: Remove select field restrictions and expand metadata to other fields, also expose kwargs to searches
Labels: name: area: vector store, name: auto:improvement

Description: 
If metadata field returned in results, previous behavior unchanged.  If metadata field does not exist in results, expand metadata to any fields returned outside of content field.

There's precedence for this as well, see the retriever: https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/retrievers/azure_cognitive_search.py#L96C46-L96C46

Issue: 
#9765 - Ameliorates hard-coding in case you already indexed to cognitive search without a metadata field but rather placed metadata in separate fields.

@hwchase17
DOC: SQLDatabaseSequentialChain Does Not Exist in langchain_experimental.sql
Labels: name: area: doc loader, name: auto:bug, name: auto:documentation

### Issue with current documentation:

when I write the following code

`from langchain_experimental.sql import  SQLDatabaseSequentialChain
`
I get the following error:
Cannot find reference 'SQLDatabaseSequentialChain' in '__init__.py' 

### Idea or request for content:

How to fix the problem so I can use **SQLDatabaseSequentialChain** class
adding new chain for logical fallacy removal from model output in chain
Labels: name: lgtm, name: auto:enhancement, name: area: models

Description: new chain for logical fallacy removal from model output in chain and docs
Issue: n/a see above
Dependencies: none
Tag maintainer: @hinthornw in past from my end but not sure who that would be for maintenance of chains
Twitter handle: no twitter feel free to call out my git user if shout out j-space-b 

Note: created documentation in docs/extras
langchain.llms.Databricks does not save necessary params (e.g. endpoint_name, cluster_driver_port, etc.) to recover from its config
Labels: name: auto:bug, name: area: models

### System Info

I used langchain=0.0.246 on Databricks, but this bug is due to the lack of implementation of `Databricks._identifying_params()`, so system info should not impact. 

### Who can help?

I see that @nfcampos contributed to most of the Databricks model serving wrapper, so tagging you here.
@hwchase17 @agola11 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
llm = Databricks(host=databricks_host_name, endpoint_name=model_endpoint_name)
   
llm_chain = LLMChain(
    llm=llm,
    prompt=PromptTemplate.from_template(prompt_template),
    verbose=True

llm_chain.save("path/to/llm_chain.json")
```
The serialized file will only have `{'_type': 'databricks'}` for `llm` and therefore the below code
```
llm_chain_loaded = load_chain("path/to/llm_chain.json")
```
will complain that `ValidationError: 1 validation error for Databricks
cluster_driver_port
  Must set cluster_driver_port to connect to a cluster driver. (type=value_error)`. 

This is because `llm_chain.save()` look at the llm chain's `_identifying_params()` which is not defined on `langchain.llms.Databricks` 



### Expected behavior

`llm_chain_loaded = load_chain("path/to/llm_chain.json")` should recover the `langchain.llms.Databricks` instance correctly.

All fields on `langchain.llms.Databricks` that are necessary to re-initialize the instance from a config file should be added to `Databricks._identifying_params()`
Avoid `type: ignore` suppression by adding mypy type hint.
Labels: name: area: doc loader, name: auto:improvement

Mypy was not able to determine a good type for `type_to_loader_dict`,
since the values in the dict are functions whose return types are
related to each other in a complex way. One can see this by adding a
line like `reveal_type(type_to_loader_dict)` and running mypy, which
will get mypy to show what type it has inferred for that value.

Adding an explicit type hint to help out mypy avoids the need for a mypy
suppression and allows the code to type-check cleanly.
Fix: Nested Dicts Handling of Document Metadata
Labels: name: area: vector store, name: auto:bug

## Description
When the `MultiQueryRetriever` is used to get the list of documents relevant according to a query, inside a vector store, and at least one of these contain metadata with nested dictionaries, a `TypeError: unhashable type: 'dict'` exception is thrown.
This is caused by the `unique_union` function which, to guarantee the uniqueness of the returned documents, tries, unsuccessfully, to hash the nested dictionaries and use them as a part of key.
```python
unique_documents_dict = {
    (doc.page_content, tuple(sorted(doc.metadata.items()))): doc
    for doc in documents
}
```

## Issue
#9872 (MultiQueryRetriever (get_relevant_documents) raises TypeError: unhashable type: 'dict' with dic metadata)

## Solution
A possible solution is to dump the metadata dict to a string and use it as a part of hashed key.
```python
unique_documents_dict = {
    (doc.page_content, json.dumps(doc.metadata, sort_keys=True)): doc
    for doc in documents
}
```
rfc: mem session
Labels: name: area: memory, name: auto:improvement

not pretty but curious if folks think this approach has legs
<img width="951" alt="Screenshot 2023-08-28 at 1 47 32 PM" src="https://github.com/langchain-ai/langchain/assets/22008038/0c634637-9c3a-4d58-8e37-420d16ffc888">
How to send ConversationalRetrievalChain object as a response of api call in javascript
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

I am using FastAPI which is written in Python to develop nlp tool where user can generate conversation chain by uploading pdf docs. Data extraction and Preprocessing logic written in backend API. I want to send a conversation chain object which is not JSON serializable. Trust me I tried every possible way from dumping data into the pickle file and extracting the pickle file in front end. nothing is working. because there is no npm package available to parse pickle data. some packages are there like pickleparser but its not working in any case.

### Suggestion:

_No response_
MultiQueryRetriever (get_relevant_documents) raises TypeError: unhashable type: 'dict' with dic metadata
Labels: name: area: vector store, name: auto:bug

### System Info

MultiQueryRetriever will fail to call `_get_relevant_documents` if the Document objects have metadata which are dictionaries.

```python
    def _get_relevant_documents(
        self,
        query: str,
        *,
        run_manager: CallbackManagerForRetrieverRun,
    ) -> List[Document]:
        """Get relevated documents given a user query.

        Args:
            question: user query

        Returns:
            Unique union of relevant documents from all generated queries
        """
        queries = self.generate_queries(query, run_manager)
        documents = self.retrieve_documents(queries, run_manager)
        unique_documents = self.unique_union(documents)
        return unique_documents
```


The following error gets raised: TypeError: unhashable type: 'dict'

As we try to hash a dict as one of the keys in `unique_union`

This is mostly due to the mechanism in: 

```python
    def unique_union(self, documents: List[Document]) -> List[Document]:
        """Get unique Documents.

        Args:
            documents: List of retrieved Documents

        Returns:
            List of unique retrieved Documents
        """
        # Create a dictionary with page_content as keys to remove duplicates
        # TODO: Add Document ID property (e.g., UUID)

        unique_documents_dict = {
            (doc.page_content, tuple(sorted(doc.metadata.items()))): doc
            for doc in documents
        }
        unique_documents = list(unique_documents_dict.values())
        return unique_documents
```

unique keys should be generated based on something else than the metadata in order to avoid this behavior.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Run the examples in the: [MultiQueryRetriever Documentation](https://python.langchain.com/docs/modules/data_connection/retrievers/MultiQueryRetriever)

and use a vectordb which contains Documents that has dictionaries in its metadata.


### Expected behavior

TypeError: unhashable type: 'dict' will be raised.