delayedkarma/langchain-issues huggingface dataset:
Es knn index search  5346

# Create elastic_vector_search.ElasticKnnSearch class

This extends `langchain/vectorstores/elastic_vector_search.py` by adding a new class `ElasticKnnSearch`

Features:
- Allow creating an index with the `dense_vector` mapping compataible with kNN search 
- Store embeddings in index for use with kNN search (correct mapping creates HNSW data structure)
- Perform approximate kNN search
- Perform hybrid BM25 (`query{}`) + kNN (`knn{}`) search
- perform knn search by either providing a `query_vector` or passing a hosted `model_id` to use query_vector_builder to automatically generate a query_vector at search time

Connection options
- Using `cloud_id` from Elastic Cloud
- Passing elasticsearch client object

search options
- query
- k
- query_vector
- model_id
- size
- source
- knn_boost (hybrid search)
- query_boost (hybrid search)
- fields


This also adds examples to `docs/modules/indexes/vectorstores/examples/elasticsearch.ipynb`


Fixes # [5346](https://github.com/hwchase17/langchain/issues/5346)

cc: @dev2049

 -->
Add parameters to send_message() call for vertexai chat models (PaLM2)

# Ensure parameters are used by vertexai chat models (PaLM2)

The current version of the google aiplatform contains a bug where parameters for a chat model are not used as intended.

See https://github.com/googleapis/python-aiplatform/issues/2263

Params can be passed both to start_chat() and send_message(); however, the parameters passed to start_chat() will not be used if send_message() is called without the overrides.  This is due to the defaults in send_message() being global values rather than None (there is code in send_message() which would use the params from start_chat() if the param passed to send_message() evaluates to False, but that won't happen as the defaults are global values).

Fixes # 5531

@hwchase17
@agola11
DOC: mlflow logging for GraphCypherQAChain

### Issue with current documentation:

I see there is a bit of documentation about the [GraphCypherQAChain](https://python.langchain.com/en/latest/modules/chains/examples/graph_cypher_qa.html). There is also a documentation about using [mlflow with langchain](https://sj-langchain.readthedocs.io/en/latest/ecosystem/mlflow_tracking.html). However, there is no documentation about how to implement both. I tried to figure it out, but I failed. Is there a way to do it or does it have to be implemented?

### Idea or request for content:

A description on using GraphCypherQAChain together with prompt logging as provided by mlflow.
Documentation fixes (linting and broken links)
Labels: name: documentation, name: lgtm

# Lint sphinx documentation and fix broken links

This PR lints multiple warnings shown in generation of the project documentation (using "make docs_linkcheck" and "make docs_build"). Additionally documentation internal links to (now?) non-existent files are modified to point to existing documents as it seemed the new correct target.

The documentation is not updated content wise.
There are no source code changes.

Fixes # (issue)

- broken documentation links to other files within the project
- sphinx formatting (linting)

## Before submitting

No source code changes, so no new tests added.

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Issue: does the stream of load_summarize_chain work?

### Issue you'd like to raise.

I tried adding stream to reduce_llm like the code below to make it work but it doesn't seem to work.

```
reduce_llm = OpenAI(
    streaming=True,
    verbose=True,
    callback_manager=callback,
    temperature=0,
    max_tokens=1000,
)

llm = OpenAI(
    temperature=0,
    max_tokens=500,
    batch_size=2,
)

summarize_chain = load_summarize_chain(
    llm=llm,
    reduce_llm=reduce_llm,
    chain_type="map_reduce",
    map_prompt=map_prompt,
    combine_prompt=combine_prompt,
)
         
return await summarize_chain.arun(...)
```
```
/lib/python3.10/site-packages/langchain/llms/base.py:133: RuntimeWarning: coroutine 'AsyncCallbackManager.on_llm_start' was never awaited
  self.callback_manager.on_llm_start(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
lib/python3.10/site-packages/langchain/llms/openai.py:284: RuntimeWarning: coroutine 'AsyncCallbackManager.on_llm_new_token' was never awaited
  self.callback_manager.on_llm_new_token(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
/lib/python3.10/site-packages/langchain/llms/base.py:141: RuntimeWarning: coroutine 'AsyncCallbackManager.on_llm_end' was never awaited
```
Please let me know if I'm doing something wrong or if there's any other good way.
Thank you.

### Suggestion:

_No response_
prompt issue and handling answers

### Issue you'd like to raise.




```
prompt_template = """Use the following pieces of context to answer the question at the end based on the examples provided in between ++++++++ If you don't know the answer, just answer with [], don't try to make up an answer.              
++++++++
Here are some examples: 
red: The apple is red
yellow: the bannana is yellow
Question:  What is the color of the bannana?
[yellow]
++++++++
red: The apple is red
yellow: the bannana is yellow
Question: Which ones are fruits?
[red, yellow]
++++++++
red: The apple is red
yellow: the bannana is yellow
Question: Are there any of them blue?
[]
++++++++
Now that you know what to do here between ``` provide a similar answer check under Helpful Answer: for more information
```
{context}              
Question: {question}
{format_answer}
```
The examples in between ++++++++ are to understand better the way to produce the answeres

The section between ``` are the ones that you will have to elaborate and provide answers.
Helpful Answer: 
Your answer should be [document1] when document1 is the pertinent answer 
Your answer should be [document2] when document2 is the pertinent answer 
The answer should be [document1, document2] when document1 and document2 include the answer or when the answer could be both of the,
Not interested why document1 or document2 are better I just need to know which one
When you cannot find the answer respond with []"""
```

I'm providing 
```
format_answer = output_parser.get_format_instructions()
prompt = PromptTemplate(
    template=prompt_template,
    input_variables=["context", "question"],
    partial_variables={"format_answer": format_answer}
)
```

I'm providing context and question 
with the format 
document1: text text text
document2: text text

Question: question goes here?
{format_answer}

But Azure Open AI answer sometimes with the format specified and some times just spit out the text provided with document1 and document2 

Any help?


### Suggestion:

I would like to understand more about 
1) How to generate consistent answers from the LLM to the same question 
==> Good luck with that

2) I would like to understand how to validate the answers better than the "parser" available here.
Corrects inconsistently misspelled variable name.

Corrects a spelling error (of the word separator) in several variable names.  Three cut/paste instances of this were corrected, amidst instances of it also being named properly, which would likely would lead to issues for someone in the future.

Here is one such example:

```
        seperators = self.get_separators_for_language(Language.PYTHON)
        super().__init__(separators=seperators, **kwargs)
```
becomes
```
        separators = self.get_separators_for_language(Language.PYTHON)
        super().__init__(separators=separators, **kwargs)
```

Make test results below:

```
============================== 708 passed, 52 skipped, 27 warnings in 11.70s ==============================
```
Extendable SequentialChain 

### Feature request

I'd like to extend `SequentialChain` to create pre-built class which has a pre-populated list of `chains` in it. Consider the following example:
```
class CoolChain(LLMChain):
    ...
    
class CalmChain(LLMChain):
    ...
    
class CoolCalmChain(SequentialChain):
    llm: BaseLanguageModel
    chains: List[LLMChain] = [
        CoolChain(
            llm=llm
        ),
        CalmChain(
            llm=llm
        )
    ]
``` 
This unfortunately cannot happen because the `root_validator` for `SequentialChain` raises error that it cannot find `chains` being passed.
```
Traceback (most recent call last):
  File "/home/yash/app/main.py", line 17, in <module>
    ai = CoolCalmChain(llm=my_llm)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1050, in pydantic.main.validate_model
  File "/home/yash/.venv/lib/python3.10/site-packages/langchain/chains/sequential.py", line 47, in validate_chains
    chains = values["chains"]
KeyError: 'chains'
```

I request that we find a way to bypass the stringent check so that this class can be easily extendable to support pre-built Chains which can be initialised on-the-go.

### Motivation

Motivation is to create multiple such "Empty Sequential Classes" which can be populated on-the-go. This saves us from populating parameters that might change for the same `SequentialChain`, such as the `llm` and `memory`, while defining the class itself.
I tried to define another `root_validator` to override the SequentialChain one and even that did not work.  

### Your contribution

Few solutions that I could think of are as follows:
- Removing `pre` from the `define_chains` **root_validator** in the `SequentialChain` class. 
- Using `@validator('chains')` instead, so that one can override it by simply using `pre`.
Create word.py

Loader that loads word files, such as .doc or .docx file

# Add a new DocumentLoader to support word file.

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

## @eyurtsev

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Query With Multiple Collections

Hi,
I am  using langchain to create collections in my local directory after that I am persisting it using below code 

from langchain.vectorstores import Chroma                                   
from langchain.embeddings import OpenAIEmbeddings                               
from langchain.text_splitter import RecursiveCharacterTextSplitter , TokenTextSplitter            
from langchain.llms import OpenAI                                             
from langchain.chat_models import ChatOpenAI                             
from langchain.chains import VectorDBQA, RetrievalQA                                        
from langchain.document_loaders import TextLoader, UnstructuredFileLoader, DirectoryLoader

loader = DirectoryLoader("D:/files/data")  
docs = loader.load()                            
text_splitter = TokenTextSplitter(chunk_size=1000, chunk_overlap=200)
texts = text_splitter.split_documents(docs)
embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key)                                 
vectordb = Chroma.from_documents(texts, embedding=embeddings, persist_directory = persist_directory, collection_name=my_collection)    
vectordb.persist()                           
vectordb = None

I am using above code for creating different different collection in the same persist_directory by just changing the collection name and the data files path, now lets say I have 5 collection in my persist directory
my_collection1
my_collection2
my_collection3
my_collection4
my_collection5

Now If I want to perform querying to my data then I have to call my persist_directory with collection_name
vectordb = Chroma(persist_directory=persist_directory, embedding_function=embeddings, collection_name=my_collection3)
qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(openai_api_key=openai_api_key), chain_type="stuff", retriever=vectordb.as_retriever(search_type="mmr"), return_source_documents=True)
qa("query")

so the issue is if I am using above code then I can perform only querying for my_collection3 but I want to perform querying to all my five collections, so can anyone please suggest, how can I do this or if it is not possible, I will be thankful to you.

I had tried without collection name for ex-
vectordb = Chroma(persist_directory=persist_directory, embedding_function=embedding)
qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(openai_api_key=openai_api_key), chain_type="stuff", retriever=vectordb.as_retriever(search_type="mmr"), return_source_documents=True)
qa("query")

but in this case I am getting 
NoIndexException: Index not found, please create an instance before querying
ConversationalRetrievalChain is changing context of input

### System Info

 The first time I query the LLM evertything is okay, the second time and in all the calls after that user query is totally changed.
For example my input was "How are you today?" and the chain while trying to make this a standalone question gets confused and totally changes the question,
file:///home/propsure/Pictures/Screenshots/Screenshot%20from%202023-06-01%2015-47-08.png
This is how I am using the chain,
`      QA_PROMPT = PromptTemplate(template=prompt_template, input_variables=["context","question", "chat_history" ])
        chain = ConversationalRetrievalChain.from_llm(
                    llm=llm, retriever=retriever, return_source_documents=False, 
                    verbose=True,
                    max_tokens_limit=2048, combine_docs_chain_kwargs={'prompt': QA_PROMPT}
`

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Run any llm locally, I am  using WizardVicuna (faced the same issue with OpenAI api), and try a conversationalretreival QA with chat_history. Also if I am doing something incorrectly please let me know much appreciated. 

### Expected behavior

there should be an option if we don't want it to condense the question, prompt. Because in context based QA we want a similar behaviour.
[Vectorstore] Added async gRPC methods to Qdrant vectorstore
Labels: name: area: vector store, name: auto:enhancement

# Async gRPC methods to Qdrant vectorstore

Changes:

- Async methods to add:
aadd_texts, asimilarity_search, asimilarity_search_with_score, amax_marginal_relevance_search_with_score

- Helper methods to add:
_document_from_scored_point_grpc, _document_from_scored_point_grpc, _build_condition_grpc, _qdrant_filter_from_dict_grpc

In addition to rest, Qdrant supports the gRPC protocol, which allows calling asynchronous methods on the client. 
gRPC is an open source remote procedure call (RPC) framework developed by Google. gRPC uses a data serialization and RPC protocol called Protocol Buffers to enable efficient communication between clients and servers.

By adding Qdrant's native asynchronous methods, we can solve the event loop blocking issue that occurs when using synchronous methods.

Since these asynchronous methods only work on remote clients, not local, I've made it so that attempting to use them in a local environment (memory, disk) will result in a NotImplementedError.

I was going to create a unit test functions, but I didn't add it for now because it requires a real Remote server to test the method, but it's working fine for now in my project.
Fixed incorrect prompt formatting for chat based ReAct agent
Labels: name: area: agent, name: auto:bug

# Fixed incorrect prompt formatting for chat based ReAct agent

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes #5390
Created two new classes (ReActStringPromptValue and ReActPromptTemplate) in order to handle correct prompt formatting for chat based models.

My twitter handle: @gultepeoguz

## Before submitting
Format/lint/tests already run.

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Improve Error Messaging for APOC Procedure Failure in Neo4jGraph
Labels: name: lgtm

## Improve Error Messaging for APOC Procedure Failure in Neo4jGraph

This commit revises the error message provided when the 'apoc.meta.data()' procedure fails. Previously, the message simply instructed the user to install the APOC plugin in Neo4j. The new error message is more specific.

Also removed an unnecessary newline in the Cypher statement variable: `node_properties_query`.

Fixes #5545 

## Who can review?
  - @vowelparrot
  - @dev2049
Issue: Improve Error Messaging When APOC Procedures Fail in Neo4jGraph

### Issue you'd like to raise.

In the current implementation, when an APOC procedure fails, a generic error message is raised stating: "Could not use APOC procedures. Please install the APOC plugin in Neo4j." This message can lead to user confusion as it suggests the APOC plugin is not installed when in reality it may be installed but not correctly configured or permitted to run certain procedures.

This issue is encountered specifically when the refresh_schema function calls apoc.meta.data(). The function apoc.meta.data() isn't allowed to run under default configurations in the Neo4j database, thus leading to the mentioned error message.

Here is the code snippet where the issue arises:
```
# Set schema
try:
    self.refresh_schema()
    except neo4j.exceptions.ClientError
    raise ValueError(
        "Could not use APOC procedures. "
        "Please install the APOC plugin in Neo4j."
    )
```

### Suggestion:

To improve the user experience, I propose that the error message should be made more specific. Instead of merely advising users to install the APOC plugin, it would be beneficial to indicate that certain procedures may not be configured or whitelisted to run by default and to guide the users to check their configurations.

I believe this will save users time when troubleshooting and will reduce the potential for confusion.
The same code, sometimes throwing an exception(ValueError: Could not parse output), sometimes running correctly

### Issue you'd like to raise.

```
loaders = [TextLoader("13.txt"), TextLoader("14.txt"), TextLoader("15.txt"),TextLoader("16.txt"), TextLoader("17.txt"), TextLoader("18.txt")]
documents = []
for loader in loaders:
    documents.extend(loader.load())

text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=150)
documents = text_splitter.split_documents(documents)
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(documents, embeddings)

qa = RetrievalQAWithSourcesChain.from_chain_type(llm=llm,
                         chain_type="map_rerank",
                         retriever=vectorstore.as_retriever(),
                         return_source_documents=True)

query = "茶艺师报名条件是什么，使用中文回答"
chat_history = []
result = qa({'question': query})
```

sometimes this code raise ValueError: Could not parse output，but when I rerun result = qa({'question': query}), this may return the anwser. how can I fix this?

### Suggestion:

I wonder why this happens, and how to fix it. please help me!!
Incorrect argument count handling
Labels: name: lgtm

Throwing ToolException when incorrect arguments are passed to tools so that that agent can course correct them.

# Incorrect argument count handling

I was facing an error where the agent passed incorrect arguments to tools. As per the discussions going around, I started throwing ToolException to allow the model to course correct.

## Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->