delayedkarma/langchain-issues huggingface dataset:
add test for ensemble retriever to ensure multiple retrievers
Labels: name: auto:improvement

Add tests to ensemble retriever to ensure it works with combination of multiple retrievers

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: Add tests to ensemble retriever to ensure it works with combination of multiple retrievers
  - Issue: None
  - Dependencies: None
  - Tag maintainer: @rlancemartin, @eyurtsev
  - Twitter handle: @Mlopsj

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Sql tool cardinality distinct
Labels: name: area: agent, name: auto:improvement

- Description: This adds functionality to the SQLDatabaseToolkit agent_toolkit to address a shortcoming in the agent's capability to better understand and utilize the data in the database for filtering when required. Previously, the model would hallucinate filter values or just hard code exactly what a user asks for, which often times would return no results. This Loop was fine for users used to working with data who could iteratively prompt to get distinct values and subsequently ask another question to filter, but not as useful for users who are not experienced with working with databases and SQL. This change makes the natural language agent a bit more autonomous and self healing by being better able to gain an understanding of the data it's working with.
- Issue: N/A,
- Dependencies: none,
- Tag maintainer: @hinthornw (changing a tool/toolkit/agent), @hwchase17, @baskaryan (changing a prompt), also made a tweak to the makefile for lint_diff (should a check for only changed files not [follow imports](https://mypy.readthedocs.io/en/stable/running_mypy.html#following-imports)?)
- Twitter handle: @bpafoshizle

[Unit Test](https://github.com/bpafoshizle/langchain/blob/f093d0c7d948186fc3501301105603cdc5a0c00e/libs/langchain/tests/unit_tests/test_sql_database.py#L128)

[Example notebook](https://github.com/bpafoshizle/langchain/blob/sql-tool-cardinality-distinct/docs/extras/integrations/toolkits/sql_database.ipynb)

Passed format, lint (without following imports), and tests (including test I added).
Questions Regarding load_summarize_chain Implementation and LLM Models Usage
Labels: name: auto:question

### Issue you'd like to raise.

Hello, I'm currently using the load_summarize_chain function to summarize extensive documents, specifically those about 250 pages or more. In this process, I'm employing the map_reduce chain type along with a custom prompt.

The process works as follows:

In the initial step, the chain sends parallel requests to LLM to summarize chunks of data.
Upon receiving all responses, the summary chain collects them, combines them into a single prompt, and generates the final output.

I'm curious about two aspects of this process:
In the second step, the chain employs the StuffDocumentsChain method to create the final summary despite setting mapreduce in chaintype. Is it possible to use an alternative chain type for this? If yes, could you recommend any?

Would it be feasible to utilize different Language Learning Models (LLMs) in the two steps? For instance, could I use GPT3.5 for the first step and GPT4 for generating the final summary?

I appreciate your help and look forward to your response.

Thanks in advance!

### Suggestion:

_No response_
Improve README.md Documentation
Labels: name: auto:documentation

This pull request improves the README.md documentation for better readability and clarity. The updates provide a clear introduction to LangChain's purpose, organized sections for each use case, and detailed explanations of core concepts. The formatting is consistent, and interactive links enhance user navigation. Additionally, we've added contributing guidelines and clear contact information. These changes aim to empower users, both newcomers and experienced, to utilize LangChain effectively and build powerful applications with Large Language Models.

@baskaryan
can you provide agent examples for multiple rounds of dialogue?
Labels: name: area: agent, name: auto:documentation

For example: we have two tool, one is search tool, used to search transaction, the other is date tool
for search tool, we have three parameters, if user input less parameters, search tool will feedback user to input missing information, then user will provide supplementary information, agent need to combine dialogue history to route to search tool until the information is complete, finally search tool return the query results.
Bagatur/from texts bug fix
Labels: name: area: vector store, name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
how can i get response of agent
Labels: name: area: agent, name: auto:question

i just wrote a custom agent to classify intent to choose different tool, but i don't know how to get response from agent, when i print answer, it shows Agent stopped due to iteration limit or time limit. Code is shown as below, thanks
tools = [SearchTool(),DateTool()]
agent = IntentAgent(tools=tools, llm=llm)
agent_exec = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True, max_iterations=1)
answer=agent_exec.run(prompt)
DOC: Custom LLM Wrappers not functional
Labels: name: area: doc loader, name: auto:bug, name: auto:documentation

### Issue with current documentation:

I have tried to make a wrapper around my LLMs but the class cant be instantiated. 

Can't even get the example here to work:

https://python.langchain.com/docs/modules/model_io/models/llms/custom_llm

Can someone help out with this?

### Idea or request for content:

_No response_
chain.arun() for summarization no faster than chain()
Labels: name: area: embeddings, name: auto:bug

### System Info

(h2ogpt) jon@pseudotensor:~/h2ogpt$ pip freeze | grep langchain
langchain==0.0.235
langchainplus-sdk==0.0.20

Python 3.10

(h2ogpt) jon@pseudotensor:~/h2ogpt$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal


### Who can help?

@agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [x] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [x] Chains
- [ ] Callbacks/Tracing
- [x] Async

### Reproduction

https://github.com/h2oai/h2ogpt/pull/551/commits/cc3331d897f4f7bab13c7f9644e7a7d7cd35031e

The above shows my introduction of async from before not having it.

The text generation inference server is set to have a large concurrency, but is showing requests are coming in back-to-back.

### Expected behavior

I expect the summarization part to be parallel, like stated here:

https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chains/combine_documents/map_reduce.py#L210-L213

But perhaps I misunderstand something.  Or perhaps it's not really parallel:

https://github.com/langchain-ai/langchain/issues/1145#issuecomment-1586234397

There's lots of discussion w.r.t. hitting rate limit with OpenAI:

https://github.com/langchain-ai/langchain/issues/2465
https://github.com/langchain-ai/langchain/issues/1643

So I presume this works, but I'm not seeing it.  In OpenAI case it seems to be done via batching, which is possible in HF TGI server but not implemented.  But I would have thought that all the reduction tasks could have been in parallel with asyncio.

https://github.com/langchain-ai/langchain/pull/1463#issuecomment-1566391189
DOC: how to setup APIs in my project to receive requests and returns responses from and to end devices
Labels: name: auto:question, name: auto:documentation

### Issue with current documentation:

I would like to deploy my langchain project in a shared host server , so I will need tobe able APIs to receive request and return response to the end devices. I think this feature is already done , however, I do not how to implement it in my project

### Idea or request for content:

_No response_
Minor fixes to enhance notebook usability:
Labels: name: area: vector store, name: auto:documentation

- Install langchain
- Set Pinecone API key and environment as env vars
- Create Pinecone index if it doesn't already exist
---
  - Description: Fix a couple minor issues I came across when running this notebook, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: none,
  - Tag maintainer: @rlancemartin @eyurtsev,
  - Twitter handle: @zackproser (certainly not necessary!)
Update tot.ipynb
Labels: name: lgtm, name: auto:documentation

Spelling error fix

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
qa_chain with retriever and input_documents. 
Labels: name: auto:question

### System Info

I wonder if the input_documents are combined with relevant_docuemnts from retriever?  I have been comparing results, and although wiki_doc provide different context, it is not reflected in response! I have tried both version, and the information by "input_doc" is not reflected in result1. I looked up in example, and I didn't find example where both "retriever" and "input_documents" are used together, it is either this OR that. 

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)
qa_chain = RetrievalQA.from_chain_type(llm, retriever=ret_para, chain_type="stuff")
result1 = qa_chain({"input_documents": wiki_doc, "query": query})
result2 = qa_chain({"query": query})


### Who can help?

@agola11

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce is:

1-load any db
2-ret = db.as_retriever()
3-llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)
4-qa_chain = RetrievalQA.from_chain_type(llm, retriever=ret_para, chain_type="stuff")
5-Create any additional_doc from some other sources 
6-result1 = qa_chain({"input_documents": additional_doc, "query": query})
7-result2 = qa_chain({"query": query})

### Expected behavior

It is expected that result1 and result2 have different output since they have different contexts, if input_documents is properly combined with relevant_documents from retriever.
Exllama integration to run GPTQ models
Labels: name: auto:enhancement

### Feature request

An integration of exllama in Langchain to be able to use 4-bit GPTQ weights, designed to be fast and memory-efficient on modern GPUs.


### Motivation

The benchmarks on the official repo speak for themselves: 
https://github.com/turboderp/exllama#results-so-far

### Your contribution

There is a fork that uses exllama with langchain here :
https://github.com/CoffeeVampir3/exllama-langchain-example/tree/master
Add LLaMa Formatter and AzureML Chat Endpoint
Labels: name: auto:enhancement

## Description

Microsoft and Meta recently [announced their collaboration](https://blogs.microsoft.com/blog/2023/07/18/microsoft-and-meta-expand-their-ai-partnership-with-llama-2-on-azure-and-windows/) on LLaMa2. This PR extends the current LLM wrapper and introduces a new Chat Model wrapper for AzureML to support LLaMa2.

## Dependencies

No dependencies added :)

## Twitter Handles

[@matthew_d13](https://twitter.com/matthew_d13)
[@prakhar_in](https://twitter.com/prakhar_in)

maintainers - @hwchase17, @baskaryan
RetrievalQA: Submit different queries to Retriever and LLM
Labels: name: area: embeddings, name: auto:enhancement

### Feature request

This is related to [HyDE](https://python.langchain.com/docs/modules/chains/additional/hyde).  But I feel my feature request puts the HyDE example in a more realistic context, and naturally extends the RetrievalQA chain.

My suggestion would be allow the text passed to the retriever to be different from the query passed to the LLM.  This would be useful as fallback when the original query, as provided by the user, yields a response of "I don't know" from the LLM; or if it returns no or poor-scoring documents during the retrieval step.

Step 1, Embed original query
Step 2, retrieve candidate documents based on embedded original query
Step 3, pass original query & retrieved documents to LLM for answer
If Step 2 yields 0/low score documents OR Step 3 yields no/low confidence answer...
Step 4, generate an alternate _hypothetical_ answer (i.e., as described in [HyDE](https://arxiv.org/abs/2212.10496)
Step 5, embed  _hypothetical_ answer
Step 6, retrieve candidate documents based on embedded  _hypothetical_ answer
Step 7, pass original query & _newly_ retrieved documents to LLM for answer

### Motivation

Perhaps I have a query that does not yield good retrieval documents from my main knowledgebase corpus.  It is poorly formed, it has spelling errors, it is vague, it is brief.  Even though the corpus does contain the factual answer to the intent of the query.  Instead, a different form of the original query would yield better retrieved documents, and therefore produce the correct & complete answer desired.

For example, very simple questions like "Who's Character X?" may yield a wide range of sub-optimal documents from a QA Retriever.  As a result, the LLM's response may be muddled or impossible to generate.

### Your contribution

Happy to refine the suggestion and provide concrete examples.
GCP Matching Engine support for public index endpoints
Labels: name: area: vector store, name: auto:improvement

### System Info

langchain==0.0.244

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Create a new Matching Engine Index Endpoint that is public.

Follow the tutorial to make a similarity search:

```
vector_store = MatchingEngine.from_components(
    project_id="",
    region="us-central1",
    gcs_bucket_name="",
    index_id="",
    endpoint_id="",
    embedding=embeddings,
)

vector_store.similarity_search("what is a cat?", k=5)
```

Error:

```

File ~/code/gcp-langchain-retrieval-augmentation/embeddings/.venv/lib/python3.9/site-packages/grpc/_channel.py:1030, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
   1021 def __call__(self,
   1022              request: Any,
   1023              timeout: Optional[float] = None,
   (...)
   1026              wait_for_ready: Optional[bool] = None,
   1027              compression: Optional[grpc.Compression] = None) -> Any:
   1028     state, call, = self._blocking(request, timeout, metadata, credentials,
   1029                                   wait_for_ready, compression)
-> 1030     return _end_unary_response_blocking(state, call, False, None)

File ~/code/gcp-langchain-retrieval-augmentation/embeddings/.venv/lib/python3.9/site-packages/grpc/_channel.py:910, in _end_unary_response_blocking(state, call, with_call, deadline)
    908         return state.response
    909 else:
--> 910     raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "DNS resolution failed for :10000: unparseable host:port"
	debug_error_string = "UNKNOWN:DNS resolution failed for :10000: unparseable host:port {created_time:"2023-07-27T20:12:23.727315699+00:00", grpc_status:14}"
>
```



### Expected behavior

It should be possible to do this. The VertexAI Python SDK supports it with the `endpoint.find_neighbors` function.

I think just changing [the wrapper](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/matching_engine.py#L178) from `.match` to `.find_neighbors` for when the endpoint is public should do it.
Add FunctionMessage to _message_from_dict
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Add support for CustomAttributes for SageMaker endpoint (required by Llama 2)
Labels: name: auto:improvement

Fixes: #8326

Description: Allows passing "CustomAttributes" to the SageMaker endpoint, see [API documentation](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html) for details.

Issue: The Llama 2 model requires the user to accept the EULA by setting the "CustomAttributes" to "accept_eula=true" in the model invocation. "CustomAttributes" are missing in the current SageMaker endpoint implementation.

Change: This PR allows passing "CustomAttributes" by setting the `custom_attributes` parameter in the endpoint instantiation, e.g.:
```
llm=SagemakerEndpoint(
    ...,
    custom_attributes="accept_eula=true"
)
```

Thanks to @sarthak10193 for reporting the issue and suggesting a fix.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->