delayedkarma/langchain-issues huggingface dataset:
CohereAPIError thrown when base retriever returns empty documents in ContextualCompressionRetriever using Cohere Rank

### System Info

- 5.19.0-42-generic # 43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 21 16:51:08 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
- langchain==0.0.180
- Python 3.10.11

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Set up a retriever using any type of retriever (for example, I used Pinecone).
2. Pass it into the ContextualCompressionRetriever.
3. If the base retriever returns empty documents,
4. It throws an error:  **cohere.error.CohereAPIError: invalid request: list of documents must not be empty**

> File "/workspaces/example/.venv/lib/python3.10/site-packages/langchain/retrievers/contextual_compression.py", line 37, in get_relevant_documents
>   compressed_docs = self.base_compressor.compress_documents(docs, query)
> File "/workspaces/example/.venv/lib/python3.10/site-packages/langchain/retrievers/document_compressors/cohere_rerank.py", line 57, in compress_documents
>   results = self.client.rerank(
> File "/workspaces/example/.venv/lib/python3.10/site-packages/cohere/client.py", line 633, in rerank
>   reranking = Reranking(self._request(cohere.RERANK_URL, json=json_body))
> File "/workspaces/example/.venv/lib/python3.10/site-packages/cohere/client.py", line 692, in _request
>   self._check_response(json_response, response.headers, response.status_code)
> File "/workspaces/example/.venv/lib/python3.10/site-packages/cohere/client.py", line 642, in _check_response
>   raise CohereAPIError(
> **cohere.error.CohereAPIError: invalid request: list of documents must not be empty**

Code is Like

```python
 
retriever = vectorstore.as_retriever()
compressor = CohereRerank()
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)
return compression_retriever
```

### Expected behavior

**no error throws** and return empty list
RFC: llm / chat model tabs
Labels: name: documentation

only in quickstart atm but could do in other places as well

https://python.langchain.com/en/dev2049-combine_quickstart/getting_started/getting_started.html#prompt-templates
<img width="927" alt="Screenshot 2023-05-26 at 3 17 59 AM" src="https://github.com/hwchase17/langchain/assets/130488702/a0daf86b-42aa-42f6-a491-0bae607fe85b">
Weaviate: Add support for other vectorizers in hybrid search

### Feature request

We should add support for the following vectorizers in the [weaviate hybrid search](https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/weaviate-hybrid.html):

1. [cohere](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-cohere)
2. [palm](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-cohere)
3. [huggingface](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-cohere)

### Motivation

more flexibility to users

### Your contribution

code review
Get the source document info with result 

### System Info

ValueError: `run` not supported when there is not exactly one output key. Got ['result', 'source_documents'
]


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

qa = RetrievalQA.from_chain_type(
    llm=OpenAI(),
    chain_type = "stuff",
    retriever=db.as_retriever(),
return_source_documents=True
)

agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)#, output_keys=['result','source_documents'])
agent_chain = AgentExecutor.from_agent_and_tools(
  agent=agent, tools=tools, verbose=True, memory=memory)# return_intermediate_steps=True)#, output_keys=['result','source_documents']
 
#)


### Expected behavior

Returns the answer and source doc as well
Issue: security concerns with `exec()` via multiple agents and Shell tool

### Issue you'd like to raise.


TL;DR: The use of exec() in agents can lead to remote code execution vulnerabilities. Some Huggingface projects use such agents, despite the potential harm of LLM-generated Python code.


#1026 and #814 discuss the security concerns regarding the use of `exec()` in llm_math chain.  The comments in #1026 proposed methods to sandbox the code execution, but due to environmental issues, the code was patched to replace `exec()` with `numexpr.evaluate()` (#2943). This restricted the execution capabilities to mathematical functionalities only. This bug was assigned the CVE number CVE-2023-29374. 

As shown in the above issues, the usage of `exec()` in a chain can pose a significant security risk, especially when the chain is running on a remote machine. This seems common scenario for projects in Huggingface. 

However, in the latest langchain, `exec()` is still used in `PythonReplTool` and `PythonAstReplTool`. 
https://github.com/hwchase17/langchain/blob/aec642febb3daa7dbb6a19996aac2efa92bbf1bd/langchain/tools/python/tool.py#L55

https://github.com/hwchase17/langchain/blob/aec642febb3daa7dbb6a19996aac2efa92bbf1bd/langchain/tools/python/tool.py#L102

These functions are called by Pandas Dataframe Agent, Spark Dataframe Agent, CSV Agent. It seems they are intentionally designed to pass the LLM output to `PythonAstTool` or `PythonAstReplTool` to execute the LLM-generated code in the machine.

The documentation for these agents explicitly states that they should be used with caution since LLM-generated Python code can be potentially harmful. For instance:
https://github.com/hwchase17/langchain/blob/aec642febb3daa7dbb6a19996aac2efa92bbf1bd/docs/modules/agents/toolkits/examples/pandas.ipynb#L12

Despite this, I have observed several projects in Huggingface using `create_pandas_dataframe_agent` and `create_csv_agent`.





### Suggestion:


Fixing this issue as done in llm_math chain seems challenging.
Simply restricting the LLM-generated code to Pandas and Spark execution might not be sufficient because there are still numerous malicious tasks that can be performed using those APIs. For instance, Pandas can read and write files.

Meanwhile, it seems crucial to emphasize the security concerns related to LLM-generated code for the overall security of LLM apps. Merely limiting execution to specific frameworks or APIs may not fully address the underlying security risks.
Implemented appending arbitrary messages
Labels: name: lgtm

# Implemented appending arbitrary messages to the base chat message history, the in-memory and cosmos ones.

<!--
Thank you for contributing to LangChain! Your PR will appear in our next release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.
-->

As discussed this is the alternative way instead of #4480, with a add_message method added that takes a BaseMessage as input, so that the user can control what is in the base message like kwargs.

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, include an integration test and an example notebook showing its use! -->

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

@hwchase17
Add an example to make the prompt more robust
Labels: name: lgtm

I am trying the opensource model for llm_math.  and I possibly found a typical problem.  chatgpt knows ** and always change a^b into  a**b(chatgpt doesn't think ^ is xor),  and opensource model prefer a^b.  so if my quesion is   29^(1/5) or the fifth root of 29,  I will get the error "_TypeError: unsupported operand type(s) for ^: 'int' and 'float'_'" from _"numexpr.evaluate("29^(1/5)")"_.  Then  I add this example, the model is able to solve questions like "a^b", "the square of a", "the fifth root of a" by using **
Better docs for weaviate hybrid search
Labels: name: lgtm

# Better docs for weaviate hybrid search

<!--
Thank you for contributing to LangChain! Your PR will appear in our next release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.
-->

<!-- Remove if not applicable -->

Fixes: NA

## Before submitting

<!-- If you're adding a new integration, include an integration test and an example notebook showing its use! -->

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

        @hwchase17 - project lead

        Tracing / Callbacks
        - @agola11

        Async
        - @agola11

        DataLoaders
        - @eyurtsev

        Models
        - @hwchase17
        - @agola11

        Agents / Tools / Toolkits
        - @vowelparrot
        
        VectorStores / Retrievers / Memory
        - @dev2049
        
 -->
@dev2049
support history in LLMChain and LLM

### Feature request

It's very userful when use the history paremeter.

```
history = [("who are you", " i'm a ai")]

llm = OpenAI()
llm("hello", history)

llm = LLMChain({ llm, prompt })
llm({"query": "hello", "history": history})
```

### Motivation

*

### Your contribution

*
SQL chain generates extra add on question if I use ChatOpenAI inplace of OpenAI

### System Info

`import os
from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
from langchain.chat_models import ChatOpenAI
from dotenv import load_dotenv
load_dotenv()
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
db = SQLDatabase.from_uri("sqlite:///data/data.db")
llm = ChatOpenAI(temperature=0, verbose=True)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)
res = db_chain.run("total sales by each region?")
print(res)`

![image](https://github.com/hwchase17/langchain/assets/52596822/fbfa65bc-398b-43f1-a506-0e7a5cd57221)



But If I use text-davinici, It generates single result.

`import os
from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
from langchain.chat_models import ChatOpenAI
from dotenv import load_dotenv
load_dotenv()
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
db = SQLDatabase.from_uri("sqlite:///data/data.db")
llm = OpenAI(temperature=0, verbose=True)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)
res = db_chain.run("total sales by each region?")
print(res)`

![text-davinici](https://github.com/hwchase17/langchain/assets/52596822/1ccd746c-5951-42c1-85f6-1530827c1029)



how to overcome this issue in **"ChatOpenAI"**?

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

import os
from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
from langchain.chat_models import ChatOpenAI
from dotenv import load_dotenv
load_dotenv()
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
db = SQLDatabase.from_uri("sqlite:///data/data.db")
llm = ChatOpenAI(temperature=0, verbose=True)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)
res = db_chain.run("total sales by each region?")
print(res)

### Expected behavior

I need one answer that is for user input query only. But after the answer It again adding a question by itself. Extra add on question and query not needed in **ChatOpenAI**
Stop logic should be optimezed to be compatible with "Conversation 1:"

### System Info

windows

### Who can help?

@hwchase17

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The llm will generate the content below in some cases:
Action 1: xxx
Action Input 1: xxx
Observation 1: xxx

regex = (
            r"Action\s*\d*\s*:[\s]*(.*?)[\s]*Action\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        )
The regex above in langchian.mrkl.output_parser can match the Action and Action Input in the following scenario:
Action 1: xxx
Action Input 1: xxx
but the stop list is still be ['\nObservation:', '\n\tObservation:'] which can not stop the generation by llm, because the llm will generate the 'Observation 1: ... '.

### Expected behavior

Optimize stop logic to solve this problem
docs: `ecosystem/integrations` update 2

# docs: ecosystem/integrations update 2

#5219 - part 1 
The second part of this update (parts are independent of each other! no overlap):

- added diffbot.md
- updated confluence.ipynb; added confluence.md
- updated college_confidential.md
- updated openai.md
- added blackboard.md
- added bilibili.md
- added azure_blob_storage.md
- added azlyrics.md
- added aws_s3.md

## Who can review?

@hwchase17@agola11
@agola11
 @vowelparrot
 @dev2049
Issue Passing in Credential to VertexAI model

### System Info

langchain==0.0.180
google-cloud-aiplatform==1.25.0

Have Google Cloud CLI and ran and logged in using `gcloud auth login`

Running locally and online in Google Colab

### Who can help?

@hwchase17 @hwchase17 @agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

https://colab.research.google.com/drive/19QGMptiCn49fu4i5ZQ0ygfR74ktQFQlb?usp=sharing

Unexpected behavior`field "credentials" not yet prepared so type is still a ForwardRef, you might need to call VertexAI.update_forward_refs().` seems to only appear if you pass in any credenitial valid or invalid to the vertexai wrapper from langchain.

### The error 

This code should not throw `field "credentials" not yet prepared so type is still a ForwardRef, you might need to call VertexAI.update_forward_refs().`. It should either not throw any errors, if the credentials, project_Id, and location are correct. Or, if there is an issue with one of params, it should throw a specific error from the `vertexai.init` call below but it doesn't seem to be reaching it if a credential is passed in.

```
vertexai.init(project=project_id,location=location,credentials=credentials,)
```
VertexAI ChatModel implementation misses few-shot "examples"

### System Info

langchain==0.0.180
python==3.10
google-cloud-aiplatform==1.25.0


### Who can help?

@hwc

### Information

- [] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Debatable if this is a bug or a missing feature, but I'd argue that the Vertex implementation is missing an important element -> Even though I'm excited to have the support now.

Using the [VertexAI documentation for chat](https://cloud.google.com/vertex-ai/docs/generative-ai/chat/test-chat-prompts), you can initialise the chat model like the below (emphasis mine). 
The list of "examples" functions as a separate instruction (few-shot), not as part of the chat history. This is different from how OpenAI does it.

The current langchain implementation doesn't seem to have an option to submit examples, instead it combining all messages in the chat-history. That would lead to unexpected results if you used if for your examples.

```
def chat_question(context=None, examples=[], chat_instruction=None):

    chat_model = ChatModel.from_pretrained("chat-bison@001")
    parameters = {
    "temperature": .0,
    "max_output_tokens": 300,
    "top_p": 0.3,
    "top_k": 3,
    }

    chat = chat_model.start_chat(
        context=context,
        **examples=examples**
    )

    response = chat.send_message(chat_instruction, **parameters)
    return response
```

### Expected behavior

Allow for a set of examples to be passed in when setting up the ChatVertexAI or when using the chat() function. 

Apologies if I've missed a way to do this.
python SDK can't query documents from an existing collection

### Feature request

In the JS SDK of Milvus, there is a function to query documents from an existing collection, while in the Python SDK, this function is not available. Instead, the collection can be constructed using the following way:
```python
vector_db = Milvus.from_documents(
    docs,
    embeddings,
    connection_args={"host": "127.0.0.1", "port": "19530"},
)
```

### Motivation

I cannot ask multiple questions

### Your contribution

no
When initializing ChatVertexAI fastapi thread pool becomes unaccessible 

### System Info

When I initialise ChatVeretexAI in a fastAPI app the thread pool never returns to idle blocking the server returning the below error.

E0526 10:18:51.289447000 4300375424 thread_pool.cc:230]                Waiting for thread pool to idle before forking

on langchain 0.180

### Who can help?

@hwchase17
@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Initialise ChatVertexAI in a fastapi app. ChatOpenAI works fine.

### Expected behavior

Don't error
added more examples for chatbots and question answering
Labels: name: wontfix

# More end-to-end examples with LangChain 🚀 

Hiya, I added more examples for chatbots and question answering 
1. [JarvisBase](https://github.com/peterw/JarvisBase): An end-to-end Customer Support assistant that transcribes user voice, performs Question Answering over a scraped documentation base, & answers in natural language. 
2. [PDF Analysis Slack Chatbot](https://github.com/hollaugo/slack-financial-analysis-chatbot): Build an end-to-end Slack chatbot that chats with multiple PDF files (financial analysis in this case).
3. [Question Answering over multiple PDFs](https://www.activeloop.ai/resources/ultimate-guide-to-lang-chain-deep-lake-build-chat-gpt-to-answer-questions-on-your-financial-data/): An intro guide for building a chat with multiple PDFs solution.

# Who can review?

forgot to tag @hwchase17 - sorry!
ZeroShotAgent fails with ShellTool due to quotes in llm output

https://github.com/hwchase17/langchain/blob/7652d2abb01208fd51115e34e18b066824e7d921/langchain/agents/mrkl/output_parser.py#L47

Due to the line above the `ShellTool` fails when using it with the `ZeroShotAgent`. In using `langchain.OpenAI` as the `llm` I encountered a scenario where ChatGPT provides a string surrounded by single quotes for `Action Input:`. This causes the ShellTool not to recognize the input command because it is surrounded by single quotes which aren't stripped (I get a command not found error). This could easily be fixed by stripping single quotes from `action_input`. 

```
return AgentAction(action, action_input.strip(" ").strip('"').strip("'"), text) 
```
use_query_checker for VertexAI fails

### System Info

langchain==0.0.180
google-cloud-aiplatform==1.25.0
SQLAlchemy==2.0.15
duckdb==0.8.0
duckdb-engine==0.7.3

Running inside GCP Vertex AI Notebook (Jupyter Lab essentially jupyterlab==3.4.8)
python 3.7

### Who can help?

@Jflick58 
@lkuligin 
@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create the Vertex AI LLM (using latest version of LangChain)

`from langchain.llms import VertexAI

palmllm = VertexAI(model_name='text-bison@001',
    max_output_tokens=256,
    temperature=0.2,
    top_p=0.1,
    top_k=40,
    verbose=True)`

2. Setup the db engine for duckdb in this case

`engine = create_engine("duckdb:///dw.db")`


2. Then create the chain using SQLDatabaseChain (Note the use of use_query_checker=True)

`#Setup the DB
db = SQLDatabase(engine=engine,metadata=MetaData(bind=engine),include_tables=[table_name])

#Setup the chain
db_chain = SQLDatabaseChain.from_llm(palmllm,db,verbose=True,use_query_checker=True,prompt=PROMPT,return_intermediate_steps=True,top_k=3)`


4. Run a query against the chain (Notice the SQLQuery: The query is correct) (It is as if its trying to execute "The query is correct" as SQL"

`> Entering new SQLDatabaseChain chain...
How many countries are there
SQLQuery:The query is correct.`

This is the error returned:

`ProgrammingError: (duckdb.ParserException) Parser Error: syntax error at or near "The"
LINE 1: The query is correct.
        ^
[SQL: The query is correct.]
(Background on this error at: https://sqlalche.me/e/14/f405)`

IMPORTANT:
- If I remove the "use_query_checker=True" then everything works well.
- If I use the OpenAI LLM and dont change anything (except the LLM), then it works with the "use_query_checker=True" setting.

This relates to [#5049](https://github.com/hwchase17/langchain/pull/5049)

### Expected behavior

I believe the intention of that flag "use_query_checker=True" is to validate the SQL and allow the chain to recover from a simple syntax error.