delayedkarma/langchain-issues huggingface dataset:
AttributeError: choice while creating python agent

### System Info

langchain version - 0.0.154
ubuntu - 18.04


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.agents.agent_toolkits import create_python_agent

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/practice_ml/lib/python3.9/site-packages/openai/openai_object.py:59, in OpenAIObject.__getattr__(self, k)
     58 try:
---> 59     return self[k]
     60 except KeyError as err:

**KeyError: 'choice'**

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
Cell In[11], line 1
----> 1 from langchain.agents.agent_toolkits import create_python_agent

File ~/practice/langchain.py:23
     16     response = openai.ChatCompletion.create(
     17         model = model,
     18         messages = messages,
     19         temperature=0
     20     )
     21     return response.choice[0].message['content']
---> 23 get_completion("what is 1 + 1")
     24 customer_email = """
     25 Arrr, I be fuming that me blender lid \
     26 flew off and splattered me kitchen walls \
   (...)
     30 right now, matey!
     31 """
     34 from langchain.chat_models import ChatOpenAI

File ~/practice/langchain.py:21, in get_completion(prompt, model)
     12 messages = [
     13     {'role' : 'user',
     14     "content" : prompt}
     15 ]
     16 response = openai.ChatCompletion.create(
     17     model = model,
     18     messages = messages,
     19     temperature=0
     20 )
---> 21 return response.choice[0].message['content']

File ~/practice_ml/lib/python3.9/site-packages/openai/openai_object.py:61, in OpenAIObject.__getattr__(self, k)
     59     return self[k]
     60 except KeyError as err:
---> 61     raise AttributeError(*err.args)

AttributeError: choice

### Expected behavior

no error .
TypeError: TextSplitter.__init__() got an unexpected keyword argument 'add_start_index'

### System Info

version = "0.0.157"


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [x] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [x] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

just run
```
from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=900,
    chunk_overlap=0,
    separators=separators,
    add_start_index = True,
    length_function=tiktoken_len,
)
```

### Expected behavior

no error
Add oobabooga/text-generation-webui support as a llm
Labels: name: enhancement, name: lgtm

Add oobabooga/text-generation-webui support as an LLM.  Currently, supports using text-generation-webui's non-streaming API interface.  Allows users who already have text-gen running to use the same models with langchain.

#### Before submitting

Simple usage, similar to existing LLM supported:

```
from langchain.llms import TextGen
llm = TextGen(model_url = "http://localhost:5000")
```
#### Who can review?

 @hwchase17 - project lead
Human Input tool is not useable in production

### Issue you'd like to raise.

The human input tool only works with agents run from the command line. As far as I can see, it is impossible to use it when the conversation is being displayed in a web UI (or, for that matter, over any other channel).

### Suggestion:

Implement a callback for handling requests for human input and a way to return the input to the tool/agent. This would allow eg requests for input to be sent to an open WebSocket chat and optionally be added to the message history.
DOC: ChatOpenAI parameters to get it to respond like ChatGPT Plus

### Issue with current documentation:

I know this is a pretty general question, but in my testing, the ChatGPT console (with GPT-4) is performing much better for text summarization than `ChatOpenAI(model='gpt-4')` is. 

Are the default settings intended to be as similar as possible, or are there other known arguments, model_kwargs, system prompts, etc. that will adjust the performance to be more in line with the console experience?
Add a new vector store - AwaDB  (#5971)

Added AwaDB vector store, which is a wrapper over the AwaDB, that can be used as a vector storage and has an efficient similarity search. Added integration tests for the vector store
Added jupyter notebook with the example

Delete a unneeded empty file and resolve the
conflict(https://github.com/hwchase17/langchain/pull/5886)

Please check, Thanks!

@dev2049
@hwchase17

---------

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Fixed typo missing "use"
Labels: name: lgtm, name: maintainer-to-merge

<!--
Fixed a simple typo on https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/vectorstore.html where the word "use" was missing.

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Implement nice pythonic @decorators [see image]

### Feature request

I think it would be a nice idea to implement the langchain logic in a minimalistic way using python decorators to quickly prototype stuff.

### Motivation

<img width="462" alt="Screenshot 2023-06-10 at 17 22 29" src="https://github.com/hwchase17/langchain/assets/34897716/a22aa5fd-be7a-4eaa-a479-cbbeeb08a15c">

like here https://github.com/srush/MiniChain

### Your contribution

Currently I do not have the time to do this but I just wanted to pop the idea :)
pydantic validation error when creating a HumanMessagePromptTemplate from a PipelinePromptTemplate

### System Info

>>> langchain.__version__
'0.0.194'

Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction
```
from langchain.prompts.pipeline import PipelinePromptTemplate
from langchain.prompts.prompt import PromptTemplate
from langchain.prompts.chat import HumanMessagePromptTemplate
pipeline_prompt = PipelinePromptTemplate(final_prompt=PromptTemplate.from_template(""), pipeline_prompts=[])
human_message_prompt = HumanMessagePromptTemplate(prompt=pipeline_prompt)
```
### Expected behavior

Code should run without raising any errors.

Instead it gives this error:
```
pydantic.error_wrappers.ValidationError: 1 validation error for HumanMessagePromptTemplate
prompt
  Can't instantiate abstract class StringPromptTemplate with abstract method format (type=type_error)
```

Which comes from the fact that HumanMessagePromptTemplate inherits from BaseStringMessagePromptTemplate and BaseStringMessagePromptTemplate requires a prompt of type StringPromptTemplate:
https://github.com/hwchase17/langchain/blob/f3e7ac0a2c0ad677e91571f59b03b55c5af52db2/langchain/prompts/chat.py#L67

I solved it this way:
```
class ExtendedHumanMessagePromptTemplate(HumanMessagePromptTemplate):
    prompt: BasePromptTemplate
human_message_prompt = ExtendedHumanMessagePromptTemplate(prompt=pipeline_prompt)
```
ConversationalRetrievalChain did not look into chat histroy while making an answer

I am trying to make a chatbot which remember the existing chat and can answer from that as well as from document. Here is what I have tried.
```
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
qa = ConversationalRetrievalChain.from_llm(
                                        ChatOpenAI(model_name='gpt-3.5-turbo'),
                                        retriever=docsearch, 
                                        memory=memory,
                                        verbose=True)


result = qa({"question": "My name is Talha. Ali is my friend. What is CNN?"})
```
My document is based on CNN, so it fetched what is CNN
Then I ask another question. 
```
result = qa({"question": "Who is Ali?"})
```
this is what happened behind the scene
```
> Entering new LLMChain chain...
Prompt after formatting:
Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language.

Chat History:

Human: My name is Talha. Ali is my friend. What is CNN?
Assistant: CNN is a tool for deep learning and machine learning algorithms used in artificial neural networks for image recognition, object detection, and segmentation.
Follow Up Input: Who is Ali?
Standalone question:

> Finished chain.


> Entering new StuffDocumentsChain chain...


> Entering new LLMChain chain...
Prompt after formatting:
Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

_few paragraphs from documents i have ingested in vector database_

Question: Ali is Talha's friend.
Answer:
```

But when i print `result['answer']` It say `"I don't know."`


Here is notebook for reproducibility
https://colab.research.google.com/drive/1UTjdpAAjoZ_ccpdAEr9gpfpqlC3pwtmT?usp=sharing
STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION with gpt-4 doesn't follow through the tasks, while gpt-3.5-turbo does.

### System Info

langchain 0.0.195

### Who can help?

@vowelparrot
@hwchase17

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
SendEmail = StructuredTool.from_function(sendEmail)
    UpdateClientProfile = StructuredTool.from_function(updateClientProfile)
    tools = [SendEmail, UpdateClientProfile]

    #llm=ChatOpenAI(openai_api_key=os.environ['OPENAI_API_KEY_GPT4'], model=pd.steps["code"]["$return_value"]["model"], temperature=0.8)
    llm=ChatOpenAI(openai_api_key=os.environ['OPENAI_API_KEY_GPT4'], model="gpt-4", temperature=0.8)

    prompt = pd.steps["code"]["$return_value"]["prompt"]

    gpt3 = initialize_agent(tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
    res = gpt3.run(prompt)


    gpt4 = initialize_agent(tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
    res = gpt4.run(prompt)
```

Here's the prompt:
<img width="714" alt="prompt" src="https://github.com/hwchase17/langchain/assets/7931903/2d15bbdf-a81d-4742-beae-0664acce53bc">

gpt-3.5-turbo correctly handle the tasks:
<img width="884" alt="gpt3 5" src="https://github.com/hwchase17/langchain/assets/7931903/0599f839-4a2e-44b5-9978-df758ea8e69c">

gpt-4 only do the first one:
<img width="880" alt="gpt4" src="https://github.com/hwchase17/langchain/assets/7931903/1f6017a4-a31c-48b6-b06b-0bf6214cddf5">



### Expected behavior

Both agents should complete two tasks :
1 - Send an email
2 - Update the customer profile

gpt-3.5-turbo correctly completes the tasks, while gpt-4 stops after the first one and outputs what it should do next:
"Now I will update the customer profile..."
ZapierToolkit Automatically Performing Unrequested Tasks

### System Info

The current behavior of the ZapierToolkit in the LangChain tool occasionally results in the execution of tasks that the user did not specifically request. For example, when a user asks the ZapierToolkit to read the most recent email, it sometimes replies to the email even when the user did not explicitly specify this action. This unexpected behavior occurs when the AgentExecutor chain automatically decides to reply to the email, regardless of whether the email requested the reply or not.  It is safe for the ZapierToolkit to adhere to the user's explicit instructions to avoid potential mishaps or undesired outcomes.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


The issue was tested with the GPT-3.5 models (openai chat completion model and text-davinci-003) with temperature=0.7, and Gmail-related Zapier NLA Development actions, such as "Gmail: Reply to Email," "Gmail: Find Email," and "Gmail: Send Email.", enabled.

Code Sample:
```Python
# get from https://platform.openai.com/
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "")

# get from https://nla.zapier.com/demo/provider/debug (under User Information, after logging in): 
os.environ["ZAPIER_NLA_API_KEY"] = os.environ.get("ZAPIER_NLA_API_KEY", "")

llm = OpenAI()
zapier = ZapierNLAWrapper()
toolkit = ZapierToolkit.from_zapier_nla_wrapper(zapier)
agent = initialize_agent(toolkit.get_tools(), llm, 
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True);

try:
    agent.run("Read the most recent email")
except Exception as e:
    print(e)
    exit(1)
```


### Expected behavior

The ZapierToolkit should only perform tasks that are explicitly requested by the user. If the user does not specify that the most recent email should be replied to, the ZapierToolkit should not automatically decide to perform this action.
Add embaas embeddings api endpoints 
Labels: name: lgtm, name: maintainer-to-merge

# Introduces embaas embeddings endpoints

In this PR, we add support for embaas embeddings endpoints to Text Embedding Models (with LLMs, and Doc loaders in different PRs coming). We currently offer the MTEB leaderboard top performers, will continue to add top embedding models and soon add support for customers to deploy thier own models.Additional Documentation + Infomation can be found [here](https://embaas.io).

While developing this integration, I closely followed the patterns established by other langchain integrations. Nonetheless, if there are any aspects that require adjustments or if there's a better way to present a new integration, let me know! :)

#### Who can review?
  - @hwchase17
  - @agola11
Allow use of non-default QDrant vector key

- Context #2594
- This is continuation of #2751 which diverged so much from origin and was better to be re-started.

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes #2594

#### Who can review?
@kacperlukawski
pydantic.error_wrappers.ValidationError for args_schema

### System Info

Latest langchain & openai 0.27.5

https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

 https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html

class CalculatorInput(BaseModel):
    question:str = Field()

tools.append(
    Tool.from_function(
        func=llm_math_chain.run,
        name="Calculator",
        description="useful for when you need to answer questions about math",
        args_schema=CalculatorInput
        # coroutine= ... <- you can specify an async method if desired as well
    )

Got this error msg: 

File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Tool
args_schema extra fields not permitted (type=value_error.extra)


### Expected behavior

Should enter Agent excution
Issue: Is there some detail documents about chroma.from_documents args?

### Issue you'd like to raise.

I was trying to store different vectorstore to chroma with different ids like "vectorstore = Chroma.from_documents(documents, embeddings, ids="test1",)", but it keeps telling Number of embeddings 9 must match number of ids 1
and how i can erase some of vectorstore inside chroma, if i can store many into it?


### Suggestion:

_No response_
Get start index and end index of a chunk in the original doc with recursivecharactersplit.

### Feature request

make recursivecharactersplit return start index and end index of a chunk in the original doc
then user can get  snnipt by  `original_doc[start_index:end_index]`

### Motivation

This is important info of the trunk.

### Your contribution

I can not do it now, but if I have time, I will try to do it.
Add a new vector store - AwaDB 

Added AwaDB vector store, which is a wrapper over the AwaDB, that can be used as a vector storage and has an efficient similarity search.
Added integration tests for the vector store
Added jupyter notebook with the example

Delete a unneeded empty file and resolve the conflict(https://github.com/hwchase17/langchain/pull/5886)

Please check, Thanks!

@dev2049
@hwchase17
Create Loaders for Major Music Streaming Services

Fixes # (issue)
Create loaders for library endpoints of major music streaming services

#### Before submitting

| - | - |
| ----------- | ----------- |
| a test for the integration      | ❌ REST APIs       |
| an example notebook showing its use   | ✅        |

[twitter](https://twitter.com/JamesSJackson)

#### Who can review?

Tag maintainers/contributors who might be interested:
@eyurtsev
`UnstructuredPDFLoader` suddenly can't parse scanned PDFs

### Issue you'd like to raise.

I'm using `Unstructured` for parsing PDFs and have it installed through a docker dev container. It was working a few months ago but after I rebuilt the container for deployment, it suddenly breaks.

Here's my dockerfile:
```
FROM python:3.9-slim-buster

# Update package lists
RUN apt-get update && apt-get install ffmpeg libsm6 libxext6 gcc g++ git build-essential libpoppler-cpp-dev libmagic-dev pkg-config poppler-utils tesseract-ocr libtesseract-dev -y

# Make working directories
RUN  mkdir -p  /app
WORKDIR  /app

# Copy the requirements.txt file to the container
COPY requirements.txt .

# Install dependencies
RUN pip install --upgrade pip

RUN pip install torch torchvision torchaudio

RUN pip install unstructured-inference

RUN pip install -r requirements.txt

RUN pip install 'git+https://github.com/facebookresearch/detectron2.git@e2ce8dc#egg=detectron2'

# Copy the .env file to the container
COPY .env .

# Copy every file in the source folder to the created working directory
COPY  . .

# Expose the port that the application will run on
EXPOSE 8080

# Start the application
CMD ["python3.9", "-m", "uvicorn", "main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "8080", "--workers", "4"]
```

and `requirements.txt`:
```
fastapi
uvicorn
langchain
python-poppler
pytesseract
unstructured[local-inference]
psycopg2-binary
pgvector
openai
tiktoken
python-dotenv
pypdf
```

I've verified that detectron is in my system but it seems like it's not being used by unstructured as it defaults to using `pdfminer`

### Suggestion:

_No response_
feat: :sparkles: Added filtering option to FAISS vectorstore

Inspired by the filtering capability available in ChromaDB, added the same functionality to the FAISS vectorestore as well. Since FAISS does not have an inbuilt method of filtering used the approach suggested in this [thread](https://github.com/facebookresearch/faiss/issues/1079)
Langchain Issue inspiration: https://github.com/hwchase17/langchain/issues/4572

- [x] Added filtering capability to semantic similarly and MMR
- [x] Added test cases for filtering in `tests/integration_tests/vectorstores/test_faiss.py`

#### Who can review?

Tag maintainers/contributors who might be interested:

  VectorStores / Retrievers / Memory
  - @dev2049
  - @hwchase17