delayedkarma/langchain-issues huggingface dataset:
ConversationBufferWindowMemory returns empty string on empty history instead of empty array.
Labels: name: auto:bug

### System Info

Python 3.10
Langchain 0.0.226

Windows 11 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction
```
import os, langchain

os.environ['SERPAPI_API_KEY'] = ""
os.environ['OPENAI_API_KEY'] = ""

from langchain.chains import LLMChain
from langchain.agents import ConversationalChatAgent, SelfAskWithSearchChain, AgentExecutor
from langchain.memory import ConversationBufferWindowMemory
from langchain.tools import Tool
from langchain.llms import OpenAI

conversation_buffer_window_memory: ConversationBufferWindowMemory = ConversationBufferWindowMemory(
    input_key="input", memory_key="chat_history")

search = langchain.SerpAPIChain()
self_ask_and_search = SelfAskWithSearchChain(llm=OpenAI(), search_chain=search, verbose=True)

tools = [
    Tool(
        name="Search",
        func=self_ask_and_search.run,
        description="useful for when you need to answer questions about current events",
    )
]

prompt = ConversationalChatAgent.create_prompt(
    tools, input_variables=["input", "chat_history", "agent_scratchpad"]
)

llm_chain = LLMChain(
    llm=OpenAI(temperature=0), prompt=prompt)

agent = ConversationalChatAgent(
    llm_chain=llm_chain, tools=tools, verbose=True)

agent_executor = AgentExecutor.from_agent_and_tools(
    agent=agent, tools=tools, verbose=True, memory=conversation_buffer_window_memory)
  
agent_executor.run('what is the capital of texas?')
```

returns the following error:
(<class 'ValueError'>, ValueError('variable chat_history should be a list of base messages, got '), <traceback object at 0x0000027BA6366E40>)

### Expected behavior

Return the LLM result while updating the memory mechanism.
Upgrade the AwaDB from 0.3.5 to 0.3.6
Labels: name: auto:improvement

Upgrade the AwaDB from 0.3.5 to 0.3.6
1. Support get, delete and update batch documents
2. Add the implementation of the max_marginal_relevance_search
3. Change the default vectors' distance computation type from L2 to InnerProduct 
Please review @rlancemartin, @eyurtsev @hwchase17.  Thanks!
Added support for stability API for bedrock
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!
This pull request is to add support for stability AI models for bedrock. 

Replace this comment with:
  - Description: Added support for bedrock stability AI models
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: @3coins 
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
update extension name
Labels: name: auto:improvement

hi @rlancemartin ,

We had a new deployment and the `pg_extension` creation command was updated from `CREATE EXTENSION pg_embedding` to `CREATE EXTENSION embedding`.

https://github.com/neondatabase/neon/pull/4646

The extension not made public yet. No users will be affected by this. Will be public next week.

Please let me know if you have any questions.

Thank you in advance 🙏
Update Pinecone Upsert method usage
Labels: name: lgtm, name: auto:improvement

Description: Refactor the upsert method in the Pinecone class to allow for additional keyword arguments. This change adds flexibility and extensibility to the method, allowing for future modifications or enhancements. The upsert method now accepts the `**kwargs` parameter, which can be used to pass any additional arguments to the Pinecone index. This change has been made in both the `upsert` method in the `Pinecone` class and the `upsert` method in the `similarity_search_with_score` class method. Falls in line with the usage of the upsert method in [Pinecone-Python-Client](https://github.com/pinecone-io/pinecone-python-client/blob/4640c4cf27b1ab935e76929c46d230dea9fe5506/pinecone/index.py#L73)
Issue: [This feature request in Pinecone Repo](https://github.com/pinecone-io/pinecone-python-client/issues/184)

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - Memory: @hwchase17
HuggingFaceEndpoint Class Bug
Labels: name: auto:bug

### System Info

the provided class
[langchain/langchain/llms/huggingface_endpoint.py](https://github.com/hwchase17/langchain/blob/370becdfc2dea35eab6b56244872001116d24f0b/langchain/llms/huggingface_endpoint.py)
 class HuggingFaceEndpoint(LLM): 

has a bug.
it should be
```python
if self.task == "text-generation":
# Text generation return includes the starter text.
text = generated_text[0]["generated_text"]
```
not
```python
text = generated_text[0]["generated_text"][len(prompt) :]
```

the current class will likely just return a 0.

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
    llm= HuggingFaceEndpoint(endpoint_url=os.getenv('ENDPOINT_URL'),task="text-generation",
                              model_kwargs={"temperature":0.7, "max_length":512})
 conversation_chain = ConversationalRetrievalChain.from_llm(
        llm=llm,
        retriever=vectorstore.as_retriever(),
        memory=memory
    )

```

### Expected behavior

the output is 0
Update README.md
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
update MosaicML inputs and outputs
Labels: name: lgtm, name: auto:improvement

As of today (July 7, 2023), the [MosaicML API](https://docs.mosaicml.com/en/latest/inference.html#text-completion-requests) uses `"inputs"` for the prompt

![image](https://github.com/hwchase17/langchain/assets/6849766/58b1fb1b-a11d-4f28-8b7a-03b36b9df2d8)

but we have

https://github.com/hwchase17/langchain/blob/0ed2da70200ada39f5319f604b4c5248bc98e81b/langchain/llms/mosaicml.py#L125

Furthermore, the output should be inside the `"outputs"` key

![image](https://github.com/hwchase17/langchain/assets/6849766/d60190d4-0e01-4ea1-9ff4-a7f1bb105aa4)

but we have

https://github.com/hwchase17/langchain/blob/0ed2da70200ada39f5319f604b4c5248bc98e81b/langchain/llms/mosaicml.py#L163-L170

This PR adds support for this new format.

`<rant>`
It is beyond me, why they apparently have versioned their API (endpoints look like `https://models.hosted-on.mosaicml.hosting/mpt-7b-instruct/v1/predict`; note the `v1` in there), but just keep changing the API without upping the version number. There is a good reason for
https://github.com/hwchase17/langchain/blob/0ed2da70200ada39f5319f604b4c5248bc98e81b/langchain/llms/mosaicml.py#L160-L161
`</rant>`

CC @hwchase17, @baskaryan
map_reduce and refine not working with RetrievalQA chain 
Labels: name: auto:bug

### System Info

Langchain Version  - 0.0.201
Platform - Windows 11
Python - 3.10.11


### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. load some text documents to a vector store, i used deeplake
2. load the db
3. call the function, summarizer(db,"Summarize the mentions of google according to their AI program")(defined in attached file)
4. run for chain_type as stuff, it will work, for map_reduce it will fail in retrieval QA Bot
[main.zip](https://github.com/hwchase17/langchain/files/11982265/main.zip)


### Expected behavior

it should work for all the chain types and give results
Issue: When implementing Cypher Search using  neo4j environment
Labels: name: auto:bug

### Issue you'd like to raise.

python environment：
neo4j 5.0.0
neobolt 1.7.17
neo4j version: neo4j-35-apoc:v20220808

报错：
File "D:\Anaconda3\envs\langchain\lib\site-packages\neo4j_sync\io_bolt3.py", line 200, in run
raise ConfigurationError(
neo4j.exceptions.ConfigurationError: Database name parameter for selecting database is not supported in Bolt Protocol Version(3, 0). Database name 'neo4

### Suggestion:

_No response_
Load OPEN API yaml file to vertex ai LLM model
Labels: name: auto:enhancement

### Feature request

I am looking for loading OPEN API yaml file into vertex ai llm model. Langchain provides that for OPENAI but not for vertex ai. This is how I currently write code in Open AI. I want a similar functionality for Vertex AI

spec = OpenAPISpec.from_file("sample.yaml")             
openai_fns, call_api_fn = openapi_spec_to_openai_fn(spec) 

### Motivation

I am creating on a GENAI chatbot for my company where customers can ask questions specific to our product. I need to return the answers of those queries using our internal API's. To query those API's, I need to know which API to call and the API parameters filled as per the user query.  For that, I need vertex ai function calling support to query it.

Is there already a way in vertex ai which does this? Kindly help me on this.

### Your contribution

I can help on the issue if anything is required.
Add context to an output parsing error on Pydantic schema to improve exception handling
Labels: name: lgtm, name: auto:improvement

## Changes

- [X] Fill the `llm_output` param when there is an output parsing error in a Pydantic schema so that we can get the original text that failed to parse when handling the exception

## Background

With this change, we could do something like this:

```
output_parser = PydanticOutputParser(pydantic_object=pydantic_obj)
chain = ConversationChain(..., output_parser=output_parser)
try:
    response: PydanticSchema = chain.predict(input=input)
except OutputParserException as exc:
    logger.error(
        'OutputParserException while parsing chatbot response: %s', exc.llm_output,
    )
```

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Issue: openai.error.RateLimitError: Rate limit reached for default-text-embedding-ada-002 in organization
Labels: name: auto:question

### Issue you'd like to raise.

I want to download a large file, 5,000,000 characters, and I get an error: openai.error.RateLimitError: Rate limit reached for default-text-embedding-ada-002 in organization org-DOqIVFPozlLEOcvlTbpvpcKt on tokens per min. Limit: 150000 / min. Current: 0 / min. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.

My code:
`class Agent:
    def __init__(self, openai_api_key: str | None = None) -> None:

        self.key = openai_api_key
        self.embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key, request_timeout=120, max_retries=10)
        self.text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)

        self.llm = ChatOpenAI(temperature=0, openai_api_key=openai_api_key, max_tokens=500, model_name="gpt-3.5-turbo-16k")

        self.chat_history = None
        self.chain = None
        self.db = None

    def ask(self, question: str) -> str:
        response = self.chain({"question": question, "chat_history": self.chat_history})
        response = response["answer"].strip()
        self.chat_history.append((question, response))
        return response

    def ingest(self, file_path) -> None:
        loader = TextLoader(file_path, encoding="utf-8")
        documents = loader.load()
        splitted_documents = self.text_splitter.split_documents(documents)

        if self.db is None:
            self.db = FAISS.from_documents(splitted_documents, self.embeddings)
            self.chain = ConversationalRetrievalChain.from_llm(self.llm, self.db.as_retriever())
            self.chat_history = []
        else:
            self.db.add_documents(splitted_documents)

    def forget(self) -> None:
        self.db = None
        self.chain = None
        self.chat_history = None`
        
Is there a solution to this problem?

### Suggestion:

_No response_
Create a built in translator for SelfQueryRetriever for Elasticsearch
Labels: name: auto:enhancement

### Feature request

Inferring query filters from natural language is powerful. The SelfQuery retriever is a great implementation but is not yet compatible with Elasticseach.

### Motivation

Choosing Elasticsearch as a vector store is interesting in terms of hybrid search.
It also makes sense when you have an established infrastructure and technical expertise.

 

### Your contribution

cc: @jeffvestal, @derickson
langchain + gpt4all 
Labels: name: auto:bug

### System Info


LangChain v0.0.225, Ubuntu 22.04.2 LTS, Python 3.10



### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I'm executing  [https://github.com/hwchase17/langchain/blob/master/docs/extras/modules/model_io/models/llms/integrations/gpt4all.ipynb](url) code on my local machine, IDE is VSCode
Getting this error - **AttributeError: 'Model' object has no attribute '_ctx'**
![Screenshot from 2023-07-07 14-36-37](https://github.com/hwchase17/langchain/assets/69832170/9b37fd68-14ba-4fee-ae0b-e56f5a464cd5)

### Expected behavior

![Screenshot from 2023-07-07 16-12-58](https://github.com/hwchase17/langchain/assets/69832170/2a1ed0a6-e24e-4f6d-9dcd-1a8bba89c289)
Issue: Whisper terminates YouTube transcriptions.
Labels: name: auto:bug

### Issue you'd like to raise.

Hello everyone,

I've tried numerous approaches, but every time I attempt to transcribe a video from Google, the Whisper task gets terminated.

Does anyone have any suggestions?

```
def youtube_transcript(query: str) -> str:
    # Use the URL of the YouTube video you want to download
    youtube_url = [get_input_writing(query)]
# -------------
    # Directory to save audio files
    save_dir = "/home/ds_logos_2/transcripts"

    # Transcribe the videos to text
    loader = GenericLoader(YoutubeAudioLoader(youtube_url, save_dir), OpenAIWhisperParser())
    docs = loader.load()    
    # Combine doc
    combined_docs = [doc.page_content for doc in docs]
    text = " ".join(combined_docs)

    # Save the transcription to a text file
    with open('/home/davesoma/transcripts/transcript.txt', 'w') as f:
        f.write(text)
```
```
[youtube] -hxeDjAxvJ8: Downloading webpage
[youtube] -hxeDjAxvJ8: Downloading ios player API JSON
[youtube] -hxeDjAxvJ8: Downloading android player API JSON
[youtube] -hxeDjAxvJ8: Downloading m3u8 information
[info] -hxeDjAxvJ8: Downloading 1 format(s): 140
[download] Destination: /home/davesoma/ds_logos_2/transcripts/Marc Andreessen： Future of the Internet, Technology, and AI ｜ Lex Fridman Podcast #386.m4a
[download] 100% of  177.41MiB in 00:00:18 at 9.52MiB/s
[FixupM4a] Correcting container of "/home/davesoma/ds_logos_2/transcripts/Marc Andreessen： Future of the Internet, Technology, and AI ｜ Lex Fridman Podcast #386.m4a"
[ExtractAudio] Not converting audio /home/davesoma/ds_logos_2/transcripts/Marc Andreessen： Future of the Internet, Technology, and AI ｜ Lex Fridman Podcast #386.m4a; fil
e is already in target format m4a                                                                                                                                          Transcribing part 1!
Killed
```
Add feature to get document ID from redis after redis search document retrieval.
Labels: name: auto:improvement

### Feature request

I have been trying to delete custom redis data using langchain. But the search feature only returns the document and the metadata with similarity score. Hence I modified the code to return the document ID along with the document data. The id will be helpful for manual deletion of certain elements. 

### Motivation

The retrieval of document ID from the redis semantic search will help me to systematically modify the wrong data with right ones after filtering it using LLMs.

### Your contribution

Yes I have fixed the redis part to retrieve the document ID from semantic search. I will fix the issue if the pull request is allowed.
Working of ConversationBufferMemory in the context of document based question answering
Labels: name: auto:question

### Issue you'd like to raise.

I am doing the pdf question answering using the below code.
Note: I am integrated the ConversationBufferMemory for keeping my chat in the memory
```
from langchain.document_loaders import PyPDFLoader
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.memory import ConversationBufferMemory
from langchain.vectorstores import Chroma
from langchain.chains import ConversationalRetrievalChain
from langchain.llms import Cohere
import os
os.environ["COHERE_API_KEY"] = "cohere key"
model = Cohere(model="command-xlarge")
embeddings = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')

pdf_path = "file.pdf"

loader = PyPDFLoader(pdf_path)
pages = loader.load_and_split()

vectordb = Chroma.from_documents(pages, embedding=embeddings,
                                 persist_directory=".")
vectordb.persist()
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
pdf_qa = ConversationalRetrievalChain.from_llm(model, vectordb.as_retriever(), memory=memory)
while True:
    query = input("Enter the question\n")
    result = pdf_qa({"question": query})
    print("Answer:")
    print(result["answer"])
```

Actually what happening here?
I am observing that memory is not keeping updated here, because if I am asking any question related to the previous context, it is unable to answer. 

Is my method is correct?




### Suggestion:

_No response_
Issue: question about agents
Labels: name: auto:question

I want pass "chat_history" to the agents like :
```python
...
agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)
response = agent.run({"input": {"chat_history": _message_to_tuple(history), "question": query}})

```
but got an error. how can i pass 'chat_history' to the agent?

### Suggestion:

_No response_