delayedkarma/langchain-issues huggingface dataset:
Resolve Buffer Clearing Issue in Summary Generation
Labels: name: area: memory, name: auto:bug

**Description:** Despite this class intending to summarize just enough messages to fit within a set context size, the product of this class always contains *ALL* messages *PLUS* a summary system message at the front, ultimately making token count longer rather than shorter. This commit addresses the issue by fixing the single bug where the buffer is constantly reset to the internal message store even after messages are summarized and removed. 

**Issues:** Couldn't find one, misread the issue I thought was similar with my last pull request.

@efriis (no LLM this time, I may have drank too much of the kool-aid)

Reason for changes in diff order:
- Local member needed to store truncated message list
- Allow buffer to use local chat list if a moving summary exists
- Let the BaseMessage object have 'system' type specifically (required to pass type checking)
- Store truncated list when `prune` runs
Resolve Buffer Clearing Issue in Summary Generation
Labels: name: area: memory, name: auto:bug

- **Description:** This pull request introduces critical updates to the ConversationSummaryBufferMemory class to address a bug where the summary buffer was not being cleared as expected. Due to this issue, the buffer continued to grow without respect to the max_token_limit, leading to potential performance bottlenecks and skyrocketing context token costs.

- **Issue:** #3922 

- **Dependencies:** N/A

Changes:

-    Addition of chat_messages List: A new list called chat_messages has been added to track the messages within the current conversation scope. This ensures a clear boundary between messages pending summarization and those already processed.

-    Buffer Selection Logic: The logic for selecting the buffer contents has been refined. The system now conditionally utilizes chat_messages when moving_summary_buffer is not empty, ensuring that only new, unsummarized messages are accounted for.

-    Summary Message Enhancement: Summary messages are now instantiated with an additional type="system" parameter, providing clarity and aiding in the identification of system-generated summaries.

-    Buffer Management in predict_new_summary Method: The predict_new_summary method has been updated to assign the current buffer to chat_messages post-summary prediction, effectively managing the buffer's size.

-    Clear Method Update: The clear method now resets the chat_messages to an empty state, which ensures that the buffer is properly cleared when needed.
VertexAI chat-bison-32k support
Labels: name: area: embeddings, name: auto:bug, name: auto:improvement, name: area: models

### System Info

google-cloud-aiplatform = "^1.36.4"
langchain = "0.0.336"

python 3.11

```
pydantic.v1.error_wrappers.ValidationError: 1 validation error for VertexAI
__root__
  Unknown model publishers/google/models/chat-bison-32k; {'gs://google-cloud-aiplatform/schema/predict/instance/text_generation_1.0.0.yaml': <class 'vertexai.preview.language_models._PreviewTextGenerationModel'>} (type=value_error)
```

```
(Pdb) model_id
'publishers/google/models/codechat-bison-32k'
(Pdb) _publisher_models._PublisherModel(resource_name=model_id)
<google.cloud.aiplatform._publisher_models._PublisherModel object at 0xffff752ba110>
resource name: publishers/google/models/codechat-bison-32k
```

Public Preview: https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/chat-bison?hl=en
|name|released|status|
| -------------- | ------------ | -------------|
|chat-bison-32k | 2023-08-29 | Public Preview|


### Who can help?

@ey

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
        llm = VertexAI(
            model_name="chat-bison-32k",
            max_output_tokens=8192,
            temperature=0.1,
            top_p=0.8,
            top_k=40,
            verbose=True,
            # streaming=True,
        )
```

### Expected behavior

chat-bison-32k works

There may be some other models released recently that may also need similar update.
little hack to make elasticsearch search OK on indexes WITHOUT any "metadata" field
Labels: name: area: vector store, name: auto:improvement, name: integration: elasticsearch, name: size:S


**Description:**
This is for enabling the whole thing for people (like me) who just **don't have "metadata" field in their index**... these extra things I put them at the same level as content in the doc, so using your tools was asking me to add a useless metadata field... I prefered to share this.
I Tested it, looks ok! The search returns the expected doc without any metadata part

**Issue:**
Blocking the search if docs DONT have the "metadata" field.

**Dependencies:**
no change
Issue: minor naming discrepency of MLflowAIGateway
Labels: name: area: embeddings, name: auto:nit, name: area: models

### Issue you'd like to raise.

The MLflowAIGateway module names should be consistent with `MLflow` (upper case ML and lowercase f):
`from langchain.chat_models import ChatMLflowAIGateway`

For embeddings and completions, current module names are:
```
from langchain.llms import MlflowAIGateway
from langchain.embeddings import MlflowAIGatewayEmbeddings
```
should be:
```
from langchain.llms import MLflowAIGateway
from langchain.embeddings import MLflowAIGatewayEmbeddings
```

### Suggestion:

_No response_
inconsistency parameter requirements for chat_models.ChatMLflowAIGateway
Labels: name: area: embeddings, name: auto:bug, name: area: models

### System Info

databricks Machine Learning Runtime: 13.3
langchain==0.0.319
python == 3.9

### Who can help?

@agola11  @hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. create a route following [this instruction](https://mlflow.org/docs/latest/python_api/mlflow.gateway.html)
```
gateway.create_route(
        name="chat",
        route_type="llm/v1/chat",
        model={
            "name": "llama2-70b-chat",
            "provider": "mosaicml",
            "mosaicml_config": {
                "mosaicml_api_key": <key>
            }
        }
    )
```
 
3. use the template code from the documentation page [here](https://python.langchain.com/docs/integrations/providers/mlflow_ai_gateway#chat-example)

```
from langchain.chat_models import ChatMLflowAIGateway
from langchain.schema import HumanMessage, SystemMessage

chat = ChatMLflowAIGateway(
    gateway_uri="databricks",
    route="chat",
    params={
        "temperature": 0.1
    }
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that translates English to French."
    ),
    HumanMessage(
        content="Translate this sentence from English to French: I love programming."
    ),
]
print(chat(messages))
```

This will complain parameter `max_tokens` is not provided. Similarly, if we update the model 
```
chat = ChatMLflowAIGateway(
    gateway_uri="databricks",
    route="chat",
    params={
        "temperature": 0.1,
         "max_tokens": 200
    }
)
```
it complains about parameter `stop` is not provided.

However, both parameters are supposed to be non-required parameters based on MLflow's doc [here](https://mlflow.org/docs/latest/llms/gateway/index.html#chat)

### Expected behavior

expecting the example code below should be executed successfully
```
from langchain.chat_models import ChatMLflowAIGateway
from langchain.schema import HumanMessage, SystemMessage

chat = ChatMLflowAIGateway(
    gateway_uri="databricks",
    route="chat",
    params={
        "temperature": 0.1
    }
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that translates English to French."
    ),
    HumanMessage(
        content="Translate this sentence from English to French: I love programming."
    ),
]
print(chat(messages))
```
Update azuresearch.py following recent change from azure-search-documents library
Labels: name: lgtm, name: area: vector store, name: auto:bug, name: size:L

  - **Description:** 

Reference library azure-search-documents has been adapted in version 11.4.0:

1. Notebook explaining Azure AI Search updated with most recent info
2. HnswVectorSearchAlgorithmConfiguration --> HnswAlgorithmConfiguration
3. PrioritizedFields(prioritized_content_fields) --> SemanticPrioritizedFields(content_fields)
4. SemanticSettings --> SemanticSearch
5. VectorSearch(algorithm_configurations) --> VectorSearch(configurations)

--> Changes now reflected on Langchain: default vector search config from langchain is now compatible with officially released library from Azure.

  - **Issue:**
Issue creating a new index (due to wrong class used for default vector search configuration) if using latest version of azure-search-documents with current langchain version
  - **Dependencies:** azure-search-documents>=11.4.0,
  - **Tag maintainer:** ,
fix for `integratons/document_loaders` sidebar
Labels: name: auto:documentation

The current `integrations/document_loaders/` sidebar has the `example_data` item, which is a menu with a single item: "Notebook".
It is happening because the `integrations/document_loaders/` folder has the `example_data/notebook.md` file that is used to autogenerate the above menu item.
- removed an example_data/notebook.md file. Docusaurus doesn't have simple ways to fix this problem (to exclude folders/files from an autogenerated sidebar). Removing this file didn't break any existing examples, so this fix is safe.
fixed: Missing escape in llms.utils.enforce_stop_tokens()
Labels: name: auto:bug, name: area: models

**Problem:**
The problem with the enforce_stop_tokens method as currently implemented is that it directly joins the stop words with the | character for the re.split function. This approach works well if the stop words don't contain any regex special characters. However, if they do, this can lead to unintended behavior because the special characters are interpreted as part of the regex pattern, not as literal characters.

**Solution**
To fix this issue, each stop word should be escaped so that any special characters it contains are treated as literals. 


fixed: #13397
Introduce `search_kwargs` for Custom Parameters in BingSearchAPIWrapper
Labels: name: auto:improvement

Added a `search_kwargs` field to BingSearchAPIWrapper in `bing_search.py`, enabling users to include extra keyword arguments in Bing search queries. This update, like specifying language preferences, adds more customization to searches. The `search_kwargs` seamlessly merge with standard parameters in `_bing_search_results` method.
ImportError: cannot import name 'Row' from 'sqlalchemy'
Labels: name: auto:bug, name: area: models

### System Info

Version: langchain 0.0.336
Version: sqlalchemy  2.0.1

  File "/Users/anonymous/code/anon/anon/utils/cache.py", line 40, in set_cache
    from langchain.cache import SQLiteCache
  File "/Users/anonymous/dotfiles/virtualenvs/aiproject/lib/python3.11/site-packages/langchain/cache.py", line 45, in <module>
    from sqlalchemy import Column, Integer, Row, String, create_engine, select
ImportError: cannot import name 'Row' from 'sqlalchemy' (/Users/anonymous/dotfiles/virtualenvs/aiproject/lib/python3.11/site-packages/sqlalchemy/__init__.py)



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.cache import SQLiteCache

### Expected behavior

Doesn't throw exception
ConversationalRetrievalChain
Labels: name: area: memory, name: area: embeddings, name: auto:bug, name: area: models

### System Info

I am facing an issue during the testing of my chatbot, here is below i attached a screenshot which can show you the exact issue.

![image](https://github.com/langchain-ai/langchain/assets/76678681/40ecc2d1-c704-47a3-8cea-f4e6b3323700)


But when i hit the api again with first small letter 'what' instead of 'What' then it return correct response. 
![image](https://github.com/langchain-ai/langchain/assets/76678681/a10c1940-0b63-4de7-a51b-fe5473c7fdc8)


Could someone please help me identify the exact issue? thanks

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [x] LLMs/Chat Models
- [x] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [x] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

{
    "question": "what are the reactions associated with hydromorphone use in animals?"
}






### Expected behavior

![image](https://github.com/langchain-ai/langchain/assets/76678681/a10c1940-0b63-4de7-a51b-fe5473c7fdc8)
Issue: Sitemap Loader not fetching
Labels: name: area: doc loader, name: auto:bug

### Issue you'd like to raise.

Wrote the following code in Google Colab but it is unable to fetch the data. Kindly help.

from langchain.document_loaders.sitemap import SitemapLoader
sitemap_loader = SitemapLoader(web_path="https://langchain.readthedocs.io/sitemap.xml")
docs = sitemap_loader.load()
print(docs)

![image](https://github.com/langchain-ai/langchain/assets/148566663/599ae1dc-51ce-455d-93ba-f22704e5e029)


### Suggestion:

_No response_
Output being rephrased
Labels: name: area: memory, name: auto:question, name: area: models, name: integration: chroma

@dosu-bot

1 more question for my code thats below.

This is my code:
loader = PyPDFLoader(file_name)
documents = loader.load()

llm = ChatOpenAI(temperature = 0, model_name='gpt-3.5-turbo', callbacks=[StreamingStdOutCallbackHandler()], streaming = True)

text_splitter = RecursiveCharacterTextSplitter(
chunk_size=300,
chunk_overlap=50,
)
chunks = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings()
persist_directory = "C:\Users\Asus\OneDrive\Documents\Vendolista"
knowledge_base = Chroma.from_documents(chunks, embeddings, persist_directory = persist_directory)
#save to disk
knowledge_base.persist()
#To delete the DB we created at first so that we can be sure that we will load from disk as fresh db
knowledge_base = None
new_knowledge_base = Chroma(persist_directory = persist_directory, embedding_function = embeddings)

knowledge_base = Chroma.from_documents(chunks, embeddings, persist_directory=persist_directory)
knowledge_base.persist()
prompt_template = """
Text: {context}

Question: {question}
Answer :
"""
PROMPT = PromptTemplate(
template=prompt_template, input_variables=["context", "question"]
)

memory = ConversationBufferMemory(memory_key='chat_history', return_messages=True)

conversation = ConversationalRetrievalChain.from_llm(
llm=llm,
memory = memory,
retriever=knowledge_base.as_retriever(search_type = "similarity", search_kwargs = {"k":2}),
chain_type="stuff",
verbose=False,
combine_docs_chain_kwargs={"prompt":PROMPT}
)
def main():
chat_history = []

while True:
    query = input("Ask me anything about the files (type 'exit' to quit): ")

    if query.lower() in ["exit"] and len(query) == 4:
        end_chat = "Thank you for visiting us! Have a nice day"
        print_letter_by_letter(end_chat)
        break

    if query != "":
        # with get_openai_callback() as cb:
        llm_response = conversation({"question": query})
if name == "main":
main()

Below is an example of my terminal in vs code when I ask my AI model a question.
Ask me anything about the files (type 'exit' to quit): How do I delete a staff account
How can I delete a staff account?To delete a staff account, you need to have administrative privileges. As an admin, you have the ability to delete staff accounts when necessary.
Ask me anything about the files (type 'exit' to quit):
Langchain SQLDatabase is not able to access  table information correctly from private schema of PostgreSQL database
Labels: name: area: embeddings, name: auto:bug, name: auto:question, name: area: models

### System Info


Langchain Version: 0.0.336
OS: Windows

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Using "SQLDatabase.from_uri" I am not able to access table information from private schema of PostgreSQL database. But using the same I am able to access table information from public schema of PostgreSQL DB.  How can I access table information from all the schema's ? Please find the code below. Can someone help me?

db = SQLDatabase.from_uri(f"postgresql+psycopg2://{username}:{password}@{host}:{port}/{database}",sample_rows_in_table_info=5) 

print(db.get_table_names())

### Expected behavior

I expected it give information of tables from all the schemas.
LangChain(OpenAI) not returning JSON format output
Labels: name: auto:bug, name: area: models

### System Info

**System Information:**
- Python: 3.10.13
- Conda: 23.3.1
- Openai: 0.28.1 
- LangChain: 0.0.330
- System: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
- Platform: AWS SageMaker Notebook

**Issue:**
- Few hours ago everything was working fine but now all of a sudden OpenAI isn't generating JSON format results
-  I can't upgrade OpenAI or LangChain because another issue related to ChatCompletion due to OpenAI's recent update comes in which LangChain hasn't resolved
- Screenshot of Prompt and the output is below

**Screenshot:**
![image](https://github.com/langchain-ai/langchain/assets/43797457/9a02d5f8-63fd-4af4-8f55-e65aabe13f8a)


### Who can help?

@hwchase17 @agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

 **Just ask the OpenAI to generate a JSON format output:**
           
prompt = """
            You're analyzing an agent's calls performance with his customers. 
            Please generate an expressive agent analysis report by using the executive summary, metadata provided to you and by 
            comparing the agent emotions against the top agent's emotions. Don't make the report about numbers only.
            Make it look like an expressive and qualitative summary but keep it to ten lines only. Also generate only 
            five training guidelines for the agent in concise bullet points. 
            The output should be in the following JSON format:
            
            {
                "Agent Analysis Report" : "..."
                "Training Guidelines" : "..."
            }
            """

### Expected behavior

**Output should be like this:**

Output = {
"Agent Performance Report": "Agent did not perform well....",
"Training Guidelines": "Agent should work on empathy..."
}
Can't load docx file with directory in content.
Labels: name: area: doc loader, name: auto:bug

### System Info

Error info:

<CT_SectPr '<w:sectPr>' at 0x2c9fa6c50> is not in list

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


1. create a new word ( .docx ) file,  
2. wirte some content inside, then insert a directory in some where and save file.  
3. upload .docx file.
4. click Button 'add file to Knowledge base'
5. check the file added. 
6. you could see "DocumentLoader, spliter is None" and  Document count is 0.

### Expected behavior

load docx file successful.
VertexAI LLM count_tokens method requires list of prompts
Labels: name: auto:bug, name: area: models

I encountered this during summarization with VertexAI. I was receiving an INVALID_ARGUMENT error, as it was trying to send a list of about 17000 single characters. 

The [count_tokens method](https://github.com/googleapis/python-aiplatform/blob/main/vertexai/language_models/_language_models.py#L658) made available by Google takes in a list of prompts. It does not fail for small texts, but it does for longer documents because the argument list will be exceeding Googles allowed limit. Enforcing the list type makes it work successfully.

This change will cast the input text to count to a list of that single text so that the input format is always correct.

[Twitter](https://www.x.com/stijn_tratsaert)
Astra DB: minor improvements to docstrings and demo notebook
Labels: name: area: vector store, name: auto:documentation

This PR brings a few minor improvements to the docs, namely class/method docstrings and the demo notebook.

- A note on how to control concurrency levels to tune performance in bulk inserts, both in the class docstring and the demo notebook;
- Slightly increased concurrency defaults after careful experimentation (still on the conservative side even for clients running on less-than-typical network/hardware specs)
- renamed the DB token variable to the standardized `ASTRA_DB_APPLICATION_TOKEN` name (used elsewhere, e.g. in the Astra DB docs)
- added a note and a reference (add_text docstring, demo notebook) on allowed metadata field names.

Thank you!