delayedkarma/langchain-issues huggingface dataset:
Fix the exception from langchain.utilities import ArceeWrapper
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->


- **Description:** Fix the exception from langchain.utilities import ArceeWrapper
i am trying to optimize the cost of SQLDatabaseToolkit calls 
Labels: name: area: agent, name: auto:question, name: auto:improvement, name: area: models

### System Info

aws

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

here's my PROMPT and code:

from langchain.prompts.chat import ChatPromptTemplate

updated_prompt = ChatPromptTemplate.from_messages(
[
("system",
"""
You are a knowledgeable AI assistant specializing in extracting information from the 'inquiry' table in the MySQL Database.
Your primary task is to perform a single query on the schema of the 'inquiry' table and table and retrieve the data using SQL.

     When formulating SQL queries, keep the following context in mind:

     - Filter records based on exact column value matches.
     - If the user inquires about the Status of the inquiry fetch all these columns: status, name, and time values, and inform the user about these specific values.
     - Limit query results to a maximum of 3 unless the user specifies otherwise.
     - Only query necessary columns.
     - Avoid querying for non-existent columns.
     - Place the 'ORDER BY' clause after 'WHERE.'
     - Do not add a semicolon at the end of the SQL.

     If the query results in an empty set, respond with "information not found"

     Use this format:

     Question: The user's query
     Thought: Your thought process
     Action: SQL Query
     Action Input: SQL query
     Observation: Query results
     ... (repeat for multiple queries)
     Thought: Summarize what you've learned
     Final Answer: Provide the final answer

     Begin!
     """),
    ("user", "{question}\n ai: "),
]
)

llm = ChatOpenAI(model=os.getenv("OPENAI_CHAT_MODEL"), temperature=0) # best result
sql_toolkit = SQLDatabaseToolkit(db=db, llm=llm)
sql_toolkit.get_tools()

sqldb_agent = create_sql_agent(
llm=llm,
toolkit=sql_toolkit,
agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=True,
handle_parsing_errors=True,
)

sqldb_agent.run(updated_prompt.format(
question="What is the status of inquiry 123?"
))

### Expected behavior

This is my current cost

response: The inquiry  123 is  Completed on August 21, 2020, at 12 PM.
Total Tokens: 18566
Prompt Tokens: 18349
Completion Tokens: 217
Total Cost (USD): $0.055915

I want to reduce the cost to less than > $0.01 

any suggestions will help.
Issue with OpenAI tokenizer not functioning as expected in streaming mode
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

I am developing an app utilizing the streaming functionality in Python 3.10. I've crafted my own generator and callback for this purpose. However, I've stumbled upon an issue when setting llm.streaming to true and utilizing a callback - the OpenAI tokenizer appears to malfunction, displaying a token count of 0$ which is not the expected behavior.

### Who can help?

@agola11 
@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

    def arun(self, g, chat_history, query:str, config:dict, agent_id:str):
        try:
            llm = ChatOpenAI(
                temperature=0, 
                verbose=False, 
                model='gpt-4',  
                streaming=True,  
                callbacks=[ChainStreamHandler(g)]
            )
            with get_openai_callback() as cb:
                custom_agent = customAgent.run(g, llm, chat_history, query, config, agent_id)
                print(cb)
                return custom_agent
        finally:
            g.close()
            
 

Tokens Used: 0
        Prompt Tokens: 0
        Completion Tokens: 0
Successful Requests: 0
Total Cost (USD): $0.0

### Expected behavior

I aim to have the tokenizer working correctly to gauge the token usage and its corresponding cost. Any guidance or suggestions to rectify this problem would be greatly appreciated
Unable to filter results in Neo4j Vectorstore.as_retriever() method using filter paramter in search_kwargs
Labels: name: area: vector store, name: auto:bug, name: integration: chroma

### Issue you'd like to raise.

Hello Team,

I am trying to build a QA system with text files as its knowledge base. While creating embeddings I have provided different metadata tags to documents.

I want to use this metadata to filter documents before sending them to LLM to generate answer.

I am using as_retriever() method to filter the documents based on 'key':'value' pair. Code below
```
vectordb.as_retriever(
    search_type="similarity",
    search_kwargs = {
        'filter' : {'key1' : 'value1'}
    }
)
```

This method works great to filter out the documents when I am using ChromaDB as VectorStore, but does not work when I use Neo4j as VectorStore. It returns the same results with or without filter using Neo4j

To figure out the issue, I checked langchain's source code for implementation of ChromaDB and Neo4j Vectorstore.

In ChomaDB the similarity_search() method is implemented as follows - 

https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/chroma.py
![image](https://github.com/langchain-ai/langchain/assets/19748080/8c2e2feb-baa0-45bc-9f19-73aa4c86044d)

Here the filter parameter is passed to the next function - self.similarity_search_with_score(query, k, filter=filter).


But when I checked the same method for Neo4j
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/neo4j_vector.py
![image](https://github.com/langchain-ai/langchain/assets/19748080/834ff3b7-bd20-454d-904e-83a765a1986f)


Here the filter parameter or the **kwargs is not passed to the next function -
self.similarity_search_by_vector(
            embedding=embedding,
            k=k,
            query=query,
     )


Please check if this is what is causing the issue.

Also let me know if something is wrong in my implementation.

Thanks in Advance



### Suggestion:

_No response_
Update README.md
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Fix json key output parser in partial (streaming) mode
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Add `openaichat` to llms class mapping
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

  - **Description:**
The `get_type_to_cls_dict` method provides type to class mapping. This PR fixes the missing `openaichat` type for importer for `OpenAIChat`.
Issue: Can't get verbose output in a variable in python
Labels: name: auto:bug, name: area: models

### Need Route Name : Coming in console but not in output

I am using Router chain to route my chain with MultiPromptChain. It gets routed and shows in console to which route it went but it doesn't come in output variable. It just shows in console. I want that in a variable. I used every available method  like return_intermediate_steps=True and return_route_name=True, But it is not working
Below is the code:

`destination_chains = {}
    for p_info in prompt_infos:
        name = p_info["name"]
        prompt_template = p_info["prompt_template"]
        prompt = ChatPromptTemplate.from_template(template=prompt_template)
        chain = LLMChain(llm=llm, prompt=prompt)
        destination_chains[name] = chain  
        
    destinations = [f"{p['name']}: {p['description']}" for p in prompt_infos]
    destinations_str = "\n".join(destinations)
    
    default_prompt = ChatPromptTemplate.from_template("{input}")
    default_chain = LLMChain(llm=llm, prompt=default_prompt)
    
    MULTI_PROMPT_ROUTER_TEMPLATE = """Given a text input to a \
    language model select the model prompt best suited for the input. \
    You will be given the names of the available prompts and a \
    description of what the prompt is best suited for. \
    
    << FORMATTING >>
    Return a markdown code snippet with a JSON object formatted to look like:
    ```json
    {{{{
        "destination": string \ name of the prompt to use or "DEFAULT"
        "next_inputs": string \ a potentially modified version of the original input
    }}}}
    ```
    REMEMBER: "destination" MUST be one of the candidate prompt \
    names specified below.
    REMEMBER: "next_inputs" can just be the original input \
    if you don't think any modifications are needed.
    
    << CANDIDATE PROMPTS >>
    {destinations}
    
    << INPUT >>
    {{input}}
    
    Output should be only in below format:
         Destination used: Output of it
    """
    
    router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(
        destinations=destinations_str
    )
    router_prompt = PromptTemplate(
        template=router_template,
        input_variables=["input"],
        output_parser=RouterOutputParser(),
    )
        
    router_chain = LLMRouterChain.from_llm(llm, router_prompt)
    
        
    chain = MultiPromptChain(router_chain=router_chain, 
                              destination_chains=destination_chains, 
                               default_chain=default_chain, verbose=True
                            )
   
    inputQues="How to turn on TV"
    output = chain.run(input=inputQues)`

### Suggestion:

_No response_
return_source documents does not work
Labels: name: auto:bug, name: area: models

### System Info

Mac os sonoma

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just use any chain that supports return_source_documents, try setting it to true. 

### Expected behavior

I will say too many output keys and throw the error, please fix this otherwise metadata for items does not make sense, since you cannot return source docs anyway.
add allowed_operators property in QdrantTranslator
Labels: name: lgtm, name: auto:bug

  - **Description:** 
This PR adds `allowd_operators` property to `QdrantTranslator` to fix the `TypeError: can only join an iterable` bug. This property is required in `get_query_constructor_prompt` in `query_constructor\base.py`:
```
allowed_operators=" | ".join(allowed_operators),
```
  - **Issue:** 
#12061
Feature/implement multi query rerank fusion retrieval
Labels: name: auto:improvement

Hi, I added the `MultiQueryRankFusionRetriever`, which inherits from the `MultiQueryRetriever` but uses a different merge strategy, i.e. rerank fusion instead of unique union. For the rerank algorithm itself I reused the implementation in EnsembleRetriever (I moved it in a separate file to reuse it). 
To save some code duplication, I adapted the return types of the `retrieve_documents` function to return a `List[List[Document]]`, so that the rerank fusion works out of the box.

Additional changes:
* Optionally append original query to query variations
* Fix docstrings
* Use kwargs in classmethod to allow some freedom in the class creation

Thanks for your reviews @hwchase17, @baskaryan
Issue: Similarity_search on Vector Store does not work.
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: auto:question, name: integration: chroma

### Issue you'd like to raise.

I am using Langchain + Chroma to do a Q&A program with a csv document as its knowledge base.

The CSV file looks like below:
[![enter image description here][1]][1]

Here is the CSV file:
https://1drv.ms/u/s!Asflam6BEzhjgbkdegCGfZ7FI4O1Og?e=2X6ior

And code for creating embedding:

    from langchain.document_loaders.csv_loader import CSVLoader
    from langchain.embeddings.openai import OpenAIEmbeddings
    from langchain.vectorstores.chroma import Chroma
    from langchain.text_splitter import RecursiveCharacterTextSplitter as RCTS
    
    file_path = "Test.csv"
    doc_pages = []
    
    csv_loader = CSVLoader(file_path)
    doc_pages = csv_loader.load()
    print(f"Extracted {file_path} with {len(doc_pages)} pages...")
    
    splitter = RCTS(chunk_size = 3000, chunk_overlap = 300)
    splitted_docs = splitter.split_documents(doc_pages)
    
    embedding = OpenAIEmbeddings()
    persist_directory = "docs_t/chroma/"
    
    vectordb = Chroma.from_documents(
        documents=splitted_docs,
        embedding=embedding,
        persist_directory=persist_directory
    )
    
    vectordb.persist()
    
    print(vectordb._collection.count())

Here is the Testing code:

    result = vectordb.similarity_search("what is the Support Item Name for 01_003_0107_1_1", k=3)
    for r in result:
        print(r.page_content, end="\n\n")


And I see this testing code returns all other non-relevant information.

Which part leads to this issue?

  [1]: https://i.stack.imgur.com/Ev3cn.png

### Suggestion:

_No response_
Qianfan/arguments pass through
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Add `init_kwargs` to qianfan endpoints' parameter list, 
  - **Issue:** None,
  - **Dependencies:** None,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
 -->
GPT-4 reads the images
Labels: name: area: doc loader, name: auto:question, name: auto:enhancement, name: area: models

### System Info

GPT-4 can now read images. How to combine images and text in PDF or WORD when calling the API?

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

document loader?

### Expected behavior

maybe we can have a document loader that loads text and images at the same time.
How do I define the stop logic for a custom llm in agent
Labels: name: area: agent, name: auto:question, name: area: models

I defined a custom LLM and a custom search, and used the agent for testing, but I found some problems in parsing the llm response, which was prone to result parsing errors after multiple rounds of dialogue. I suspect that there is something wrong with the stop logic of the custom llm, please help me analyze it.

```
    if __name__ == '__main__':
    gpt35_llm = gpt35_llm()
    tools = [GoogleSearchTool(), BaiduSearchTool()]
    agent = initialize_agent(tools, gpt35_llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, handle_parsing_errors=True,verbose=True)
    print(agent.run('杭州亚运会的奖牌榜前三名是哪些国家？杭州亚运会的第一枚金牌是谁获得的？'))
```

```
   class gpt35_llm(LLM):
    model: str = "gpt-3.5"

    @property
    def _llm_type(self) -> str:
        return "gpt-3.5"

    def _call(
            self,
            prompt: str,
            stop: Optional[List[str]] = None,
            run_manager: Optional[CallbackManagerForLLMRun] = None,
            **kwargs: Any,
    ) -> str:
        response = gpt_35_chat(prompt, None)
        if stop is not None:
            print(f'stop:{stop}')
            content = response['content']
            output = content[:content.find(stop[0])]
            return output
        if response['code'] != 200:
            raise ValueError(response['content'])
        return response['content']

    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        """Get the identifying parameters."""
        return {"model": self.model}```
Fix Typo in CONTRIBUTING.md
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->


I have corrected the typos, grammar, and formatting issues.
Conversation Buffer Memory clear histroy
Labels: name: area: memory, name: auto:question

### System Info

langchain version: 0.0320
python version: 3.10

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I want to clear history of ConversationBufferMemory. I used ConversationBufferMemory to maintain chat history but right now I don't  know how can I delete history of that . If anyone knows these please help me out of this.



### Expected behavior

ConversationBufferMemory history should be clear.
Fix SupbaseVectoreStore write operation timeout
Labels: name: area: vector store, name: auto:improvement

**Description**
This small change will make chunk_size a configurable parameter for loading documents into a Supabase database.

**Issue**
https://github.com/langchain-ai/langchain/issues/11422

**Dependencies**
No chanages

**Twitter**
@ j1philli

**Reminder**
If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.