delayedkarma/langchain-issues huggingface dataset:
feat: 1. Add system parameters, 2. Align with the QianfanChatEndpoint for function calling
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: size:M

 - **Description:** 
     1. Add system parameters to the ERNIE LLM API to set the role of the LLM. 
     2. Add support for the ERNIE-Bot-turbo-AI model according from the document https://cloud.baidu.com/doc/WENXINWORKSHOP/s/Alp0kdm0n. 
     3. For the function call of ErnieBotChat, align with the QianfanChatEndpoint.

With this PR, the `QianfanChatEndpoint()` can use the `function calling` ability with `create_ernie_fn_chain()`. The example is as the following: 

```
from langchain.prompts import ChatPromptTemplate
import json
from langchain.prompts.chat import (
    ChatPromptTemplate,
)

from langchain.chat_models import QianfanChatEndpoint
from langchain.chains.ernie_functions import (
    create_ernie_fn_chain,
)

def get_current_news(location: str) -> str:
    """Get the current news based on the location.'

    Args:
        location (str): The location to query.
    
    Returs:
        str: Current news based on the location.
    """

    news_info = {
        "location": location,
        "news": [
            "I have a Book.",
            "It's a nice day, today."
        ]
    }

    return json.dumps(news_info)

def get_current_weather(location: str, unit: str="celsius") -> str:
    """Get the current weather in a given location

    Args:
        location (str): location of the weather.
        unit (str): unit of the tempuature.
    
    Returns:
        str: weather in the given location.
    """

    weather_info = {
        "location": location,
        "temperature": "27",
        "unit": unit,
        "forecast": ["sunny", "windy"],
    }
    return json.dumps(weather_info)

template = ChatPromptTemplate.from_messages([
    ("user", "{user_input}"),
])

chat = QianfanChatEndpoint(model="ERNIE-Bot-4")
chain = create_ernie_fn_chain([get_current_weather, get_current_news], chat, template, verbose=True)
res = chain.run("北京今天的新闻是什么？")
print(res)
```

The result of the above code:
```
> Entering new LLMChain chain...
Prompt after formatting:
Human: 北京今天的新闻是什么？
> Finished chain.
{'name': 'get_current_news', 'arguments': {'location': '北京'}}
```

For the `ErnieBotChat`, now can use the `system` parameter to set the role of the LLM.

```
from langchain.prompts import ChatPromptTemplate
from langchain.chains import LLMChain
from langchain.chat_models import ErnieBotChat

llm = ErnieBotChat(model_name="ERNIE-Bot-turbo-AI", system="你是一个能力很强的机器人，你的名字叫 小叮当。无论问你什么问题，你都可以给出答案。")
prompt = ChatPromptTemplate.from_messages(
    [
        ("human", "{query}"),
    ]
)
chain = LLMChain(llm=llm, prompt=prompt, verbose=True)
res = chain.run(query="你是谁？")
print(res)
```

The result of the above code:

```
> Entering new LLMChain chain...
Prompt after formatting:
Human: 你是谁？
> Finished chain.
我是小叮当，一个智能机器人。我可以为你提供各种服务，包括回答问题、提供信息、进行计算等。如果你需要任何帮助，请随时告诉我，我会尽力为你提供最好的服务。
```
Agent not calling tool with the right data.
Labels: name: area: agent, name: auto:question

### System Info

langchain 0.0.330

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
class SummarizationTool(BaseTool):
    name = "summarization_tool"
    description = '''This tool must be used at the very end.
        It is used to summarize the results from each of other tools.
        It needs the entire text of the results from each of the previous tool.'''
    llm: BaseLanguageModel
    return_direct = True
 
    def _run(self, query: str, run_manager: Optional[CallbackManagerForToolRun] = None) -> str:
        print ('\n in summarization tool. query:', query, 'query type:', type(query), query[0], query[1])

```

I can see that this tool is called at the very end, as expected.
The output from the query is:
in summarization tool. query: [text from openai_search tool, text from wikipedia_search tool] query type: <class 'str'> [ t

Instead of " [text from openai_search tool, text from wikipedia_search tool]", I want the actual text.
How do I get it to pass the actual text?

### Expected behavior

Instead of " [text from openai_search tool, text from wikipedia_search tool]", I want the actual text.
How do I get it to pass the actual text?
docs: Mendable AI Floating Button
Labels: name: auto:documentation, name: size:M

Hey yall,

As discussed, here is the Mendable AI chat triggered by a small floating button on the right of the screen. That way users can continue to get the value out of the assistant while you experiment. The implementation is very similar to what we have done with Snapchat, Llamaindex, AutoGPT.

Let me know if you have any questions and feedback, happy to iterate! If this looks good, we will PR in the langchain js docs as well.

Thank you!
DOC:  UnstructuredExcelLoader code comment‘s error
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

In  UnstructuredExcelLoader code comment,  introduce 'elements' mode for twice, but nothing about 'single' mode.
Original code comment are following:
  Unstructured loaders, UnstructuredExcelLoader can be used in both "single" and "elements" mode. **_If you use the loader in "elements" mode_**,  each sheet in the Excel file will be a an Unstructured Table element. **_If you use the loader in "elements" mode_**, an HTML representation of the table will be available in the "text_as_html" key in the document metadata.

### Idea or request for content:

_No response_
Handling inline references to .ipynb files
Labels: name: area: doc loader, name: auto:documentation, name: size:M

### Description

I found that [the OpenAI Adapter document](https://python.langchain.com/docs/integrations/adapters/openai) cannot correctly jump to other internal links. Clicking on the link will download it as an ipynb file.

If it works normally on both jupyter notebook and doc websites, additional processing needs to be done, that is, replacing the link before rendering.

I added 2 scripts and tested `make docs_build` and `./vercel_build.sh`. In both cases, the link can be generated correctly without affecting the original ipynb file content.

@efriis Please review this.
ImportError: cannot import name 'BaseCache' from 'langchain'
Labels: name: auto:bug, name: area: models

### System Info

Every time I use Langchain, something is wrong with it.  This is just the latest iteration.  If you guys want people to use your library you seriously need to clean things up.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

To reproduce:

from llama_index import SimpleDirectoryReader, LLMPredictor, ServiceContext, GPTVectorStoreIndex

### Expected behavior

Run without error
DOC: ParentDocumentRetriever without InMemoryStore
Labels: name: area: doc loader, name: auto:question, name: auto:documentation

### Issue with current documentation:

Hi, I'm not quite sure how to translate the `ParentDocumentRetriever` examples to ingest documents to OpenSearch in one phase, and then reconnect to it by instantiating a retriever at a later point.

The examples use an `InMemoryStore()` for the parent documents. Is the idea then that it would be necessary if I wanted to use OpenSearch to create two different OpenSearch clusters, one for the parent docs and one for the child docs? Or is there a more simple way to do this?
Image prompt template
Labels: name: lgtm, name: auto:improvement, name: size:XL

Builds on Bagatur's (#13227). See unit test for example usage (below)

```python
def test_chat_tmpl_from_messages_multipart_image() -> None:
    base64_image = "abcd123"
    other_base64_image = "abcd123"
    template = ChatPromptTemplate.from_messages(
        [
            ("system", "You are an AI assistant named {name}."),
            (
                "human",
                [
                    {"type": "text", "text": "What's in this image?"},
                    # OAI supports all these structures today
                    {
                        "type": "image_url",
                        "image_url": "data:image/jpeg;base64,{my_image}",
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "data:image/jpeg;base64,{my_image}"},
                    },
                    {"type": "image_url", "image_url": "{my_other_image}"},
                    {
                        "type": "image_url",
                        "image_url": {"url": "{my_other_image}", "detail": "medium"},
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "https://www.langchain.com/image.png"},
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "data:image/jpeg;base64,foobar"},
                    },
                ],
            ),
        ]
    )
    messages = template.format_messages(
        name="R2D2", my_image=base64_image, my_other_image=other_base64_image
    )
    expected = [
        SystemMessage(content="You are an AI assistant named R2D2."),
        HumanMessage(
            content=[
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{other_base64_image}"
                    },
                },
                {
                    "type": "image_url",
                    "image_url": {"url": f"{other_base64_image}"},
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"{other_base64_image}",
                        "detail": "medium",
                    },
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "https://www.langchain.com/image.png"},
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "data:image/jpeg;base64,foobar"},
                },
            ]
        ),
    ]
    assert messages == expected
```
Bagatur/community
Labels: name: area: agent, name: auto:enhancement, name: size:XXL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Feature/Add LLMs Integration For Oracle Cloud Infrastructure(OCI) Data Science Model Deployment Endpoint
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: size:XL

- **Description:** 
    - [OCI Data Science](https://docs.oracle.com/en-us/iaas/data-science/using/home.htm) is a fully managed and serverless platform for data science teams to build, train, and manage machine learning models in the Oracle Cloud Infrastructure. This PR add integration for using LangChain with an LLM hosted on a [OCI Data Science Model Deployment](https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-about.htm). To authenticate, [oracle-ads](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html) has been used to automatically load credentials for invoking endpoint.
- **Issue:** None
- **Dependencies:** `oracle-ads`
- **Tag maintainer:** @baskaryan
- **Twitter handle:** None
Make LogFileLoader more generic
Labels: name: area: doc loader, name: auto:enhancement, name: size:L

**Description:**
Introducing a flexible log file loader class, `LogFileLoader`, designed to handle various log file formats. This enhancement addresses the need for a versatile tool that adapts to diverse log structures. The class allows users to define custom parsing functions for timestamps, severity levels, and log messages, providing a robust solution for log file processing.

**Issue:** 
N/A

**Dependencies:**
- [loguru](https://pypi.org/project/loguru/): A logging library for flexible and easy-to-use logging.
- [dateutil](https://pypi.org/project/python-dateutil/): Part of the Python standard library, used for parsing timestamps.
- [termcolor](https://pypi.org/project/termcolor/): A library for adding colored output to the terminal.

**Tag maintainer:** 
@ancjainil
Lang chain not able to connect to SQL Anywhere 17 
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

Name: langchain
Version: 0.0.344

Python 3.9.6 
--(tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32

OS - Window 11 

Programming language - Python 

Database - SQL Anywhere 17

### Who can help?

@hwchase17
@agola11


**I have SAP SQL Anywhere 17 database server** which I want to talk to **using Langchain and OpenAI in python language** 

I am successful able to do this with MS SQL **but failing to do with SQL Anywhere** 

**Code below** 

 
import os
import pyodbc
import tkinter as tk
import tkinter.ttk as ttk
from langchain.agents import create_sql_agent
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.sql_database import SQLDatabase
from langchain.llms.openai import OpenAI
from langchain.agents import AgentExecutor

DATABASE_SERVER = 'sqldemo'
DATABASE_NAME = 'demo'
DATABASE_USERNAME = 'dba'
DATABASE_PASSWORD = 'sql'
DRIVER = '{SQL Anywhere 17}'
    
    // For Microsoft SQL DB - working 
   // conn_uri = f"mssql+pyodbc://{DATABASE_SERVER}/{DATABASE_NAME}?driver=ODBC+Driver+18+for+SQL+Server&TrustServerCertificate=yes&Trusted_Connection=yes"
    
 //For SQL Anywhere 17  -  Not working 
    conn_uri = f"**sqlanywhere+pyodbc:**//{DATABASE_USERNAME}:{DATABASE_PASSWORD}@{DATABASE_SERVER}/{DATABASE_NAME}?driver=SQL+Anywhere+17"     
 
    db = SQLDatabase.from_uri(conn_uri)   #Error line 
    llm = OpenAI(api_key=os.environ['OPENAI_API_KEY'], temperature=0)  
    toolkit = SQLDatabaseToolkit(db=db, llm=llm)   
    agent_executor = create_sql_agent(
        llm=OpenAI(temperature=0),
        toolkit=toolkit,
        verbose=True
    )
    return agent_executor

================================
**Error I am getting as below** 
Exception has occurred: NoSuchModuleError
**Can't load plugin: sqlalchemy.dialects:sqlanywhere.pyodbc**
![image](https://github.com/langchain-ai/langchain/assets/8278919/879a9f8e-ab78-4f06-a7eb-5061ae9530a2)

I can be reached at cloudme50@gmail.com if needed 

### Information

- [] The official example notebooks/scripts
- [X ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Code can be found here 
Please download and check 

https://github.com/developer20sujeet/Self_GenerativeAI/blob/main/Langchain_OpenAI_SQLAnywhere_Chat/sql_anywhere_error.py
### Expected behavior

able to connect and query SQL anyhwere 17
Use deepcopy in RunLogPatch
Labels: name: lgtm, name: auto:bug, name: size:M

This PR adds deepcopy usage in RunLogPatch. I am not confident that this is the appropriate place for the fix.

I included a unit-test that shows an issue that was caused in LangServe in the RemoteClient.

```python
import jsonpatch

s1 = {}
s2 = {'value': []}
s3 = {'value': ['a']}

ops0 = list(jsonpatch.JsonPatch.from_diff(None, s1))
ops1 = list(jsonpatch.JsonPatch.from_diff(s1, s2))
ops2 = list(jsonpatch.JsonPatch.from_diff(s2, s3))
ops = ops0 + ops1 + ops2

jsonpatch.apply_patch(None, ops)
{'value': ['a']}

jsonpatch.apply_patch(None, ops)
{'value': ['a', 'a']}

jsonpatch.apply_patch(None, ops)
{'value': ['a', 'a', 'a']}
```
Request: ability to set callbacks with LCEL at instantiation
Labels: name: auto:improvement, name: area: models

### Feature request

With `LLMChain`, it was possible to instantiate with `callbacks`, and just pass around the `LLMChain`.

With LCEL, the only way to handle `callbacks` is to pass them to every `invoke` call.  This requires one to pass around both the runnable LCEL object as well as the `callbacks`.

### Motivation

It's preferable to bake the `callbacks` into the LCEL object in advance at instantiation, then they get called each `invoke`.

### Your contribution

I can contribute something if I can get a confirm this is desirable.  The callbacks would be inserted at the base of the LCEL:

```python
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain.schema.callbacks.stdout import StdOutCallbackHandler
from langchain.schema.runnable import RunnableConfig

config = RunnableConfig(callbacks=[StdOutCallbackHandler()])
prompt = PromptTemplate.from_template(
    "What is a good name for a company that makes {product}?"
)
runnable = config | prompt | ChatOpenAI()
runnable.invoke(input={"product": "colorful socks"})
```
Remove deprecated Embaas API
Labels: name: lgtm, name: area: doc loader, name: auto:refactor, name: size:L

  - **Description:** We're removing the doc api (since nobody uses it) of Embaas and are cleaning this up as well., 
  - **Issue:** - 
  - **Dependencies:** -,
  - **Tag maintainer:** -,
  - **Twitter handle:** -
Fixed OpenAIFunctionsAgent not returning when receiving AgentFinish
Labels: name: area: agent, name: lgtm, name: auto:bug, name: size:XS

**Description:** The way the condition is checked in the `return_stopped_response` function of `OpenAIAgent` may not be correct, when the value returned is `AgentFinish` from the tools it does not work properly.


Thanks for review, @baskaryan, @eyurtsev, @hwchase17.
Issue: Recently, the DuckduckGo search tool seems not working. VQDExtractionException: Could not extract vqd. keywords="keywords"
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

from langchain.agents import AgentType, initialize_agent
from langchain.chat_models import ChatOpenAI
from langchain.tools import DuckDuckGoSearchRun

llm = ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo-16k")

web_search = DuckDuckGoSearchRun() 
tools = [
    web_search,
    # other tools
]

agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
 
agent.run("<Some question>")

VQDExtractionException: Could not extract vqd. keywords="blah blah blah"

---
The code above worked fine several weeks ago, but it is not working any more. I tried every version of LangChain released after September.

Maybe there is a change in DuckDuckGo's API. Please help me, thanks.

### Suggestion:

_No response_
similarity_search_with_relevance_scores returns incoherent relevance scores with Neo4jVector
Labels: name: area: vector store, name: auto:bug

### System Info

python 3.11.6
langchain==0.0.345
langchain-core==0.0.9
jupyter_client==8.6.0
jupyter_core==5.5.0
ipykernel==6.27.0
ipython==8.17.2

on mac M2

### Who can help?

@baskaryan @tomasonjo @hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Add a first node to the graph
`first_node = Node(
    id="1",
    type="Sentence",
    properties={
        "description": "This is my first node"
    }
)

graph_document = GraphDocument(
    nodes= [first_node],
    relationships= [],
    source= Document(page_content="my first document")
)

graph.add_graph_documents([graph_document])`

2. Add a second node to the graph
`second_node = Node(
    id="2",
    type="Sentence",
    properties={
        "description": "I love eating spinach"
    }
)

graph_document = GraphDocument(
    nodes= [second_node],
    relationships= [],
    source= Document(page_content="second doc")
)
graph.add_graph_documents([graph_document])`

3. Create a hybrid index
`existing_graph = Neo4jVector.from_existing_graph(
    embedding=OpenAIEmbeddings(),
    url=url,
    username=username,
    password=password,
    index_name="sentence_index",
    keyword_index_name="sentence_kindex",
    node_label="Sentence",
    text_node_properties=["description"],
    embedding_node_property="embedding",
    search_type="hybrid"
)`

4. Do a similarity search
`existing_graph.similarity_search_with_score("It is the end of the world. Take shelter")`

5. This yields a relevance score of 1 for the 1st result
`[(Document(page_content='\ndescription: This is my first node'), 1.0),
 (Document(page_content='\ndescription: I love eating spinach'),
  0.8576263189315796)]`

6. Test the strategy in use: 
`existing_graph._distance_strategy`
which returns `<DistanceStrategy.COSINE: 'COSINE'>`

### Expected behavior

The relevance score should return the cosine similarity score.

In this particular case the cosine similarity score is 0.747

`def get_embedding(text):
    response = openai.embeddings.create(input=text, model="text-embedding-ada-002")
    return response.data[0].embedding

def cosine_similarity(vec1, vec2):
    return dot(vec1, vec2) / (norm(vec1) * norm(vec2))

embedding1 = get_embedding("This is my first node")
embedding2 = get_embedding("It is the end of the world. Take shelter")

similarity = cosine_similarity(embedding1, embedding2)
print(f"Cosine Similarity: {similarity}")`

returns
`Cosine Similarity: 0.7475260325549817`