delayedkarma/langchain-issues huggingface dataset:
Issue: Need a trace or debug feature in Lanchain DirectoryLoader
Labels: name: area: doc loader, name: auto:improvement

### Issue you'd like to raise.

I've set "langchain.debug=True"; however, it does not work for the DirectoryLoader. I have a notebook that tried to load a dozen or more PDFs, and typically, at least one of the files fails (see attached). I looked at the code, and as far as I can tell, there is no trace or debug feature in (https://github.com/hwchase17/langchain/tree/master/langchain/document_loaders). 

My issue is that the loader code is a black box. I can't tell which file is failing; therefore, I have to process each one individually to find out which one is failing. It would be beneficial if a trace/debugger could help me identify which file it's failing on. 

TIA
<img width="912" alt="Screen Shot 2023-07-14 at 9 04 56 AM" src="https://github.com/hwchase17/langchain/assets/457288/fd5b7732-1040-4c73-91dc-abc41fb9cadd">


### Suggestion:

Please make a debug option for "https://github.com/hwchase17/langchain/tree/master/langchain/document_loaders" code.
cube loader dev
Labels: name: auto:documentation

**PR Description:**

This pull request introduces several enhancements and new features to the `CubeSemanticLoader`. The changes include the following:

1. Added imports for the `json` and `time` modules.
2. Added new constructor parameters: `load_dimension_values`, `dimension_values_limit`, `dimension_values_max_retries`, and `dimension_values_retry_delay`.
3. Updated the class documentation with descriptions for the new constructor parameters.
4. Added a new private method `_get_dimension_values()` to retrieve dimension values from Cube's REST API.
5. Modified the `load()` method to load dimension values for string dimensions if `load_dimension_values` is set to `True`.
6. Updated the API endpoint in the `load()` method from the base URL to the metadata endpoint.
7. Refactored the code to retrieve metadata from the response JSON.
8. Added the `column_member_type` field to the metadata dictionary to indicate if a column is a measure or a dimension.
9. Added the `column_values` field to the metadata dictionary to store the dimension values retrieved from Cube's API.
10. Modified the `page_content` construction to include the column title and description instead of the table name, column name, data type, title, and description.

These changes improve the functionality and flexibility of the `CubeSemanticLoader` class by allowing the loading of dimension values and providing more detailed metadata for each document.


@rlancemartin, @eyurtsev
DOC: Tools\How to\Defining Custom Tools\Handling Tool Errors Import error with ToolException
Labels: name: auto:documentation

### Issue with current documentation:

In the Tools\How to\Defining Custom Tools\Handling Tool Errors, it seems that the ToolException import has not been updated to the latest module architecture. (langchain**0.0.233)

```
from langchain.schema import ToolException

from langchain import SerpAPIWrapper
from langchain.agents import AgentType, initialize_agent
from langchain.chat_models import ChatOpenAI
from langchain.tools import Tool

from langchain.chat_models import ChatOpenAI
```

### Idea or request for content:

```
from langchain.tools.base import ToolException

from langchain import SerpAPIWrapper
from langchain.agents import AgentType, initialize_agent
from langchain.chat_models import ChatOpenAI
from langchain.tools import Tool

from langchain.chat_models import ChatOpenAI
```
DOC: `ToolException` cannot be imported as mentioned on "Defining Custom Tools" page - Python
Labels: name: auto:documentation

### Issue with current documentation:

The [Handling Tool Errors section](https://python.langchain.com/docs/modules/agents/tools/how_to/custom_tools#handling-tool-errors) on Defining Custom Tools page has following line as part of the first code cell:

```py
from langchain.schema import ToolException
```

When running this, I get this error:
```py
ImportError: cannot import name 'ToolException' from 'langchain.schema' (<path>\venv\lib\site-packages\langchain\schema\__init__.py)
```



### Idea or request for content:

A quick search for `ToolException` shows that it's defined in `langchain\tools\base.py`, perhaps the docs need to be updated to

```py
from langchain.tools.base import ToolException
```
Updated references and listed on updated audio.py
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: Changed and listed the new audio.
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!
pip-audit detects numerous vulnerabilities
Labels: name: auto:bug

### System Info

Using Lanchain version .233 currently, and each time I make an update and run tests on my project, pip-audit is returning an additional vulnerability. I use Gitlab for the project and I am adding commands to ignore these vulnerabilities in gitlab-ci.yml, but currently the command looks like:

`- pip-audit --ignore-vuln PYSEC-2023-109 --ignore-vuln PYSEC-2023-110 --ignore-vuln PYSEC-2023-98 --ignore-vuln PYSEC-2023-91 --ignore-vuln PYSEC-2023-92`

these are all langchain vulnerabilities. 

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce behavior:

1. Use Gitlab's CI-CD flow with a gitlab-ci.yml file
2. use pip-audit from pip-tools as part of the testing
3. push a change to production

### Expected behavior

Most well-known packages don't get any vulnerabilities, or when they do they get fixed shortly.
handle_tool_error in @tool decorator
Labels: name: auto:improvement

### Feature request

we have the `@tool` decorator and the `Tool.from_function` function.

But they are (or seem to me) inconsistent. The kwargs `handle_tool_error` and `return_direct` are only available in `Tool.from_function` .

Shouldn't be available in both?

Shouldn't be `@tool` just a mirror of `Tool.from_function` ?

### Motivation

I want to be able to do `@tool(handle_tool_error=True)`

### Your contribution

If this is not me just reading wrong the docs. And needs development, happy to do a PR
Client argument ignored in HuggingFaceTextGenInference constructor
Labels: name: auto:bug

### System Info

Lancghain version 0.0.233.


The arguments Client as well as async_client are ignored since they are set in the pydantic [root_validator ](https://github.com/hwchase17/langchain/blame/master/langchain/llms/huggingface_text_gen_inference.py#L104).

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain import HuggingFaceTextGenInference

llm = HuggingFaceTextGenInference(client=my_client)

assert llm.client is my_client
```

### Expected behavior

The llm should use the client or async client provided to the constructor instead of just ignoring it.

```
from langchain import HuggingFaceTextGenInference

llm = HuggingFaceTextGenInference(client=my_client)

assert llm.client is my_client
```
Could not parse LLM output when using 'create_pandas_dataframe_agent' with open source models (any model other than OpenAI models)
Labels: name: auto:bug

### System Info

I keep getting an error 'Could not parse LLM output' when using the create_pandas_dataframe_agent with Vicuna 13B as the LLM.
Any solution to this?

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

agent = create_pandas_dataframe_agent(llm, df, verbose=True, agent_kwargs={"format_instructions": FORMAT_INSTRUCTIONS}, handle_parsing_errors="Check your output and make sure it conforms!")
agent.run(input='How many rows are there?')

### Expected behavior

There are 15 rows
Implement async API for Qdrant vector store
Labels: name: auto:enhancement

Inspired by #5550, I implemented full async API support in Qdrant. The docs were extended to mention the existence of asynchronous operations in Langchain. I also used that chance to restructure the tests of Qdrant and provided a suite of tests for the async version. Async API requires the GRPC protocol to be enabled. Thus, it doesn't work on local mode yet, but we're considering including the support to be consistent.

Tagging @baskaryan @rlancemartin, @eyurtsev
Allow passing auth objects in TextRequestsWrapper
Labels: name: auto:improvement

  - Description: This allows passing auth objects in request wrappers. Currently, we can handle auth by editing headers in the RequestsWrappers, but more complex auth methods, such as Kerberos, could be handled better by using existing functionality within the requests library. There are many authentication options supported both natively and by extensions, such as requests-kerberos or requests-ntlm.
  
  - Issue: Fixes #7542
  - Dependencies: none
  - Tag maintainer: @baskaryan
Update pgembedding.ipynb
Labels: name: auto:documentation

Update the extension name. It changed from pg_hnsw to pg_embedding.

Thank you. I missed this in my previous commit.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Enabled nesting chain group
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Issue: Cannot use CombinedMemory with VectorStoreRetrieverMemory and ConversationTokenBufferMemory
Labels: name: auto:bug, name: auto:improvement

### Issue you'd like to raise.

Here is an (incomplete) modification of the documentation for using `CombinedMemory`, but with `VectorStoreRetrieverMemory` instead of `ConversationSummaryMemory`.

```python
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import ConversationChain
from langchain.memory import (
    ConversationBufferMemory,
    CombinedMemory,
    ConversationSummaryMemory,
)


conv_memory = ConversationBufferMemory(
    memory_key="chat_history_lines", input_key="input"
)
# not shown: define retriever as shown in the [Chroma docs](https://python.langchain.com/docs/modules/data_connection/vectorstores/integrations/chroma)
vector_memory = VectorStoreRetrieverMemory(retriever=retriever, memory_key="history", input_key="input")
    
# Combined
memory = CombinedMemory(memories=[vector_memory, summary_memory])
_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Summary of conversation:
{history}
Current conversation:
{chat_history_lines}
Human: {input}
AI:"""
PROMPT = PromptTemplate(
    input_variables=["history", "input", "chat_history_lines"],
    template=_DEFAULT_TEMPLATE,
)
llm = OpenAI(temperature=0)
conversation = ConversationChain(llm=llm, verbose=True, memory=memory, prompt=PROMPT)
```

What happens is, [on this line](https://github.com/hwchase17/langchain/blob/master/langchain/memory/vectorstore.py#L58), `inputs.items()` looks something like this:

```python
dict_items([('input', 'How are you?'), ('chat_history_lines', 'Human: wow\nAI: Wow!'), ('history', 'Human: wow\nAI: Wow!\nHuman:yes\nAI: Yes!\nHuman: hello\nAI: Hi!')])
```

When adding documents to the vectorstore retriever memory, all items are added except for `self.memory_key` (history). Thus, `chat_history_lines` is included in the created documents, without a way to prevent that.

### Suggestion:

One approach for supporting this would be to add a property to VectorStoreRetrieverMemory that allows the caller to specify which input keys should be included in the created documents. Another approach may be to only create documents with input_key from inputs.
SQLDatabaseChain runs under the GPT4 model and reports an error
Labels: name: auto:bug

### System Info

### my python code 

from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
from langchain.chains import SQLDatabaseSequentialChain

db = SQLDatabase.from_uri("clickhouse://xx:xx@ip/db",
                         include_tables=include_tables,
                         custom_table_info=custom_table_schemas,
                         sample_rows_in_table_info=2)
        
llm = OpenAI(temperature=0, model_name="**gpt-4-0613**", verbose=True, streaming=True, openai_api_base="https://xxx.cn/v1")

db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True, use_query_checker=True, return_intermediate_steps=True, top_k=3)

instruction = "Statistics EL_C1 device uptime today"
result = db_chain(instruction)
result["intermediate_steps"]

### run result

![image](https://github.com/hwchase17/langchain/assets/5253435/3677ef99-5ff5-4d08-a9a8-51585a5770a4)
DatabaseException: Orig exception: Code: 62. DB::Exception: Syntax error: failed at position 1 ('The') (line 1, col 1): The original query seems to be correct as it doesn't have any of the common mistakes mentioned. Here is the reproduction of the original query:

I don't know why these strings are being run as SQL


if set use_query_checker = False

![image](https://github.com/hwchase17/langchain/assets/5253435/6ecee8fe-7200-45cf-a73d-cf615ba4919a)
There is an extra double quote in SQL

DatabaseException: Orig exception: Code: 62. DB::Exception: Syntax error: failed at position 1 ('"SELECT SUM(`value`) FROM idap_asset.EL_MODEL_Run_Time WHERE `asset_code` = 'EL_C1' AND toDate(CAST(`window_end` / 1000, 'DateTime')) = today()"'): "SELECT SUM(`value`) FROM idap_asset.EL_MODEL_Run_Time WHERE `asset_code` = 'EL_C1' AND toDate(CAST(`window_end` / 1000, 'DateTime')) = today()".


**All these errors are only generated under the GPT4 model. If the default model is used, no errors are generated**




### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

llm = OpenAI(temperature=0, model_name="**gpt-4-0613**", verbose=True, streaming=True, openai_api_base="https://xxx.cn/v1")

db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True, use_query_checker=False, return_intermediate_steps=True, top_k=3)

As long as the GPT4 model is used, there will be problems through the db chain, and the string returned by the model will be executed as SQL



### Expected behavior

![image](https://github.com/hwchase17/langchain/assets/5253435/1e898a60-beed-49ee-87a3-4ab6c5a80614)

The hope is to generate correct SQL execution, do not treat the string returned by the model as SQL execution
[Small upgrade] Allow document limit in AzureCognitiveSearchRetriever
Labels: name: lgtm, name: auto:improvement

Multiple people have asked in #5081 for a way to limit the documents returned from an AzureCognitiveSearchRetriever. This PR adds the `top_n` parameter to allow that.

Maintainer responsibilities:
DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev

Twitter handle:
 [@UmerHAdil](twitter.com/umerHAdil)
Fix same issue #7524 in FileCallbackHandler
Labels: name: lgtm, name: auto:bug

Fix for Serializable class to include name, used in FileCallbackHandler as same issue #7524 

Description: Fixes the Serializable class to include 'name' attribute (class_name) in the dict created,
This is used in Callbacks, specifically the StdOutCallbackHandler, FileCallbackHandler.
Issue: As described in issue #7524
Dependencies: None
Tag maintainer: SInce this is related to the callback module, tagging @agola11 @idoru 
Comments:

Glad to see issue #7524 fixed in pull #6124, but you forget to change the same place in FileCallbackHandler
feat(chains): adding ElasticsearchDatabaseChain for interacting with analytics database
Labels: name: lgtm, name: auto:enhancement

This pull request adds a ElasticsearchDatabaseChain chain for interacting with analytics database, in the manner of the SQLDatabaseChain.

It can be used in this way:

```py
from elasticsearch import Elasticsearch

from langchain.chains.elasticsearch_database import ElasticsearchDatabaseChain
from langchain.chat_models import ChatOpenAI

client = Elasticsearch("http://localhost:9200")

OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY')
llm = ChatOpenAI(temperature=0, openai_api_key=OPENAI_API_KEY, model_name='gpt-3.5-turbo-16k')

db_chain = ElasticsearchDatabaseChain.from_llm(llm=llm, db=client, verbose=False, sample_documents_in_index_info=10)

result = db_chain.run(question)
print(result)
```

Maintainer: @samber
Twitter handler: samuelberthe

@baskaryan
Executor calling CustomTool with no parameter needed insists to call the tool with a parameter coming from nowhere
Labels: name: auto:bug

### System Info

langchain==0.0.232
Google Colab

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
def get_current_oil_price():
    ticker_data = yf.Ticker("CL=F")
    recent = ticker_data.history(period='1d')
    return {"price": recent.iloc[0]["Close"], "currency": ticker_data.info["currency"]}
```
```
def get_oil_price_performance(days):
    past_date = datetime.today() - timedelta(days=int(days))
    ticker_data = yf.Ticker("CL=F")
    history = ticker_data.history(start=past_date)
    old_price = history.iloc[0]["Close"]
    current_price = history.iloc[-1]["Close"]
    return {"percent_change": ((current_price - old_price) / old_price) * 100}
```
```
class CurrentOilPriceTool(BaseTool):
    name = "get_oil_price"
    description = "Get the current oil price. No parameter needed from input"


    def _run(self):
        price_response = get_current_oil_price()
        return price_response


    def _arun(self):
        raise NotImplementedError("get_oil_price does not support async")
```

```
class CurrentOilPerformanceTool(BaseTool):
    name = "get_oil_performance"
    description = "Get the current oil price evolution over a given number of days. Enter the number of days."


    def _run(self, days):
        performance_response = get_oil_price_performance(days)
        return performance_response


    def _arun(self):
        raise NotImplementedError("get_oil_performance does not support async") 
```

```
llm = ChatOpenAI(model="gpt-3.5-turbo-0613", temperature=0)

tools = [CurrentOilPriceTool(), CurrentOilPerformanceTool()]

agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=True
)
```

`agent.run("What is the oil price?")`


### Expected behavior

```
> Entering new AgentExecutor chain...

Invoking: `get_oil_price`
```

But instead, the executor insists on adding a parameter to the tool function !!!

```
> Entering new AgentExecutor chain...

Invoking: `get_oil_price` with `USD`
```

provoking an obvious error:

`TypeError: CurrentOilPriceTool._run() takes 1 positional argument but 2 were given`