delayedkarma/langchain-issues huggingface dataset:
Qdrant Client: Expose instance for creating client
Labels: name: area: vector store, name: auto:improvement

Expose classmethods to convenient initialize the vectostore.

The purpose of this PR is to make it easy for users to initialize an empty vectorstore that's properly pre-configured without having to index documents into it via `from_documents`.

This will make it easier for users to rely on the following indexing code: https://github.com/langchain-ai/langchain/pull/9614
to help manage data in the qdrant vectorstore.
Bagatur/redis refactor
Labels: name: area: vector store, name: auto:refactor

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
DOC: Custom Templates issue with Pydantic v2
Labels: name: area: doc loader, name: auto:bug, name: auto:documentation

### Issue with current documentation:

The "Creating a Custom Prompt Template" documentation is outdated with Pydantic v2.

```
class FunctionExplainerPromptTemplate(StringPromptTemplate, BaseModel):
    """A custom prompt template that takes in the function name as input, and formats the prompt template to provide the source code of the function."""

    @validator("input_variables")
    def validate_input_variables(cls, v):
        """Validate that the input variables are correct."""
        if len(v) != 1 or "function_name" not in v:
            raise ValueError("function_name must be the only input_variable.")
        return v
```
        
The above code raises following TypeError:

```
>   class FunctionExplainerPromptTemplate(StringPromptTemplate, BaseModel):
E   TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

```
Frankly, I have no idea how to get the prompt templates to work with new Pydantic, even after changing @validator to @field_validator.

### Idea or request for content:

Update the documentation to include new version of Pydantic in examples.
Do not share executors between parent and child tasks
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
adapt ChatGLM in llm model and chat model
Labels: name: auto:improvement, name: area: models

using sample:
```
endpoint_url = API URL
ChatGLM_llm = ChatGLM(
    endpoint_url=endpoint_url,
    api_key=Your API Key by ChatGLM
)
print(ChatGLM_llm("hello"))
```

```
model = ChatChatGLM(
    chatglm_api_key="api_key",
    chatglm_api_base="api_base_url",
    model_name="model_name"
)
chain = LLMChain(llm=model)
```
Description: The call of ChatGLM has been adapted.
Issue: The call of ChatGLM has been adapted.
Dependencies: Need python package `zhipuai` and `aiostream`
Tag maintainer: @baskaryan, @hwchase17
Twitter handle: None
Calude models not able to output more than 8191 tokens.
Labels: name: auto:bug, name: area: models

### System Info

Platform - AWS
Python version: 3.11.4
OS - Mac


### Who can help?

@3coins , @hwchase17 @agola11 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The code sample which produces the bug:

```
body = json.dumps({"prompt": prompt, "max_tokens_to_sample": 8192})
    accept = "application/json"
    contentType = "application/json"

    response_claudeV2 = boto3_bedrock.invoke_model(
        body=body, modelId="anthropic.claude-v2", accept=accept, contentType=contentType
    )
    response_body_claudeV2 = json.loads(response_claudeV2.get("body").read())

    print(response_body_claudeV2.get("completion"))
```

When the above code snippet is executed with boto3_client we get the error:
` botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: 8192 is not less or equal to 8191, please reformat your input and try again.`

As per the doc of anthropic claude the claude-instant can output 9k tokens, claude-v1 can output 12k tokens and claude-v2 can output 12k tokens but when 'max_token_sample' parameter exceeds 8191 values they give the above error.

### Expected behavior

Expected behaviour is that the model should give the output as it can produce max 12K tokens.
No way to Close an open connection in PGVector.py
Labels: name: area: vector store, name: auto:improvement

### Feature request

Initialization with Database Connection: When an instance of the PGVector class is created, it automatically establish a connection with the PostgreSQL Vector database.

Method for Closing Connection: we need to implement a method within the PGVector class that allows you to close the established connection with the PostgreSQL database.

`def __del__(self):
        # Close the session (and thus the connection) when the instance is destroyed.
        self.session.close()`

### Motivation

The problem is, I am unable to close a connection and the pool get overload with multiple connections and hence the service starts throwing error

### Your contribution

I guess, may be.
Agent by AgentType.OPENAI_FUNCTIONS cannot output message content.
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

It will always output 
```
responded: {content}
```

I suppose this [line](https://github.com/langchain-ai/langchain/blob/v0.0.271/libs/langchain/langchain/agents/openai_functions_agent/base.py#L130) is not correct. 




### Suggestion:

Is the following code the way you need? 
```
content_msg = f"responded: {message.content}\n" if message.content else "\n"
```
Add .with_config() method to Runnables, Add run_id, run_name to RunnableConfig
Labels: name: auto:improvement

- with_config() allows binding any config values to a Runnable, like .bind() does for kwargs

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Add AliCloudPaiEAS type in langchain.llms to access alicloud pai-eas service.
Labels: name: auto:enhancement, name: area: models

### Description
  This PR aims at adding new llm type  AliCloudPaiEAS to support service accessing for alicloud product named pai-eas service. PAI-EAS can help users to deploy llm models on alicloud.  

### Usage
The user needs to start an eas service on alicloud and then set the service's access url and token as a params for AliCloudPaiEAS. Then AliCloudPaiEAS  object as a llm object is passed to the Chain to run an output. 
The usage is like:
`llm = AliCloudPaiEAS(eas_service_url="your_access_url",eas_service_token="your_access_token"`
`llm_chain = LLMChain(prompt=prompt, llm=llm)
`
`llm_chain.run("Hello there.")`

### Test and example
The integration test case is located at libs/langchain/tests/integration_tests/llms/test_alicloud_eas.py
The example notebook is located at docs/extras/integrations/lls/alicloud_api_eas.ipynb
fix setting --only-extended or --only-core flags to False if not prov…
Labels: name: lgtm, name: auto:bug

Description:
`--only-extended` and `--only-core` flags are not set correctly to `False`, if not provided

Issue:
When trying to run `make coverage` I got the following issue:
```
INTERNALERROR> ValueError: no option named 'only_extended'
```
It turned out that `config.getoption("--only-extended")` in `pytest_collection_modifyitems` function was unable to get the `--only-extended` value
Update cube_semantic.py to add new type
Labels: name: area: doc loader, name: auto:improvement

the cube_semantic example jupyter notebook didn't work, if I use curl, it has following json with type=cube, so I add this type in the code to make it work:
(langchain) ➜  langchain git:(master) ✗ curl \       
  -H "Authorization:.xxx" \
  -G \
  --data-urlencode 'query={"measures":["LineItems.count"]}' \
  https://xxx.cubecloudapp.dev/cubejs-api/v1/meta
{"cubes":[{"public":true,"name":"LineItems","type":"cube","title":"Line Items","measures":[{"name":"LineItems.count","title":"Line Items Count","shortTitle":"Count","cumulativeTotal":false,"cumulative":false,"type":"number","aggType":"count","drillMembers":[],"drillMembersGrouped":{"measures":[],"dimensions":[]},"isVisible":true},{"name":"LineItems.price","title":"Line Items Price","shortTitle":"Price","cumulativeTotal":fals



<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: Update cube_semantic.py to add new type "cube", 
  - Issue: the cube_semantic example jupyter notebook didn't work,
  - Dependencies: NA,
  - Tag maintainer: NA
  - Twitter handle: NA

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Add the option to use separate LLMs for GraphCypherQA chain
Labels: name: auto:improvement, name: area: models

The Graph Chains are different in the way that it uses two LLMChains instead of one like the retrievalQA chains. Therefore, sometimes you want to use different LLM to generate the database query and to generate the final answer.

This feature would make it more convenient to use different LLMs in the same chain.

I have also renamed the Graph DB QA Chain to Neo4j DB QA Chain in the documentation only as it is used only for Neo4j. The naming was ambigious as it was the first graphQA chain added and wasn't sure how do you want to spin it.
Issue: APIConnectionError: Error communicating with OpenAI
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

eroory msg:
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/chat/completions (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response'))).

code:
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI

llm = OpenAI()
chat_model = ChatOpenAI()

chat_model.predict("hi!")


export OPENAI_API_KEY="sk-xxxxxxujOoH";



### Suggestion:

_No response_
DQL DB Langchain : When running an db_run query based on the prompt it should execute sql query only when needed. 
Labels: name: auto:bug

![image](https://github.com/langchain-ai/langchain/assets/53115668/70a42838-7758-4c0a-be99-ba41ad861aaf)
 The Prompt was to explain tables in the data. in this case it should query into the SQL database.
An error occurred: 'DDGS' object does not support the context manager protocol
Labels: name: area: agent, name: auto:bug

### System Info

> Entering new AgentExecutor chain...

Invoking: `duckduckgo_search` with `2023年8月的新闻`


An error occurred: 'DDGS' object does not support the context manager protocol

### Who can help?

@hwchase17
@agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

# 初始化OpenAI Functions代理
agent = initialize_agent(
    # tools,

    tools=load_tools(["ddg-search"]),

    llm=llm,

    agent=AgentType.OPENAI_FUNCTIONS,

    verbose=True,
    agent_kwargs=agent_kwargs,
    memory=memory,
    max_iterations=10,
    early_stopping_method="generate",
    handle_parsing_errors=True,  # 初始化代理并处理解析错误
    callbacks=[handler],
)
> Entering new AgentExecutor chain...

Invoking: `duckduckgo_search` with `2023年8月的新闻`


An error occurred: 'DDGS' object does not support the context manager protocol

### Expected behavior

该类提供了通过 [DuckDuckGo](https://duckduckgo.com/) 搜索引擎搜索的功能。

from langchain.tools import DuckDuckGoSearchRun
search = DuckDuckGoSearchRun()
search.run("Who is winner of FIFA worldcup 2018?")
你应该期望如下输出：

The 2018 FIFA World Cup was the 21st FIFA World Cup, ... Mario Zagallo (Brazil) and Franz Beckenbauer (Germany) have also achieved the feat.
Add dashvector self query retriever
Labels: name: lgtm, name: area: vector store, name: auto:enhancement

## Description
Add `Dashvector` retriever and self-query retriever

## How to use
```python
from langchain.vectorstores.dashvector import DashVector

vectorstore = DashVector.from_documents(docs, embeddings)
retriever = SelfQueryRetriever.from_llm(
    llm, vectorstore, document_content_description, metadata_field_info, verbose=True
)
```
Allow specifying arbitrary keyword arguments in `langchain.llms.VLLM`
Labels: name: auto:improvement, name: area: models

### Feature request

Enable the option to specify arbitrary keyword arguments (e.g. gpu_memory_utilization=0.98) in langchain.llms.VLLM() constructor

### Motivation

Currently, vllm library provides many useful keyword arguments which enables a lot of usages on a wide variety of devices, but langchain doesn't.

e.g. Some models do not work with GPUs with smaller memory as the default gpu_memory_utilization = 0.9, increasing this limit could enable the use of those models on the smaller GPUs.

![image](https://github.com/langchain-ai/langchain/assets/140134804/1eb50665-d371-4695-b815-0bcfe0092906)


### Your contribution

I can contribute by submitting a PR for that.
initialize_agent not saving and returning messages in memory
Labels: name: area: memory, name: area: agent, name: auto:bug

### System Info

langchain version == 0.0.271

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

chat_history = MessagesPlaceholder(variable_name="chat_history")
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo')
agent_chain = initialize_agent(
tools=tools,
llm=llm,
agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
verbose=True,
memory=memory,
max_iterations=1,
early_stopping_method='generate',
handle_parsing_errors=True,
system_prompt="You are an assistant named Mikey",
agent_kwargs={
"memory_prompts": [chat_history],
"input_variables": ["input", "agent_scratchpad", "chat_history"],
}
)
agent_chain.run(input="My name is Dev")

agent_chain.run(input="What is my name")




### Expected behavior

Expected behavior
final answer: Your name is Dev


Output behavior:
Final answer: I don't have access to personal information like your name. Is there anything else I can help you with?


Note: What happens is that the memory is not being passed along, so the model doesn't know my name
Initialize_agent not storing messages when memory is present
Labels: name: area: memory, name: area: agent, name: auto:bug

### System Info

langchain version == 0.0.271

### Who can help?

@hw

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


chat_history = MessagesPlaceholder(variable_name="chat_history")
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo')
agent_chain = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    memory=memory,
    max_iterations=1,
    early_stopping_method='generate',
    handle_parsing_errors=True,
    system_prompt="You are an assistant named Mikey",
    agent_kwargs={
    "memory_prompts": [chat_history],
    "input_variables": ["input", "agent_scratchpad", "chat_history"],
}
)

agent_chain.run(input="What is my name")

### Expected behavior

final answer: Your name is .... 
What happens is that the memory is not being returned(the name was given to the model in a previous run) so the model doesn't know my name
Issue:  ERNIE Bot is not able to call tool
Labels: name: area: agent, name: auto:bug, name: area: models

### Issue you'd like to raise.

I tried to use Erine Bot as an agent like below

```python
tools = [faq_tool, query_order_tool]
FORMAT_INSTRUCTIONS = """要使用工具，请按照以下格式返回：

Thought: 我是否需要使用一个工具? 是
Action: 采取的action应该是以下之一：[{tool_names}]
Action Input: action的输入参数
Observation: action的返回结果

如果你不需要使用工具， 请按以下格式返回：

Thought: 我是否需要使用工具? 否
{ai_prefix}: [把你的回复放在这里]
"""

PREFIX = """用中文回答以下问题. 如果找不到答案回答 '我不知道'. 你可以使用以下工具:"""

prompt_ = ConversationalAgent.create_prompt(
     tools,
     prefix = PREFIX,
     format_instructions = FORMAT_INSTRUCTIONS,
     input_variables=["input", "chat_history","agent_scratchpad"]
)
memory = ConversationBufferWindowMemory(memory_key="chat_history", return_messages=True, k=3)
chatllmChain = LLMChain(llm=ErnieBotChat(),  callbacks=[self.logHandler], verbose = True)
agent = ConversationalAgent(llm_chain=chatllmChain)
callback_manager = CallbackManager([self.logHandler])
self.conversation_agent = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools,memory=memory, callback_manager = callback_manager, verbose=True, max_iterations=3)

``` 

Then I asked a question "你好，我应该怎么申请赔偿"
The answer returned by the agent is:

Could not parse LLM output: `我不知道，但我会将你的问题传递给对应的工具来获取答案。我发送了一个消息到[FAQ](%E8%BF%94%E5%9B%9E)和[search_order](%E8%BF%94%E5%9B%9E)。请等待他们的回复。`

Which is not correct. The correct behavior  is call the faq tool and get relevant information to form a reply. Which was proven to be work  when using Open API chat model.

@axiangcoding  Any idea of how to make Ernie Bot to be able to handle this situation? Thanks in advance!!!




### Suggestion:

_No response_
Support load ChatPromptTemplate from config dict
Labels: name: auto:enhancement, name: area: models

### Feature request

We are currently using OpenAI's ChatCompletion API with custom ChatPromptTemplate, as converting langchain's ChatPromptTemplate to dict (or vice versa) seems not working well.

So I'd like to suggest new feature, to enable loading [ChatPromptTemplate](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/prompts/chat.py) from config dict, 
by adding loading function (or classmethod in ChatPromptTemplate) to [prompts/loading.py](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/prompts/loading.py#L151-L155).

Or if there is standardize way to easily convert dict to ChatPromptTemplate, please let me know.

### Motivation

We are currently using OpenAI's ChatCompletion API with custom ChatPromptTemplate, as converting langchain's ChatPromptTemplate to dict (or vice versa) seems not working well.
As Utilizing OpenAI's ChatCompletion API (other than Completion API) becoming mainstream, I thought it might be good to support standard loading method for ChatPromptTemplate.

### Your contribution

If you allow me, I'd like to make a contribution related to this feature.