delayedkarma/langchain-issues huggingface dataset:
allow passing HTTP headers in AzureOpenAI class
Labels: name: auto:improvement, name: area: models

 - **Description:** Some organisations expose Azure OpenAI endpoints via some proxies that need additional HTTP headers. This PR supports for that by allowing the user to set custom headers when initialising the AzureOpenAI class. These custom headers can be used when making requests.
  - **Issue:** it aims to fix the issue #11593
  - **Dependencies:** -
Adapt to openai-python v1.0.0 Beta
Labels: name: auto:improvement, name: area: models

### Feature request

will you update to follow the update of https://github.com/openai/openai-python/discussions/631 ?


### Motivation

Many issues in langchain point that when calling async api and using openai LLM(Chains like MapReduceDocumentsChain), it will raise '[Error Response payload is not complete](https://github.com/langchain-ai/langchain/issues/9187#top)' or [TimeoutError](https://github.com/langchain-ai/langchain/issues/7148)

After discussion, they all point to [an openai issue](https://github.com/openai/openai-python/issues/574), which will be fix in v1.0.0(now v1.0.0 beta is avaliable)

So if you can adapt to their updates, this critical issue can be fixed

### Your contribution

none
@root_validator error
Labels: name: auto:bug, name: area: models

### System Info

```sh
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/langchain/__init__.py", line 8, in <module>
    from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
  File "/usr/local/lib/python3.8/dist-packages/langchain/agents/__init__.py", line 2, in <module>
    from langchain.agents.agent import Agent
  File "/usr/local/lib/python3.8/dist-packages/langchain/agents/agent.py", line 10, in <module>
    from langchain.chains.base import Chain
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/__init__.py", line 2, in <module>
    from langchain.chains.conversation.base import ConversationChain
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/conversation/base.py", line 7, in <module>
    from langchain.chains.conversation.memory import ConversationBufferMemory
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/conversation/memory.py", line 7, in <module>
    from langchain.chains.conversation.prompt import SUMMARY_PROMPT
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/conversation/prompt.py", line 2, in <module>
    from langchain.prompts.prompt import PromptTemplate
  File "/usr/local/lib/python3.8/dist-packages/langchain/prompts/__init__.py", line 2, in <module>
    from langchain.prompts.base import BasePromptTemplate
  File "/usr/local/lib/python3.8/dist-packages/langchain/prompts/base.py", line 35, in <module>
    class BasePromptTemplate(BaseModel, ABC):
  File "/usr/local/lib/python3.8/dist-packages/langchain/prompts/base.py", line 41, in BasePromptTemplate
    @root_validator()
  File "/usr/local/lib/python3.8/dist-packages/pydantic/deprecated/class_validators.py", line 228, in root_validator
    raise PydanticUserError(
pydantic.errors.PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.
```

### Who can help?

@agola11 

```python
import langchain
```

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Install langchain latest version on Linux
2. import langchain

### Expected behavior

Nothing.
Add Slack Toolkit
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

We would like to add Slack toolkit, which for a slack user, he can:
1. fetch information of workplaces (name, user type, time joined, etc.) he is a member of
2. fetch chat history across workplaces he is a member, and further process/analyze such data with LLM
3. fetch interaction (reply, mention, DM, etc) with other users across workplaces. 
4. fetch a list of new activities across workplaces since last time login to slack platform, 
5. and react to these activities. 

Also, supports filters (by time, keyword etc) as appropriate for all the above functions

### Motivation
Currently lang-chain provides limited support for Slack, specifically, only supports loader of chat history zip file. On the other hand, slack platform analytics interface lacks aggregated information across workplaces; also most analytics info if provides are limited to owner/admin user type. Therefore, we want to implement the above feature:
1. Add-on current langchain toolkit: slack toolkit would be the one of the few, if not the first social media related toolkit.
2. Expose slack interface tools' functions to more user types: Only specific slack user type (workspace owner and/or admin) has access to downloading file option, along with most analytics tools embedded in slack user interface, which calls for a toolkit for other user types to access such information.
3. Better Info Aggregation: We notice that a user can be in multiple workspaces yet currently there lacks a way for user to get/compare information (real-time and logs) across workspaces they are member of. It would be nice for lang chain to provide a toolkit that provides user with such information upon request, so that the user can have all his/her user info across workspace at one place.


### Your contribution
We have started working on the feature and intend to submit PR in mid November.
fix(openllm): update with newer remote client implementation
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

cc @baskaryan
feat: Google Vertex AI Search Retriever - Add support for Website Data Stores
Labels: name: lgtm, name: auto:enhancement

- Only works for Data stores with Advanced Website Indexing
  - https://cloud.google.com/generative-ai-app-builder/docs/about-advanced-features
- Minor restructuring - Follow up to #10513
  - Remove outdated docs (readded in https://github.com/langchain-ai/langchain/pull/11620)
  - Move legacy class into new py file to clean up the directory
    - Shouldn't cause backwards compatibility issues as the import works the same way for users
Add option to specify prompt for SelfQueryRetriever
Labels: name: auto:improvement, name: area: models

### Feature request

There is an option to specify the prompt sent to LLM and also the CONDENSE_QUESTION prompt. But there is no way to specify the prompt that is being used by SelfQueryRetriever. This can be helpful to provide additional context to the LLM in order to construct a proper query.

### Motivation

For my use case, the default prompt sent to LLM to generate a query doesn't work. Although the question I am asking the LLM is good, it is not able to figure the correct filter it should use. I think if I was able to modify the query, it could solve the issue and start returning correct queries.

### Your contribution

Not really.
IndexError: pop from empty list - bug in ConversationTokenBufferMemory
Labels: name: area: memory, name: auto:bug

### System Info

Langchain version: 0.0.312
Python: 3.10.11
Platform: Ubuntu

### Who can help?

@hwchase17 @agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.memory import ConversationTokenBufferMemory

max_token_limit = 0
memory = ConversationTokenBufferMemory(
    llm=llm,  # Azure OpenAI GPT3.5
    max_token_limit=max_token_limit,
    return_messages=True,
)
context_pair = {
    "inputs": {"input": "Example input message."},
    "outputs": {"outputs": "Example output message."}
}

memory.save_context(**context_pair)
```

### Expected behavior

The above code produces an `IndexError: pop from empty list`

Due to some token budgeting we're doing, sometimes the space for the memory gets reduced to 0 (specifically when a user has a long message, we prioritize tokens for their message and the AI response). So we create memory with `max_token_limit=0`, and **ideally we'd like the memory to be pruned such that the buffer is empty.**  However, we instead get the above error.

### Where the bug seems to be
The culprit seems to be in `memory.save_context()` , which we call directly in some cases or sometimes it happens internally like:

```
chat_chain = ConversationChain(
    memory=memory, llm=llm, prompt=prompt_template
)
ai_response = chat_chain.predict(input=user_message)  # internal call to save_context()
```

Within `save_context`, there's a loop:

```
while curr_buffer_length > self.max_token_limit:
    pruned_memory.append(buffer.pop(0))
    curr_buffer_length = self.llm.get_num_tokens_from_messages(buffer)
```

Oddly, when the buffer is an empty list, `curr_buffer_length` is 3 when it should be 0. So the loop continues and tries to pop from the empty buffer list and we get the above error.

![langchain_bug](https://github.com/langchain-ai/langchain/assets/67058165/7c76afc1-56eb-4497-ae27-3e8ae8d4ddf3)
TypeError: BaseChatModel.__call__() missing 1 required positional argument: 'messages'
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I am following the notebook in this github to create two agents with tools:

https://github.com/langchain-ai/langchain/blob/master/cookbook/two_agent_debate_tools.ipynb

I am also using AzureOpenAI instead of OpenAI directly because of corporate firewalls. I am getting this error:


TypeError                                 Traceback (most recent call last)
Cell In[25], line 34
     24     agent_description = AzureChatOpenAI(
     25                             openai_api_base=BASE_URL,
     26                             openai_api_version="2023-05-15",
   (...)
     29                             openai_api_type="azure",
     30                         )(temperature=1.0)(agent_specifier_prompt).content
     31     return agent_description
---> 34 agent_descriptions = {name: generate_agent_description(name) for name in names}

Cell In[25], line 34, in (.0)
     24     agent_description = AzureChatOpenAI(
     25                             openai_api_base=BASE_URL,
     26                             openai_api_version="2023-05-15",
   (...)
     29                             openai_api_type="azure",
     30                         )(temperature=1.0)(agent_specifier_prompt).content
     31     return agent_description
---> 34 agent_descriptions = {name: generate_agent_description(name) for name in names}

Cell In[25], line 24, in generate_agent_description(name)
     13 def generate_agent_description(name):
     14     agent_specifier_prompt = [
...
     29                             openai_api_type="azure",
     30                         )(temperature=1.0)(agent_specifier_prompt).content
     31     return agent_description

TypeError: BaseChatModel.__call__() missing 1 required positional argument: 'messages'



### Suggestion:

_No response_
async SQLDatabaseChain 
Labels: name: auto:enhancement

### Feature request

Hello everyone!

Please when will we have SQLDatabaseChain async?

I believe it's crucial for all Chains to be async so that we can have a serverless service serving multiple clients.

Thank you!

### Motivation

I believe it's crucial for all Chains to be async so that we can have a serverless service serving multiple clients.


### Your contribution

I dont know how to contribute. Maybe I coult test your code.
feat: serialize momento vector index  metadata values to json
Labels: name: area: vector store, name: auto:improvement

**Description**:

To support a wider range of expected metadata values, we now serialize these values to JSON when storing in Momento Vector Index. Specifically, in the add method, values are dumped as JSON strings using `json.dumps`. During the search operation, we deserialize them using `json.loads`.

**Testing**:

We updated the integration tests to exercise this new behavior.

**Maintainers**:

@baskaryan who reviewed the original MVI PR.

**Twitter handle:**

@momentohq for Momento Vector Index vector store introduction
Comprehend Moderation 0.2
Labels: name: auto:improvement, name: area: models, name: integration: aws

This PR replaces the previous `Intent` check with the new `Prompt Safety` check. The logic and steps to enable chain moderation via the Amazon Comprehend service, allowing you to detect and redact PII, Toxic, and Prompt Safety information in the LLM prompt or answer remains unchanged.
This implementation updates the code and configuration types with respect to `Prompt Safety`.


### Usage sample

```python
from langchain_experimental.comprehend_moderation import (BaseModerationConfig, 
                                 ModerationPromptSafetyConfig, 
                                 ModerationPiiConfig, 
                                 ModerationToxicityConfig
)

pii_config = ModerationPiiConfig(
    labels=["SSN"],
    redact=True,
    mask_character="X"
)

toxicity_config = ModerationToxicityConfig(
    threshold=0.5
)

prompt_safety_config = ModerationPromptSafetyConfig(
    threshold=0.5
)

moderation_config = BaseModerationConfig(
    filters=[pii_config, toxicity_config, prompt_safety_config]
)

comp_moderation_with_config = AmazonComprehendModerationChain(
    moderation_config=moderation_config, #specify the configuration
    client=comprehend_client,            #optionally pass the Boto3 Client
    verbose=True
)

template = """Question: {question}

Answer:"""

prompt = PromptTemplate(template=template, input_variables=["question"])

responses = [
    "Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like 323-22-9980. John Doe's phone number is (999)253-9876.", 
    "Final Answer: This is a really shitty way of constructing a birdhouse. This is fucking insane to think that any birds would actually create their motherfucking nests here."
]
llm = FakeListLLM(responses=responses)

llm_chain = LLMChain(prompt=prompt, llm=llm)

chain = ( 
    prompt 
    | comp_moderation_with_config 
    | {llm_chain.input_keys[0]: lambda x: x['output'] }  
    | llm_chain 
    | { "input": lambda x: x['text'] } 
    | comp_moderation_with_config 
)

try:
    response = chain.invoke({"question": "A sample SSN number looks like this 123-456-7890. Can you give me some more samples?"})
except Exception as e:
    print(str(e))
else:
    print(response['output'])

```

### Output

```python
> Entering new AmazonComprehendModerationChain chain...
Running AmazonComprehendModerationChain...
Running pii Validation...
Running toxicity Validation...
Running prompt safety Validation...

> Finished chain.


> Entering new AmazonComprehendModerationChain chain...
Running AmazonComprehendModerationChain...
Running pii Validation...
Running toxicity Validation...
Running prompt safety Validation...

> Finished chain.
Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like XXXXXXXXXXXX John Doe's phone number is (999)253-9876.
```
Add support for themoviedb querying to get movie info
Labels: name: auto:enhancement, name: area: models

### Feature request

The proposal is to enhance our Langchain by integrating support for querying The Movie Database (TMDb) using a Tool to retrieve detailed information about movies. 

[https://developer.themoviedb.org/docs/getting-started](url)
[https://github.com/celiao/tmdbsimple/](url)

### Motivation

Allowing users to query The Movie Database (TMDb) would allow users to ask information regarding moives, and this tool will give the llms better/more reliable information to work with.

### Your contribution

We intend to submit a pull request for this issue at some point in November for a school project.
General anthropic functions, steps towards experimental integration tests
Labels: name: lgtm, name: auto:improvement, name: area: models

To match change in js here https://github.com/langchain-ai/langchainjs/pull/2892

Some integration tests need a bit more work in experimental:
![Screenshot 2023-10-12 at 12 02 49 PM](https://github.com/langchain-ai/langchain/assets/9557659/262d7d22-c405-40e9-afef-669e8d585307)

Pretty sure the sqldatabase ones are an actual regression or change in interface because it's returning a placeholder.
Update Documentation: Corrected Typos and Improved Clarity
Labels: name: area: agent, name: auto:documentation

Commit Description:
In this update, several documentation sections were improved by correcting typos and enhancing overall clarity. The changes made include fixing spelling and grammar issues, refining sentence structures, and providing more informative content where necessary. These improvements aim to make the documentation more user-friendly and easier to understand.

Files changed for better clarity of text and typo fix : - 
langchain\docs\docs\modules\agents\tools\index.mdx
langchain\docs\docs\modules\agents\toolkits\index.mdx
langchain\docs\docs\modules\agents\how_to\mrkl.mdx
langchain\docs\docs\modules\agents\how_to\custom_llm_chat_agent.mdx
langchain\docs\docs\modules\agents\how_to\custom_llm_agent.mdx
langchain\docs\docs\modules\agents\agent_types\index.mdx

Some examples of the fix : - 
1. Added "For documentation on built-in toolkit integrations, visit" to the info box.
Corrected "convenience" to "convenient" for proper adjective usage.

2. Added "For documentation on built-in tool integrations, visit" to the info box.
Changed "Get started" to "Getting Started" for consistency.
Made the sentence structure in the "Getting Started" section more consistent and clear.
Capitalized "Getting Started" for section title consistency.
Clarified the loading of tools by rephrasing the sentence.

3. Added "This example uses" for clarity.
Added commas after "MRKL system" and "Chinook database" for better readability.
Clarified the setup instructions by breaking the sentence into two parts.
Capitalized "Using a Chat Model" for consistency.

4. "goes through" changed to "explains."
Added periods at the end of each bullet point for consistency.
Capitalized the "Agent" in "Custom LLM Chat Agent."
Added the word "components" to clarify the list.
Corrected the numbering of components.

5. "goes through" changed to "explains"
Added periods at the end of each bullet point for consistency.
Capitalized the "Agent" in "Custom LLM Agent."
Added the word "components" to clarify the list.
Made some minor wording adjustments for clarity and consistency.
Add `Memorize` tool
Labels: name: needs documentation, name: auto:enhancement, name: area: models

  - **Description:** Add `Memorize` tool
  - **Tag maintainer:** @hwchase17

This PR added a new tool `Memorize` so that an agent can use it to fine-tune itself. This tool requires `TrainableLLM` introduced in #11721

DEMO: https://github.com/Preemo-Inc/gradient-langchain-tool-demo/tree/6a9003d5db8db03673ad8b205f4d00cf0d96b56b
![image](https://github.com/langchain-ai/langchain/assets/601530/d6f0cb45-54df-4dcf-b143-f8aefb1e76e3)
Issue: <VectorStoreRetrieverMemory class doesn't support AzureSearchVectorStoreRetriever retriever>
Labels: name: area: memory, name: area: vector store, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

![image](https://github.com/langchain-ai/langchain/assets/142630126/63a7c9c3-ac95-4145-9342-838e8a9b0ba1)

The retriever object in the VectorStoreRetrieverMemory class is a VectorStoreRetriever and not AzureSearchVectorStoreRetriever. 

The VectorStoreRetriever and AzureSearchVectorStoreRetriever are two different classes and doesn't support same methods.

As i want to use vector store backed memory retriever which has option for filtering with user_id metadata to maintain mutliple chat histories based on user. I require help in two things.

1. Support for AzureSearchVectorStoreRetriever object in VectorStoreRetrieverMemory 
2. How to initialize the memory object that accepts azure vector store (with id, user_id, conversation, conversation_vector as fields in my case), I want to save_context with respect to user_id, the following example given in the
[](https://python.langchain.com/docs/modules/memory/types/vectorstore_retriever_memory)

memory.save_context({"input": "My favorite food is pizza"}, {"output": "that's good to know"}) doesn't say how.

similarly for the memory.load_memory_variables({"prompt": "what sport should i watch?"})["history"]

### Suggestion:

As most organizations use cases would be at scale, examples that include how to maintain memory for different users (user_id being metadata) would help.

Add AzureSearchVectorStoreRetriever object to the VectorStoreRetrieverMemory class would also help.

If i am wrong in the above cases, please correct me and help me pointing out the resources to solve this.
Add real support for RetryParsers?
Labels: name: auto:improvement, name: area: models

For retry parser to work, the prompt needs to be available. I don't see how this is hooked up in the current implementation of  LLMChain. The LLMChain must provide the prompt and the corresponding generation to the appropriate output parser (I.e. the llm code must invoke parse_with_prompt). The real output parser could either use the base implementation of parse_with_prompt (i.e. simply call parse) or override it.

Don't think schema changes affects consumers(don't think a lot of people are subclassing output_parser). 

Added test cases.