delayedkarma/langchain-issues huggingface dataset:
fix(cache): use dumps for RedisCache
Labels: name: auto:bug

# Description
Attempts to fix RedisCache for ChatGenerations using `loads` and `dumps`  used in SQLAlchemy cache by @hwchase17 . this is better than pickle dump, because this won't execute any arbitrary code during de-serialisation. 

# Issues
#7722 & #8666 

# Dependencies
None, but removes the warning introduced in #8041 by @baskaryan

Handle: @jaikanthjay46
How to use human input as a tool in wen based application
Labels: name: area: agent, name: auto:question, name: auto:improvement, name: area: models

### System Info

In current tool the input from the user is taken using CMD(Command line), but how will use  in case of web application?

### Who can help?

@hwchase17 @agola11 @eyurtsev 

### Information

- [x] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

You can reproduce by using current code given in human as a tool.

### Expected behavior

It should not take human input from CMD, as it cause issue in web based application.
textgen-silence-output-feature in terminal
Labels: name: lgtm, name: auto:improvement, name: area: models

Hello,
Added the new feature to silence TextGen's output in the terminal.

  - Description: Added a new feature to control printing of TextGen's output to the terminal., 
  - Issue: the issue #TextGen parameter to silence the print in terminal #10337  it fixes (if applicable)
  
  Thanks;
New LLM integration: Ctranslate2
Labels: name: auto:enhancement, name: area: models

## Description:

I've integrated CTranslate2 with LangChain. CTranlate2 is a recently popular library for efficient inference with Transformer models that compares favorably to alternatives such as HF Text Generation Inference and vLLM in [benchmarks](https://hamel.dev/notes/llm/inference/03_inference.html).
Supported custom ernie_api_base & Implemented asynchronous for ErnieEmbeddings
Labels: name: area: embeddings, name: auto:improvement

Description: Supported custom ernie_api_base & Implemented asynchronous for ErnieEmbeddings
 - ernie_api_base：Support Ernie Service custom endpoints
 - Support asynchronous 

Issue: None
Dependencies: None
Tag maintainer:
Twitter handle: @JohnMai95
How to use my fine-tuned Llama2 model in Langchain?
Labels: name: auto:question, name: area: models

### Feature request

Any possible ways to run a Q&A bot for my fine-tuned Llama2 model in Google Colab?

### Motivation

Any possible ways to run a Q&A bot for my fine-tuned Llama2 model in Google Colab?

### Your contribution

Any possible ways to run a Q&A bot for my fine-tuned Llama2 model in Google Colab?
Amazon Bedrock Support Streaming
Labels: name: auto:enhancement, name: area: models

### Description

- Add support for streaming with `Bedrock` LLM and `BedrockChat` Chat Model.
- Bedrock as of now supports streaming for the `anthropic.claude-*` and `amazon.titan-*` models only, hence support for those have been built.
- Also increased the default `max_token_to_sample` for Bedrock `anthropic` model provider to `256` from `50` to keep in line with the `Anthropic` defaults.  
- Added examples for streaming responses to the bedrock example notebooks.

**_NOTE:_**: This PR fixes the issues mentioned in #9897 and makes that PR redundant.
Feature bedrock streaming
Labels: name: auto:enhancement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Issue with loading xxx_prompts.yaml file specified under "examples" section in the .yaml file passed as parameter in load_prompt()
Labels: name: area: doc loader, name: auto:bug

### System Info

Langchain version: 0.0.272
Python version: 3.10
Host System: Windows 11
I'm loading Few Shot Prompts from a fewshot_prompts.yaml file by using load_prompt() function. The fewshot_prompts.yaml file has a section with the title "examples:" to load the Few Shot Prompts from file example_prompts.yaml.  The files fewshot_prompts.yaml and example_prompts.yaml both are in the same directory. But the _load_examples() function is not able to locate/load the example_prompts.yaml. There is no way to specify the path to this file.
Due to the above issue, the loading of the example_prompts.yaml file fails.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The 
_type: few_shot
input_variables:
  ["bot_response"]
prefix: 
  The following are excerpts from conversations with an AI assistant. Given an input text and a set of rules,the assistant strictly follows the rules and provides an "yes" or "no" answer.Here are some examples
example_prompt:
  _type: prompt
  input_variables:
    ["bot_response","answer"]
  template:
    "bot_response: {bot_response}\nanswer: {answer}"
examples:
    example_prompts.yaml
**************************************************************************
Unable to find the file example_prompts.yaml


### Expected behavior

1. Provide a way to specify the path to load the example_prompts.yaml file
Latest version of HazyResearch/manifest doesn't support accessing "client" directly
Labels: name: lgtm, name: auto:improvement, name: area: models

**Description:** 
The latest version of HazyResearch/manifest doesn't support accessing the "client" directly. The latest version supports connection pools and a client has to be requested from the client pool. 
**Issue:**
No matching issue was found
**Dependencies:** 
The manifest.ipynb file in docs/extras/integrations/llms need to be updated
**Tag maintainer:** 
**Twitter handle:** 
@hrk_cbe
Add Polars Dataframe Agent to langchain_experimental
Labels: name: area: agent, name: auto:enhancement

## Description
Added the Polars dataframe agent.

A new PR after https://github.com/langchain-ai/langchain/pull/10291.
This PR was added the task of moving the Pandas DataFrame to langchain_experimental.

Test code added to the use_case.
I'm not sure if I should delete the Pandas Dataframe Agent from the LangChain due to backward compatibility, so if it's okay to delete it, please do :)

## Issue
https://github.com/langchain-ai/langchain/issues/4620

## Dependencies
polars (https://pola-rs.github.io/polars-book/user-guide/)

## Tag maintainer
@baskaryan 

## Twitter handle
N/A
fix(DynamoDBChatMessageHistory): correct delete_item method call
Labels: name: area: memory, name: auto:bug

**Description**: 
Fixed a bug introduced in version 0.0.281 in `DynamoDBChatMessageHistory` where `self.table.delete_item(self.key)` produced a TypeError: `TypeError: delete_item() only accepts keyword arguments`. Updated the method call to `self.table.delete_item(Key=self.key)` to resolve this issue.

Please see also [the official AWS documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/table/delete_item.html#) on this **delete_item** method - only `**kwargs` are accepted.

See also the PR, which introduced this bug: https://github.com/langchain-ai/langchain/pull/9896#discussion_r1317899073

Please merge this, I rely on this delete dynamodb item functionality (because of GDPR considerations).

**Dependencies**: 
None

**Tag maintainer**: 
@hwchase17 @joshualwhite 

**Twitter handle**: 
[@BenjaminLinnik](https://twitter.com/BenjaminLinnik)

- [x] Linting (`make lint`)
- [x] Formatting (`make format`)
- [x] Tests (`make test`)
Resolves issue DOC: Incorrect and confusing documentation of AIMessag…
Labels: name: lgtm, name: auto:documentation

Resolves issue DOC: Incorrect and confusing documentation of AIMessagePromptTemplate and HumanMessagePromptTemplate #10378

- Description: Revised docstrings to correctly and clearly document each PromptTemplate
- Issue: #10378
- Dependencies: N/A
- Tag maintainer: @baskaryan
DOC: Incorrect and confusing documentation of AIMessagePromptTemplate and HumanMessagePromptTemplate
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

[AIMessagePromptTemplate documentation](https://api.python.langchain.com/en/latest/prompts/langchain.prompts.chat.AIMessagePromptTemplate.html#langchain-prompts-chat-aimessageprompttemplate) incorrectly and confusingly describes the message as "... This is a message that is not sent to the user."

[HumanMessagePromptTemplate documentation](https://api.python.langchain.com/en/latest/prompts/langchain.prompts.chat.HumanMessagePromptTemplate.html#langchain-prompts-chat-humanmessageprompttemplate) incorrectly and confusingly describes the message as "... This is a message that is sent to the user."

Compare to the documentation for [AIMessage](https://api.python.langchain.com/en/latest/schema/langchain.schema.messages.AIMessage.html#langchain-schema-messages-aimessage) and [HumanMessage](https://api.python.langchain.com/en/latest/schema/langchain.schema.messages.HumanMessage.html#langchain-schema-messages-humanmessage), which correctly and clearly describe each message as "A message from an AI" and "A message from a human." respectively.

### Idea or request for content:

AIMessagePromptTemplate should be described as "AI message prompt template. This is a message that is sent to the user from the AI."

HumanMessagePromptTemplate should be described as "Human message prompt template. This is a message that is sent from the user to the AI."

These are clear, concise and consistent with documentation of the message schema.

I will submit a PR with revised docstrings for each class.  This should, then, be reflected in the API reference documentation upon next build.
Couldn't import BaseMessageConverter from 
Labels: name: area: memory, name: auto:bug

### System Info

Python 3.10.8 running on mac mini in VS Code

### Who can help?

@asai95 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

attempt to import basemessage converter following customer storage guide here - https://python.langchain.com/docs/integrations/memory/sql_chat_message_history

Enter from langchain.memory.chat_message_histories.sql import BaseMessageConverter
Actual behavior - ImportError: cannot import name 'BaseMessageConverter' from 'langchain.memory.chat_message_histories.sql' (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/memory/chat_message_histories/sql.py)

### Expected behavior

Being able to import the BaseMessageConverter
Add `Runnable.astream_log()`
Labels: name: area: agent, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Why did you broke Redis completely in a minor version change
Labels: name: area: vector store, name: auto:bug

### System Info

Hi there :wave: ,

I am wondering why now redis is completely broken (most of the paramters names have been changed) and you did this is a minor version change - doesn't make any sense - and I'd like to know how to pass `k` to the retriever now and how to get the `k`; previous I could just do `retriever.k`

Thanks,

Fra

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just try to use redis now

### Expected behavior

I wouldn't except BREAKING changes from a minor version change
Add Language Detection
Labels: name: auto:enhancement

**!! Should wait for https://github.com/langchain-ai/langchain/pull/10181 to be merged, so that the example with TTS component is added !!**

### Description

Adds language detection component based on [langdetect](https://github.com/Mimino666/langdetect/tree/master/langdetect) library. It implements naive Bayesian filter and is able to recognize >50 languages. This module can be especially useful together with components that require selection of the language (e.g. text-2-speech or data-anonymizer)

### Twitter handle

@deepsense_ai, @matt_wosinski
fix ChatMessageChunk concat error
Labels: name: auto:bug

  - Description: fix ChatMessageChunk '+' error, add __add__  method for ChatMessageChunk
  - Issue: ChatMessageChunk has required field "role", however it's use BaseMessageChunk.__add__method which do not handle role field
  ```python
      # BaseMessageChunk
      def __add__(self, other: Any) -> BaseMessageChunk:  # type: ignore
        if isinstance(other, BaseMessageChunk):
            # If both are (subclasses of) BaseMessageChunk,
            # concat into a single BaseMessageChunk
            
           # this line will raise error for role not set
            return self.__class__(
                content=self.content + other.content,
                additional_kwargs=self._merge_kwargs_dict(
                    self.additional_kwargs, other.additional_kwargs
                ),
            )
        else:
            raise TypeError(
                'unsupported operand type(s) for +: "'
                f"{self.__class__.__name__}"
                f'" and "{other.__class__.__name__}"'
            )
  ````
DOC: What is the Difference between OpenAIAgent and agent=AgentType.OPENAI_FUNCTIONS
Labels: name: area: agent, name: auto:question, name: auto:documentation

### Issue with current documentation:

Overall there are so many terms on top of each other and I do understand we are trading off growing fast to having clean, set terms but lets discuss about currently what is the common way to define an AGENT that uses openAI for reasoning engine, and answers questions using tools that we define.

What is the AgentExecutor if we can also run agent.run()
What is AgentExecutorIterator

We can define an agent like this:
```
agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS,
                         agent_kwargs=agent_kwargs, verbose=True, return_intermediate_steps=True,)
```

But also we can define an agent like this: [(From this documentation: Custom LLM Agent)](https://python.langchain.com/docs/modules/agents/how_to/custom_llm_agent)
```
llm = OpenAI(temperature=0)
llm_chain = LLMChain(llm=llm, prompt=prompt)

agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
```
There are also ChatAgent, OpenAIAgent and more[ Classes](https://js.langchain.com/docs/api/agents/classes/)


I would like to have a custom agent that I use with openai api and I will give custom tools like weather  functions or finance functions etc. What is the cleanest way to do this currently ? (8 September 2023)



### Idea or request for content:

_No response_
Modified chain datatype to Chain to support for ConversationalRetrievalChain, etc
Labels: name: lgtm, name: auto:improvement, name: area: models

 **Description:**
In the class ConstitutionalChain currently supports LLMChain when using from_llm.
When we try to use it with other type of chains such as ConversationalRetrievalChain whose superclass isn't LLMChain it gives a type error.
As both the class's superclass was Chain, when replaced with it, it worked as supposed to.

**Issue:**
Found it to be relevant to
[https://github.com/langchain-ai/langchain/issues/5881](https://github.com/langchain-ai/langchain/issues/5881)
[https://github.com/langchain-ai/langchain/issues/1904](https://github.com/langchain-ai/langchain/issues/1904)

**Dependencies:**
None

**Tag maintainer:**
@wnmurphy
@hwchase17 

**Twitter handle:**
Linkdin: www.linkedin.com/in/prar

**How I used it**
```
 qa_chain = ConversationalRetrievalChain.from_llm(llm, db.as_retriever(), 
                        memory=memory, condense_question_prompt=qa_prompt)

constitutional_chain = ConstitutionalChain.from_llm(
            llm=llm,
            chain=qa_chain,
            constitutional_principles=list(principles),
        )

```
pydantic 2.0 support
Labels: name: auto:improvement

### Feature request

Please update dependencies to support installing `pydantic>=2.0. Now there's a conflict, for example pip resolver gives following error:

```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
langchainplus-sdk 0.0.20 requires pydantic<2,>=1, but you have pydantic 2.3.0 which is incompatible.
langchain 0.0.228 requires pydantic<2,>=1, but you have pydantic 2.3.0 which is incompatible.
```

### Motivation

pydantic 1.* is outdated. Starting new project and use old syntax which will be deprecated soon is unpleasant

### Your contribution

-
ENH: Add `llm_kwargs` for Xinference LLMs
Labels: name: auto:enhancement, name: area: models

- This pr adds `llm_kwargs` to the initialization of Xinference LLMs (integrated in #8171 ). 
- With this enhancement, users can not only provide `generate_configs` when calling the llms for generation but also during the initialization process. This allows users to include custom configurations when utilizing LangChain features like LLMChain.
- It also fixes some format issues for the docstrings.
ConversationalRetrievalChain is not adding chat_history to new message
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I am using ConversationalRetrievalChain for document Q/A bot and updating chat_history = []  with every message however I noticed this chat_history string is never added to final inference string. In _call method of BaseConversationalRetrievalChain class, even when  "if chat_history_str:" condition is true, new_question is never updated with the chat_history_str.

### Suggestion:

_No response_
quip doc loader 
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

like https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/document_loaders/confluence.py, we need a loader for quip document, please follow https://github.com/quip/quip-api/tree/master/python

### Motivation

I have a lot of documents in quip, want to load these quip documents. 

### Your contribution

I like to contribute this idea if no one to take.
Feature/add deepeval
Labels: name: auto:enhancement, name: area: models

Description: Adding `DeepEval` - which provides an opinionated framework for testing and evaluating LLMs
Issue: Missing Deepeval
Dependencies: Optional DeepEval dependency
Tag maintainer: @baskaryan   (not 100% sure)
Twitter handle: https://twitter.com/ColabDog 

 ✅ `make format`
 ✅`make lint` and 
 ✅`make test`

If you're adding a new integration, please include:
 ✅ a test for the integration, preferably unit tests that do not rely on network access, 
 ✅ an example notebook showing its use. These live is docs/extras directory.
Stop generating undesired tokens from GPT4All
Labels: name: auto:bug, name: area: models

@baskaryan  In this PR, I made a small modification in gpt4all.py in the project folder libs/langchain, more precisely in the method `_call`. I added a unit test for the additional function `remove_stop_words` in libs/lanchain/llms/utils.py

It fixes the issue https://github.com/langchain-ai/langchain/issues/10345 

There is no additional packages.

By executing  `make format`, `make lint` and `make test` locally, it works.
Issue: GPT4All LLM continue to generate undesired tokens even if stop attribute has been specified
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I was led to use GPT4All for building a personal chatbot. It generated undesired output. More precisely, it generates the all conversation, after writing the first prompt. For solving this problem, I instantiated GPT4All with a not empty list value for attribute `stop` : 

`
llm = GPT4All(
    model=local_path,
    callbacks=callbacks,
    verbose=True,
    streaming=True,
    stop=["System:"],
)
`

However, it continues to generate these undesired output. 

### Suggestion:

Finally I may have found a small fix to solve this problem for this type of LLM. I intent to create a PR very soon, if you agree with this.
Fix colab link(missing graph in url) and comment to match the code fo…
Labels: name: area: doc loader, name: auto:documentation

  - Description: Fixing Colab broken link and comment correction to align with the code that uses Warren Buffet for wiki query 
  - Issue: None open
  - Dependencies: none
  - Tag maintainer: n/a
  - Twitter handle: Not a PR change but: kcocco