delayedkarma/langchain-issues huggingface dataset:
add streaming to huggingfacepipeline models
Labels: name: needs test, name: needs work, name: auto:improvement, name: area: models, name: size:M

**Description:** 
This PR includes modifications to the `HuggingFacePipeline` class in the `langchain` library. Changes include the removal of unused imports, a minor cleanup in the code for better readability, and the addition of an `_astream` method for asynchronous streaming support. The `_astream` method integrates the `TextIteratorStreamer` from the `transformers` library and ensures inputs and the model are on the same device (cuda). Additionally, a print statement has been added in the generation loop for debugging purposes.

**Issue:** 
N/A

**Dependencies:** 
- `transformers` library

**Tag maintainer:**
@baskaryan, @eyurtsev, @hwchase17 for review.
Bagatur/lcel why doc
Labels: name: area: doc loader, name: auto:documentation, name: size:XXL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Pydantic error field required (type=value_error.missing) in LLMChainExtractor or ContextualCompressionRetriever
Labels: name: area: vector store, name: auto:bug, name: area: models

### System Info

langchain          0.0.343
langchain-core     0.0.7


### Who can help?

_No response_
### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
retriever = AzureCognitiveSearchRetriever(content_key="content", top_k=5)

llm = AzureChatOpenAI(
    openai_api_version="2023-05-15",
    azure_deployment="larger-chat-35",
)

template = """
I want you to act as a research assistant. you will answer questions about the context.

Context: {context}
Question: {question}?
"""

prompt = PromptTemplate(
    input_variables=["question", "context"],
    template=template,
)

compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(base_compressors=compressor,
                                                         base_retriever=retriever)
```

### Expected behavior

I'd expect the compression_retriever to be instantiated. But instead I receive the following error:

```
pydantic.v1.error_wrappers.ValidationError: 1 validation error for ContextualCompressionRetriever
base_compressor
  field required (type=value_error.missing) 
```
RunnableWithFallbacks should have a parameter to choose whether to raise first error or last error
Labels: name: auto:improvement

### Feature request

The `invoke()` method for RunnableWithFallbacks is only raising the first_error caught in the runnables. I would like to have an option to choose whether to raise the first error or last error.
Can this be a parameter for `invoke()`?

### Motivation

When using fallbacks, the first error could be related to the first Runnable. However, the error I'm actually interested to handle is the last error from the end of my fallbacks.

The last error is the most downstream error from my chains, and it's the error I want to handle in my app business logic.

### Your contribution

N/A
`BaseOutputParser` breaks pydantic `PrivateAttr`
Labels: name: auto:bug, name: area: models

### System Info

langchain== 0.0.340
pydantic==2.4.2
pydantic_core==2.10.1

### Who can help?

@hwchase17 @eyur

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.schema import BaseOutputParser
from pydantic import PrivateAttr

class MyParser(BaseOutputParser[list[int]]):
    _list: list[int] = PrivateAttr(default_factory=list)

    def parse(self, text: str) -> list[int]:
        return self._list

parser = MyParser()
assert parser.parse("test") == []
```

### Expected behavior

I expect `PrivateAttr` to still work within `BaseOutputParser`.
in AWS langchain Bedrock, how do I set temperature?
Labels: name: auto:question, name: area: models, name: integration: aws

### System Info

in AWS langchain Bedrock, how do I set temperature?

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Tried putting it as a parameter of Bedrock and gave error that such parameter does not exist. Then put is as paramenters of kwargs and it had no effect

### Expected behavior

Expected this to be a direct parameter of Bedrock, but it seems that it is not
Combine langchain retrievers
Labels: name: area: vector store, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

I am using ConversationalRetrievalChain. I have created two vector stores and I want the query from ConversationalRetrievalChain to be compared against both vector stores and results from both vector stores to be used to create the final answer.

So I have decided to create two retrievers 

retriever1 = vectorstore1.as_retriever()
retriever2 = vectorstore2.as_retriever()

How can I now override the Retriever class so that when the query is compared against my custom_retriever, it is compared against documents from both retrievers, and documents from both retrievers are used to create the prompt.

Note: i don't want to merge the vectorstores because that messes up the similarity search.


### Suggestion:

_No response_
Random Seed when using LLM Agent
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

Mac OS

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When I am using Agent, random seed doesn't work.

model_kwargs = {
    "seed": 2139
}

llm_agent = ChatOpenAI(temperature=0, 
                 openai_api_key='sk-',
                 model_name='gpt-4-1106-preview', 
                 model_kwargs=model_kwargs)

llm_cypher = ChatOpenAI(temperature=0, 
                 openai_api_key='sk-',
                 model_name='gpt-4-1106-preview', 
                 model_kwargs=model_kwargs)

llm_graph = ChatOpenAI(temperature=0, 
                 openai_api_key='sk-',
                 model_name='gpt-4-1106-preview', 
                 model_kwargs=model_kwargs)

tools = [
    Tool(
        name="GraphDB Search tool",
        func=chain,
        description=tool_description
    )
] 

 chain = CustomGraphCypherQAChain.from_llm(
    top_k=100,
    llm=llm_graph, 
    return_direct=True, # this return the observation from cypher query directly without rethinking over observation
    return_intermediate_steps=True, # this is only returned when it's call by chain() not by chain.run()
    cypher_llm=llm_cypher,
    graph=graph, 
    verbose=True,
    cypher_prompt=CYPHER_GENERATION_PROMPT
)


def init_agent():
    agent = initialize_agent(
    tools,
    llm_agent,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, 
    verbose=True, 
    handle_errors=True, 
    handle_parsing_errors=True,
    return_intermediate_steps=True,
    max_iterations=10,
    )
    return agent

this agent don't reproduce for same input

### Expected behavior

I expect Agent keep resulting in same answer for the same input prompt
Support for Perplexity's PPLX models: `pplx-7b-online` and `pplx-70b-online`
Labels: name: auto:enhancement, name: area: models

### Feature request

Support for Perplexity's new [PPLX models](https://blog.perplexity.ai/blog/introducing-pplx-online-llms).

### Motivation

Boosting online LLM support, particularly for Perplexity-based models, would be highly impactful and useful.

### Your contribution

Currently working on a PR!
Fix: (issue #14066) DOC: Summarization output broken
Labels: name: lgtm, name: auto:documentation, name: area: models, name: size:XS

  - **Description:** : As described in the issue below, 
https://python.langchain.com/docs/use_cases/summarization  
I've modified the Python code in the above notebook to perform well. 

I also modified the OpenAI LLM model to the latest version as shown below. 
`gpt-3.5-turbo-16k --> gpt-3.5-turbo-1106`
This is because it seems to be a bit more responsive.
  - **Issue:** : #14066
MyCallbackHandler can't show logs explicitly
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Hello everyone,

We are having request timeout on our calls and they are being getting retries after failing but we want to see how many times doing these retries and all their logs.

When the requests are timing out, nothing has been sending callbackhandler and we want to see how many retries had been done under the ChatOpenAI LLM model.

### Suggestion:

_No response_
Dependency Issues with sentence-transformers and chromadb
Labels: name: area: embeddings, name: area: doc loader, name: area: vector store, name: auto:bug, name: integration: chroma

### System Info

LangChain Version: 0.0.339
ChromaDB 0.4.18
sentence-transformers 2.2.1
Python 3.12.0


### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Create a requirements file with these entries:
langchain
chromadb
sentence-transformers

Run pip install -r requirements.txt

### Expected behavior

I expect the install to complete correctly.
Chat model for Llamacpp runtime
Labels: name: needs documentation, name: auto:enhancement, name: area: models, name: size:XL

  **Description:**
  LllamaCpp runtime has been supported by langchain for some time. However, it lacked appropriate chat model to go with it. This PR adds chat model built around llamacpp llm, using some of the common facilities of chat models.

**Issue:**
Lack of chat model to use with llamacpp runtime.

  **Dependencies:** 
  N/A
DOC: Summarization output broken
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

At end of [summarization documentation](https://python.langchain.com/docs/use_cases/summarization) the output is shown with a ValueError. Line that fail:
```python
summarize_document_chain.run(docs[0])
``` 

### Idea or request for content:

Looks like the docs[0] object is not what excepected.
Disable HTTP Request logging in Langchain
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

When using `langchain.chat_models` such as `ChatAnthropic` and `ChatOpenAI`, I am seeing a bunch of HTTP logs that look like the following:
`2023-11-30 16:43:09 httpx INFO: HTTP Request: POST https://api.anthropic.com/v1/complete "HTTP[/1.1] 200 OK"`

Any suggestions on how to disable them?

### Suggestion:

_No response_
How to have agent decide what param should be pass into a tool?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

I've created a custom tool to query the equity daily quote, I would like to have two parameters in the tool function: stock_name and trade_date, and I would expect  LLM Agent will automatically decide how many days quotes or which day's quote it should retrieve. 

```
class SearchSchema(BaseModel):
    stock_name: str = Field(description="should be the name of the equity.")
    trade_date: str = Field(description="should be the trading date of the equity.")


class DailyQuoteSearch(BaseTool):
    name = "equity daily quote search"
    description = "useful for equity daily quote retrieval"
    return_direct = False
    args_schema: Type[SearchSchema] = SearchSchema

    def _run(self, stock_name: str, trade_date: str) -> str:
        output = self.query_equity_daily_quote(stock_name, trade_date)
        return output
```

However, LLM doesn't seem to know how to pass in the trade date? Is it possible to have LLM think about the what  trade_date should be passed? or I expect too much on agent intelligence?
```
pydantic_core._pydantic_core.ValidationError: 1 validation error for SearchSchema
trade_date
  Field required [type=missing, input_value={'stock_name': 'xxxx'}, input_type=dict]
```

### Suggestion:

_No response_
Add openai v2 adapter
Labels: name: lgtm, name: needs documentation, name: auto:improvement, name: area: models, name: size:XL

### Description

Starting from [openai version 1.0.0](https://github.com/openai/openai-python/tree/17ac6779958b2b74999c634c4ea4c7b74906027a#module-level-client), the camel case form of `openai.ChatCompletion` is no longer supported and has been changed to lowercase `openai.chat.completions`. In addition, the returned object only accepts attribute access instead of index access:

```python
import openai

# optional; defaults to `os.environ['OPENAI_API_KEY']`
openai.api_key = '...'

# all client options can be configured just like the `OpenAI` instantiation counterpart
openai.base_url = "https://..."
openai.default_headers = {"x-foo": "true"}

completion = openai.chat.completions.create(
    model="gpt-4",
    messages=[
        {
            "role": "user",
            "content": "How do I output all files in a directory using Python?",
        },
    ],
)
print(completion.choices[0].message.content)
```

So I implemented a compatible adapter that supports both attribute access and index access:

```python
In [1]: from langchain.adapters import openai as lc_openai
   ...: messages = [{"role": "user", "content": "hi"}]

In [2]: result = lc_openai.chat.completions.create(
   ...:     messages=messages, model="gpt-3.5-turbo", temperature=0
   ...: )

In [3]: result.choices[0].message
Out[3]: {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}

In [4]: result["choices"][0]["message"]
Out[4]: {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}

In [5]: result = await lc_openai.chat.completions.acreate(
   ...:     messages=messages, model="gpt-3.5-turbo", temperature=0
   ...: )

In [6]: result.choices[0].message
Out[6]: {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}

In [7]: result["choices"][0]["message"]
Out[7]: {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}

In [8]: for rs in lc_openai.chat.completions.create(
    ...:     messages=messages, model="gpt-3.5-turbo", temperature=0, stream=True
    ...: ):
    ...:     print(rs.choices[0].delta)
    ...:     print(rs["choices"][0]["delta"])
    ...:
{'role': 'assistant', 'content': ''}
{'role': 'assistant', 'content': ''}
{'content': 'Hello'}
{'content': 'Hello'}
{'content': '!'}
{'content': '!'}

In [20]: async for rs in await lc_openai.chat.completions.acreate(
    ...:     messages=messages, model="gpt-3.5-turbo", temperature=0, stream=True
    ...: ):
    ...:     print(rs.choices[0].delta)
    ...:     print(rs["choices"][0]["delta"])
    ...:
{'role': 'assistant', 'content': ''}
{'role': 'assistant', 'content': ''}
{'content': 'Hello'}
{'content': 'Hello'}
{'content': '!'}
{'content': '!'}
...
```

### Twitter handle

[lin_bob57617](https://twitter.com/lin_bob57617)
Azure OpenAI, gpt-4 model returns gpt-35-turbo
Labels: name: auto:bug, name: area: models

### System Info

I am using gpt-4 deployed on AzureOpenAI.

I want to get the model used. From openai, I will get the model we used. But when I tried with langchain I got an older model.

Basically, I am integrating with other services. So it is essential to get the model name intact for the cost calculations.

What could be the issue with the langcahin? 
It works great with openai chat completion. 

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce:
```
import os
from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint = "{}",
    api_key="{}",
    api_version="{}",
    azure_deployment="gpt4",
)

response = client.chat.completions.create(
    model="gpt4",
    messages=[{"role": "user", "content": "tell me a joke"}]
)

response.model
```
> 'gpt-4'

```
from langchain.schema import HumanMessage
from langchain.chat_models.azure_openai import AzureChatOpenAI

llm = AzureChatOpenAI(
    azure_endpoint = "{}",
    openai_api_key="{}",
    openai_api_version="{}",
    azure_deployment="gpt4",
    model_version="0613",
    temperature=0
)

response = llm.generate(messages=[[HumanMessage(content="tell me a joke")]])
response.llm_output["model_name"]
```
> 'gpt-3.5-turbo'

```
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

prompt = PromptTemplate(
    input_variables=["joke"],
    template="Tell me a {joke}?",
)
chain = LLMChain(llm=llm, prompt=prompt)
response = chain.generate([{"joke":"joke"}])
response.llm_output["model_name"]
```
> 'gpt-3.5-turbo'

### Expected behavior

From all we should be getting gpt-4 as I have deployed only got-4 on azure openai
fix(vectorstores): incorrect import for mongodb atlas DriverInfo
Labels: name: lgtm, name: area: vector store, name: auto:bug, name: size:XS

  - **Description:** fix `import` issue for `mongodb atlas` vectore store integration
  - **Issue:** none
  - **Dependencies:** none

while trying to follow official `langchain`'s [mongodb integration guide](https://python.langchain.com/docs/integrations/vectorstores/mongodb_atlas), an import error will happen. 

It's caused by incorrect import location:
- `from pymongo import DriverInfo` should be `from pymongo.driver_info import DriverInfo`
- reference: [pymongo's DriverInfo class](https://pymongo.readthedocs.io/en/stable/api/pymongo/driver_info.html#pymongo.driver_info.DriverInfo)

Thanks!
Create Closed Captioning Chain for .mp4 videos
Labels: name: auto:enhancement, name: area: models, name: size:XXL

Description: Video imagery to text (Closed Captioning)
This pull request introduces the VideoCaptioningChain, a tool for automated video captioning. It processes audio and video to generate subtitles and closed captions, merging them into a single SRT output. 

Issue: https://github.com/langchain-ai/langchain/issues/11770
Dependencies: opencv-python, ffmpeg-python, assemblyai, transformers, pillow, torch, openai
Tag maintainer:
@baskaryan
@hwchase17


Hello!  We are a group of students from the University of Toronto (@LunarECL, @TomSadan, @nicoledroi1,  @A2113S)  that want to make a contribution to the LangChain community! We have ran make format, make lint and make test locally before submitting the PR. To our knowledge, our changes do not introduce any new errors.

Thank you for taking the time to review our PR!
Don't proceed with chatting until the user provides his name and phone number
Labels: name: auto:question, name: auto:improvement, name: area: models

### Issue you'd like to raise.

I want to customize a LLM (a GPT-3.5-turbo) for chatting but don't proceed with chatting until the user provides his name and his phone number and validates the count of his phone number.

Please help me I tried many methods but without any significant result

### Suggestion:

_No response_
Add unit tests for Huggingface dataset loader
Labels: name: lgtm, name: area: doc loader, name: auto:improvement, name: size:L

  - **Description:** Add unit tests for huggingface dataset loader and sample huggingface dataset for future tests. Updates dependencies for `datasets` module.
  - Adds coverage for [previous pull request](https://github.com/langchain-ai/langchain/pull/13864)
  - **Tag maintainer:** @hwchase17
Bagatur/lcel get started
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->