Repository: langchain
Issue: ChatHuggingFace + HuggingFaceEndpoint does not properly implement `max_new_tokens`
Labels: 🤖:bug

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in LangChain rather than my code.
- [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

### Example Code

```python
from transformers import AutoTokenizer
from langchain_huggingface import ChatHuggingFace
from langchain_huggingface import HuggingFaceEndpoint

import requests

sample = requests.get(
    "https://raw.githubusercontent.com/huggingface/blog/main/langchain.md"
).text


tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-70B-Instruct")


def n_tokens(text):
    return len(tokenizer(text)["input_ids"])


print(f"The number of tokens in the sample is {n_tokens(sample)}")

llm_10 = HuggingFaceEndpoint(
    repo_id="meta-llama/Meta-Llama-3-70B-Instruct",
    max_new_tokens=10,
    cache=False,
    seed=123,
)
llm_4096 = HuggingFaceEndpoint(
    repo_id="meta-llama/Meta-Llama-3-70B-Instruct",
    max_new_tokens=4096,
    cache=False,
    seed=123,
)

messages = [
    (
        "system",
        "You are a smart AI that has to describe a given text in to at least 1000 characters.",
    ),
    ("user", f"Summarize the following text:\n\n{sample}\n"),
]

# native endpoint
response_10_native = llm_10.invoke(messages)
print(f"Native response 10: {n_tokens(response_10_native)} tokens")
response_4096_native = llm_4096.invoke(messages)
print(f"Native response 4096: {n_tokens(response_4096_native)} tokens")

# make sure the native responses are different lengths
assert len(response_10_native) < len(
    response_4096_native
), f"Native response 10 should be shorter than native response 4096, 10 `max_new_tokens`: {n_tokens(response_10_native)}, 4096 `max_new_tokens`: {n_tokens(response_4096_native)}"

# chat implementation from langchain_huggingface
chat_model_10 = ChatHuggingFace(llm=llm_10)
chat_model_4096 = ChatHuggingFace(llm=llm_4096)

# chat implementation for 10 tokens
response_10 = chat_model_10.invoke(messages)
print(f"Response 10: {n_tokens(response_10.content)} tokens")
actual_response_tokens_10 = response_10.response_metadata.get(
    "token_usage"
).completion_tokens

print(
    f"Actual response 10: {actual_response_tokens_10} tokens (always 100 for some reason!)"
)

# chat implementation for 4096 tokens
response_4096 = chat_model_4096.invoke(messages)
print(f"Response 4096: {n_tokens(response_4096.content)} tokens")
actual_response_tokens_4096 = response_4096.response_metadata.get(
    "token_usage"
).completion_tokens

print(
    f"Actual response 4096: {actual_response_tokens_4096} tokens (always 100 for some reason!)"
)


# assert that the responses are different lengths, which fails because the token usage is always 100
print("-" * 20)
print(f"Output for 10 tokens: {response_10.content}")
print("-" * 20)
print(f"Output for 4096 tokens: {response_4096.content}")
print("-" * 20)
assert len(response_10.content) < len(
    response_4096.content
), f"Response 10 should be shorter than response 4096, 10 `max_new_tokens`: {n_tokens(response_10.content)}, 4096 `max_new_tokens`: {n_tokens(response_4096.content)}"
```

This is the output from the script:
```
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
The number of tokens in the sample is 1809
Native response 10: 11 tokens
Native response 4096: 445 tokens
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Response 10: 101 tokens
Actual response 10: 100 tokens (always 100 for some reason!)
Response 4096: 101 tokens
Actual response 4096: 100 tokens (always 100 for some reason!)

--------------------
Output for 10 tokens: The text announces the launch of a new partner package called `langchain_huggingface` in LangChain, jointly maintained by Hugging Face and LangChain. This package aims to bring the power of Hugging Face's latest developments into LangChain and keep it up-to-date. The package was created by the community, and by becoming a partner package, the time it takes to bring new features from Hugging Face's ecosystem to LangChain's users will be reduced.

The package integrates seamlessly with Lang
--------------------
Output for 4096 tokens: The text announces the launch of a new partner package called `langchain_huggingface` in LangChain, jointly maintained by Hugging Face and LangChain. This package aims to bring the power of Hugging Face's latest developments into LangChain and keep it up-to-date. The package was created by the community, and by becoming a partner package, the time it takes to bring new features from Hugging Face's ecosystem to LangChain's users will be reduced.

The package integrates seamlessly with Lang
--------------------
```


### Error Message and Stack Trace (if applicable)

AssertionError: Response 10 should be shorter than response 4096, 10 `max_new_tokens`: 101, 4096 `max_new_tokens`: 101

### Description

There seems to be an issues when using `langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint` together with the `langchain_huggingface.chat_models.huggingface.ChatHuggingFace` implementation.

When just using the `HuggingFaceEndpoint`, the parameter `max_new_tokens` is properly implemented, while this does not work properly when wrapping inside `ChatHuggingFace(llm=...)`. The latter implementation always returns a response of 100 tokens, and I am unable to get this to work properly after searching the docs + source code. 

I have created a reproducible example using `meta-llama/Meta-Llama-3-70B-Instruct` (as this model is also supported for serverless). 

### System Info

System Information
------------------
> OS:  Darwin
> OS Version:  Darwin Kernel Version 23.5.0: Wed May  1 20:19:05 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8112
> Python Version:  3.12.3 (main, Apr  9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]

Package Information
-------------------
> langchain_core: 0.2.10
> langchain: 0.2.6
> langchain_community: 0.2.5
> langsmith: 0.1.82
> langchain_anthropic: 0.1.15
> langchain_aws: 0.1.7
> langchain_huggingface: 0.0.3
> langchain_openai: 0.1.9
> langchain_text_splitters: 0.2.2
> langchainhub: 0.1.20

Packages not installed (Not Necessarily a Problem)
--------------------------------------------------
The following packages were not found:

> langgraph
> langserve