noamaanMulla-03/transformers-issues huggingface dataset:
ffmpeg_microphone_live hangs if anything else is using STDIN
Author: mikamerath
Pull request: no
Labels: name: Feature request

### System Info

- `transformers` version: 4.39.3
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.11.7
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.2+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

When running ffmpeg_microphone_live in an environment where anything else is using STDIN, ffmpeg_microphone_live stops processing and prints `Enter command: <target>|all <time>|-1 <command>[ <argument>]`.

I believe this is related to this issue:
https://stackoverflow.com/questions/12543628/bash-script-stops-execution-of-ffmpeg-in-while-loop-why?rq=4

![ffmpeg_microphone_live](https://github.com/huggingface/transformers/assets/10928325/408f7078-84cc-4762-8ff3-507c0fdde25c)


### Who can help?

@sanchit-gandhi 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. Run a process that uses stdin
2. Run a Python script that uses ffmpeg_microphone_live
3. Observe the behavior from the screenshot of outputting: `Enter command: <target>|all <time>|-1 <command>[ <argument>]`

### Expected behavior

I would hope that ffmpeg_microphone_live would have an option to use a different file for live reading or a -nostdin option with another file to support running as a background process

Comments:
`self.pre_layrnorm` should be `self.pre_layernorm`?
Author: devymex
Pull request: no

https://github.com/huggingface/transformers/blob/745bbfe4bb2b61491dedd56e1e8ee4af8ef1a9ec/src/transformers/models/clip/modeling_clip.py#L821

Is this obviously a typographical error, but will correcting it affect existing model weights?

Comments:
This is indeed a typo but will be hard to change now :/
Limit number of parametes logged with `MLflowCallback`
Author: cecheta
Pull request: no
Labels: name: trainer

name: Feature request

### Feature request

Add a new environment variable, such as `MLFLOW_MAX_LOG_PARAMS`, which can limit the number of parameters logged by the `MLflowCallback`.

### Motivation

When using mlflow in Azure ML, there is a limit of 200 parameters that can be logged in a single run, meaning that when attempting to run a training job, the callback needs to be disabled entirely, or the module needs to be "monkeypatched" to limit the number of params logged.

### Your contribution

I will submit a PR

Comments:
cc @ydshieh what do you think? :)
fix error in _get_eval_sampler when group_by_length enabled
Author: akakakakakaa
Pull request: yes

# What does this PR do?

https://github.com/huggingface/transformers/pull/33514
this PR adds group_by_length support for evaluation. But, this part uses self.eval_dataset instead of eval_dataset. So, if eval_dataset is dictionary it fails.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
- trainer: @muellerzr and @SunMarc

Comments:
should probably remove `self` at [line 968](https://github.com/huggingface/transformers/pull/34237/files#diff-ed55888e6665791fe92cc8fc0c499da54f4ace6738551cd9a2591881cda076deR968) as well

Thank you for pointing out what I missed
[docs] Fix GenerationConfig params
Author: stevhliu
Pull request: yes

Fixes #34273 #33756  to display all the `GenerationConfig` parameters as shown [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34297/en/main_classes/text_generation). Can be merged after https://github.com/huggingface/transformers/pull/34293 which fixes the failing `build_pr_documentation` test.

Comments:
Just a note that [this won't be expanded](https://github.com/huggingface/transformers/blob/93352e81f5019abaa52f7bdc2e3284779e864367/src/transformers/generation/configuration_utils.py#L173). Maybe an opportunity to list the cache methods explicitly and provide one-liners for each one, in a followup PR? Happy to contribute if you're busy.
🌐 [i18n-KO] Translated `executorch.md` to Korean
Author: ahnjj
Pull request: yes

# What does this PR do?

Translated the `executorch.md file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->
@chhaewxn, @ahnjj, @jun048098, @fabxoe

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
 @stevhliu May you please review this PR? 

Comments:
🌐 [i18n-KO] Translated `bert japanese.md` to Korean
Author: ahnjj
Pull request: yes

# What does this PR do?

Translated the `bert japanese.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
<!-- @SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd -->
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->
@chhaewxn, @ahnjj, @jun048098, @fabxoe

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR? 

Comments:
🌐 [i18n-KO] Translated `model_doc/bartpho.md` to Korean
Author: Jwaminju
Pull request: yes

# What does this PR do?

Translated the `model_doc/bartpho.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
@junejae, @Jwaminju, @010kim, @4N3MONE, @boyunJang 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
@stevhliu May you please review this PR?

Comments:
add Glm
Author: Cyrilvallez
Pull request: yes
Labels: name: run-slow

GLM model!

Comments:
Hey! 🤗 Thanks for your contribution to the `transformers` library!

Before merging this pull request, slow tests CI should be triggered. To enable this:
- Add the `run-slow` label to the PR
- When your PR is ready for merge and all reviewers' comments have been addressed, push an empty commit with the command `[run-slow]` followed by a comma separated list of all the models to be tested, i.e. `[run_slow] model_to_test_1, model_to_test_2`
  - If the pull request affects a lot of models, put at most 10 models in the commit message
- A `transformers` maintainer will then approve the workflow to start the tests

(For maintainers) The documentation for slow tests CI on PRs is [here](https://www.notion.so/huggingface2/CI-for-pull-requests-8335bd9217d24d1e8ed1d3a016f39804).

Ready for last review @ArthurZucker, `setup_and_quality` fail because of the `__all__` issue, but will pass once https://github.com/huggingface/transformers/pull/33859 is merged.

Thank you very much for your help. I also saw this [huggingface](https://huggingface.co/THUDM/glm-4-9b-chat/discussions/81) PR.
I have replied, and some of the code may need to be modified. Perhaps we can work together to improve it and merge this work.

Thank you again for your support!

Of course! 

> LGTM anything missing before we merge?

No, only issue are the docstrings in the configuration, but this will be solved with the auto-docstrings. In the meantime, I just moved the config outside modular to please the CIs.

Confimed that slow tests pass for the model. Merging.

@Cyrilvallez Hi Cyril, you PR for the 1M version of the model got an unexpected generation. Please refer to here for more information: [https://huggingface.co/THUDM/glm-4-9b-chat-1m/discussions/17](https://huggingface.co/THUDM/glm-4-9b-chat-1m/discussions/17).
Tokenizer encode with fast or not get different tokens
Author: bugm
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.44.2
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.9.19
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.32.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.4.1+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A40


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

when run code below (where /data/mengzhangyuan/model/llama2-tokenizer includes tokenizer files download from 
https://huggingface.co/meta-llama/Llama-2-7b-hf )
```
text = "▁<unk>Hey"
t = AutoTokenizer.from_pretrained("/data/mengzhangyuan/model/llama2-tokenizer",use_fast=False)
print(t.encode(text))
print(t.tokenize(text))
print("*"*10)
t2 = AutoTokenizer.from_pretrained("/data/mengzhangyuan/model/llama2-tokenizer",use_fast=True)
print(t2.encode(text))
print(t2.tokenize(text))
```
The results show
```
[1, 259, 0, 29950, 1032]
['▁▁', '<unk>', 'H', 'ey']
**********
[1, 259, 0, 18637]
['▁▁', '<unk>', '▁Hey']
```

With same text,  the tokens are different.


### Expected behavior

use_fast = True and use_fast = False produce same result

Comments:
pre tokenization differs in both the tokenizers
thus subword tokenization differs AFAIK

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Stronger bias from attention_mask in BERT
Author: fkrasnov2
Pull request: no
Labels: name: bug

### System Info

all versions

### Who can help?

@Artu

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
# Since attention_mask is 1.0 for positions we want to attend and 0.0 for
# masked positions, this operation will create a tensor which is 0.0 for
# positions we want to attend and -10000.0 for masked positions.
adder = (1.0 - tf.cast(attention_mask, tf.float32)) * -10000.0

# Since we are adding it to the raw scores before the softmax, this is
# effectively the same as removing these entirely.
attention_scores += adder
```

### Expected behavior

Hey there, 

I'm kind of stuck on this attention layer thing and I'm a bit confused about the difference between our implementation and Jacob's original code. It's clear that the attention mask does some biasing before the softmax , but in Jacob's code, it seems like the bias is a lot stronger.

[**Jacob Davlin:**](https://github.com/google-research/bert/blob/eedf5716ce1268e56f0a50264a88cafad334ac61/modeling.py#L712)

![image](https://github.com/user-attachments/assets/050e26b5-501f-40e6-99b7-e43944e084c3)

[**Transformers:**](https://github.com/huggingface/transformers/blob/53fad641cfdb5105e2470bcf3ef17ea8e25cc300/src/transformers/models/bert/modeling_bert.py#L331)

![image](https://github.com/user-attachments/assets/d71e8d76-4177-446f-a28f-9fbf6ae8c8e1)

In the Transformers code, only 1 and 0 are added from attention_mask. 

Maybe I'm wrong. Is it a mistake? What do you think?

Best, 
Fedor

Comments:
Hey Fedor, the attention mask preperation happens elsewhere and depends on the attention implementation (eager, flash attn, sdpa). The only difference to the original version here in Bert is that it uses torch's min dtype instead of the hardcoded -10000.

See the attention preparation here (the last line is for eager for example):
https://github.com/huggingface/transformers/blob/53fad641cfdb5105e2470bcf3ef17ea8e25cc300/src/transformers/models/bert/modeling_bert.py#L1086-L1114
and the inversion in eager you're looking for is over here:
https://github.com/huggingface/transformers/blob/343c8cb86f2ab6a51e7363ee11f69afb1c9e839e/src/transformers/modeling_utils.py#L1188
PEFT fix: T5 prefix-tuning with new cache format 
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes prefix tuning for T5 models which started breaking after the recent compile-compatibility PR. However, note this will not enable correct prefix tuning unless https://github.com/huggingface/peft/pull/2096#pullrequestreview-2385144033 is merged

The main issue was that in new cache format we don't concatenate new key/values with cached key/values if `past_key_values.is_updated`. That issue is fixed on PEFT side by init a cache object and setting `is_updated=False`.

Another issue is the mask shape mismatch since we have to extend `cross_attention_mask` to account for new virtual tokens. Current PR adds enables it but I am also thinking if it will be possible to do in PEFT and pass already extended attention masks?

Tested with the below code for T5:
```python
import torch
from transformers import AutoModelForSeq2SeqLM
from peft import PrefixTuningConfig, get_peft_model

inputs = {
    "input_ids": torch.tensor([[1, 2, 3, 4, 5, 6, 7]]),
    "decoder_input_ids": torch.tensor([[1, 2, 3, 4, 5]]),
    "attention_mask": torch.tensor([[1, 1, 1, 1, 1, 1, 1]]),
}
model_id = "ybelkada/tiny-random-T5ForConditionalGeneration-calibrated"
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
model(**inputs)

config = PrefixTuningConfig(num_virtual_tokens=20, task_type="SEQ_2_SEQ_LM")
model = get_peft_model(model, config)

output = model(**inputs)

```

cc @BenjaminBossan wdyt?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34312). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Just to update, we're internally discussing other options.

Closing as we realized encoder-decoder models work differently for prefix-tuning and found a fix on PEFT side
Boolean as tool input
Author: paulblgr
Pull request: no
Labels: name: Feature request

name: Agents

### Feature request

It would be great if `boolean` was authorized as input to a `Tool`

### Motivation

I am willing to use my own tools with transformers CodeAgent ; using the method `tool`
I have a proper function `func` with typing and doc-strings as required. One of the input of the function is a `bool`.
When I try to run `tool(func)` I get: `Exception: Input 'perte_de_salaire': type 'boolean' is not an authorized value, should be one of ['string', 'integer', 'number', 'image', 'audio', 'any']`.

The Exception is rather clear, but why wouldn't a type as basic as `boolean` not be allowed? Especially since any is authorized. This is clearly a limitation to using the library.

### Your contribution

I seems like a few lines of code to change in tools.py (https://github.com/huggingface/transformers/blob/main/src/transformers/agents/tools.py)

Comments:
cc @aymeric-roucher 

Please assign this to me
Support boolean tool args
Author: aymeric-roucher
Pull request: yes
Labels: name: Agents

# What does this PR do?

Fixes #34056 by adding support for boolean tool args.

cc @LysandreJik @paulblgr




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34208). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Mamba `torch.export` `RuntimeError: Found <class 'transformers.cache_utils.MambaCache'> in output, which is not a known type.`
Author: Wheest
Pull request: no

I was trying to export `state-spaces/mamba-2.8b-hf` using `torch.export`.

I used the following code:

```python
# Load tokenizer and model
model_name = "state-spaces/mamba-2.8b-hf"
name = model_name.split("/")[-1]
cache_dir = "/models/"
model = model_name
# Load tokenizer and model using the specified cache directory
model = MambaForCausalLM.from_pretrained(model_name, cache_dir=cache_dir)
tokenizer = AutoTokenizer.from_pretrained(model_name, cache_dir=cache_dir)

base_prompt = "How many hours are in a day?"
base_inputs = tokenizer(base_prompt, return_tensors="pt")
input_ids = base_inputs.input_ids

print("Exporting model using torch export...")
exported = torch.export._trace._export(
    model,
    (input_ids,),
    strict=False,
    _allow_complex_guards_as_runtime_asserts=True,
)
```

However this was failing as the `torch.export` stage with the error `RuntimeError: Found <class 'transformers.cache_utils.MambaCache'> in output, which is not a known type.`

Clearly MambaCache is not a valid object type in the context of torch.export.

[This PR](https://github.com/huggingface/transformers/pull/31247) recently added `torch.compile` support for Mamba in Transformers.  It made reference to the MambaCache.

[This person had an issue](https://github.com/huggingface/transformers/issues/29505) with the cache in Mamba, and someone else recommended setting `use_cache = False`.  There was [a fix](https://github.com/huggingface/transformers/pull/31116) which let Mamba use this parameter.

Using this feature `model.config.use_cache = False` in my code allows the export to run successfully.

I'm closing this issue since this is a solution, but leaving it here as information for the future.

Comments:
skip `test_pipeline_depth_estimation` temporarily
Author: ydshieh
Pull request: yes

# What does this PR do?

Failing after #32550

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34316). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

It's mentioned 

https://github.com/huggingface/transformers/pull/32550#issuecomment-2429494566

with the contributor replied.

I will open the issue if there is no further action from there this week.
[docs] Fix Korean toctree
Author: stevhliu
Pull request: yes

One more fix to the Korean `toctree` which was missing a `title` key

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34324). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix red CI: benchmark script
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Skips init container when your PR does not have the `run-benchmark` tag

Comments:
Let me sync

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34351). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.