noamaanMulla-03/transformers-issues huggingface dataset:
adds: extra_repr() to MambaRMSNorm to include hidden size / size of weights in the layer
Author: rohitdwivedula
Pull request: yes

# What does this PR do?
Printing out a MAMBA model right now looks something like this:

```
> model = MambaForCausalLM.from_pretrained("state-spaces/mamba-130m-hf")
> print(model)
MambaForCausalLM(
  (backbone): MambaModel(
    (embeddings): Embedding(50280, 768)
    (layers): ModuleList(
      (0-23): 24 x MambaBlock(
        (norm): MambaRMSNorm()
        (mixer): MambaMixer(
          (conv1d): Conv1d(1536, 1536, kernel_size=(4,), stride=(1,), padding=(3,), groups=1536)
          (act): SiLU()
          (in_proj): Linear(in_features=768, out_features=3072, bias=False)
          (x_proj): Linear(in_features=1536, out_features=80, bias=False)
          (dt_proj): Linear(in_features=48, out_features=1536, bias=True)
          (out_proj): Linear(in_features=1536, out_features=768, bias=False)
        )
      )
    )
    (norm_f): MambaRMSNorm()
  )
  (lm_head): Linear(in_features=768, out_features=50280, bias=False)
)
```

All of the other layers (e.g. `Linear`, `Conv1d`, `Embedding`), when printed as a string show the size of the hidden states within the paranthesis. The `MambaRMSNorm`, when printed, does not include the `hidden_size` (or the `eps`) it was initialized with - this change would change the way this layer is printed from `MambaRMSNorm()` to `MambaRMSNorm(768, eps=1e-05)`. This format matches the convention that other normalization layers in PyTorch (e.g. `torch.nn.LayerNorm`) use when being printed. 

For example,

```python3
> from torch.nn import LayerNorm
> l = LayerNorm(10)
> print(l)
LayerNorm((10,), eps=1e-05, elementwise_affine=True)
```


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32171). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
GPT2 Tokenizer got an expected argument `skip_special_tokens`
Author: sashank06
Pull request: no

## 🐛 Bug

Model I am using  -> GPT2

Language I am using the model on - >English

The problem arise when using:
I keep running into this error when trying to use the GPT2 model and GPT2 tokenizer while decoding.
Keep getting the following error when I run the piece of code below:
tokenizer.decode(response_ids, skip_special_tokens=True)
Error:
TypeError: decode() got an unexpected keyword argument 'skip_special_tokens'

* OS: Ubuntu
* Python version: 3.6.8
* PyTorch version: 1.1.0
* PyTorch Transformers version (or branch): 1.0.0
* Using GPU - Yes



Comments:
I'm having a hard time reproducing it on my side on a clean install. Could you provide a sample that throws the error?

@LysandreJik I will give you the code with sample input and output asap. 

I met the same problem. Could you tell me how you solved that?
Using `AutoTokenizer.from_pretrained`'s `.encode()` function fails to add BOS token for new Llama-3.1 model
Author: m18coppola
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.43.1
- Platform: Linux-6.9.3-76060903-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.24.1
- Safetensors version: 0.4.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Are the differences in the `.encode()` method between Llama-3 and Llama-3.1 expected?

Reproduce:
```python
from transformers import AutoTokenizer

llama_3_tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
llama_3_1_tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")

text = "Hello"

print(llama_3_tokenizer.encode(text))
print(llama_3_1_tokenizer.encode(text))
```
Output:
```
[128000, 9906]
[9906]
```

### Expected behavior

Expected output:
```
[128000, 9906]
[128000, 9906]
```

Comments:
Right! We need to add this: 
https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/commit/339ce92d052f002cdbac4a4bd551d1c61dd8345e
to all tokenizers

It should be done now! 

New output:
```
[128000, 9906]
[128000, 9906]
```

Thanks!
Remove conversational pipeline tests
Author: amyeroberts
Pull request: yes

# What does this PR do?

Conversational pipeline has been fully deprecated. The tests for the pipeline were accidentally added back in #31308 (my bad). This PR removes them again. 

Fixes #32096 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32099). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Backwards compatibility broken for RoPE: "rope_type"
Author: snarayan21
Pull request: no
Labels: name: bug

### System Info

transformers v4.43.1

### Who can help?

@ArthurZucker @gante 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Using an older rope_config with new transformers version

### Expected behavior

Hey, there seems to be an issue when trying to use an older model that has the previous rope_config. Specifically, the rope_config used to have the "type" key, and while this has now been changed to "rope_type", the key is not being extracted correctly. At this line, the "type" key is extracted for backwards compatibility but is not set inside the `rope_scaling` dict. And all the validation functions expect `rope_scaling` to have "rope_type" as a key (for example, [here](https://github.com/huggingface/transformers/blob/c85510f958e6955d88ea1bafb4f320074bfbd0c1/src/transformers/modeling_rope_utils.py#L371)).

Could this be addressed? Would likely require another patch release as well. Thank you!

Comments:
Related issue #32167
RoPE: relaxed rope validation
Author: gante
Pull request: yes

# What does this PR do?

Fixes #32167 
Fixes #32166 

Converts most exceptions into warnings, leaving the user to determine whether the warning is relevant to their use case.

There are a two cases that still throw exceptions:
- if a missing mandatory option is missing for a given rope type (e.g. linear scaling requires `'factor'` to be set)
- if a user tries to instantiate a rope layer with both `**kwargs` and a `config` (mutually exclusive, `**kwargs` is for BC and will be removed in v4.45 if everything goes well)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32182). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

When will the patch release go out? today?

Yes
Fix failing test with race condition
Author: Rocketknight1
Pull request: yes

The `test_push_to_hub_dynamic_processor` calls `create_repo` with a fixed repo name, and often throws an error in the CI. I suspect this happens when multiple test runners try to create the repo at the same time.

I rewrote the test to use a randomized repo name, which should hopefully avoid these collisions - though I don't know how the dummy user works, so I hope these still get cleaned up!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32140). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ready for review @amyeroberts! In case you're worried about the `try/finally` thing, or clogging up the dummy user, see [this discussion](https://huggingface.slack.com/archives/C02EMARJ65P/p1721657352954859).

> Thanks for handling @Rocketknight1
> 
> c.f. #31973 #32025 cc @ydshieh

Thank you. I didn't touch anything about `test_processor_auto` and didn't add any new `create_repo`. But yes, probably I forgot to use `finally`.

Anyway, I will rework this kind of thing in #32025
:rotating_light: No more default chat templates
Author: Rocketknight1
Pull request: yes

We've been putting increasingly loud deprecation warnings on default chat templates telling users that they'd be removed in v4.43. It's now time to remove them entirely, since that's the next release.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31733). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix: Replaced deprecated `unittest method` with the correct one
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
Small fix, `assertEquals(a, b)` is deprecated and replaced with `assertEqual(a, b)`
Reference: https://docs.python.org/3/library/2to3.html#to3fixer-asserts

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts 

Comments:
Metrics - Pipeline
Author: kotari
Pull request: no
Labels: name: Feature request

### Feature request

Output token generation metrics/inference-time as part of invoking pipeline. For text-generation pipeline having metrics like generated-tokens, prompt-tokens, inference-time will give insights into expected response time when using the model.

### Motivation

While evaluating models for different scenarios, development/testing happens using the examples provided in the model card. One dimension in evaluating the model is the response. Having an additional dimensions of performance metrics would be useful in understanding the trade-off between model response vs time.
These metrics are available as part of text-generation-inference server as http headers. 

### Your contribution

I haven't thought about contributing but this is something I am willing to do if similar work is not in the pipeline.

Comments:
Hi @kotari, 

The pipeline is intended to be any easy entry point into using our models for inference i.e. a single line call. It's not intended as a full bells-and-whistles feature and we deliberately want to avoid adding to many features. 

For any more advanced use cases, I'd suggest using calling the modeling API directly and building metrics around that e.g.:

```
import time
from transformers import AutoTokenizer, GemmaForCausalLM

model = GemmaForCausalLM.from_pretrained("google/gemma-7b")
tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b")

prompt = "What is your favorite condiment?"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
with torch.no_grad():
    start = time.time()
    generate_ids = model.generate(inputs.input_ids, max_length=30)
    end = time.time()

generation_running_time = end - start
```

cc @Rocketknight1 

Agree with @amyeroberts - we definitely want to keep the pipeline API simple, and leave that kind of performance benchmarking for the lower-level API!

@amyeroberts Thanks for the guidance and keeping true to the philosophy to keep things simple. I was able to generalize the suggestion provided. I had to re run tokenizer as I do not have all the fields needed to generate the metrics (This generalized code has a bug with a line break and I can live with it)

```
pipe = pipeline("text-generation", model=model_name, device_map="auto",) 
start = time.time()
response = pipe(prompt, **kwargs)
end = time.time()
process_time = end - start
prompt_tokens = len(pipe.tokenizer(prompt).input_ids)
generated_tokens = (
    len(pipe.tokenizer(response[0]["generated_text"]).input_ids)
    + prompt_tokens
)
logger.info("inference-time: %d", round(process_time * 1000))
logger.info("prompt-tokens: %d", prompt_tokens)
logger.info("generated-tokens: %d", generated_tokens)
logger.info(
    "time-per-token: %d",
    round(process_time * 1000 / (generated_tokens - prompt_tokens)),
)
```
[whisper] fix short-form output type
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Currently, the output type from Whisper short-form generate is inconsistent depending on whether or not we do temperature fallback:
```python
from datasets import load_dataset
from transformers import WhisperForConditionalGeneration, AutoProcessor

# load model + processor
processor = AutoProcessor.from_pretrained("openai/whisper-tiny.en")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-tiny.en")

# load dataset
dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
sample = dataset[0]["audio"]

# pre-process inputs
inputs = processor(sample["array"], sampling_rate=sample["sampling_rate"], return_tensors="pt")

# generation **without** fallback
pred_ids = model.generate(**inputs, max_new_tokens=8)
print(type(pred_ids))

# generation **with** temperature fallback
pred_ids = model.generate(**inputs, max_new_tokens=8, logprob_threshold=-1.0, temperature=[0.0, 0.1])
print(type(pred_ids))
```
**Print Output:**
```
<class 'torch.Tensor'>
<class 'transformers.generation.utils.GenerateEncoderDecoderOutput'>
```
=> we see that generating without fallback gives a torch tensor, whereas generating with fallback gives `GenerateEncoderDecoderOutput`.

This can be fixed by changing one line in the post-processing of the generation outputs, in order to respect the setting `return_dict_in_generate` passed by the user:
```
<class 'torch.Tensor'>
<class 'torch.Tensor'>
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32178). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks @kamilakesbi - added a test and fixed the failing ones!
Fix float8_e4m3fn in modeling_utils
Author: SunMarc
Pull request: yes

# What does this PR do ?

This PR fixes a bug that was introduced in the latest release. We need to check that torch.float8_e4m3fn exist (only after torch 2.1)
Fixes https://github.com/huggingface/transformers/issues/32185

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32193). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

local file changed according to https://github.com/huggingface/transformers/pull/32193/files  
but the error still exists
```
  File "/opt/conda/lib/python3.10/site-packages/safetensors/torch.py", line 310, in load_file
    result[k] = f.get_tensor(k)
AttributeError: module 'torch' has no attribute 'float8_e4m3fn'
```
so , i wonder can  #32193 solve above torch error??

Hi @tanghengjian, can you verify which version of transformers you're running in your python env? 

`python -c "import transformers; print(transformers.__version__)"`
Update question_answering.py
Author: avlewis
Pull request: yes

# What does this PR do?
Fixes issue with select_starts_ends function within question_answering.py

Error
`
    undesired_tokens = np.abs(p_mask.numpy() - 1)
    AttributeError: 'list' object has no attribute 'numpy'
`
## Before submitting
- [ y ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ y ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ n ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ n ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ n ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
cc @Rocketknight1, do you mind giving this one a look?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32208). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[BigBird Pegasus] set _supports_param_buffer_assignment to False
Author: kashif
Pull request: yes

# What does this PR do?

sets the `_supports_param_buffer_assignment` to `False` to fix the model loading error.

similar to https://github.com/huggingface/transformers/pull/32149

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32222). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Follow up for #31973
Author: ydshieh
Pull request: yes

# What does this PR do?

@amyeroberts Could you take a look to see such design is fine 🙏 ?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32025). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts If you are happy with this current version, I will apply to all related places
Allow a specific microphone to be used by the ffmpeg audio pipeline utility functions. Default to using the currently active microphone on Mac
Author: jrhe
Pull request: yes

Adds a `ffmpeg_input_device` parameter to `ffmpeg_microphone` and `ffmpeg_microphone_live` to specify the microphone to be used as input. 
Changes the default input device to be the currently active device on Mac/Darwin, rather than device 0.

Addresses issues #31820, #31074 , #28154, #31393.

I unfortunately can't get the tests pass on my machine with a freshly checked out repository, so cannot comment on if this breaks any tests. From what I can tell this code is untested so it shouldn't break any. 

I tested the changes work as expected locally using existing code.

These functions are not referenced in the main transformers documentation.

@kamilakesbi 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31846). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@jrhe Have you had time to iterate on this? :) 

@jrhe great! thanks for iterating! 

@amyeroberts this should be ready for final review and merge :) 

Sorry, missed the documentation style guide! Have merged those bits fixed some other bits of documentation (missing colons, and a typo).