noamaanMulla-03/transformers-issues huggingface dataset:
recommend DeepSpeed's Argument Parsing documentation
Author: BurnzZ
Pull request: yes

# What does this PR do?

Clarify how to properly set the arguments passed by `deepspeed` when running in CLI.

For example the following errors might be raised when running something like `deepspeed --num_gpus=2 fine-tune.py google/flan-t5-xxl` due to args passed by `deepspeed`:

```
usage: fine-tune.py [-h] model_id                                                                                                      
fine-tune.py: error: unrecognized arguments: --local_rank=0 --deepspeed llms/flan-t5-fp16-z3.json                                      
usage: fine-tune.py [-h] model_id                                                                                                      
fine-tune.py: error: unrecognized arguments: --local_rank=1 --deepspeed llms/flan-t5-fp16-z3.json
```



## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@stas00 @sgugger 


Comments:
cc @pacman100 

_The documentation is not available anymore as the PR was closed or merged._
Question about generate method for AutoModelForCausalLM
Author: alimirgh75
Pull request: no

Hi, 
I am trying to use the git model from the pretrained to pass to captum API for calculation of the attribution score.


`
### Initialize the attribution algorithm
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("microsoft/git-base")
ig = IntegratedGradients(model)
`

However, in order for the IG algorithm to work, the "model" should be the forward function of the model.
I need to understand how the output of the model 
`
outputs = model(input_ids=training_batch["input_ids"],
                attention_mask=training_batch["attention_mask"],
                pixel_values=training_batch["pixel_values"],
                labels=training_batch["input_ids"])
`
corresponds with output of the generate method `generated_ids = model.generate(pixel_values=pixel_values, max_length=80)`
?

Comments:
Hi, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.
CI with layers=2
Author: ydshieh
Pull request: yes

# What does this PR do?

Running a (sub) set of 24315 tests (given by test fetcher) - only tests in `test_modeling_xxx.py`.
(for a full run like nightly run, it doesn't seem change anything about running time - need more investigation)

Running time:

- num_layers = mixed (2, 3, 4, 5, 6) - currently `main`
  - torch: 16m
  - tf: : 8m
  - flax: 11m30

- num_layers = 2
  - torch: 12m30
  - tf: 8m (not sure nothing change)
  - flax: 8m30

Comments:
_The documentation is not available anymore as the PR was closed or merged._
CI with `pytest_num_workers=8` for torch/tf jobs
Author: ydshieh
Pull request: yes

We set `pytest_num_workers` to `3` for `torch_job` and 6 for `tf_job` to avoid OOM. With the recent efforts of reducing model size in CI, we can actually set `pytest_num_workers=8`.

- The full suite: all 3 jobs (PT/TF/Flax): `12-15 minutes`
- On the latest nightly CI (without all PRs merged today): `PT: 37 min | TF: 25 min | Flax: 20 min)`

The `torch_job` reach `95%` of RAM (peak), and `tf_job` is at `80%` of RAM. The `torch_job` with `n8` is a bit dangerous, but I think I have a way to further improve things in follow PR(s).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Represent query_length in a different way to solve jit issue
Author: jiqing-feng
Pull request: yes

Hi @ArthurZucker @younesbelkada  @sgugger 

Thanks for contributing to the MPT model. I found a jit issue with this model.
```python
from transformers import AutoModelForCausalLM, AutoConfig
from optimum.intel.generation.modeling import jit_trace

model = AutoModelForCausalLM.from_pretrained("mosaicml/mpt-7b")

jit_model = jit_trace(model=model, task="text-generation", use_cache=True)
```
When I tried to trace mpt model, I got this error
![image](https://github.com/huggingface/transformers/assets/107918818/67244f76-625a-4080-aa54-34b8b032545a)

This is because float values like seq_length and query_length are detected as tensor in trace mode. When we set `query_length = seq_length` and `seq_length += past_key_value[0].shape[2]`, the `seq_length` is changed too which is unexpected.
![MicrosoftTeams-image (1)](https://github.com/huggingface/transformers/assets/107918818/8a6cff61-3a07-4f1a-96d6-d17a7efe9292)

So I use a more clean way to set `query_length` and it can also avoid the jit issue.
`query_length = seq_length if past_key_value is None else seq_length + past_key_value[0].shape[2]`

Would you please help me review it? Thanks!




Comments:
_The documentation is not available anymore as the PR was closed or merged._
Docs: Update list of `report_to` logging integrations in docstring
Author: tomaarsen
Pull request: yes

# What does this PR do?

## Pull Request overview
* Add missing `dagshub`, `codecarbon` and `flyte` integrations to `TrainingArguments` docstring.
* Update `report_to` type hint to allow strings.

## Details
I also converted the ordering back to alphabetical. 

I considered using a typing `Literal` as the type hint to help users via their IDE, but I haven't implemented it here as to not clash with the existing style. 

## Before submitting
- [x] This PR fixes a typo or improves the docs

## Who can review?

@sgugger

- Tom Aarsen


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Docs: separate generate section
Author: gante
Pull request: yes

# What does this PR do?

A conclusion of the latest doc brainstorming section with @patrickvonplaten was that generate-related doc discoverability will become harder as we add more guides. The plan would envision a tutorial page and a few new developer guides -- in addition to the existing task pages, developer guide, and API reference.

As such, we converged on the need for a new doc section, under which most new docs will reside (see #24575 for the plan), with a focus on the first L of LLMs.

There is no section that would fit perfectly, this is (IMO) the best compromise: it contains a bit of "task", "developer guide", and "performance and scalability", but "task" is the most obvious place to look for this information 🤗 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update bark doc
Author: ylacombe
Pull request: yes

# What does this PR do?

Bark can be greatly optimized with a few lines of code, which is discussed and explained in more detail in this [blog post](https://github.com/huggingface/blog/pull/1353). To encourage adoption and promote the use of optimization, I've added a few lines to the Bark documentation to reflect this.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


## Before submitting
- [x] This PR fixes a typo or improves the docs

## Who can review?

@sanchit-gandhi , @sgugger, @MKhalusova, feel free to comment on what can improved or clearer!
many thanks!

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @MKhalusova and @sanchit-gandhi , I've updated the docs according to your comments!
Thanks for the review!

Thanks @ylacombe for the recent round of changes!
device_map="auto" -> uninitialized parameters
Author: BramVanroy
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.11.4
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu118 (True)


### Who can help?

@ArthurZucker @younesbelkada
Maybe also @sgugger because this is a general use-case about PyTorch models

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am encountering an issue that worries me slightly. When I load a model with `device_map`, everything goes fine - no warnings.

```python
from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained("BramVanroy/flan-t5-small-amr-en")
```

Howver, when I do use the device_map, I get the warning that some weights are not initialized

```python
from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained("BramVanroy/flan-t5-small-amr-en", device_map="auto")
```

Result:
> Some weights of T5ForConditionalGeneration were not initialized from the model checkpoint at BramVanroy/flan-t5-small-amr-en and are newly initialized: ['decoder.embed_tokens.weight', 'encoder.embed_tokens.weight']
> You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

However, I am wondering whether this isn't a false positive because the model performance seems the same with/without. My model repo contains both safetensors and the PyTorch *.bin, if that has something to do with it?

### Expected behavior

Either a warning in both or no warning in either.

Comments:
I think this should have been fixed by #25101 Could you try again with a source install?
(Yes it is a false positive, just tied weights where the copies are not present in the state dict.)

Awesome, that works. Was afraid that I was messing something up with converting to safetensors. Glad that that is not the case.

Thanks for the prompt response! @sgugger
Add support for timeout parameter for load_image
Author: rolisz
Pull request: no

### Feature request

Add a parameter timeout to the `image_utils.py:load_image` function, which would enable setting the timeout for the requests call. This parameter should be plumbed in through all the ways to call that function (so add support for it in all the image related pipelines).

Alternatively, you should add a `requests_params` parameter, which should be a dictionary, to enable passing any parameters to requests.get. 

### Motivation

When using requests, the default timeout is None, which means that the request will wait (hang) until the connection is closed. Some servers for whatever reason don't return anything, but also don't close the connection. It would be useful to be able to set a timeout for these cases. 

### Your contribution

I can contribute a PR for this. 

Comments:
cc @amyeroberts 
Sounds like a good idea, so if you want to open a PR, please go ahead!

Do you have any preferences, keeping it simple, add just the timeout parameter, or adding a more general `requests_params`?

I think keeping it simple is probably for the best.
Add timeout parameter to load_image function
Author: rolisz
Pull request: yes

# What does this PR do?

This PR adds a timeout parameter to all pipelines that can fetch images from remote URLs. Without a timeout, the request can hang indefinitely. 

Fixes #25168


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x ] Did you write any new necessary tests?


## Who can review?

@amyeroberts



Comments:
_The documentation is not available anymore as the PR was closed or merged._

@amyeroberts I've done the requested changes
[JAX] Bump min version
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Bumps the minimum version of JAX to [0.4.1](https://jax.readthedocs.io/en/latest/changelog.html#jax-0-4-1-dec-13-2022), the earliest version where the new `jax.Array` API is introduced, replacing the deprecated `jax.numpy.DeviceArray` API. This allows compatibility with the latest JAX version [0.4.14](https://jax.readthedocs.io/en/latest/changelog.html#jax-0-4-14-july-27-2023), where `jax.numpy.DeviceArray` is removed entirely.

Related: #24875


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Minor typo referencing RoBERTa
Author: MrGeislinger
Pull request: no

"Roberta" should use the correct capitalization: "RoBERTa"

https://github.com/huggingface/transformers/blob/d27e4c18fe2970abcb9a48dcb8a824e48083b15f/docs/source/en/tokenizer_summary.md?plain=1#L144

Should be a simple fix.

Comments:
Fix typo: Roberta -> RoBERTa
Author: MrGeislinger
Pull request: yes

# What does this PR do?

Small typo in docs: "Roberta" should have the correct capitalization "RoBERTa".

Fixes #25301


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
<!--
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?
-->

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @sgugger, @stevhliu and @MKhalusova



Comments:
_The documentation is not available anymore as the PR was closed or merged._
use _make_causal_mask in clip/vit models
Author: kashif
Pull request: yes

# What does this PR do?

Uses the `_make_causal_mask` helper to build the causal attention mask which works for `bfloat16` dtype.

the current `_build_causal_attention_mask` uses the `torch.triu_` which is not supported in pytorch release  fixed recently in main: https://github.com/pytorch/pytorch/pull/101414 

should fix https://github.com/huggingface/diffusers/issues/3453


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Llama 2: NaN values when torch_dtype=torch.float16 and padding_side="left"
Author: ValeKnappich
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-4.18.0-477.13.1.el8_8.x86_64-x86_64-with-glibc2.28
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker @younesbelkada 

Llama 2 seems to produce NaN-values when `torch_dtype=torch.float16` and `padding_side="left"`. The behavior seems very odd to me. See the MWE below.

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig

WITH_DTYPE_KWARG = True
LEFT_PADDING = True

model_id = "meta-llama/Llama-2-7b-hf"

if WITH_DTYPE_KWARG:
    model = AutoModelForCausalLM.from_pretrained(model_id, low_cpu_mem_usage=True, torch_dtype=torch.float16).cuda()
else:
    model = AutoModelForCausalLM.from_pretrained(model_id, low_cpu_mem_usage=True).cuda()

if LEFT_PADDING:
    tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="left")
else:
    tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="right")

tokenizer.pad_token = tokenizer.eos_token

inputs = [
    "Short input",
    "Long long long input with lots of tokens so that there is a padding",
    "Short input",
    "Long long long input with lots of tokens",
]

enc = tokenizer(inputs, padding=True, return_tensors="pt").to(model.device)
o = model(**enc)
assert o.logits.isnan().sum().item() == 0
```

The assertion fails only when both WITH_DTYPE_KWARG  and LEFT_PADDING  are true. However, there also seems to be some interaction with the input, the problem does not appear for every input.

### Expected behavior

All variants should work or a meaningful error message should be shown.

Comments:
Closing as duplicate of #25065, but thanks a lot for putting effort in having a nice small reproducer!
Script to convert huggingface models to deepspeed/megatron checkpoints
Author: ShivamSharma2705
Pull request: no

Do we have a script to convert the huggingface pretrained models on the server to deepspeed/megatron style model checkpoints?

Comments:
We don't at the moment. You can probably try to reverse the existing conversion scripts.

@tjruwase (Deepspeed) has been working on doing various conversions between Meg-LM, Meg-DS and HF Transformers:
https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/239
so perhaps it'd fit there the best, but it's a slow going, as we keep asking Tunji to work on other things every time we try to go back to work on the checkpoints

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi there. 
Just wondering any updates of the conversion from HF models to DS models. 
Thanks! 

how to convert llama2 hf format model to megatron-ds format?
Show a warning for missing attention masks when pad_token_id is not None
Author: hackyon
Pull request: yes

# What does this PR do?

Fixes #16136

Shows a one-time warning message when the pad_token_id is not None and no attention masks are given. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? #16136 
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante @ydshieh

Comments:
@ydshieh @gante

I've added the warning message as per #21916 and also a short test. Please let me know if you have any suggestions. If it looks good, I can copy/paste the warning to more models as well. Thanks!


@hackyon LGTM 👍 

But before you spend time propagating the change to the other models, let's also get a green light from a core maintainer (cc @sgugger )

Thanks for the review. 

Would it be better to check for the presence of the pad_token_id inside input_ids first before throwing the error, as per
https://github.com/huggingface/transformers/pull/17444/files? If so, I can make the change to reflect that here.



@sgugger 

Thanks for the input. I changed my pull request up to be more like #17444. Let me know what you think. Thanks!


Thanks, I've updated the code accordingly.

@gante could you have a second look here?

_The documentation is not available anymore as the PR was closed or merged._

@hackyon Thank you for the contribution! Would you like to add it to the remaining models? 🤗 

Sure, I'll look into it 👍

Thanks @ydshieh for fixing the flaky test!

I was busy in July, but will now add the warning to more models over the next couple of days.
Use of logging.warn is deprecated in favour of logging.warning
Author: PeterJCLaw
Pull request: no

There are a few places where `transformers` uses the deprecated `warn` method on a logger, while most of the library uses `warning`. While this works for now, it will presumably be removed at some point (calling it emits a `DeprecationWarning`) and it means that strict test runners (such as `pytest`) complain about some codepaths.

As far as I can tell, all versions of Python supported by `transformers` support the new spelling (`warning` has been around for a _long_ time) so the upgrade should be simple.

I'd be happy to have a go at a PR for this.

Comments:
@PeterJCLaw Indeed! Happy to review a PR :)