noamaanMulla-03/transformers-issues huggingface dataset:
Update examples with image processors
Author: amyeroberts
Pull request: yes

# What does this PR do?

Updates all of the feature extractor references to image processors in the `examples` folder.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This broke the deepspeed tests. Fix is here: https://github.com/huggingface/transformers/pull/21283

when modifying examples and breaking back compat please scan the slow tests and adjust those too. Thank you!

The reason this is important is because Deepspeed CI runs our slow deepspeed tests as normal CI so when we break things their CI is broken.

@stas00  - thanks for applying a quick fix and apologies about the break. I'll make sure to remember the slow tests next time! 

Thank you, Amy!

`tests/extended/` and `tests/deepspeed` are the ones that heavily rely on `examples/pytorch`
[docs] TrainingArguments default label_names is not what is described in the documentation
Author: fredsensibill
Pull request: no

### System Info

- `transformers` version: 4.25.1
- Platform: macOS-12.6.1-arm64-arm-64bit
- Python version: 3.8.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: No

### Who can help?

@sgugger, @stevhliu and @MKhalusova

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. Create a model with a `forward` that has more than one label. For example:

```
 def forward(
        self,
        input_ids,
        bbox,
        attention_mask,
        token_type_ids,
        labels,
        reference_labels
)
```

2. Create a trainer for your model with `trainer = Trainer(model, ...)`. Make sure to not set `label_names` and let it default.
3. Check `trainer.label_names` and see that it returns `["labels", "reference_labels"]`

### Expected behavior

[The documentation](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.Seq2SeqTrainingArguments.label_names) states that:

> Will eventually default to ["labels"] except if the model used is one of the XxxForQuestionAnswering in which case it will default to ["start_positions", "end_positions"].

[This PR](https://github.com/huggingface/transformers/pull/16526) changed the behaviour that the documentation describes.

Comments:
Indeed. Do you want to open a PR to fix the documentation?
Fix `TrainingArguments.label_names` docs to reflect the correct default value behaviour
Author: fredtcaroli
Pull request: yes

Fixes #21287
@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This was never flagged as a breaking change (indeed I only found this because it broke one of my scripts). I wonder if I should add "🚨 🚨 🚨" to the PR name to indicate a breaking change

The breaking change is in the PR that changed the default to `label_names` a while ago, not in this one :-)
`trainer.predict(dataset)` drops samples when used with multi-gpu setup 
Author: fgbelidji
Pull request: no

### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-4.19.0-23-cloud-amd64-x86_64-with-glibc2.28
- Python version: 3.9.13
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.11.0+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes


### Who can help?

@sgguger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

When running on a multi-gpu setup, the output of `trainer.predict()` misses samples even if the parameter `dataloader_drop_last` is set to `False` in the `TrainingArguments`. 

[Here is an example colab notebook](https://drive.google.com/file/d/1pvnDkAhkMjLkUZVvindsFqhA_DRLveUk/view?usp=sharing) that could be run on a multi-gpu instance to reproduce the issue.

As an example, I had 2 gpus on my machine, my dataset had a size of 25000 samples, batch size per device was set to 256 (then 512 in total). When calling `trainer.predict()`, the output I got had a size of 24951.

The number of missing samples (25000-24951= 49) corresponds to the size of the dataloader (output of `len(trainer.get_test_dataloader()`)

### Expected behavior

Output of `trainer.predict(dataset)` should be the same length as dataset

Comments:
The model used in this notebook (`Roberta0`) does not follow the requirements of the `Trainer` as described [here](https://huggingface.co/docs/transformers/main_classes/trainer) (see the big square in red). The output of the model should be a tuple, a dictionary or a `ModelOutput`, but it can't be a simple tensor. This is the reason for the problem seen.

Thanks for your quick response @sgugger
from transformers import T5Model -> No module named 'torch._C'
Author: ndvbd
Pull request: no

### System Info

Prompt says to use "transformers-cli env", but it's not clear where is the documentation for installing transformers-cli on Ubuntu...

python version: 3.10.6
system: ubuntu 20 (no gpu, laptop)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import T5Model

### Expected behavior

Should give no errors, but for me it gives:
```

  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1101, in __getattr__
    value = getattr(module, name)
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1100, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1112, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.t5.modeling_t5 because of the following error (look up to see its traceback):
No module named 'torch._C'

Process finished with exit code 1

```

Comments:
Solved it by:

```
pip3 uninstall torch
pip3 install torch

```
Very weird.
[Mask2Former] Add doc tests
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR ensures that the code snippet's in Mask2Former's docs work as intended, and are tested.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh not sure why CI is failing, running `make fixup` locally doesn't result in any updates. My version is black 22.3

@NielsRogge mine is also 22.03, but it reformates the modeling file. Not sure why though, do you want me to push? I can also post the whole content of `pip freeze` for you to check the package versions.

Feel free to push a commit :)

I pushed a commit. Actually, you are right. `make fixup` will change the files twice in the run, and that 2 changes cancel each other's change. I am not sure why. After running `make style` to fix some issues, it then works for `make fixup` too.
How to create distil-opt/bloom
Author: omerarshad
Pull request: no

Is there any script to create a distilled version of opt or bloom model?

Comments:
This is not an issue, could you maybe ask the question in the [forum](https://discuss.huggingface.co/)? Also, the answer is no. @younesbelkada worked a bit on this so he can answer if you ping him on the forum.
Licensing of LayoutLM models
Author: athuljays
Pull request: no

Hi, Can you clarify the licensing of the layoutLM models provided in HuggingFace?

It is very ambiguous right now. 

Ref: [https://github.com/microsoft/unilm/issues/352](url)



Comments:
Hello @athuljays, thank you for your report. We're currently looking into it and will remove any ambiguity in the coming days.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I've updated the license of LayoutLMv2 and LayoutXLM on the hub to CC BY-NC-SA 4.0 as stated on [this page](https://github.com/microsoft/unilm/tree/master/layoutlmft).

@NielsRogge,
How about the usage of just the code from HuggingFace? Could we use LayoutLMv2 code from HuggingFace, pretrain our own model from scratch and use it in production? 

@NielsRogge,
Can someone from your team confirm on the above comment: https://github.com/huggingface/transformers/issues/13932#issuecomment-1387206937
A bug with subscript while maintaining beam_indices.
Author: jicoder-nwpu
Pull request: no

#### This is the position with this bug: https://github.com/huggingface/transformers/blob/31d452c68b34c2567b62924ee0df40a83cbc52d5/src/transformers/generation/utils.py#L2875
#### The problem caused by this bug:
I want to get all decoder_hidden_states (with dimension [max_seq_len * (1 + decoder_layers) * (num_beam * batch_size) * embedding_size]) and beam_indices to keep track of those hidden states of each word generated.
After some example early stop (\<eos\> is generated), some beam_indices will become zero incorrectly. This bug is caused by the wrong subscript (`beam_indices[beam_idx[i]]`) when the model update the beam_indices and it should be `beam_indices[i]`.

Comments:
cc @gante, but note that there is little we can do to help without a reproducer of the bug.

Hey @aj666aj 👋 

Can I have a script to reproduce your problem? I'll struggle to pinpoint the issue without it, which will make me deprioritize this issue :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix `EfficientFormer`
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix `EfficientFormer`:

  - correct checkpoints
  - fix an device issue regarding `EfficientFormerSelfAttention.ab`, see [failed job run page](https://github.com/huggingface/transformers/actions/runs/3992425421/jobs/6848316487)

    The error:
    ```bash
    (line 136)  RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
    ```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Hubert] Fix Hubert processing auto
Author: younesbelkada
Pull request: yes

# What does this PR do?

Currently on the `main` branch, the scripts provided on the docstring of `Hubert` fails:
```
from transformers import AutoProcessor, HubertModel
from datasets import load_dataset
import soundfile as sf

processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft")
model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft")

def map_to_array(batch):
  speech, _ = sf.read(batch["file"])
  batch["speech"] = speech
  return batch

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
        
ds = ds.map(map_to_array)

input_values = processor(ds["speech"][0], return_tensors="pt").input_values  # Batch size 1
hidden_states = model(input_values).last_hidden_statehidden_states = model(input_values).last_hidden_state
```
With the PR #21225 all custom `xxxProcessor` have been removed in favor of `AutoProcessor` in docstrings. Since `Hubert` was not included inside the processor automapping the script above lead into a bug, since if the model is not present in the auto mapping dictionary, the script will try to load a tokenizer: https://github.com/huggingface/transformers/blob/255257f3ea0862cbb92ea9fa1113cbee1898aadd/src/transformers/models/auto/processing_auto.py#L275 / Hence, `Wav2vec2CTCTokenizer` was loaded instead of `Wav2vec2Processor` that is supposed to be the target object to be loaded.

This PR fixes this by adding `hubert` inside the automapping class for `AutoProcessor`

This PR also fixes 2 failing doctests for `HubertModel`, link to failing job: https://github.com/huggingface/transformers/actions/runs/4002271138/jobs/6869333719

cc @sgugger @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks!
Add metric_key_prefix from training_args
Author: marctorsoc
Pull request: no

### Feature request

Today, if we create a `Trainer` as in
```
trainer = Trainer(
            model=self.model,  # the instantiated 🤗 Transformers model to be trained
            args=training_args,  # training arguments, defined above
            train_dataset=data_cls["train"],
            eval_dataset=data_cls["develop"],
            compute_metrics=partial(
                compute_metrics,
                fbeta_beta=self.config.early_stopping.fbeta_beta,
            ),
            data_collator=collate_chunks,  # type: ignore
            callbacks=callbacks,  # type: ignore
)
```

and do `trainer.train()`:
1. There's no way to change the prefixes for the `train` dataset metrics (at least that I'm aware of)
2. One can change the prefix for the evaluation dataset from the default `eval/` into anything by changing the above into
```
prefix = "other"
Trainer(
         ....
         eval_dataset={prefix: data_cls["develop"]},
         ...
)
```
However, doing this creates `eval/other_accuracy` due to the way [rewrite_logs](https://github.com/huggingface/transformers/blob/e2e393c6f25205739b5dc9fddd460d7bfab85150/src/transformers/integrations.py#L540) works. Ideally I'd like it to be `other/accuracy`.

My request is to have a clear way in the training_args to add any prefixes to the metrics, either for train or eval datasets.

### Motivation

I want to train multiple models within the same wandb run. As things stand right now, the metrics clash

### Your contribution

I've contributed to other OS projects but I'm not very familiar with the codebase to do the contribution directly. If someone guides me around with a high-level description, happy to do it myself

Comments:
cc @sgugger 

This seems like a very niche feature which can be achieved by customizing your callback (you can use your own instead of the default ones).

@sgugger could you elaborate a bit more about what callback(s)? I guess I have to remove one and add mine?

(oh sorry, I just found https://huggingface.co/docs/transformers/main_classes/callback)
Updates to computer vision section of the Preprocess doc
Author: MKhalusova
Pull request: yes

This PR expands the Computer Vision section of the Preprocess doc to include a small explainer on the difference between image augmentation and image preprocessing, and what ImageProcessor handles. It also refactors the code example to use `ImageProcessor` for normalizing and converting images to tensors instead of `torch.transforms`. 
 It mentions padding for certain cases (DETR), and the availability of post-processing methods for some models/tasks. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Notebook examples grouping and update
Author: MKhalusova
Pull request: yes

This PR groups the notebook examples on [this page](https://huggingface.co/docs/transformers/main/en/notebooks) by modality for easier navigation. It also adds a few notebooks from the official repo that were not previously listed, e.g. fine-tuning models for image classification, semantic segmentation, video classification, image similarity, and time series. 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add: TensorFlow example for semantic segmentation task guide
Author: MKhalusova
Pull request: yes

This PR adds a TensorFlow example to the existing [Semantic Segmentation task guide](https://huggingface.co/docs/transformers/main/en/tasks/semantic_segmentation) using the same dataset and fine-tuning steps. 

This example supplements the existing guide and can be helpful to those who choose TensorFlow over PyTorch and would like to use Transformers for semantic segmentation.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Supported pipeline tasks update
Author: MKhalusova
Pull request: yes

The docstring of the `transformers.pipeline` listed only 16 supported tasks while `SUPPORTED_TASKS` contains 24 tasks. This PR adds the missing tasks to the docstrings so that the generated reference docs accurately list all of the supported tasks here - https://huggingface.co/docs/transformers/main/en/main_classes/pipelines#transformers.pipeline 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Adding `evaluate` to the list of libraries required in generated notebooks
Author: MKhalusova
Pull request: yes

This PR is based on the discussion in [doc-builder/Add custom first cell #50](https://github.com/huggingface/doc-builder/pull/50#issuecomment-1359312952).
It modifies the config file that defines the contents of the first cell for the Colab notebooks generated from the doc pages. This change adds `evaluate` to the list of libraries that are installed in the first cell of every generated notebook. 
Currently, only `transformers` and `dataset` libraries are installed by default. However, many notebooks also require `evaluate`. See examples: 
https://huggingface.co/docs/transformers/tasks/sequence_classification
https://huggingface.co/docs/transformers/tasks/semantic_segmentation

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger No problem! I don't have permissions to merge though :)
Refactoring of the text generate API docs
Author: MKhalusova
Pull request: yes

This is part 2 of the refactoring efforts in the text generation docs. The first part (here - https://github.com/huggingface/transformers/pull/21090) adds an introductory guide with examples. 
The second part of the refactor (this PR) reduces repetitive examples and somewhat trims down the API reference doc. Only documentation is affected by this PR.

The text generation API doc page can be trimmed down even further If we remove the docstrings of individual methods like greedy_search(), contrastive_search(), etc. At the moment in 99% of the cases, one can use generate() directly. If it was 100%, I would remove these from the docs. However, I have kept them in the doc at the moment.  



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add: tensorflow example for image classification task guide
Author: MKhalusova
Pull request: yes

This PR addresses https://github.com/huggingface/transformers/issues/21037
It adds a Tensorflow example to the existing task guide on image classification. 

State of the PR: 
The example illustrates preprocessing in TF, training, and pushing to Hub. The code samples have been tested and they work/reproduce. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you all for the reviews! 
I have added the final part - data augmentation (kudos to @sayakpaul for helping me troubleshoot the issues I was having). The example is now complete. Let me know if it looks good enough to be merged :) 
cc @amyeroberts @sgugger
Add: An introductory guide for text generation
Author: MKhalusova
Pull request: yes

Context: current [text generation doc](https://huggingface.co/docs/transformers/main/en/main_classes/text_generation) is large, difficult to navigate, and may be overwhelming to users who are new to text generation.  

I suggest splitting the document into two parts: 
1. an introductory guide that explains the basics and provides some examples
2. trimmed down API reference to be used for looking up specific parameter descriptions (https://github.com/huggingface/transformers/pull/21112)
 
This PR is the first part. It adds a guide that introduces readers to the text generation strategies, explains the generation configuration defaults, and provides some examples. The second part will follow in a separate PR.


 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi! Thank you all for the invaluable feedback! I have addressed all the comments and suggestions. Please let me know if there's anything else I need to improve or if we can merge this PR and https://github.com/huggingface/transformers/pull/21112. 

I think both can be merged. Thank you @MKhalusova ❤️
Small fix to ExponentialDecayLengthPenalty docstring
Author: njhill
Pull request: yes

## What does this PR do?

Currently, the `ExponentialDecayLengthPenalty` doc string incorrectly states that its `exponential_decay_length_penalty` tuple parameter is optional.

Also changed the corresponding type hint to be more specific.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).



## Who can review?

@gante @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._