noamaanMulla-03/transformers-issues huggingface dataset:
Move AutoClasses under Main Classes
Author: stevhliu
Pull request: yes
Labels: name: Documentation

This PR proposes moving `Auto Classes` under the `Main Classes` section instead. As pointed out by @patrickvonplaten, the docs look a little strange because the `Auto Classes` doc isn't bold like the rest of the section and the header name doesn't exactly fit. Since bolded titles mean it is expandable we can't put `Auto Classes` in bold to align it with the rest of the section. `Auto Classes` also includes other things besides models (config, tokenizer, processor, etc.), so it doesn't exactly fit under `Models`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Focus doc around preprocessing classes
Author: stevhliu
Pull request: yes
Labels: name: Documentation

This PR refocuses `preprocessing.mdx` around the preprocessor classes themselves instead of focusing on the different modalities. I think this puts more emphasis on what the library offers as opposed to focusing on something more generic like the different modalities. I know we're going to use `ImageProcessor` for image preprocessing soon, so I can update this again whenever that is ready.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix torch.fx supports (ViT Model)
Author: dwlim-nota
Pull request: yes

# What does this PR do?

Fixes #19209


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@michaelbenayoun 


## Contents

Originally, torch.fx was not working on vit model
because torch.fx proxy object does not working on if-statement
for example,
```if num_channels != self.num_channels:```
can raise error.

1. I replaced if-statements with torch._assert function.

2. below code should be splitted to two lines.
```
if height != self.image_size[0] or width != self.image_size[1]:
```
because `torch._assert(`statement1` and `statement2`, "error message")` will not work.
so, I wrote as below.

```
err_message = f"Input image size ({height}*{width}) doesn't match model({self.image_size[0]}*{self.image_size[1]})."
torch._assert(height == expected_height, err_message)
torch._assert(width == expected_width, err_message)
```



Comments:
_The documentation is not available anymore as the PR was closed or merged._

I replied on the issue thread. It seems like we do not need that.

Yes, I agree with that
I didn't know that
thank you for your comment

I'll close this PR :)
Fix confusing working directory in Push CI
Author: ydshieh
Pull request: yes

# What does this PR do?

⚠️ ~~Let me launch a tiny run to make sure before merge~~ 🙏

Previously, the job `setup` (to prepare the tests to run) in push CI ran on `ubuntu-latest` runners, and has to clone `transformers` with `actions/checkout@v2`.

In #19054, it is changed to use the docker image (and run on our runner). This is no particular reason for it, just to make the sequence of jobs looks more linear.

But I forgot to change the working directory in #19054. The job still run without failure (as the 2 `transformers` directory exist at the same time), but it looks somehow confusing. This PR is to clean this.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
NaN in XGLM Softmax with FP16
Author: gsarti
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.21.0.dev0
- Platform: Linux-5.3.0-1017-x86_64-with-glibc2.27
- Python version: 3.9.13
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.0+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes, 4x V100

### Who can help?

@ydshieh @patrickvonplaten 

### Reproduction

Related to the fixes in #17437 most likely.

I am using an example similar to `test_batched_nan_fp16` in `test_modeling_opt.py`, but for an XGLM model. The only difference with that test is the `torch.cuda.amp.autocast` usage, which I found necessary to perform inference (otherwise I would get an error saying "expected scalar type Float but found Half" coming from the forward of XGLM)

```python
import torch
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer

# Tested with xglm-564M and 7.5B (the second using `infer_auto_device_map` and 
# `load_checkpoint_and_dispatch` from `accelerate`.
tokenizer = AutoTokenizer.from_pretrained("facebook/xglm-564M", padding_side="left")
model = AutoModelForCausalLM.from_pretrained("facebook/xglm-564M", torch_dtype=torch.float16, use_cache=True).cuda()

batch = tokenizer(["Who are you?", "Joe Biden is the president of"], padding=True, return_tensors="pt")

input_ids = batch["input_ids"].cuda()
attention_mask = batch["attention_mask"].cuda()

with torch.no_grad():
    with torch.cuda.amp.autocast():
        outputs = model(input_ids, attention_mask=attention_mask)
        assert not torch.isnan(outputs.logits[0]).any().item() # Raises an AssertionError
```

### Expected behavior

I would expect the model to have normal logits when using FP16. The spotting of this bug was prompted by an issue of garbage generation when doing batching, despite the left padding and a valid attention mask.

Comments:
cc @patil-suraj for XLGM

From `padding=left` in the code snippet, I would guess this is similar to issue #17433 and the fix in #17437 should fix it. @gsarti Would you like to try it and maybe also open a PR?

P.S I actually thought yesterday if I should apply #17437 to all models, and tried a few models like GPT2, Bart which are fine. Maybe it is indeed better to do it for all models.




Opened PR #18057 with the suggested fix @ydshieh, should be good to go!

Can we close this one? cc @ydshieh ? 

I re-opened that PR #18057. Let's see if @gsarti would like to continue the work, otherwise I can take it.
The necessary fix is minimal.
Fixes HFTracer getattr for PyTorch 1.13+
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

There was an API change where `torch.fx.Tracer._module_getattr` became `torch.fx.Tracer.getattr`, causing issues when using the `HFTracer` with `torch-nightly`. This PR fixes that.

Fixes: https://github.com/pytorch/pytorch/issues/84840 (@pbelevich)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @Chillee
Fix opt softmax small nit
Author: younesbelkada
Pull request: yes

# What does this PR do?

- Following the PR #18057 we found out that the argument `dtype_attn_weights ` is not defined outside the condition `if attention_mask is not None:` which makes the modeling code prone to potential bugs. Also, the solution provided in #18057 is clearer so this PR addresses the same change to make the implementations consistent across models that suffers from the same Softmax issue. 

cc @ydshieh @sgugger 

Can also confirm slow tests pass!

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use `hf_raise_for_status` instead of deprecated `_raise_for_status`
Author: Wauplin
Pull request: yes

# What does this PR do?

Replace `huggingface_hub.utils._errors._raise_for_status` by the officially supported `huggingface_hub.utils.hf_raise_for_status`.

Related to https://github.com/huggingface/huggingface_hub/pull/1019#issuecomment-1233864547.

Requires `huggingface_hub>=0.10.0`.

Comments:
Yep true. Thanks for letting me know. Is it also required to run `make deps_table_update` ? I did it in [my last commit ](https://github.com/huggingface/transformers/pull/19244/commits/f0dcac9fdf512105f01a5dd55bc1229a63cb6e73) but I can remove it.

_The documentation is not available anymore as the PR was closed or merged._

Ok, I should be good now. @sgugger I let you merge it when it's green :)
TrainingArguments._n_gpus serialization
Author: atturaioe
Pull request: no

I'm not sure, but I think that `TrainingArguments._n_gpu` attribute is not supposed to be serialized in `TrainingArguments.to_dict()` .
Or even basically all the attributes that are defined as `dataclass.field(init=False...)` should be filtered out while doing `to_dict()`.

What do you think?

@sgugger 

Comments:
Indeed, this argument shouldn't be serialized.

I'm on it
Fix TrainingArgs argument serialization
Author: atturaioe
Pull request: yes

# What does this PR do?
Filter out arguments in TrainingArguments.to_dict() such as `field(init=False)` so arguments such as `_n_gpu` won't be serialized.

Fixes #19236 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix test fetching for examples
Author: sgugger
Pull request: yes

# What does this PR do?

When re-working the circleCI config to execute test fetching first, I didn't do anything special for the examples, which means that if only an example file is modified, no tests are run (by default only tests found in the tests subfolder are kept, for the examples we need to add `--filters tests examples`.

You can check on [this report](https://github.com/huggingface/transformers/runs/8604472112) that when only example files are modifier, the tests for all examples are still run (next step would be to properly filter between TF/Flax and PyTorch but this is going to take slightly longer).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Enable numpy behaviour across the TF codebase
Author: Rocketknight1
Pull request: yes

This PR enables TF Numpy behaviour for all of our models, which will hopefully reduce the number of PRs we have to make to fix dtype incompatibilities. This flag mainly relates to type promotion (TensorFlow is strict by default, and won't automatically promote if you e.g. add an `int32` tensor to an `int64`), but it also enables some Numpy-like indexing that the standard TF indexing doesn't allow.

It shouldn't break anything, and despite being marked as experimental it's been in TF since 2.4 and actively promoted by Chollet on his Twitter, so I think it's stable and here to stay.

Relevant TF documentation is [here](https://www.tensorflow.org/api_docs/python/tf/experimental/numpy/experimental_enable_numpy_behavior)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19246). All of your documentation changes will be reflected on that endpoint.

Update: We realized some issues related to float promotion that would make this very messy, closing this PR for now!
Adding a documentation to save the best checkpoint during the training in summarization example project
Author: shamanez
Pull request: no

### Feature request

I am working with the latest [latest summarization example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/summarization). But after the training loop ends, I can't find an easy way to find what is the best checkpoint saved. 

### Motivation

I would be easy to keep a separate checkpoint for the best one.

### Your contribution

If there's already a way, please let me know, so that we can document it. Else I can contribute to this feature. I think we can use trainer_arguments. 

Comments:
Hi @shamanez  👋 As per our [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md), we reserve GitHub issues for bugs in the repository and/or feature requests. For any other matters, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Cast TF generate() inputs
Author: Rocketknight1
Pull request: yes

Enable TF `generate()` to support any integer dtype by adding casts at the beginning of the main method.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@gante Let me know if you're okay with the slightly cheeky `-1`, and other than that I'm happy to merge whenever!

haha the cheeky `-1` is fine for me. 

I do have one additional nit: can it be moved to after the code in step `0`? The idea would be to run a quick validation of arguments and classes before doing any actual work. (My bad, I should have been more precise in my prior comment 🤦 )

@gante Done! The failing tests are hub server-side issues.

Thanks 🙏
Add and save job names in Past CI artifacts
Author: ydshieh
Pull request: yes

# What does this PR do?

In order to ease the fix process for Past CI, it would be very helpful to have links to jobs for the failed tests.

The GitHub API could give a list of jobs and a list of artifacts, but there is no ensured and easy way to get the  correspondence between each artifact/job, see [jobs here](https://api.github.com/repos/huggingface/transformers/actions/runs/3145556680/jobs) and [artifacts here](https://api.github.com/repos/huggingface/transformers/actions/runs/3145556680/artifacts).

Moreover, when the jobs are running, there is no context evn. variable that gives the job run link (or anything helpful to get this information)

However, the list of jobs given by the API contains the **job names** and the **job run page** link (given by `html_url`).

This PR therefore saves the job names in the corresponding test artifacts. Once the past CI is done:

- we extract the error information together the job names
- we fetch the list of jobs
- For each artifact, we find the corresponding job in the job list using the job name saved in the artifact
- We get the job run link by looking `html_url`
- We add the job run link in the error info.**

To check the effect of this PR, see the contents in the artifacts in [this run](https://github.com/huggingface/transformers/actions/runs/3145556680)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update Past CI report script
Author: ydshieh
Pull request: yes

# What does this PR do?

To make the fix process easier:
- A single file `errors.json` (instead of 2 files) that contains `line of error`, `error`, `failed test` and `job_link` together for each element. See below.
- The table for reporting on GitHub page get a new `status` column - in order to update the progress.

Elements in `errors.json` looks now

```python
[
    "/transformers/src/transformers/configuration_utils.py:642",
    "OSError: gs555750 is not a valid git identifier (branch name, tag name or commit id) that exists for this model name. Check the model page at 'https://huggingface.co/bigscience/bloom-350m' for available revisions.",
    "tests/models/bloom/test_modeling_bloom.py::BloomModelTest::test_batch_generation",
    "https://github.com/huggingface/transformers/actions/runs/3145556680/jobs/5112996392"
],
```

Having a job link attached to each element, so we can click or copy/paste to go the the job run page for the full traceback. However, this information is not available yet. ~~I will try to obtain them~~ #19235 is for this goal.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Wav2Vec2] Fix None loss in doc examples
Author: rbsteinm
Pull request: yes

# What does this PR do?

Doc examples in [Wav2Vec2ForPreTraining](https://huggingface.co/docs/transformers/model_doc/wav2vec2#transformers.Wav2Vec2ForPreTraining.forward.example) and  [Wav2Vec2ConformerForPreTraining](https://huggingface.co/docs/transformers/model_doc/wav2vec2-conformer#transformers.Wav2Vec2ConformerForPreTraining.forward.example) produce None loss due to missing sampled_negative_indices parameter in the model. See #15232 

* pass sampled_negative_indices parameter to the model to avoid getting a None loss
* The sequence length is a tensor when it should be an integer. Add .item() call to address this issue.

Fixes #15232 


## Who can review?

@patrickvonplaten 





Comments:
_The documentation is not available anymore as the PR was closed or merged._

@patrickvonplaten Could you take a look at this PR for Wav2Vec2, thanks :-)
Updated README and docs with ESM-1b and ESM-2 separately
Author: tomsercu
Pull request: yes

# Improve ESM documentation
We split out the papers for ESM-1b and ESM-2 separately, and improve the documentation of esm model_doc.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@Rocketknight1 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @tomsercu! I checked with other team members and we prefer to have only one entry in the list per model class (which would include ESM-1b as well as ESM-2), but it's totally okay to have a longer entry that includes the citations for ESM-1b and ESM-2, and optionally ESM-1v as well.

Gotcha that makes sense @Rocketknight1 , updated to a single entry in all 5 places. Lmk if this is too much :D 

@tomsercu Don't worry, we have some scripts to handle the copies, so you only have to worry about the English one! I'll run them now and see if I can get these tests to pass.

Remaining issues seem to just be from the specific commit this was branched from - will merge into my PR and hopefully they'll go away. Thanks for this!
Add MarkupLM
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR implements the [MarkupLM](https://arxiv.org/abs/2110.08518) model by Microsoft Research.

MarkupLM is very similar to LayoutLM(v1), namely a Transformer encoder pre-trained on another domain than just text (HTML web pages). Similar to how LayoutLM adds additional embeddings for the layout information, MarkupLM adds additional embeddings for the XPATH information of nodes within an HTML string.

To do:

- [x] add soft dependency check for `MarkupLMFeatureExtractor`. cc @sgugger, will need some help here. New dummy objects probably need to be created for bs4 (Beautiful Soup 4). These currently make the entire CI fail :(
- [x] fix fast tokenizer

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Ok, so I added requires_backends("bs4"), however the dependency check is not working for me:

```
(env) niels@brutasse:~/python_projects/transformers$ python
Python 3.9.10 (main, Jan 25 2022, 09:49:28) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bs4
>>> from transformers.utils import is_bs4_available
>>> is_bs4_available()
False
```
The import check is defined as follows:

```
_bs4_available = importlib.util.find_spec("bs4") is not None
try:
    _bs4_version = importlib_metadata.version("bs4")
    logger.debug(f"Successfully imported bs4 version {_bs4_version}")
except importlib_metadata.PackageNotFoundError:
    _bs4_available = False
```

@sgugger I've addressed all your comments, feel free to approve :)
Using Huggingface Trainer in Colab -> Disk Full
Author: kforcodeai
Pull request: no

https://discuss.huggingface.co/t/using-huggingface-trainer-in-colab-disk-full/5951/2 

I faced the same issue mentioned in this discussion. Please give it a look.

[## Environment info
- `transformers` version: 4.15.0
- Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.10.0+cu111 (True)
- Tensorflow version (GPU?): 2.7.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help


- Trainer: @sgugger


## Information
This issue is model agnostic
I believe it's task agnostic too

## To reproduce

Steps to reproduce the behavior:

1. Run this notebook in colab
 https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb#scrollTo=YpvnFFmZJD-N

## Expected behavior
1. Disk should not go out of space,  Because I used `save_total_limit=1`  in the training argument. So previous checkpoints should be deleted


Comments:
I guess I found the reason, 
on deleting the previous checkpoint, it goes to the google drive bin and the bin does not delete it then (deletes after 30 days) and this results in occupied space. 
One way to solve this will be overriding the `_rotate_checkpoint` method of the `Trainer` to also clean the drive bin, but this would require authentication from the user every time to empty the bin.
Any idea around implementing the same?


Looks like an issue on Colab that they don't let the user choose to delete things without going in that bin (or clear that bin when space is needed).

The only workaround I can think of is to use a very high value as a saving step, or to disable the saving altogether during training with `save_strategy="no"`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.


> Looks like an issue on Colab that they don't let the user choose to delete things without going in that bin (or clear that bin when space is needed).
> 
> The only workaround I can think of is to use a very high value as a saving step, or to disable the saving altogether during training with `save_strategy="no"`.


Still facing the same issue.

+Having save_strategy = 'no' means that we are not able to pick the best model based on our evaluation metric :(


You can raise the issue on Colab. There is really nothing we can do on our side since the problem is that Colab does not let us delete older checkpoints.