noamaanMulla-03/transformers-issues huggingface dataset:
CircleCI branch filtering on push-ci branch
Author: ydshieh
Pull request: yes

To avoid the branch `push-ci` (the branch to launch the actual Push CI on GitHub actions) to run `CircleCI` tests, as those are already run when the commits are merged into `main`.

The `filters` could not be added directly in the job definitions, like
```
    run_tests_torch_and_tf_all:
        filters:
            branches:
                ignore:
                    - circleci_branch_filtering
        working_directory: ~/transformers
```
nor
```
    run_tests_torch_and_tf_all:
        <<: *job_filters
        working_directory: ~/transformers
```
otherwise I got error
```
... [#/jobs/check_code_quality] extraneous key [filters] is not permitted|   |   Permitted keys:
```

See
https://stackoverflow.com/a/57366603

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I will close this PR, once #17692 is merged (as it is preferred).
Update GPT2 I/O definition to be recognized by ORT optimizer.
Author: mfuntowicz
Pull request: yes

This PR aims at making the GPT2 + past more compatible with ONNX Runtime optimizer (especially attention fusion).
Past key values should be presented as a single tensor with both key and value stacked on the leading axis.

It also provide a concat mecanism to merge the resulting past key values so it's a single tensor too.

- [x] I/O shapes changes
- [x] I/O dtype changes
- [ ] Past keys wrapper
- [ ] Monkey Patching
- [ ] Unittests

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16754). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add support for moving Trainer out of GPU memory
Author: ClonedOne
Pull request: no

# 🚀 Feature request

Hello! I couldn't find any way of doing this, so I'm opening this issue. I'm sorry if there is already support for this and I couldn't find it.

It would be useful to have the ability to completely remove a Trainer and all associated objects (model, optimizer, etc..) out of the GPU memory without having to destroy the Trainer object. This should also allow the reverse operation, that is, once the Trainer is on CPU, it should be possible to move it back on GPU -- including all components and accounting for multi-GPU parallelism.

Essentially, it would be nice to have a `trainer.move_to_device()` function that took care of everything.

## Motivation

In my particular use case I have 2 relatively small GPUs and I would like to train multiple models in the same script in an interleaved fashion.  Something like:
``` python
# Ugly semi-pseudocode
while not done:
  for trainer in trainer_list:
    trainer.move_to_device(cuda)
    for epoch in range(epochs):  
      trainer.train()
    trainer.move_to_device(cpu)
```

I suspect it could also be useful for other scenarios where VRAM is limited and someone could want to train multiple models.

ps: I'm aware of `_move_model_to_device()`, but that seems to be limited to the model weights.

## Your contribution

I don't know much about how the Trainer handles device placement, but I'd be happy to test any proposed solution. 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Sorry for not seeing this earlier! Pinging @sgugger for advice.

This might be useful indeed, but is not something I will have time to work on any time soon. Always happy to review a PR however!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Flax implementation of DPT
Author: younesbelkada
Pull request: yes

# What does this PR do?

I tried to implement DPT (Dense Prediction with Transformers) in Flax during my free time! 🚀 
By the way it is the first Segmentation and Depth Estimation model implemented in Flax!

Nits/TODOs:
- [x] Figure out how to properly call `BatchNorm` and `Dropout` inside a `Sequential` 
- [ ] Test equivalency tests 
- [ ] Write documentation - For now they're just copy/pasted
Quetions:
- Why the loss is not implemented in `modeling_dpt.py` ? I can probably help on that since I have already implemented the loss for a university project: https://github.com/antocad/FocusOnDepth/blob/master/FOD/Loss.py 
 
cc @NielsRogge @sanchit-gandhi @patil-suraj 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Attempt to change Push CI to workflow_run
Author: ydshieh
Pull request: yes

# What does this PR do?

This is a fix for #17692 :

- Use the correct properties to get the information (branch/commit-SHA, etc.) for both `push` and `workflow_run` event type 
  - Even if we consider only the `workflow_run` event triggered by a push to `main` branch, we still need to use `github.event.workflow_run.head_sha` to get the correct SHA (otherwise, in the case where 2 PRs merged into `main` in a very short time period, the first one will get the latest commit SHA)
  - Currently, the push CI could still be triggered by `push` event if the branch is **NOT** `main`. The main purpose is for testing a particular branch. This is why I need to consider both event type.

**NOTE**: I have verified the change extensively in my own (dummy) repo. However, the part regarding the actual CI tests + the part of slack report are not verified. **The part regarding preparing the necessary information for slack report is verified.** Since the `workflow_run` could be launched only when the PR is merged into `main`, I hope there is no other unexpected issue in this PR 🙏.




Comments:
_The documentation is not available anymore as the PR was closed or merged._

I will merge this tomorrow (so unlikely to have other PRs merged), and revert it if anything breaks (hopefully not, I am running out of ideas 😄 )

This works well, one good example to check is

[TF: BART compatible with XLA generation](https://github.com/huggingface/transformers/commit/132402d752044301b37e54405832738b16f49df6)
GPT-NEOX RuntimeError
Author: yupei9
Pull request: no

Hi, when I ran the model GPT-NEOX, I got the "RuntimeError: batch1 dim 2 must match batch2 dim1" in modeling_gpt_neox.py, line 212. 
So I tried to debugg and fix this problem, I found the code "present = None if use_cache else (key, value)" in modeling_gpt_neox.py, line 146. 
Is that logical wrong? and the correct coding should be "present = None if not use_cache else (key, value)" ?


Comments:
Hey @yupei9 - great catch! I think you're 100% right - do you want to open a PR to fix it? Also cc @sgugger 

Ahah sorry, I had a PR ready already since I wanted to test if the fix worked.
deprecate is_torch_bf16_available
Author: stas00
Pull request: yes

This is a follow up to https://github.com/huggingface/transformers/pull/17734 where @pacman100 discovered that the IPEX PR made `is_torch_bf16_available` ambiguous, as it went from gpu-only checks to cpu or gpu which is undefined behavior. 

So this PR deprecates this function in favor of the very specific `is_torch_bf16_gpu_available` and `is_torch_bf16_cpu_available` that were added in https://github.com/huggingface/transformers/pull/17734

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

does it look good now, Sylvain? 

Yes, all good :-) Thanks again!
Fix cache for GPT-Neo-X
Author: sgugger
Pull request: yes

# What does this PR do?

As pointed out on #17745, there is a problem with the logic in the cache for GPT-Neo-X. Can confirm I can use it for generation after this PR, but not before.

Fixes #17745

Comments:
Thanks a lot! If it's not too much work could you maybe try running this test: https://github.com/huggingface/transformers/blob/522a9ece4baeb5abfec8953ef76469a530e987d5/tests/models/gpt_neox/test_modeling_gpt_neox.py#L144 

I think right now it's not run (seems like the test function was removed)

_The documentation is not available anymore as the PR was closed or merged._

I can also do it also otherwise :-)

Added tests that were removed and had a corresponding `check` function. Thanks for flagging this!
Fx support for Deberta-v[1-2], Hubert and LXMERT
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

Adds `torch.fx` tracing support for:

- Deberta v1
- Deberta v2
- Hubert
- LXMERT

Comments:
_The documentation is not available anymore as the PR was closed or merged._
LayoutLMv2 Fast Tokenizer improperly aligns labels for non-first subwords with 0 offsets
Author: timothyjlaurent
Pull request: no
Labels: name: bug

### System Info

```shell
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.16.0
- Platform: macOS-12.3.1-x86_64-i386-64bit
- Python version: 3.8.9
- PyTorch version (GPU?): 1.8.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

 @NielsRogge

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
In [1]: from transformers import LayoutLMv2TokenizerFast
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

In [2]: tokenizer = LayoutLMv2TokenizerFast.from_pretrained("microsoft/layoutlmv2-base-uncased")

In [4]: toks = tokenizer(['그'], boxes=[[1,2,3,4]], word_labels=[2])

In [5]: toks
Out[5]: {'input_ids': [101, 1455, 30017, 102], 'token_type_ids': [0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1], 'bbox': [[0, 0, 0, 0], [1, 2, 3, 4], [1, 2, 3, 4], [1000, 1000, 1000, 1000]], 'labels': [-100, 2, 2, -100]}

In [9]: toks.labels
Out[9]: [-100, 2, 2, -100]

```

### Expected behavior

```shell
Since the single word '그' was split into 2 tokens, the second subword should be assigned a -100 value.

This is happening because of logic here:

https://github.com/huggingface/transformers/blob/3f936df66287f557c6528912a9a68d7850913b9b/src/transformers/models/layoutlmv2/tokenization_layoutlmv2_fast.py#L599


where a token's offset is used to determine whether it is the first subword or not.

The problem with '그' is that it is split into 2 subword: 'ᄀ', '##ᅳ' that share an offset of (0,1) so both given the label of the word.

By using solely the word_index to decide which is a subword this problem can be avoided.
```


Comments:
https://github.com/huggingface/transformers/blob/3f936df66287f557c6528912a9a68d7850913b9b/src/transformers/models/layoutlmv2/tokenization_layoutlmv2_fast.py#L599



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
For model long-t5-tglobal-x, fix 'float' object cannot be interpreted as an integer
Author: bjascob
Pull request: yes

On line 180, `torch.tensor(-1.0, dtype=global_block_ids.dtype)` gives the error __TypeError: 'float' object cannot be interpreted as an integer__ .  This is because the dtype here is `int64`.  For `dtype=int64`, this needs to simply be `-1`.  

This impacts the `long-t5-tglogbal-x` model.  It does not impact the `long-t5-local-x` version which does not appear to call this line in the code.

The torch version where I see this is 1.11.0+cu113.  I'm not certain if older, or non-gpu versions of torch allowed this but 1.11.0+cu113 does not.

Note that torch does not complain when casting an int to a float so it should be safe to change this to `-1` even if there are occasions where `global_block_ids.dtype` is a float.

# What does this PR do?

Fixes # (no issue # created). 
There is a simple error in the code where torch fails when trying to create a constant int64 tensor using `-1.0` instead of `-1`.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
This model is new.  I would suggest someone from the original upload team review this.  Here are the first 3 in the file history.. 
@stancld, @PhungVanDuy, @sgugger



Comments:
_The documentation is not available anymore as the PR was closed or merged._

@patil-suraj could you maybe take a look here? :-)

Also cc @stancld in case you're interested and have an idea what the problem could be

Interesting. Looks like this is a change in python, not torch.  UBT 22.04 uses Python 3.10.4 and this is fully broken for that version.

Great looks good to me than as well!
Is there Any difference of performance when finetuning bert use the huggingface or the google official code?
Author: Doragd
Pull request: no

Hi, 
I tend to finetune BERT with a simple text classification task.
However, I got different results when using the huggingface library (torch 1.8.1+cu111) and google's official code (tf 1.15).
I wonder if there is any optimization in huggingface for fine-tuning bert?

By the way, I believe that I use the same hyper-parameters. But I got the higher performance using the huggingface library

Comments:
Hi @Doragd  👋 As per our [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md), we reserve GitHub issues for bugs in the repository and/or feature requests. For any other matters, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗

all right. Thanks anyway. @gante
How to checkpoint TFAutoModelForSequenceClassification every k batches
Author: preethiseshadri518
Pull request: no

I am following the Huggingface Tensorflow Notebook to [train a BERT model on MNLI](https://github.com/huggingface/notebooks/blob/main/examples/text_classification-tf.ipynb) (the task itself is irrelevant). I am interested in checkpointing models locally every k batches, so that I end with 20 or 30 intermediate checkpoints over the course of training. The notebook uses model.fit() to train the model, but my understanding is that using a [callback](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpoint) to save intermediate checkpoints saves a SavedModel file. 

I am currently not saving intermediate checkpoints, only the final model, and use model.save_pretrained(ckpt). However, I now want the model to be checkpointed as a part of training. If I try loading a checkpoint directory in the SavedModel format using `model = tf.keras.models.load_model()`, the expected input format is not compatible with how I feed in inputs for `model = TFAutoModelForSequenceClassification.from_pretrained()`. I get the following message (32 is batch size, I think 109 is the input token length for the longest example in the batch). 
<img width="686" alt="Screen Shot 2022-06-17 at 7 20 14 PM" src="https://user-images.githubusercontent.com/60128552/174418985-4331fca5-4b1b-4486-90f2-4f98b98a35d1.png">


My question is how to checkpoint a `TFAutoModelForSequenceClassification` model with a specified frequency (every k batches). Either I have to change how I am feeding the input data into `model = tf.keras.models.load_model()` or change how I am saving the intermediate checkpoints, but I am having trouble figuring out what to do and how to proceed. 
Note: I am not interested in pushing models to the hub, only saving models locally. 

Comments:
Hey @preethiseshadri518 👋  We have an outstanding set of issues related to the `SavedModel` format, resulting in the errors you see. There are ways to work around it, by manually specifying portions of the graph at save time -- you can find them if you search in closed issues here :)

A much simpler workaround, if it suits your problem, is to store/load the weights with [`save_weights`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#save_weights) and [`load_weights`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#load_weights), which is what our API uses. Let us know if it sorted your problem!

Hi Joao! Yes, this solves the issue. If I use `save_weights` and save them in the h5 format, then I can use  `TFAutoModelForSequenceClassification.from_pretrained()`. Thanks for your comment!
Add MVP model
Author: StevenTang1998
Pull request: no
Labels: name: New model

### Model description

Hi, I want to add my model MVP to hugging face. My model is very similar with BART. The difference is that it has pre-trained soft prompt in the format of Prefix-tuning.

We have pretrained a series of prompts combining with backbone transformer. And users can choose which prompt to load. So I wonder where I can change the behaviour of loading which prompt?

To put it simply, our model is composed of a big nn.Transformer and a small nn.Linear. We have pre-trained a Transformer and _**n**_ Linear. Users can choose which Linear to load.
One solution can be that, we upload _**n**_ chechpoints, including the common Transformer and respective Linear. And users just write `model.from.pretrained('model1/2/3')` to load which prompt. However, the it is a bit waste to upload and download the common Transformer **_n_** times.

So I wonder how I can solve it with a simple method. Such as our uploaded model is consist of Transformer and **_n_** prompts, and users can choose which prompt to use through config.





### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Hello! I think this is a situation where the code-on-the-hub approach is likely to be the best way to share it! See this guide: [Sharing custom models](https://huggingface.co/docs/transformers/custom_models).

This will give you the most amount of freedom with regards to your model.

cc @sgugger 

Thank you! I will take a look.

Great! Please let us know if you run into any issues, or if anything is unclear from the guide. Thanks!

Hi, sorry I still don't know how to handle a situation like mine.

My model is 
```
(
BertModel()
nn.Linear()
)
```

And my checkpoint is composed of
```
(
BertModel()
'a': nn.Linear()
'b': nn.Linear()
'c': nn.Linear()
)
```

And my expected effect is that user can determine which Linear to use through the config, such as `XXX.from_pretrained('my_model', linear='a' or 'b' or 'c')` to load according linear.
However, the model initilization is before the model loading. So I don't know how to solve it.

A quick solution would be to have three architectures, one with `a`, one with `b`, and one with `c` linear layers and load the one you want accordingly.

Do you mean I upload three models, named my-model-a, my-model-b and my-model-c, then load the one I want?



You can upload one single checkpoint, but three different architectures that each use the `a`, `b`, and `c`. Similarly to models trained on sequence classification (or any other task): they can be loaded in base models, in question answering models, etc. It will only load the layers it needs, and ignore the rest.

Thanks for your response! I understand what you mean.

Actually, my situation is a little different from the task head. The prompt is the part of the encoder and decoder, and I have 8 different prompts rather than 3. So I wonder if there is any other way, considering they have the same architecture.
Error in TAPAS Tokenizer 
Author: shivangibithel
Pull request: no

### System Info

```shell
Google Colab CPU
```


### Who can help?

@SaulLu  @NielsRogge

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

# TODO: should this be in the base class?
https://github.com/huggingface/transformers/blob/3f936df66287f557c6528912a9a68d7850913b9b/src/transformers/tokenization_utils.py#L507

Giving Error in the above function , though the code should not be called

from bs4 import BeautifulSoup
from urllib.request import Request, urlopen
import re
import regex
import pandas as pd # library for data analysis
import requests # library to handle requests
from bs4 import BeautifulSoup # library to parse HTML documents
import numpy as np
from transformers import TapasTokenizer, TapasModel
import pandas as pd

tokenizer = TapasTokenizer.from_pretrained("google/tapas-base")
model = TapasModel.from_pretrained("google/tapas-base")

table_class="wikitable sortable jquery-tablesorter"
response=requests.get("https://en.wikipedia.org/wiki/2017_EFL_Trophy_Final")
soup = BeautifulSoup(response.text, 'html.parser')
indiatable=soup.find('table',{'class':"wikitable"})
if indiatable:
    count +=1        
    df=pd.read_html(str(indiatable))
    df=pd.DataFrame(df[0])   
    df = df.astype(str, errors='ignore')
    queries=["2017 EFL Trophy Final"]     
    inputs = tokenizer(table=df, queries=queries, do_lower_case=False, do_basic_tokenize=False, padding="max_length", 
    return_tensors="pt", truncation = True)
        


![image](https://user-images.githubusercontent.com/19774925/170099245-50cb73e5-e009-4a31-9b53-fc4844890dad.png)


### Expected behavior

```shell
This attribute is set false explicitly, thus should not be called
```


Comments:
Hello @shivangibithel,

You are entering the following loop:
https://github.com/huggingface/transformers/blob/71e602725b90f63f404109bae9f72cbdf755477b/src/transformers/tokenization_utils.py#L508-L514

because `do_lower_case` is a paramer that need to be filled in when initializing the tokenizer (or redefined in `from_pretrained`) but not in the `__call__` method. I'm also taking this opportunity to highlight that `do_basic_tokenize` is also a parameter that can't be changed in  the `__call__` method.

I hope this will help you! :relaxed: 

I also take this opportunity to share [some tips](https://github.com/huggingface/transformers/blob/main/ISSUES.md#the-github-issues) that would help us a lot to read quickly your issue :slightly_smiling_face: . 

Closing this issue due to inactivity :slightly_smiling_face: