noamaanMulla-03/transformers-issues huggingface dataset:
Selecting specific GPU CUDA devices
Author: kenghweeng
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.0
- Platform: Linux-4.15.0-65-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.5.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: 

### Who can help
Hello @sgugger,

Steps to reproduce the behavior:

1. I would like to use selected CUDA GPU cores among 8 of them in the HF `Trainer` class. I've written something along the following lines: 
2. So, I've done `export CUDA_VISIBLE_DEVICES=1,8` to select specific GPU devices, and ran:

```
training_args = TrainingArguments(
            output_dir=self._output_dir,
            overwrite_output_dir=True,
            num_train_epochs=epochs,
            per_device_train_batch_size=batch_size,
            per_device_eval_batch_size= int(batch_size/2), # since evaluation per 
            logging_steps = 20,
            save_total_limit = 20,
            warmup_steps=500, 
            weight_decay=0.01,
            evaluation_strategy = "steps",
            load_best_model_at_end = True,
            eval_accumulation_steps = 1,
            logging_dir = "logs"
        )
        
trainer = Trainer(
            model=self._model,
            args=training_args,
            tokenizer=self._tokenizer,
            data_collator=self._data_collator,
            train_dataset=self._train,
            eval_dataset = self._test,
        )
        
print("Devices used are:")
print(training_args.device)
```

## Expected behavior
I was under the impression that the `training_args.device` should return me cuda:1,8 or something along those lines, but it still reverted back to cuda:0. Are there any arguments I could specify to select a particular core.

Thank you in advance!


Comments:
When you do `CUDA_VISIBLE_DEVICES=1,8`, CUDA will still call the two available GPUs 0 and 1, 0 will correspond to 1 and 1 to 8. I f you look at the output of `nvidia-smi`, you will see the training will only run on GPUs 1 and 8.

Thank you, silly me. I'll close the issue, thanks!
GPT2 medium config n_ctx is wrong I guess?
Author: s4sarath
Pull request: no

Hi Guys,
For gpt2-medium ```n_ctx: 4096``` right ?
But in config it is showing ```n_ctx: 1024``` . 

Comments:
No, 1024 is correct. It refers to the sequence length of the model. 

Oh sorry.
Then whats the config parameter of ```intermediate projection after attention```, which is ```4096``` in gpt2-medium.

Looking at the [config attributes of GPT-2](https://huggingface.co/transformers/model_doc/gpt2.html#transformers.GPT2Config), there's an attribute called `n_inner` which is defined as "Dimensionality of the inner feed-forward layers. None will set it to 4 times `n_embd`". 

Apparently, the `n_embd` attribute of the medium-sized GPT-2 model is 1024. So this times 4 equals 4096. 

Oh my bad. Thanks. There is too much of inconsistencies between different model configs.
Fix a condition in test_generate_with_head_masking
Author: stancld
Pull request: yes

Fix a glitch in a condition in `test_generate_with_headmasking`, i.e.

```diff
- if set(head_masking.keys()) < set([*signature.parameters.keys()]):
+ if set(head_masking.keys()) > set([*signature.parameters.keys()]):
    continue
```

This PR also fixes usage of head_mask for bigbird_pegasus and speech2texy models.

**Reviewer:** @patrickvonplaten 

Comments:
[Flax] Adding Visual-Transformer
Author: jayendra13
Pull request: yes
Labels: name: Flax

# What does this PR do?

This PR adds the `ViT` model in JAX/Flax

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #11948


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


Comments:
Awesome! 
The equivalence tests are still failing, from CI logs
```
FAILED tests/test_modeling_flax_vit.py::FlaxViTModelTest::test_equivalence_flax_to_pt
FAILED tests/test_modeling_flax_vit.py::FlaxViTModelTest::test_equivalence_pt_to_flax
```

Could you fix these, otherwise lmk and I will take care of it :) 

sorry having a busy week, will back to this on the weekend :weary:
[examples] add desc to `dataset.map` to improve tqdm bars
Author: stas00
Pull request: no

https://github.com/huggingface/datasets/pull/2374 has been merged - we should deploy this feature in our examples, which would tell the user what's being processed and the tqdm bar is for.

Currently we get a bunch of bars that are absolutely meaningless and hard to understand what they do. See also: https://github.com/huggingface/datasets/issues/2330

The only issue is how to depend on `datasets` dev version, might have to wait for a new `datasets` release 1.6.3 to be able to merge such PR.

A new release should be made in the next few days I'm being told, so a PR can be made.

Comments:
[Question] How to move and reuse preprocessed dataset?
Author: AtmaHou
Pull request: no

Hi, I am training a gpt-2 from scratch using run_clm.py.

I want to move and reuse the preprocessed dataset (It take 2 hour to preprocess),

I tried to :
1. copy `path_to_cache_dir/datasets` to  `new_cache_dir/datasets` 
2. set `export HF_DATASETS_CACHE="new_cache_dir/"`

but the program still re-preprocess the whole dataset without loading cache.

I also tried to `torch.save(lm_datasets, fw)`, but the saved file is only 14M.

What is the proper way to do this?

Comments:
cc @lhoestq the preprocessed dataset should be cached, right?

Hi ! Yes it should, as long as you didn't change any parameter passed to the `map` function. They must be exactly the same.

Could you open an issue on the `datasets` repo at https://github.com/huggingface/datasets if you want to discuss this caching issue in more details ?

> Hi ! Yes it should, as long as you didn't change any parameter passed to the `map` function. They must be exactly the same.
> 
> Could you open an issue on the `datasets` repo at https://github.com/huggingface/datasets if you want to discuss this caching issue in more details ?

Sure thanks, the new issue is at here:
https://github.com/huggingface/datasets/issues/2345

> Hi ! Yes it should, as long as you didn't change any parameter passed to the `map` function. They must be exactly the same.
> 
> Could you open an issue on the `datasets` repo at https://github.com/huggingface/datasets if you want to discuss this caching issue in more details ?

I tried to re-run the example [script ](https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_clm.py) after a success running (preprocess finished and start training) it still re-preprocess all data.

**Details:**
(1) It re-preprocess data even after showing: `05/11/2021 11:47:01 - WARNING - datasets.builder -   Reusing dataset text (/home/cache/text/default-7083a0557f2cff9e/0.0.0/e16f44aa1b321ece1f87b07977cc5d70be93d69b20486d6dacd62e12cf25c9a5)`
(2) I didn't --overwrite_cache

So, how to reuse preprocessed data? Is there any option I need to open for the scripts? @lhoestq 


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add text_column_name and label_column_name to run_ner and run_ner_no_trainer args
Author: kumapo
Pull request: yes

# What does this PR do?

it's nice to enable to specify which columns are for `text` and `label` for run_ner with its arguments.
especially it's useful when we train it on not csv (i.e. json) dataset because `text` and `label` columns are determined by column order if default columns are missing.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

```
% python -m pytest -n auto --dist=loadfile -s -v ./examples/
...
Results (1653.79s):
      18 passed
       3 skipped
```

## Who can review?

- maintained examples (not research project or legacy): @sgugger, @patil-suraj



Comments:
@sgugger thank you for your suggestions! 
I've pushed a commit based on it.

I'm appreciated for your fix too!
BERT Huggingface trainer api: ValueError: expected sequence of length 128 at dim 1 (got 314)
Author: quest4next
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I'm using new trainer api in HuggingFace Transformers to train on a GLUE task (QQP). This error shows up during training.

This is the example I'm using https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/trainer/01_text_classification.ipynb

Only change I made is the task. In the notebook GLUE task is MNLI which I changed to QQP. While original MNLI task runs without errors, QQP task fails.

ValueError: expected sequence of length 128 at dim 1 (got 314)

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->

Please check my Stackoverflow question for more details
**https://stackoverflow.com/questions/62675482/bert-huggingface-trainer-api-valueerror-expected-sequence-of-length-128-at-dim**:


Comments:
I can reproduce! Thank you for opening an issue, I'm looking into it now.

Same error (StackOverflow--> https://stackoverflow.com/questions/67004233/typeerror-zeros-like-argument-input-when-fine-tuning-on-mlm) what was the fix in the end @LysandreJik ?

@LysandreJik Is there any way to fix this (for what I presume is a model pretrained on the old HuggingFace version)?

Sorry just seeing this now - @neel04 are you still facing the issue? @msamogh can you open a new issue and fill in the issue template (with full error, environment, code run)? Thanks

I don't really remember what I was trying to do :sweat_smile: Sorry, couldn't help you more. I think the problem was some changes in the API, while the example notebooks weren't updated at that time - so the `max_length` argument (which took `int`) didn't work leading to that error. Now, its changed to a `str` whic represents padding strategy - and since it works now with my current problem, I personally don't think the issue remains anymore :hugs: 

I haven't done the training though, so I would surely update you if I encouter it again!

**EDIT:** My training works (albeit with small batch size), so you must have processed your data wrongly @msamogh. Check out the updated example notebooks to get an idea on how to build your datasets with the :hugs: `Datasets`  lib
`TypeError: TextInputSequence must be str` from Fast Tokenizer
Author: ElvaHuang
Pull request: no

### Bug

On version 4.5.1, trying to use fast tokenizer for Roberta, and got the above error. Weirdly, saw `transformers/models/gpt2/tokenization_gpt2_fast.py` in the traceback even no `gpt2` models are involved. 

**Script to reproduce**

```
from transformers import AutoTokenizer
from transformers.data.processors.utils import InputExample

if __name__ == "__main__":
    tokenizer = AutoTokenizer.from_pretrained(
        "roberta-base", use_fast=True
    )

    MAX_LENGTH = 256
    LABEL_LIST = [0, 1]
    OUTPUT_MODE = "classificaiton"

    inputs = ["Ututu goes public.", "This moon is huge."]
    examples = [InputExample(guid=str(index), text_a=text, label=None) for index, text in enumerate(inputs)]

    # this throws TypeError: TextInputSequence must be str
    batch_encoding = tokenizer(
        [(example.text_a, example.text_b) for example in examples],
        max_length=MAX_LENGTH,
        padding='max_length',
        truncation=True
    )
    print(batch_encoding)
```

**Traceback**

```
Traceback (most recent call last):
  File "tests/integration/tmp_test.py", line 22, in <module>
    truncation=True
  File "/usr/local/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2271, in __call__
    **kwargs,
  File "/usr/local/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2456, in batch_encode_plus
    **kwargs,
  File "/usr/local/lib/python3.7/site-packages/transformers/models/gpt2/tokenization_gpt2_fast.py", line 163, in _batch_encode_plus
    return super()._batch_encode_plus(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/transformers/tokenization_utils_fast.py", line 388, in _batch_encode_plus
    is_pretokenized=is_split_into_words,
TypeError: TextInputSequence must be str
```

Saw a similar issue raised for QnA pipeline, but I'm not doing QnA here. Thoughts?

Comments:
You're sending this to your tokenizer: `[(example.text_a, example.text_b) for example in examples]`

But this is: `[('Ututu goes public.', None), ('This moon is huge.', None)]`.

A tokenizer cannot handle `None`, only text.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Temporary files from an interrupted download litter the disk
Author: Gurkenglas
Pull request: no

## Environment info
- `transformers` version: 4.5.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.4
- PyTorch version (GPU?): 1.8.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

## Information

Model I am using (Bert, XLNet ...): EleutherAI/gpt-neo-125M

The problem arises when using:
* the official example scripts

## To reproduce

Steps to reproduce the behavior:

1. run the transformers library code snippet from https://huggingface.co/EleutherAI/gpt-neo-2.7B:
```
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-2.7B")
model = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-neo-2.7B")
```
2. abort as it's downloading
3. rerun: it start's anew
4. behold a tmp file remaining until you remove it manually

## Expected behavior

The download should resume where it left off, or the temp file should be deleted when the process is aborted, or at least when it is rerun. (It would also be nice if the download location was allocated on the same disk partition as the calling .py - I put my pycharm project on a partition with enough space expecting the model file to also go there; instead the remaining space on my primary partition was eaten.) (It would also be nice if the files ended up with the same name as the model.)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
DPR with ELECTRA models
Author: julian-risch
Pull request: no

# 🚀 Feature request

Dense Passage Retrieval should work with any language model as encoder, in particular also with ELECTRA models.
However, the current implementation of the class `DPREncoder` seems to be limited to BERT models:

https://github.com/huggingface/transformers/blob/a5d2967bd8a5ed2456c593fa9eb5d9c0d726ae7a/src/transformers/models/dpr/modeling_dpr.py#L155

In my understanding, ELECTRA does not provide a pooled_output by default. Therefore, the following line breaks because there are not enough values to unpack (expected 2, got 1):

https://github.com/huggingface/transformers/blob/a5d2967bd8a5ed2456c593fa9eb5d9c0d726ae7a/src/transformers/models/dpr/modeling_dpr.py#L181

## Motivation
I would like to train a Dense Passage Retrieval model with transformers and FARM. It works with BERT models as encoders of queries and passages but not with ELECTRA models. I expect the ELECTRA DPR model to outperform the BERT DPR model with regard to retrieval performance. The code for training DPR is similar to this example [here](https://github.com/deepset-ai/haystack/blob/master/tutorials/Tutorial9_DPR_training.py).

If the handling of the pooled_output is really the only reason that prevents ELECTRA models from being used, I could imagine to work on this myself but I'd be happy to get some guidance first. Maybe there are other problems that I oversaw.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add from_pretrained to dummy timm objects
Author: LysandreJik
Pull request: yes

Closes https://github.com/huggingface/transformers/issues/12091

cc @NielsRogge 

@sgugger Am I missing something relative to specifying that these dummy items should be generated with the `from_pretrained` method?

Comments:
Failure of the templates came from the failure of `make quality` on master, fixed in a commit, so this is good to merge!

Thanks a lot @sgugger!
MIssmatch between problem_type and loss functions in DistillBert for sequence classification
Author: sidhantls
Pull request: no

## Environment info
- Platform: Windows 
- Python version: 3.8
- PyTorch version (GPU?): 1.8

### Who can help
@LysandreJik

Models:
Distillbert for sequence classification

## Information
The problem_type in config is not correct with the loss function applied. Can be seen [here](https://github.com/huggingface/transformers/blob/242ec31aa59b358e631d981b545fd08330584ea8/src/transformers/models/distilbert/modeling_distilbert.py#L649) 

Eg. for the problem type "single_label_classification", the CrossEntropyLoss is used, and the logic to detect "single_label_classification" indicates there should be multiple classes. So its just that the labels of problem_type are switched. Hence, if someone uses actually uses this config of probelm_type to choose  their task (binary, or multiclass), the applied loss would be inccorect 

## Expected behavior
For problem_type, the "single_label_classification", the BCE loss should be applied


Comments:
cc @abhi1thakur