noamaanMulla-03/transformers-issues huggingface dataset:
Trainer : AttributeError: 'str' object has no attribute '_memory_tracker'
Author: KhalidAlt
Pull request: no

## Environment info
- `transformers` version: 4.6.0
- Platform: Linux-5.8.0-53-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0+cu111 (True)
- Tensorflow version (GPU?): 2.4.1 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

Library:

- trainer: @sgugger 

## Information

Model I am using (Bert, XLNet ...):

T5ForConditionalGeneration

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Training a T5 from scratch over 20GB of data for one epoch and save the model checkpoints using Trainer library
2. Trying to (resume_from_checkpoint="./checkpoint-1320000")


The code :

```
model=T5ForConditionalGeneration.from_pretrained('./checkpoint-1320000/')

%%time
Trainer.train("./T5_model_result/checkpoint-1320000/")
```

The Error message : 

```
AttributeError                            Traceback (most recent call last)
<timed eval> in <module>

~/anaconda3/envs/py38/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, **kwargs)
    998 
    999         # memory metrics - must set up as early as possible
-> 1000         self._memory_tracker.start()
   1001 
   1002         args = self.args

AttributeError: 'str' object has no attribute '_memory_tracker'
```

## Expected behavior

The Traianer library should resume from the last epoch to continue the learning. 





Comments:
The `Trainer` will resume from the last epoch and continue the learning... if you create it. You are using the class directly, not a `Trainer` object.
[AutomaticSpeechRecognitionPipeline] CUDA support
Author: francescorubbo
Pull request: no

## Environment info

- `transformers` version: 4.6.0
- Platform: Linux-4.15.0-106-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu101 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
- pipelines: @LysandreJik

## Information

Model I am using (Bert, XLNet ...): Wav2Vec2

The problem arises when using:
* [ X ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)
* [ X ]  Automatic Speech Recognition 

## To reproduce

Steps to reproduce the behavior:

1. Instantiate AutomaticSpeechRecognitionPipeline with device set to GPU
2. Run pipeline inference on example  audio input

```
import transformers
model = transformers.Wav2Vec2ForCTC.from_pretrained("facebook/wav2vec2-large-xlsr-53-spanish")
tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/wav2vec2-large-xlsr-53-spanish")
feature_extractor = transformers.AutoFeatureExtractor.from_pretrained("facebook/wav2vec2-large-xlsr-53-spanish")
pl = transformers.AutomaticSpeechRecognitionPipeline(feature_extractor=feature_extractor, model=model, tokenizer=tokenizer, framework='pt',device=0)

pl('waveform.wav')
```

The snippet above results in the following error:

`RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same`

## Expected behavior

Inputs should be converted to CUDA tensor. 
I believe this is happening because the feature extractor doesn't preserve the device.
I'm able to solve the issue if I add 
`processed = self.ensure_tensor_on_device(**processed)`
after https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines/automatic_speech_recognition.py#L136-L138

If this solution is acceptable, I'm happy to open a PR.

Comments:
Sure, feel free to open a PR! Thanks @francescorubbo
[AutomaticSpeechRecognitionPipeline] Ensure input tensors are on device
Author: francescorubbo
Pull request: yes

# What does this PR do?

Enables using AutomaticSpeechRecognitionPipeline on GPU. 
The feature extractor does not create tensors on the appropriate device, so we call `ensure_tensor_on_device` before feeding the processed inputs to the model.

Fixes #11829 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?
 @LysandreJik are there tests running on GPU? The other pipelines do not seem to test GPU inference, either.

## Who can review?
@LysandreJik, @Narsil 

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
Fix usage of head masks by TF encoder-decoder models' `generate()` function
Author: stancld
Pull request: yes

TF counterpart to #11621 

**Description:** It is necessary to fix head masking for LED and T5 models.

Edit: Fix for T5 - #11857

<hr>

**Reviewers:** @patrickvonplaten @Rocketknight1 

Comments:
Thanks for the addition @stancld ! 

I think once we fix the tests in led + T5 we can merge this one :-)

It also looks good to me!

Hey @patrickvonplaten, I haven't implemented head masking for the `generate` method for LED and T5 intentionally. The reason is that TF LED and T5 does not use head masks properly (there's an old glitch that the decoder uses encoder's `head_mask` instead of `cross_attn_head_mask`).  Maybe, I can fix this issue in other PRs and then enable testing for these two models? :)

> Hey @patrickvonplaten, I haven't implemented head masking for the `generate` method for LED and T5 intentionally. The reason is that TF LED and T5 does not use head masks properly (there's an old glitch that the decoder uses encoder's `head_mask` instead of `cross_attn_head_mask`). Maybe, I can fix this issue in other PRs and then enable testing for these two models? :)

Good for me!
Correcting comments in T5Stack to reflect correct tuple order 
Author: talkhaldi
Pull request: yes

# What does this PR do?

In order to match the actual order (line 513 and 516, and as accessed in 968), I've changed the order mentioned in comments L962 and L966-967.

## Before submitting
- [ x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. (Seems like @patrickvonplaten, @patil-suraj are suggested for T5)


Comments:
Hi, @talkhaldi  thanks for correcting the other comment!

There is one code quality check failing, could run `make style && make quality` and push the code again?

Hi!

Thanks for your comment! The command you mentioned was requiring a lot of dependencies, some of which conflicted with the venv I was using, so instead of creating a new venv to run it, I removed the extra trailing space which technically was the only extra thing, and hoped it would work, but seems not ^^' 

I created a new venv and running the command now...

Hi @patil-suraj,

So it seems make style works well, but I get this error with make quality:

```
> make quality
black --check� �amples tests src utils
839 files would be left unchanged.
isort --check-only examples tests src utils
python utils/custom_init_isort.py --check_only
flake8 examples tests src utils
make extra_quality_checks
make[1]: Entering directory '/mnt/berry/home/alkhaldi/contribcode/transformers'
python utils/check_copies.py
python utils/check_table.py
python utils/check_dummies.py
python utils/check_repo.py
Traceback (most recent call last):
  File "utils/check_repo.py", line 22, in <module>
    from transformers.models.auto import get_values
ImportError: cannot import name 'get_values' from 'transformers.models.auto' (unknown location)
make[1]: *** [Makefile:33: extra_quality_checks] Error 1
make[1]: Leaving directory '/mnt/berry/home/alkhaldi/contribcode/transformers'
make: *** [Makefile:42: quality] Error 2
```

Do you have an idea of what's wrong?

hi @talkhaldi

please make sure you have all the dev deps before running make

to do that you could run `pip install -e ".[dev]"` from the root of the repo.

Hi @patil-suraj,

I have applied that command successfully, but it seems the tests are still failing :o What could be the problem?

Note: That command changed many files, but I only added modeling_t5.py to the commit. Should I include the changes of all files even though I haven't changed them manually?

Hi @talkhaldi 

I think this is because we upgraded the version of `black`. Could you rebase your branch with master and then push again ?

Hi @patil-suraj,

The check_code_quality test passes now, but another test fails. Can you please tell me what's wrong?
Potential incorrect application of layer norm in BlenderbotSmallDecoder
Author: sougata-ub
Pull request: no

In BlenderbotSmallDecoder,  layer norm is applied only on the token embeddings, and not on the hidden_states, whereas in the BlenderbotSmallEncoder, layer norm is applied after adding the input_embeds and positional embeds

BlenderbotSmallEncoder:
`hidden_states = inputs_embeds + embed_pos`
`hidden_states = self.layernorm_embedding(hidden_states)`

BlenderbotSmallDecoder:
`inputs_embeds = self.layernorm_embedding(inputs_embeds)`
`hidden_states = inputs_embeds + positions`

Comments:
cc @patrickvonplaten @patil-suraj 

Great catch @sougata-ub! This indeed seems like a bug. Will check the original implementation and fix this.

Hey @sougata-ub - did you try to change the order in the `BlenderbotSmallDecoder` manually and run a couple of examples to see whether the model performs better? Think, we can check this in a first step -> If we get much worse results than the previous implementation was probably correct. If results seem to be better, it's a strong hint that there is a bug actually.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
【 run_mlm.py 】attention_mask will be set to [1,1,...1] with DataCollatorForLanguageModeling 
Author: xieyuchen13
Pull request: no

tokenized_datasets has been padded when padding="max_length"
when get a dataloader, we will use DataCollatorForLanguageModeling with tokenizer.pad at first
tokenizer.pad will set attention_mask to all 1 because input_ids have already been padded
so i want to know whether the attention mask meet expectations.

Comments:
Sorry, I don't really understand the question here. Could you clarify a bit? In case this is a question about the behavior of `DataCollatorForLanguageModeling`, it would be awesome if you could use the forum: https://discuss.huggingface.co/ .
Otherwise it would be great if you provide a code snippet showcasing the unexpected behavior. Thanks!

in run_mlm.py

first use:
```python
def tokenize_function(examples):
    # Remove empty lines
    examples["text"] = [line for line in examples["text"] if len(line) > 0 and not line.isspace()]
    return tokenizer(
                examples["text"],
                padding=padding,
                truncation=True,
                max_length=data_args.max_seq_length,
                # We use this option because DataCollatorForLanguageModeling (see below) is more efficient when it
                # receives the `special_tokens_mask`.
                return_special_tokens_mask=True,
     )
```

if i set padding="max_length", the inputs will be padded. that means the inputs will already be padded to [1, 4, 5, 6, 2, 0, 0..] and attention_mask will be set to [1, 1, 1, 1, 1, 0, 0...]. so when use DataCollatorForLanguageModeling, the inputs will be padded again (tokenizer.pad). the inputs will not change but the attention_mask will be [1, 1, 1, 1, 1, 1, 1..].
if i set padding="false", tokenizer.pad in DataCollatorForLanguageModeling will not pad the inputs to max_length. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hey @xieyuchen13 - I don't think the `attention_mask` will change from [1,1,1,...0,0,0] to [1,1,1,....1,1,1] => could you show me an example that proves otherwise? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Output probability from model.generate
Author: tomdzh
Pull request: no

# 🚀 Feature request

Do we have the option to output the probability of the generated sequence from model.generate function? It will be super useful for evaluating the confidence score of the generated sequence. Thanks so much!

Comments:
You'll have it soon 😉 , once #9150 is merged 

> You'll have it soon 😉 , once #9150 is merged

That's awesome. Thanks!

@patil-suraj great, it got merged but how does translates now to your question_generation repo? How do I get the output probability/confidence score to the the predicted answers?

I only found this comment from you https://discuss.huggingface.co/t/text-generation-pipeline-output-scores-parameter/3294/2: 

*the text-generation pipeline doesn’t return scores, however you could the generate method directly, to get the scores, this should help*

Would be great if you could elaborate on that.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

@patil-suraj Any chance that `generate_tf_utils` will get the same functionality? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
changing find_batch_size to work with tokenizer outputs
Author: joerenner
Pull request: yes

trainer_pt_utils.find_batch_size currently does not recognize the batch size of BatchEncoding objects. This can cause an error when a trainer relies on find_batch_size to report the number of observed examples in the evaluation loop, which is the case when the eval dataset is Iterable.

# What does this PR do?

Very simple change that lets find_batch_size find the batch size of BatchEncoding objects.

Fixes # (11882)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x ] Was this discussed via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
        https://github.com/huggingface/transformers/issues/11882

- [x ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x ] Did you write any new necessary tests?


@LysandreJik @sgugger




Comments:
Mmm, I don't think I am allowed to push commits on your branch and the CI decided to not run on your PR. Could you push an empty commit to trigger it?
```
git commit --allow-empty -m "Trigger CI"
```
Should do this (and then push).


ah looks like it needs approval:

```First-time contributors need a maintainer to approve running workflows```

No that's for the Git actions (and I clicked yes). Your empty commit did trigger circle CI so all is good, just have to wait for the green tick :-)
Link official Cloud TPU JAX docs
Author: avital
Pull request: yes

# What does this PR do?

Adds a link to the new official Cloud TPU VM JAX docs.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->


Comments:
Update deepspeed config to reflect hyperparameter search parameters
Author: Mindful
Pull request: yes

# What does this PR do?

This PR adds a few lines of code to the Trainer so that it rebuilds the Deepspeed config when running hyperparameter_search. As is, if you run hyperparameter_search while using Deepspeed the TrainingArguments are updated but the Deepspeed config is not, the two become out of sync, and Deepspeed effectively ignores the parameters of any hyperparameter search trials which are set by the Deepspeed config. 
This fixes #11894


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. --> https://github.com/huggingface/transformers/issues/11894
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

I ran the Deepspeed tests and Trainer tests locally; everything passed except for `test_stage3_nvme_offload` but I think that was a hardware compatibility issue on my local machine. 

## Who can review?

@stas00 (and maybe whoever implemented hyperparameter_search() in the Trainer)


Comments:
Code quality check passes on my local machine 🤔 
Happy to change formatting if necessary, just not sure what to change.

You probably have a different version of `black`.

Please try:

```
cd transformers
pip install -e .[dev]
make fixup
```

this should re-align the versions.

Turns out I just ran the style check on the wrong branch the first time; my bad. 
Should be fixed now. 

the doc job failure is unrelated - we will re-run it when other jobs finish - the CI has been quite flakey...

Thanks for your PR!
[mBART] one slow integration test is failing on master
Author: patil-suraj
Pull request: no

The `test_enro_generate_batch` slow test is now failing on master

command
```bash
RUN_SLOW=1 pytest tests/test_modeling_mbart.py::MBartEnroIntegrationTest::test_enro_generate_batch
```

Traceback
```
tests/test_modeling_mbart.py F                                           [100%]

=================================== FAILURES ===================================
______________ MBartEnroIntegrationTest.test_enro_generate_batch _______________

self = <tests.test_modeling_mbart.MBartEnroIntegrationTest testMethod=test_enro_generate_batch>

    @slow
    def test_enro_generate_batch(self):
        batch: BatchEncoding = self.tokenizer.prepare_seq2seq_batch(self.src_text, return_tensors="pt").to(
            torch_device
        )
        translated_tokens = self.model.generate(**batch)
        decoded = self.tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)
>       assert self.tgt_text == decoded
E       AssertionError: assert ['Şeful ONU d...e de oameni.'] == ['Şeful ONU d...e de oameni.']
E         At index 1 diff: 'Secretarul General Ban Ki-moon declară că răspunsul său la intensificarea sprijinului militar al Rusiei pentru Siria este că "nu există o soluţie militară" la conflictul de aproape cinci ani şi că noi arme nu vor face decât să înrăutăţească violenţa şi mizeria pentru milioane de oameni.' != 'Secretarul General Ban Ki-moon declară că răspunsul său la intensificarea sprijinului militar al Rusiei pentru Siria este că "nu există o soluţie militară" la conflictul de aproape cinci ani şi că noi arme nu vor face decât să înrăutăţească violenţa şi mizeria a mi...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

tests/test_modeling_mbart.py:366: AssertionError
```

cc @patrickvonplaten 

Comments:
Yeah this test is failing for a while now (even before the Bart split PR) -> think we should just adapt the text

This issue has been stale for 1 month.

Is this fixed? I think we just need to update the test here

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.