noamaanMulla-03/transformers-issues huggingface dataset:
QA Documentation: I got error just copy and pasting documentation
Author: andreabac3
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.1
- Platform:Manjaro Linux
- Python version: 1.5.1
- PyTorch version (GPU?): Yes
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger 

## Information
I am trying the to train a QA model following the huggingface documentation, I just copied and pasted the code in my machine (and in Colab) but I was not able to proceed in the training phase because I got None value.
## To reproduce

Steps to reproduce the behavior:

1. Go to the documentation: https://huggingface.co/transformers/custom_datasets.html at Squad training section
2. Copy and paste the code as you can see from my pastebin: https://pastebin.com/hZvq7Zs7
3. And you got the following error
`File "/home/andrea/PycharmProjects/qa-srl/test.py", line 78, in __getitem__
    return {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
RuntimeError: Could not infer dtype of NoneType
`
4. My naive solution was modifying the __getitem__ method from the SquadDataset class in order to avoid to serve the val[idx] == None


Comments:
Pinging @joeddav on this one, since he wrote this tutorial :-)

Thank you @sgugger for the reply.
Ok I can wait for the answer from @joeddav.

Have a nice day. 

Figured it out. `answer_end` is the character position immediately _after_ the answer, so end_position should be derived from `answer_end - 1`. I'm not sure why I was able to run it without this error previously (perhaps a resolved tokenizer bug?), but this should be correct.

```python
def add_token_positions(encodings, answers):
    start_positions = []
    end_positions = []
    for i in range(len(answers)):
        start_positions.append(encodings.char_to_token(i, answers[i]['answer_start']))
        end_positions.append(encodings.char_to_token(i, answers[i]['answer_end'] - 1))
 
        # if start position is None, the answer passage has been truncated
        if start_positions[-1] is None:
            start_positions[-1] = tokenizer.model_max_length
            end_positions[-1] = tokenizer.model_max_length

    encodings.update({'start_positions': start_positions, 'end_positions': end_positions})
```

Closed by #10217 

Thank you @joeddav the posted code works perfectly. 


Sorry for bothering you @joeddav again, I have a question related to the code posted by you here. 
I am still getting None with the dataset built by myself using this code. My dataset works perfectly with the run_squad original script.
In this snipped posted by you I encounter None in the vector of end_positions and I don't know how fix it. I saw the condition in which there's a None the start_positions but what I have to do in the case the None is only in the end_positions vector?

Kind regards,
Andrea
fix deprecated reference `tokenizer.max_len` in glue.py
Author: poedator
Pull request: yes

This is to fix deprecated reference to `tokenizer.max_len` with `tokenizer.model_max_length` - similar to [issue 8739](https://github.com/huggingface/transformers/issues/8739) and [PR 8604](https://github.com/huggingface/transformers/pull/8604). 
See error example [in Colab here](https://colab.research.google.com/gist/poedator/f8776349e5c625ce287fc6fcd312fa1e/tokenizer-max_len-error-in-transformers_glue.ipynb).  it causes `AttributeError: 'BertTokenizer' object has no attribute 'max_len'`
The error happens when `glue_convert_examples_to_features()` is called without `max_length` parameter specified. In that case [line 119](https://github.com/huggingface/transformers/blob/master/src/transformers/data/processors/glue.py#L119) with wrong reference gets called. This simple fix should  do it.


Comments:
deprecated reference `tokenizer.max_len` in glue.py (PR #10220)
Author: poedator
Pull request: no

There is a deprecated reference to `tokenizer.max_len` with `tokenizer.model_max_length` - similar to [issue 8739](https://github.com/huggingface/transformers/issues/8739) and [PR 8604](https://github.com/huggingface/transformers/pull/8604). 
See error example [in Colab here](https://colab.research.google.com/gist/poedator/f8776349e5c625ce287fc6fcd312fa1e/tokenizer-max_len-error-in-transformers_glue.ipynb).  it causes `AttributeError: 'BertTokenizer' object has no attribute 'max_len'`
The error happens when `glue_convert_examples_to_features()` is called without `max_length` parameter specified. In that case [line 119](https://github.com/huggingface/transformers/blob/master/src/transformers/data/processors/glue.py#L119) with wrong reference gets called. 

I submitted a [simple PR #10220](https://github.com/huggingface/transformers/pull/10220). It should be able to fix this issue.

Comments:
Thanks for opening a PR which fixes the issue! I just merged it.
[trainer] move secondary methods into a separate file
Author: stas00
Pull request: yes

We are trying to keep `trainer.py` to a manageable size and as of recent it has been getting new helper methods which should remain methods, but aren't really important for understanding how the Trainer works, so we propose to move them into the utils file and then import them using this nifty idea presented at https://stackoverflow.com/a/47562412/9201239 where instead of subclassing and mixing in, we import the desired methods into the class.

See if you like it. 

And if yes please let me know if there are any other candidates to move.

@sgugger 

Comments:
Why should `attn_weights` be reshaped twice in BartAttention ?
Author: shenfe
Pull request: no

Can anybody help to understand that? https://github.com/huggingface/transformers/blob/3437d12134893dd7b45737e422e105e511341297/src/transformers/models/bart/modeling_bart.py#L238-L244


Comments:
@patrickvonplaten excuse me, could you share any idea please

See this PR for more information: https://github.com/huggingface/transformers/pull/8747

> See this PR for more information: #8747

Almost understand, although it's still weird. As #8747 explained,
```
This ensures that the returned hidden state tensors lie upstream in the graph from the model outputs (allowing their gradients to be computed)
```

Thanks!
ConvBERT fix torch <> tf weights conversion
Author: abhishekkrthakur
Pull request: yes

(from @patrickvonplaten):

This PR corrects the shape of the grouped linear layer weight so that the general conversion function does not have to be changed.

All models are tested to work correctly as follows:

```python
from transformers import ConvBertModel, TFConvBertModel
import tensorflow as tf
import torch

input_ids = [[1, 2, 3, 4, 5]]

tf_input_ids = tf.convert_to_tensor(input_ids)
pt_input_ids = torch.tensor(input_ids)

for name in ["conv-bert-base", "conv-bert-medium-small", "conv-bert-small"]:
    model_tf = TFConvBertModel.from_pretrained(f"YituTech/{name}", from_pt=True)
    model = ConvBertModel.from_pretrained(f"YituTech/{name}")

    assert abs(model_tf(tf_input_ids)[0].cpu().numpy().sum() - model(pt_input_ids)[0].cpu().numpy().sum()) < 1e-2, "Error"
```

Changing the size and the name of a weight means that all tf weights have to be updated, but I think this is ok here since the TF models (if I understood correctly) were not behaving as expected before anyways.

I also checked that the conversion the other way around works as expected (`...from_tf=True`)

Comments:
I'll push the fixed weights and remove from_pt in test.

> Ok with the change!
> 
> Should we do a patch release for this?

Think it's a good idea

Thanks @patrickvonplaten 

Ok will do a patch this afternoon

cc @stefan-it and @mrm8488 that have been playing with the model. We'll release v4.3.3 very soon which will contain that patch.

Should I re-convert the TF model then :thinking: 

I think you should, once v4.3.3 is released (in a few minutes)

v4.3.3 has been released!
Model templates tests are run twice
Author: LysandreJik
Pull request: no

The CI currently runs the model template tests twice when opening a PR from a branch of the huggingface/transformers repo.

The `pull_request_target` should only trigger on external pull requests, or we should remove the `push` target so that the suite only runs once.

Additionally, the `pull_request_target` suite doesn't take into account the changes the PR is doing.

ETA until resolved ~ 2 weeks.

Comments:
TF2 support for Longformer
Author: Pringled
Pull request: no

# 🚀 Feature request

Hi, 

I'm currently working on a project involving long documents (6000+ tokens).  I normally work with Tensorflow and I was wondering if there are any plans for adding a Longformer TF model in the near future? My PyTorch knowledge is fairly limited, but given the potential of the Longformer for my project, I would want to learn the basics if there are no plans for adding TF support. 

Kind regards




Comments:
Yes, we are planning to add this in ~1 month

Great, looking forward to it, thanks!

+1 on this @patrickvonplaten any news? :)

+2 on this

Finished by end of the week :-) See https://github.com/huggingface/transformers/pull/5764. It's almost finished :-) 

@patrickvonplaten any reference on how to train unsupervised model for longformer (not fine-tuning)?
Rework the AMP for TF XLNet
Author: jplu
Pull request: yes

# What does this PR do?

This PR reworks the AMP of XLNet to remove some useless casts for better and less confusing AMP compliancy.

Comments:
Yes, `bfloat16` is only for TPU. Hence, we cannot really test it elsewhere than inside a TPU context. I have added the `bfloat16` condition only if XLNet is run on TPU because we were handling a specific case when the model is run under AMP.
Diverse Beam Search decoding
Author: dakshvar22
Pull request: no

# 🚀 Feature request

Currently, with all the decoding strategies available in this library, there isn't any decoding strategy which helps in producing diverse outputs across the beam of the decoder. Such diversity is useful for tasks such as noisy data augmentation where you want to generate multiple possible outputs. [Diverse Beam Search](https://arxiv.org/abs/1610.02424) paper introduces an extremely simple trick to accomplish this and it works really well.
It is already implemented in the fairseq library and would be cool to have it in transformers too.

## Motivation

Having a decoding strategy which promotes more diversity across the beam.

## Your contribution

I would like to submit a PR for it, but before that I would like to know if there has already been some work internally on exploring this. I would appreciate if someone from the team could guide me on what parts of the code are relevant for this to work.

cc @patrickvonplaten would love some feedback from you since your wrote this [blog](https://huggingface.co/blog/how-to-generate) and you may have seen some work around this.


Comments:
Hey @dakshvar22,

Thanks for posting this Diverse Beam Search paper. Looks cool :-) 
At the moment, I won't find the time to take a deeper look, but feel free to open a PR to add it to our generate() method and I'm happy to discuss possible design choices :-) 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


My understanding is that this should have been closed by https://github.com/huggingface/transformers/pull/9006?
Ignore unexpected weights from PT conversion
Author: LysandreJik
Pull request: yes

Some weights resulting from the conversion from a PyTorch model to a TensorFlow model are throwing an unnecessary warning.

To see for yourself, the following code throws a warning before the PR:

```py
from transformers import BertForPreTraining, BertConfig, TFBertForPreTraining

pt = BertForPreTraining(BertConfig())
pt.save_pretrained("here")
tf = TFBertForPreTraining.from_pretrained("here", from_pt=True)
```

Fix https://github.com/huggingface/transformers/issues/10348

Comments:
Issue Loading bert-based-german-cased
Author: George-Ogden
Pull request: no

Message on the website is:
 Can't load tokenizer using from_pretrained, please update its configuration: 400 Client Error: Bad Request for url: https://int-deepset-models-bert.s3.eu-central-1.amazonaws.com/pytorch/bert-base-german-cased-vocab.txt


Comments:
Which code caused this error?

This issue comes from the hosted API https://huggingface.co/bert-base-german-cased?text=Ich+bin+%5BMASK%5D

@tholor The url above currently loads for me, but to be future-proof should we cp the files currently loaded from that S3 bucket to the corresponding model repo (here, https://huggingface.co/bert-base-german-cased)?

cc'ing @LysandreJik 

@julien-c Sure, let's copy them from our S3 to the model repo.  

copied to the model repo in 
https://huggingface.co/bert-base-german-cased/commit/876457621368b8c955478cfe1cdee634f47ea34c

Changed hardcoded url in https://github.com/huggingface/transformers/pull/10353

@Narsil  could you please check if the inference widget works for this model when you get a chance to upgrade the transformers dependency in the API? Thanks!

⚠️ Can't load tokenizer using from_pretrained, please update its configuration: 400 Client Error: Bad Request for url: https://int-deepset-models-bert.s3.eu-central-1.amazonaws.com/pytorch/bert-base-german-cased-vocab.txt
It's still not working!

Hi @George-Ogden, the change was merged two days ago and is therefore available on the `master` branch, but not yet in a release.

Do you still get the same error when installing from source?

This is on the inference API on the website I haven't tried it from source.
NER Pipeline not working
Author: GuillemGSubies
Pull request: no



I just followed the token classification notebook and created a pipeline from the model I trained there. Here you can see the full notebook: https://colab.research.google.com/drive/1OzfFTgZwjxdIikbQ8lVJbA2SRF3IeJ5F?usp=sharing

The only thing that I changed (apart from creating the pipeline and calling it) s the number of training steps so the training is faster.

In the last cell you can see the error:

```
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
```


### Who can help

- pipelines: @LysandreJik


Comments:
Hello! The fix for this was merged a few days ago: https://github.com/huggingface/transformers/pull/10184

I recommend you install from source while no new version is available for Transformers (a new version should be out in ~2 weeks). Sorry for the inconvenience.

Ok thank you for the fast response. I always open the issues before checking master branch :sweat: 

No worries, better to have too much issues reported than not enough!
Make Barthez tokenizer tests a bit faster
Author: sgugger
Pull request: yes

# What does this PR do?

Currently, CI is pretty slow because of this:
```
93.44s call     tests/test_tokenization_barthez.py::BarthezTokenizationTest::test_add_special_tokens
77.20s call     tests/test_tokenization_barthez.py::BarthezTokenizationTest::test_pretokenized_inputs
77.00s call     tests/test_tokenization_barthez.py::BarthezTokenizationTest::test_maximum_encoding_length_single_input
76.66s call     tests/test_tokenization_barthez.py::BarthezTokenizationTest::test_maximum_encoding_length_pair_input
75.77s call     tests/test_tokenization_barthez.py::BarthezTokenizationTest::test_internal_consistency
```

This is cause by the BarthezTokenizer conversion from slow to fast being pretty slow, so this PR saves the fast tokenizer to make those tests faster. To be even more efficient a new sentencepiece model with a mask token and a pad token should be added and used here.

Comments:
Remove unused variable in example for Q&A
Author: abhishekkrthakur
Pull request: yes

This PR removed unsed `text_tokens = tokenizer.convert_ids_to_tokens(input_ids)` from pytorch and tensorflow examples of Question Answering: https://huggingface.co/transformers/usage.html#extractive-question-answering

Comments:
Thanks a lot for the fix!

What is this all about why am I attached can someone help me


On Thu, Feb 25, 2021, 8:19 AM Lysandre Debut <notifications@github.com>
wrote:

> Merged #10392 <https://github.com/huggingface/transformers/pull/10392>
> into master.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/10392#event-4376807276>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AS5YU5XRPNEC22Y7RAP4W6TTAZL7LANCNFSM4YGIRPLA>
> .
>
Fix run_glue evaluation when model has a label correspondence
Author: sgugger
Pull request: yes

# What does this PR do?

The `run_glue` script uses the correspondence id to label stored in a given model but when using
```
AutoModelForSequenceClassication.from_pretrained(xxx, num_labels=x)
```
that correspondence is reset. This PR fixes that, along with a few other bugs in the script. To confirm MNLI evaluation does take the correspondence in a model config
```bash
python examples/text-classification/run_glue.py --model_name_or_path roberta-large-mnli --task_name mnli --max_seq_length 128 --output_dir ~/tmp/test-mnli --do_eval
```
gices 90.6%/90.1% accuracy (matched/mismatched) after this PR, vs 4.28%/4.86% accuracy on current master.

Comments:
How to train on shards of bookcorpus + wikipedia + openwebtext on 1 TB disk.
Author: gaceladri
Pull request: no

# 🚀 Feature request

Hello, I am trying to pretrain from scratch a custom model on bookcorpus + wikipedia + openwebtext but I only have a 1TB disk. I tried to merge 20% of each one and then reload the training on other 20% of each, but I am having issues with the learning rate scheduler. So if I hardcode the max_steps to the total size of the dataset (100% of all concatenated) it do various passes to the 20%. The same that putting 5 epochs. But I have to deal with lots of points like LambdaLR that is in pure pytorch to set the epoch, current step and all the states. It's a little pain! 

Any suggestion?

## Motivation

I wan to train from scratch a linear attention model with some modifications

## Your contribution

The idea on how to train medium models with big datasets and regular hardware.


Comments:
Training with constant_warmup would be an option since it does not do learning rate decay with respect to dataset size. But I am a bit afraid of end having a poor trained model after 72H of training.

Closed since the new `dataset.set_transform()` lazy loading. Thanks!
SageMaker Model Parallel: cluttered tensorboard plots
Author: mansimane
Pull request: no

## Environment info

- `transformers` version: master
- Platform: SageMaker
- Python version: 3.6
- PyTorch version (GPU?): 1.6
- Using GPU in script?: Y
- Using distributed or parallel set-up in script?: Y

### Who can help


Models:All 

Library:SageMaker Model parallel


## Information


The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: MLM


## Expected behavior

Currently, SM trainer class inherits `is_world_process_zero` variables from main trainer class. In main trainer class, these variables are derived using `self.args.local_rank` or  `dist.get_rank()`, which are not unique for SMMP. This cause multiple processes writing tensorboard summaries because of which there are loops in the tensorboard graph. The `is_world_process_zero` can be implemented in SageMakerTrainer as below. This makes sure that only single process is writing tensorboard summaries.

```python
    def is_world_process_zero(self) -> bool:
        """
        Whether or not this process is the global main process (when training in a distributed fashion on several
        machines, this is only going to be :obj:`True` for one process).
        """
        if self.is_model_parallel_enabled:
            return smp.rank() == 0 and smp.local_rank() == 0 and smp.mp_rank() == 0 and smp.dp_rank() == 0
        else:
            return super.is_world_process_zero()
```




Comments:
Hello! Thank you for opening an issue and for offering a code sample!

Could you open a PR with your code changes?

Thank you!

@LysandreJik  Please find the PR here: https://github.com/huggingface/transformers/pull/10403/files 

Cool, thanks for fixing! Just merged the PR.
MLflow Trainer Callback
Author: noise-field
Pull request: no

# 🚀 Feature request

A callback to log hyperparameters, metrics and cofings/weights to MLFlow, like the existing wandb and Tensorboard callbacks. 

## Motivation

I use MLFlow as my primary experiment tracking tool. It is convenient to run on a remote server and log the results from any of your training machines, andit also facilitates collaboration. 

Trainer is an amazing tool, it makes it very simple to train models, however, the only way to modify the training loop to include custom logging seems to add a callback.

## Your contribution

I can contribute a PR.


Comments:
Happy to get a PR on this!

@noise-field  is remote tracking  with remote server uri with authentication also enabled as part of this feature request ?

@RahulKulhari well, this is not part of the feature request, but you can certainly do remote tracking with mlflow callback. However, you will need to use environment variables (MLFLOW_TRACKING_URI , MLFLOW_TRACKING_USERNAME, MLFLOW_TRACKING_PASSWORD) in advance to configure your connection to the remote server.