noamaanMulla-03/transformers-issues huggingface dataset:
fix typing error for HfArgumentParser for Optional[bool]
Author: bfineran
Pull request: yes

`TrainingArguments` uses the `Optional[bool]` type for [a couple arguments](https://github.com/huggingface/transformers/blob/master/src/transformers/training_args.py#L443).  I ran into the following error when using transformers v4.3.3 with python 3.8:

`"TrainingArguments" TypeError: issubclass() arg 1 must be a class`

Comments:
No, `Optional[bool]` are dealt with later on at [this line](https://github.com/huggingface/transformers/blob/master/src/transformers/hf_argparser.py#L101) and need to stay wrapped until then.

Without more information on your error it's hard to help find the right fix.

Hi @sugger, thank you for the quick response and for pointing that out.  Sorry I did not catch it.

I also see that you put a [tentative fix](https://github.com/huggingface/transformers/commit/fa1a8d102f273ee8118546f7b84133ab58032ac5) for this issue, but also just wanted to check to make sure that `Optional[bool]` values are caught by the expected if statement.

Looking more into the issue, I'm actually seeing that `disable_tqdm` has its typing changed from `Optional[bool]` to `Union[bool, None]` during the `dataclasses.fields` parsing.  So it is not getting caught by the line you reference and then the error occurs on the next if statement.

I'm running Python 3.8 on ubuntu 18.04, maybe dataclasses parses optionals differently in other versions.

Printout of `disable_tqdm` fields value:
```
>>> [field for field in dataclasses.fields(TrainingArguments) if "tqdm" in field.name][0]
Field(name='disable_tqdm',type=typing.Union[bool, NoneType],default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7f2e6e7ff1f0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({'help': 'Whether or not to disable the tqdm progress bars.'}),_field_type=_FIELD)
```

dataclasses parsing of generic optional bool type:
```
>>> @dataclass
... class OptionalBool:
...     value: Optional[bool]
...
>>> dataclasses.fields(OptionalBool)
(Field(name='value',type=typing.Union[bool, NoneType],default=<dataclasses._MISSING_TYPE object at 0x7f2e6e7ff1f0>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f2e6e7ff1f0>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),_field_type=_FIELD),)
```

To address this, would it be fine to change the `is Optional[bool]` check [here](https://github.com/huggingface/transformers/blob/master/src/transformers/hf_argparser.py#L101) to `==`?

```
>>> Optional[bool] is Union[bool, None]
False
>>> Optional[bool] == Union[bool, None]
True
```

Thanks again for looking into this so quickly.

I think this change is acceptable, thanks! Trying to check what the failure in the test is and if it's spurious.
Issue when customizing loss in Trainer
Author: LedaguenelArthur
Pull request: no

Hi everyone,

I am a student and therefore not yet very familiar with the way issues report work on git, so I aplogize in advance if this is not the proper place to post this message.
I'm trying to customize the loss to use a weighted CrossEntropyLoss, I browsed the issues reports and saw that this matter was already mentionned and a solution was brought by the developpers (I think it was @sgugger) of the Transformers library, I tried to follow their code snipets as much as possible but always ended up with the same error.

I'm working with the last version of Transformers and this is my code:

```
config = AutoConfig.from_pretrained("bert-base-cased", num_labels=2, finetuning_task="SST-2")

# Test with modified trainer for weighted CrossEntropyLoss

model = AutoModelForSequenceClassification.from_pretrained(
        "dmis-lab/biobert-base-cased-v1.1",
        from_tf=False,
        config=config)

from torch import FloatTensor

classDistribution_raw = [97, 3]
classDistribution = [0.8, 0.2]
normedWeights = [1 - (x / sum(classDistribution)) for x in classDistribution]
normedWeights = FloatTensor(normedWeights).cuda()

from torch.nn import CrossEntropyLoss

class MyTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):
        
        if "labels" in inputs:
          labels = inputs.pop("labels")
        
        outputs = model(**inputs)
        logits = outputs.logits
        loss_function = CrossEntropyLoss(weight = normedWeights)

        if self.args.past_index >= 0:
            self._past = outputs[self.args.past_index]

        if labels is not None:
            loss = loss_function(logits, labels)
        else:
            # We don't use .loss here since the model may return tuples instead of ModelOutput.
            loss = outputs["loss"] if isinstance(outputs, dict) else outputs[0]

        return (loss, outputs) if return_outputs else loss

trainer = MyTrainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        compute_metrics=compute_metrics_fn,
        tokenizer=tokenizer,
    )
```
And this is the error I keep getting:
`'NoneType' object has no attribute 'detach'`

I'm probably doing something wrong but I can't understand what.

Thanks in advance for your answers, I stay availble if u need any more details about my set up or my code,
Best regards,
Arthur Ledaguenel

Comments:
I think you may be experiencing a bug that was fixed since then (I would need the whole error message to be sure) so before we dive further, could you see if an [install from source](https://huggingface.co/transformers/installation.html#installing-from-source) solves your problem?

Hi @sgugger,

Thank you very much for the quick answer, I tested with the installation from source and he it worked !
Possible bug in RAG Tokenizer
Author: krishanudb
Pull request: no

On this line 
input_dict = tokenizer.prepare_seq2seq_batch(question, return_tensors="pt")
the following error is being generated 
AttributeError: 'super' object has no attribute 'prepare_seq2seq_batch'


Comments:
hi @krishanudb 

Thank you for reporting this @krishanudb !

Is there any update on this issue? @patil-suraj 

It's fixed now on master!
see #10167

The issue persists in transformers 4.3.3

/usr/local/lib/python3.7/dist-packages/transformers/models/rag/tokenization_rag.py in prepare_seq2seq_batch(self, src_texts, tgt_texts, max_length, max_target_length, **kwargs)
     75         if max_target_length is None:
     76             max_target_length = self.generator.model_max_length
---> 77         return super().prepare_seq2seq_batch(
     78             src_texts, tgt_texts, max_length=max_length, max_target_length=max_target_length, **kwargs
     79         )

AttributeError: 'super' object has no attribute 'prepare_seq2seq_batch'

Hi @rajasekar-venkatesan 

The issue is fixed on master after the `4.3.3` release.  This fix will be available in the next release.
TensorFlow tests: having from_pt set to True requires torch to be installed.
Author: LysandreJik
Pull request: yes

Some tests were executed without having torch installed, while they require torch. Namely, all the tests that have a `from_pt=True` requirement require torch to be installed.

This is a draft PR as several of the requirements to merge this PR are not met:
- The Marian models do not have their tensorflow variant available on the hub
- Neither do the RAG models

The easy option is to only set `@requires_torch`, but since we have no slow test suite that runs both PT + TF that's not a good workaround.

How do you want to proceed @patrickvonplaten ?


Comments:
@LysandreJik - I'll upload the respective weights today and then check that all these slow tests here work without `from_pt`

Uploaded all the TF weights and checked that:

`RUN_SLOW=1 pytest tests/test_modeling_tf_rag.py`
`RUN_SLOW=1 pytest tests/test_modeling_tf_blenderbot.py` 

pass.

For some reason `RUN_SLOW=1 pytest tests/test_modeling_tf_marian.py` now throws an error. I've opened a new issue for this here: https://github.com/huggingface/transformers/issues/10680
Add auto_wrap option in fairscale integration
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds support for the `auto_wrap` function added by fairscale to automatically wrap the model's modules in the `FSDP` container (necessary for ZeRO-DP3).

cc @stas00 So you are informed for when you want to experiment more with fairscale.

Comments:
expanduser path in Trainer
Author: PaulLerner
Pull request: no

the `output_dir` passed to TrainingArguments is not expanded (the behaviour is probably the same for logging_dir)
### Who can help

Library:

- trainer: @sgugger


## To reproduce
Directly using os.makedirs but this is what happens in Trainer
```py
In [7]: !mkdir ~/foo

In [8]: !cd ~/foo
/mnt/beegfs/home/lerner/foo

In [10]: os.makedirs("~/bar")

In [14]: !realpath "~/bar"
/mnt/beegfs/home/lerner/foo/~/bar

```

## To fix
Call os.path.expanduser before making dir
```py
In [10]: os.makedirs(os.path.expanduser("~/bar"))

In [18]: cd
/mnt/beegfs/home/lerner

In [21]: !realpath bar
/mnt/beegfs/home/lerner/bar
```



Comments:
Sounds reasonable. Would you like to make a PR with this change?

Can do.
I should expand the path in TrainingArguments right? for logging_dir also?

Yes, `output_dir` and `logging_dir`, preferable in the postinit of `TrainingArguments` so it's done as early as possible.
Ray Tune Integration Bug Fixes
Author: amogkam
Pull request: yes

# What does this PR do?
Fixes resource allocation and checkpointing bugs with the Ray Tune `hyperparameter_search` integration.

@sgugger @richardliaw @krfricke 

Comments:
@amogkam There is an `s` missing in line 202 in `src/transformers/integrations.py` in `{kwargs['keep_checkpoint_num']}` which should be `{kwargs['keep_checkpoints_num']}`, which is causing the logger to crash instead of just a warning. Thanks for the fixes btw!
[wip] [deepspeed] AdamW is now supported by default
Author: stas00
Pull request: yes
Labels: name: DeepSpeed

This PR syncs with changes in DeepSpeed since `deepspeed==0.3.10` and can only be merged when `deepspeed==0.3.11` or higher is released. So it may sit here for a while aggregating adjustments

* [x] AdamW is now supported by default so we can remove the now redundant config options and comments https://github.com/microsoft/DeepSpeed/pull/670


Comments:
Add new PET Model
Author: sagarreddypatil
Pull request: no
Labels: name: New model

name: Feature request

# 🌟 New model addition

## Model description

A new article just landed on ArXiv: https://arxiv.org/pdf/2009.07118.pdf
An implementation will eventually be available at https://github.com/timoschick/pet

Authors are @timoschick and Hinrich Schutze.

I didn't see any pre-trained models linked on the GitHub README, but the model is pretty small and easy to train.

Update: the code is available open source along, and it can presumably use pretrained BERT models(I do not know how this works, bu the GitHub page states that the roberta-large pretrained model can be used). The model also works unsupservised.

## Open source status

* [x] the model implementation is available: (give details)
* [x] the model weights are available: (give details)
* [x] who are the authors: (mention them, if possible by @gh-username)


Comments:
The readme in the repo still says this:

> :rotating_light: This repository does not yet contain the modifications to PET introduced in "[It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://arxiv.org/abs/2009.07118)" but will be updated soon.

Looks like the authors updated the repo and added the necessary model.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


While I don't have the time to add PET to this repository myself, I'm always happy to help if someone wants to take it on :)
Token Classification: How to tokenize and align labels with overflow and stride?
Author: oliverguhr
Pull request: no

Hello Huggingface,
I try to solve a token classification task where the documents are longer than the model's max length. 
I modified the `tokenize_and_align_labels` function from [example token classification notebook](https://github.com/huggingface/notebooks/blob/master/examples/token_classification.ipynb). I set the tokenizer option `return_overflowing_tokens=True` and rewrote the function to map labels for the overflowing tokens:

```python
tokenizer_settings = {'is_split_into_words':True,'return_offsets_mapping':True, 
                        'padding':True, 'truncation':True, 'stride':0, 
                        'max_length':tokenizer.model_max_length, 'return_overflowing_tokens':True}

def tokenize_and_align_labels(examples):
    tokenized_inputs = tokenizer(examples["tokens"], **tokenizer_settings)

    labels = []
    for i,document in enumerate(tokenized_inputs.encodings):
       doc_encoded_labels = []
       last_word_id = None
       for word_id  in document.word_ids:            
           if word_id == None: #or last_word_id == word_id:
               doc_encoded_labels.append(-100)        
           else:
               document_id = tokenized_inputs.overflow_to_sample_mapping[i]
               label = examples[task][document_id][word_id]               
               doc_encoded_labels.append(int(label))
           last_word_id = word_id
       labels.append(doc_encoded_labels)
    
    tokenized_inputs["labels"] = labels    
    return tokenized_inputs
```

Executing this code will result in an error:

```
exception has occurred: ArrowInvalid
Column 5 named task1 expected length 820 but got length 30
```
It looks like the input 30 examples can't be mapped to the 820 examples after the slicing. How can I solve this issue?



## Environment info
Google Colab runing Code:
https://github.com/huggingface/notebooks/blob/master/examples/token_classification.ipynb

### Who can help

Library:
- tokenizers: @LysandreJik

## Information

Model I am using (Bert ):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official conll2003 task:
* [x] my own task or dataset:

## To reproduce

Steps to reproduce the behaviour:

1. Replace the tokenize_and_align_labels function with the function given above.
2. Add examples longer than max_length
3. run `tokenized_datasets = datasets.map(tokenize_and_align_labels, batched=True)` cell.


Comments:
Hi,

would it be possible to ask this question on HuggingFace's [forum](https://discuss.huggingface.co/)? Some people like Sylvain (who created the tutorial you mention) are very active there, and are happy to help you. The authors like to keep Github issues for bugs caused by the Transformers library or feature requests.

Thanks!

I moved this question to: https://discuss.huggingface.co/t/token-classification-how-to-tokenize-and-align-labels-with-overflow-and-stride/4353
Fix Wav2Vec2 classes imports
Author: jjdelvalle
Pull request: yes

# What does this PR do?

Fixes imports for the following classes:

* Wav2Vec2CTCTokenizer
* Wav2Vec2FeatureExtractor
* Wav2Vec2Processor

In order to fine tune FB's Wav2Vec2 XLSR model, these classes need to be accessible. Importing using the instructions in the current [blog post](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2) won't work, i.e. using `from transformers import Wav2Vec2CTCTokenizer` will fail. This PR fixes that.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten, I'd appreciate it if you could give this a look. Thanks!




Comments:
Trying to implement "nielsr/luke-large" gives "KeyError: 'luke'"
Author: UrosOgrizovic
Pull request: no

## Environment info

- `transformers` version: 4.1.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.3
- PyTorch version (GPU?): 1.7.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik I guess, because it's an `AutoTokenizer`-related issue.

## Information

I'm trying to use an implementation of LUKE ([paper](https://arxiv.org/abs/2010.01057)) ([implementation](https://huggingface.co/nielsr/luke-large/tree/main)).

The problem arises when using:
* my own modified scripts

The task I am working on is: 
I don't think this is relevant.

## To reproduce

Steps to reproduce the behavior:

1. `from transformers import AutoTokenizer, AutoModel`
2. `tokenizer = AutoTokenizer.from_pretrained("nielsr/luke-large")`

Running gives the following error:

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-12-c1614eef2346> in <module>
      4 
----> 5 luke_tokenizer = AutoTokenizer.from_pretrained("nielsr/luke-large")
      6 

c:\...\venv\lib\site-packages\transformers\models\auto\tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    343         config = kwargs.pop("config", None)
    344         if not isinstance(config, PretrainedConfig):
--> 345             config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
    346 
    347         use_fast = kwargs.pop("use_fast", True)

c:\...\venv\lib\site-packages\transformers\models\auto\configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    350 
    351         if "model_type" in config_dict:
--> 352             config_class = CONFIG_MAPPING[config_dict["model_type"]]
    353             return config_class.from_dict(config_dict, **kwargs)
    354         else:

KeyError: 'luke'
```

## Expected behavior
I'm expecting no error to be thrown.


Comments:
Thanks for your interest! LUKE is not part of the master branch yet.

Actually, the current implementation of LUKE is here (at my `adding_luke_v2` branch): https://github.com/NielsRogge/transformers/tree/adding_luke_v2/src/transformers/models/luke

Note that it is work-in-progress, but you can already use the base `EntityAwareAttentionModel` and the head models. It's mostly the tokenizer that needs some work.

cc'ing the original author for visibility: @ikuyamada 

Thanks, Niels!

As far as I'm concerned, this can be closed.
Is any possible with pipeline for using local model?
Author: ahnz7
Pull request: no

Hallo
I have fine tuned a model in my local PC. I have read the Doc for Pipeline to search a way to use local model in Pipeline. But didn't find. Someone who knows how should I do? 

Comments:
As mentioned in the [docs](https://huggingface.co/transformers/main_classes/pipelines.html), you can either provide a model identifier from the hub or a model that inherits from `PreTrainedModel` or `TFPreTrainedModel`. 

So suppose you have fine-tuned a model and stored it into a variable called `model`, then you can initialize a corresponding pipeline by providing this model variable at initialization. Make sure that the model you're providing is suitable for the pipeline. 

So suppose that you want to use the question answering pipeline, and you have a local `xxxForQuestionAnswering` model, then you can provide it as follows:

```
from transformers import pipeline

model = ...
nlp = pipeline (task='question-answering', model=model)
```



> docs

Thanks a lot.
enable loading Mbart50Tokenizer with AutoTokenizer 
Author: patil-suraj
Pull request: yes

# What does this PR do?


Currently `MBart50Tokenizer`, `MBart50TokenizerFast` can't be loaded using `AutoTokenizer` because they use the `MBartConfig` which is associated with `MBartTokenizer`. 

This PR enables loading `MBart50Tokenizer(Fast)` by adding them to the `NO_CONFIG_TOKENIZER` list. I've also added the `tokenizer_type` argument in the respective models' config file on the hub.

cc @Narsil 

Comments:
Very nice !
Wrong link to super class
Author: cronoik
Pull request: yes

# What does this PR do?

Documentation was referring to slow tokenizer class while it should be the fast tokenizer.




## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

Documentation: @sgugger

Comments:
Performance Issue in doing inferencing hugging face models
Author: kingafy
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:3.2.0
- Platform:windows azure with GPU
- Python version: 3.6
- PyTorch version (GPU?):CUDA 10.1
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

We have created a flask end point to cater to multiple ML models such as Longformer Question answeing,zero shot learning, paraphrasing.But when these functions are called from flask end point the CPU percentage utilisation increases to more than 97% even though the GPU is already in place. Any idea why CPU takes a hit as this is just used for inferencing purpose. 
Also is there a way to check the memory size allocated to a variable when we load the torch pretrained model in the variable and pass this as an argument in the function. If there are concurrent calls does this memory requirement increases or is the same memory is being referred.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!