noamaanMulla-03/transformers-issues huggingface dataset:
[setup.py] note on how to get to transformers exact dependencies from shell
Author: stas00
Pull request: yes

As a follow up to #9550, this PR adds a few handy one liners to quickly access the correct dependency versions from shell.

e.g if you want to install deps for a group of packages we control and with their correct versions, you just need to run:

```
pip install -U $(python -c 'import sys; from transformers.dependency_versions_table import deps; \
print(" ".join([deps[x] for x in sys.argv[1:]]))' numpy filelock protobuf requests tqdm regex \
sentencepiece sacremoses tokenizers packaging importlib_metadata)
```

that was one option for torchhub, but since it didn't have `transformers` installed it didn't work and a different solution was provided https://github.com/huggingface/transformers/pull/9552

@sgugger, @LysandreJik 

Comments:
[Model Exporting] How to export a fine tuned model to a single pytorch or tensorflow model file?
Author: farazk86
Pull request: no

Apologies if this is a very basic question, but I just cant seem to find any help or documentation for this online.

I want to use google cloud to generate text from a trained model and the maximum size for a model there is ``500MB``. Currently when finetuning a model the checkpoints folder has the ``model.bin`` file and an ``optimizer.pt`` file. These both are used when loading from pretrained.

Even when using ``distilgpt2`` the combined size of this folder is ~900MB. How do I export this model to its actual documented size of ~400MB. I assume the ``optimizer.pt`` are the weights.

So please can someone help, how do I export a checkpoint to either a tensorflow model to a pytorch model so that I can then use to generate text?

I know the latest release 4.2.0 has the function ``model.save_pretrained()``, but I am using ``transformers==2.8.0`` can a model fine tuned using ``2.8.0`` be exported using the new function?

Thanks

Comments:
_Note:_ that'd be a better question for the forums at discuss.huggingface.co

The `optimizer.pt` is a snapshot of the optimizer's internal state, for inference you can delete it and only keep your `model.bin` (= the weights)

Thank you, that is very helpful and for directing me to the forums, I did not even know it existed :)

As regards to my other question, is it possible to do the fine tuning in tensorflow? or even export to a tensorflow model? 

 or would this discussion be better suited to the forum?

Thanks
[deepspeed doc] install issues + 1-gpu deployment
Author: stas00
Pull request: yes
Labels: name: DeepSpeed

This PR extends the DeepSpeed/FairScale integration documentation to:

* add extensive general troubleshooting for CUDA-extensions (applies to fairscale, deepspeed, apex or any other python pytorch extension with CUDA C++ code) - these are very likely to be encountered by our users - all notes are based on my first hand encounters with these issues - 2 of which I run into yesterday while trying to build fairscale and deepspeed on Sylvain's hardware which he let me use to run the recent benchmarks. so I figured others are likely to have similar issues and neither fairscale nor deepspeed have these documented anywhere.
* adds deployment for 1 gpu DeepSpeed notes 
* reformats sub-headers so that it's easier to link to specific sections

@sgugger 

Comments:
Thank you for your awesome suggestions and tweaks - all done.
Converting original BERT tf checkpoints to BertForMaskedLM
Author: ethch18
Pull request: no

Hi!  I have some BERT models that I've trained using the original Google code for BERT, and I was hoping to port them over to `transformers`.  I noticed that there are two scripts to do this conversion: one for [the original tf1.x code](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/convert_bert_original_tf_checkpoint_to_pytorch.py), and one for [the new tf2 code](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/convert_bert_original_tf2_checkpoint_to_pytorch.py).

I noticed that tf2 conversion script has the following comment:
> You may adapt this script to include classification/MLM/NSP/etc. heads.

I'm using tf1.x, and that comment isn't the tf1.x conversion script.  However, the output model is a `BertForPreTraining`, and I'd like to port the entire MLM head over too.  I'm assuming that I'd need to somehow get a `BertForMaskedLM` in order to keep the MLM head.

Questions:
1. Would I have to make any modifications to the tf1.x conversion script other than swapping `BertForPreTraining` -> `BertForMaskedLM`?
2. I also noticed that the BERT configs on the model hub are slightly different than the original Google configs.  Is there any additional processing that I'd need to do to convert my configs too, so that they can be loaded by the `Auto*` classes?

Thank you!

Comments:
The `BertForPreTraining` contains two heads: the NSP and the MLM heads. Therefore, by using the tf1 conversion script, you're already porting the entire model!

For the configuration, it should align pretty seamlessly to Google's configurations, but you can check the expected field here: https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/configuration_bert.py#L120-L138

Great, thank you!  For anyone who's curious, I ended up updating my config file with a few extra fields from here to make `Auto*` detection work: https://huggingface.co/bert-base-multilingual-cased/blob/main/config.json
Is it possible to export a pytorch .pt file after finetuning a model?
Author: farazk86
Pull request: no

Hi,

Considering how I am having troubles with the tflite interpreter at issue https://github.com/huggingface/transformers/issues/9392 . I was wondering if I will have better luck trying Pytorch mobile, since the base models are pytorch to begin with.

But to use the pytorch converter I need the saved ``.pt`` file. The checkpoints saved during training are in ``.bin`` format. Is there any way to get the exported pytorch ``.pt`` file from a checkpoint folder? 

Thanks

Comments:
Just rename your `.bin` to `.pt`

Hi @julien-c 

Sorry to come back to this but I am having trouble with this. As the .bin file also has a much larger accompanying ``optimizer`` file that I assume holds the weights.

I am trying to deploy a fine tuned model to google cloud. And even when using a custom prediction routine to load the entire folder for distil GPT2 the folder size exceeds the limit of ``500MB``.

Is there a way to export the fine tuned model to be used alone. Either export to a standalone pytorch model or tensorflow model.

I searched, but could not find any documentation on this. Would appreciate any help on this, or even direct me towards relevant documentation that can help me.

I'm using ``transformers==2.8.0``

Thank you

The optimizer file does not contain the weights of your model, but the state of the optimizer during your training.

If you do not plan on continuing training, then you can safely discard that file. You can have more information about [optimizers here](https://pytorch.org/docs/stable/optim.html).
change masked_bias to -inf
Author: xu-song
Pull request: yes

# What does this PR do?

change masked_bias to -inf


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?



Comments:
Hi, thank you for opening a PR!

Our goal is to stay as close to the initial implementation as possible. The original implementation by OpenAI uses -1e4, so we will keep it this way.

I find the initial implementation is `-1e10` in https://github.com/openai/gpt-2/blob/master/src/model.py#L88

```py
w = w*b - tf.cast(1e10, w.dtype)*(1-b)
```
related issue #9594

I am not quite sure, I guess `1e-10` is not compatible with `fp16`, that may be the reason behind huggingface implementation.
Convert ckpt from TFTrainer to huggingface format.
Author: kiyoungkim1
Pull request: no

I trained models with ```Trainer(pytorch)``` and ```TfTrainer(tensorflow)```, repectively.
With ```Trainer```, everything is ok. Saved models are directly applicable to huggingface pipeline (eg, AutoModel('model_name')).

But with save models from ```TFTrainer``` (ckpt format), I can not do that with ```AutoModel``` and ```TFAutoModel``` .
I can restart the training, so files do not have a problem.
I guess the problem is ckpt file all contains weight and other parameters related to optimizer.

How can I transform my ckpt file to huggingface-applicable format like ```tf_model.h5``` or convert to pytorch?

@jplu 

Comments:
Hello!

You have to use the `save_model` method of the trainer.

ok. thanks!
Gradient accumulation for TFTrainer
Author: kiyoungkim1
Pull request: yes

# What does this PR do?
```TFTrainer``` does not work with ```gradient_accumulation_steps``` > 1 (I am doing with ```TFGPT2LMHeadModel```).
Similar treatment of #6479 is done for labels.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

 tensorflow: @jplu 

Comments:
Sorry we have to revert this PR as `labels` is not a dict but a tensor and makes fails all our examples.

Thanks a lot for having spotted this cas, a more adapted fix will be available here #9616 very sorry for the inconvenience.

Never mind. Thanks again for the fix.
Remove unused token_type_ids in MPNet
Author: jplu
Pull request: yes

# What does this PR do?

This PR adds a warning when the argument `token_type_ids` is given a show a message saying that this argument is never used. I just supressed the internal appearance of this argument without modifying the method signatures in order to do not integrates a breaking change.

Should I update the Tokenizer to make it returns only `attention_mask`?


Comments:
I'm totally fine to definitely suppress this argument in once if this is prefered (I would prefer as well)

Not an expert neither. Maybe @LysandreJik knows better.

> Thanks for adapting @jplu!
> 
> Not an expert on the tokenization part, but is the method `build_inputs_with_special_tokens` still necessary in that case? (It's in both tokenizers files.)

I think it's still required as it puts *e.g.* the [sep] token correctly between two sentences. I don't think that `build_inputs_with_special_tokens` necessarily has something to do with `token_type_ids`
Ignore lm_head decoder bias warning
Author: LysandreJik
Pull request: yes

Removes the warning that's currently happening when importing `xlm-roberta-base` with any of the XLM-R models.

Closes https://github.com/huggingface/transformers/issues/9579

Comments:
Is it normal that this bias is missing?

By answering your question I realized this could be upstreamed directly in the RoBERTa model, which I just did.

You can take a look at my answer this morning to a similar question: https://github.com/huggingface/transformers/issues/6193#issuecomment-760797867.

XLM-R is an alias of the RoBERTa model, hence why they both need this.
Some weights of XLMRobertaForMaskedLM were not initialized from the model checkpoint at xlm-roberta-base and are newly initialized
Author: Syavaprd
Pull request: no

Where can I find weight that won't give the following error?
The code:
model = XLMRobertaForMaskedLM.from_pretrained('xlm-roberta-base')

Comments:
Could you put the full error in the description of the issue rather than in the title? We don't know which weights are not initialized.

> Could you put the full error in the description of the issue rather than in the title? We don't know which weights are not initialized.
Code:
model = XLMRobertaForMaskedLM.from_pretrained('xlm-roberta-base')
Warning:
Some weights of XLMRobertaForMaskedLM were not initialized from the model checkpoint at xlm-roberta-base and are newly initialized: ['lm_head.decoder.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

You don't need to worry about that message, it lets you know that the bias in the LM head is not initialized - it will be initialized to all zeros.

I'm removing this warning in #9615.
[deepspeed] --gradient_accumulation_steps fix
Author: stas00
Pull request: yes
Labels: name: DeepSpeed

This PR fixes deepspeed integration to run `self.deepspeed.step()` instead of `optimizer.step()` + adds test. As it was failing when `--gradient_accumulation_steps 2` was added.

Thank you @jncasey for detecting this bug in https://github.com/microsoft/DeepSpeed/issues/671

@sgugger 

Comments:
Pass kwargs to Pipeline's tokenizer call
Author: guyrosin
Pull request: yes

# What does this PR do?
When calling a Pipeline, the `kwargs` argument is not passed to the tokenizer (it is actually not used at all).
I think the intended behavior is to pass it (as the base tokenizer's `__call__()` method already supports `kwargs`), and that's what this PR does.
[Related to #8180]

The call order is:

```Python3
SpecificPipeline.__call__(..., **kwargs)
# Which calls
Pipeline.__call__(..., **kwargs)
# Which calls
SpecificPipeline._parse_and_tokenize(..., **kwargs)
# Which in turn calls
self.tokenizer(...) # No kwargs in this call
```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@sgugger @LysandreJik

Comments:
cc @Narsil who probably knows better.

Hi @guyrosin , we actually don't want to enable that.
The problem is that kwargs are used both by `_parse_and_tokenize` and by `generate`/`forward`.
See discussion here: https://github.com/huggingface/transformers/pull/9432#discussion_r552550844

I'm guessing you want to override a tokenizer argument at runtime in the pipeline. The best way to do that is to whitelist all arguments of the `tokenizer` (like we did with truncation). and *only* pass `**kwargs` to generate. That's the best way to isolate arguments of both functionalities without creating a mess. **Hopefully** there won't be any arguments with the same name in both function calls..


The `**kwargs` in the function signature, is legacy for now as it simply captures previous arguments that used to be sent, and prevents triggering an error for previously written code.



Ohh, got it. Thanks for the explanation @Narsil!
Load saved Pytorch model into Tensorflow or convert from Pytorch model to TF
Author: Jess0-0
Pull request: no

Hi,

Thanks for this awesome framework!
I have trained and saved an XLMRoberta model in PyTorch and I'm wondering if there is any way I can load the model into TFRobertaForSequenceClassification class. Or if there are ways to convert the checkpoint to TensorFlow checkpoints so that it can be load by TF2.

I came across this file https://github.com/huggingface/transformers/blob/master/src/transformers/convert_pytorch_checkpoint_to_tf2.py
and I'm wondering if there are any docs or instructions to use the script.

Thanks!

Comments:
Hey @Jess0-0,

You can load TF checkpoints into PT and vice-versa via:

`XLMRoberta.from_pretrained(...., from_tf=True)`

or 

`TFXLMRoberta.from_pretrained(...., from_pt=True)`.

Since `XLMRobertaForSequenceClassification` is just an alias of `RobertaForSequenceClassification` there should be no problem in doing 

```python
TFRobertaForSequenceClassification.from_pretrained("<your/path/to/saved/xlm/roberta/pytorch/dir>", from_pt=True)
```
How can we freeze the last few layers of a  BERT model using tf 2.0(or higher)
Author: soumya997
Pull request: no
Labels: name: wontfix

How can we freeze the last few layers of a  BERT model using tf 2.0(or higher)?
I just want to take the pre-trained layers and train the last few layers for fine-tuning on my own data in min time.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[Model] mT5 Cross-Lingual Model
Author: sumanthd17
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

## Model description

<!-- Important information -->

Multilingual T5 (mT5) is a massively multilingual pretrained text-to-text transformer model, trained following a similar recipe as T5.

Weights, code are available.

Github Repo: [mT5 Weights and Code](https://github.com/google-research/multilingual-t5)

Paper: [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/abs/2010.11934)

## Open source status

* [x] the model implementation is available: [Implementation](https://github.com/google-research/multilingual-t5)
* [x] the model weights are available: [checkpoints](https://github.com/google-research/multilingual-t5#released-model-checkpoints)
* [x] who are the authors: (@craffel, @adarob)


Comments:
Will be a part of #6285

Hey, @sumanthd17 any update on this? 

@julien-c thanks for your amazing nlp lib. 
When do you plan to support mT5 ?
When #6285 will be release ?
Cheers
Philippe 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
error  'ascii' codec can't decode byte 0xc3 in position 6550: ordinal not in range(128)\n", when running finetune_trainer.py on multiple tpus 
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi
I am running finetune_trainer.py on multiple tpus getting the following error, thank you for your help 
{
  "textPayload": "Traceback (most recent call last):\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py\", line 330, in _mp_start_fn\n    _start_fn(index, pf_cfg, fn, args)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py\", line 324, in _start_fn\n    fn(gindex, *args)\n  File \"/workdir/seq2seq/finetune_trainer.py\", line 299, in _mp_fn\n    app.run(main, flags_parser=parse_flags)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/absl/app.py\", line 300, in run\n    _run_main(main, args)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/absl/app.py\", line 251, in _run_main\n    sys.exit(main(argv))\n  File \"/workdir/seq2seq/finetune_trainer.py\", line 200, in main\n    if training_args.do_train\n  File \"/workdir/seq2seq/utils.py\", line 128, in __init__\n    self.src_lens = self.get_char_lens(self.src_file)\n  File \"/workdir/seq2seq/utils.py\", line 147, in get_char_lens\n    return [len(x) for x in Path(data_file).open().readlines()]\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/encodings/ascii.py\", line 26, in decode\n    return codecs.ascii_decode(input, self.errors)[0]\nUnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6550: ordinal not in range(128)\n",
  "insertId": "5rl5rhwn8m5snsv0h",
  "resource": {
    "type": "k8s_container",
    "labels": {
      "namespace_name": "ruse-xgcp",
      "project_id": "try-ideas-for-rmi",
      "pod_name": "20201104.seq2seq.7685c.0-md6t7",
      "container_name": "seq2seq",
      "location": "europe-west4-a",
      "cluster_name": "xcloud-v3-donut-europe-west4-a"
    }
  },
  "timestamp": "2020-11-04T23:53:22.599603005Z",
  "severity": "ERROR",
  "labels": {
    "k8s-pod/jobowner": "ruse-xgcp",
    "k8s-pod/serviceName": "xc-20201104-seq2seq-7685c-0",
    "k8s-pod/controller-uid": "32981a05-9c69-4234-b64e-6750f0afde11",
    "k8s-pod/app": "xcloud",
    "k8s-pod/job-name": "20201104.seq2seq.7685c.0"
  },
  "logName": "projects/try-ideas-for-rmi/logs/stderr",
  "receiveTimestamp": "2020-11-04T23:53:27.836579865Z"
}

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
OOMKilled with exit code 137 with finetune_trainer.py
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi
I am trying to train on WMT with TPUs using seq2seq_trainer.py, and I always get "OOMKilled with exit code 137" on the line when it tries to use xm.spawn to run the main function on multiple TPUs. I decreased the batch size as much as possible, still the error is there. Do you have an idea why this is happening? thank you. I appreciate your help. 

Comments:
I even set batch_size to 1 and still the error is there. any suggestion? 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Does Tokenizer provide parameters to split the number？
Author: wulaoshi
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
I want to split "2004年06月25日" into [2, 0, 0, 5, 年, 0, 6, 月, 2, 5,日],not [2005, 年,  06, 月, 25, 日]. 
How can i do it the easiest?
now I use these API : 
BertTokenizer
BertTokenizerFast
tokenizer.tokenize("2004年06月25日")
Thanks.

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.