noamaanMulla-03/transformers-issues huggingface dataset:
Support additional dictionaries for BERT Japanese tokenizers
Author: singletongue
Pull request: yes

This PR is to support additional dictionaries for BERT Japanese tokenizers.

Specifically, we add support for `unidic_lite` and `unidic` dictionaries.
Both dictionaries are pip-installable like `ipadic` and compatible with the `fugashi` package introduced in #6086 by @polm.

(We are going to release newly pre-trained BERT models using these dictionaries as well.)

Comments:
Thank you, @JetRunner and @polm.

I've fixed the test-related issues and it should be OK now.
Small bug on website
Author: tanmaypandey7
Pull request: no

Hi, I am not sure if this is the correct place to report this but all the web pages of https://huggingface.co/transformers/master/index.html are having some issues with scrolling. Scrolling the main text (right side) also scrolls the table of contents(left part) 

Comments:
Hi! Yes, this isn't an issue, this is the intended behavior. It's the standard behavior with Sphinx/ReadTheDocs. You can see a similar example with the [PyTorch docs](https://pytorch.org/docs/stable/tensors.html).
[model_cards] Fixed some typing mistakes in usage sections in model cards.
Author: abdullaholuk-loodos
Pull request: yes
Labels: name: model card

Loodos model cards had errors on "Usage" section. It is fixed. Also "electra-base-turkish-uncased" model removed from s3 and re-uploaded as "electra-base-turkish-uncased-discriminator". Its README added.


Comments:
(ONNX) Error while converting the model: bad allocation
Author: shinishiho
Pull request: no

I was trying to convert gpt2-xl model to onnx model using convert_graph_to_onnx.py.
It ran for a while and stopped with some errors:
`TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!`
` w = w / (float(v.size(-1)) ** 0.5)` (in modeling_gpt2.py:151)
`TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!`
`mask = self.bias[:, :, ns - nd : ns, :ns]` (in modeling_gpt2.py:151)
And the last one:
`Error while converting the model: bad allocation`
I googled about this problem but there was no effective solution (for me) at all.

Please help me, thank you in advance.

Comments:
Ok, I know it was my fault. I didn't add the argument `--use-external-format` (gpt2-xl is more than 2GB)
Actually I had to open the convert_graph_to_onnx.py file and read each argument's description
Thanks again, I'm closing the issue now.
F16 support for DistilBert
Author: xuxingya
Pull request: no

# 🚀 Feature request
DIstilBert model donesn't support f16 or mixed precision training.
The hard-coded uses of float32 from the tensorflow implementation of BERT and ELECTRA has been fixed by #3320 .
The same problem in DistilBert needs a fix.


Comments:
Fixed by #6858
Remove hard-coded uses of float32 to fix mixed precision use in Distilbert
Author: ydshieh
Pull request: no

In this commit [Remove hard-coded uses of float32 to fix mixed precision use](https://github.com/huggingface/transformers/commit/4fca874ea995f3d23ad7062b07b5ed7c4f87c0cd#diff-e3ab4f29f29fe1d243a6b55fafaab097), the mixed precision issue is fixed for [modeling_tf_bert.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_bert.py).

However, for [modeling_tf_distilbert.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_distilbert.py#L171), the line 171 is not fixed yet, and we get 

>    173         embeddings = inputs_embeds + position_embeddings  # (bs, max_seq_length, dim)
>    --> 174         embeddings = self.LayerNorm(embeddings)  # (bs, max_seq_length, dim)
>    InvalidArgumentError: cannot compute AddV2 as input #1(zero-based) was expected to be a bfloat16 tensor but is a float tensor

while using `mixed_bfloat16 mixed precision` with `TPU`.

A very quick fix is the same as the fix for `modeling_tf_bert.py`:

    position_embeddings = tf.cast(self.position_embeddings(position_ids), inputs_embeds.dtype)

@schmidek 

Comments:
Indeed! Do you want to open a PR to fix this?

@LysandreJik 

I can do that. However @patrickvonplaten  has already self-assigned for this. How do you think, @patrickvonplaten?

Hey @chiapas, it would be great if you can open a PR for it :-) 

Hi @patrickvonplaten , OK, that would be my first contribution to transformers :)
Training script for other language (except English)
Author: Tortoise17
Pull request: no

Dear Friends. I have found this very nice and great. 
I would like to ask, is there any script which can be used to train other language model?
and how much minimum GPU power is required for such training?

If you can guide . ! I am thinking to test for German language. 



Comments:
You could try either https://huggingface.co/blog/how-to-train or https://discuss.huggingface.co

^^ Forum is better for open-ended questions like those ones.
Native integration with pytorch/serve
Author: MFreidank
Pull request: no
Labels: name: wontfix

First off, many thanks to @julien-c for sharing my [blog post](https://medium.com/analytics-vidhya/deploy-huggingface-s-bert-to-production-with-pytorch-serve-27b068026d18) for an example custom handler integration of a BERT sequence classifier model with [pytorch/serve](https://github.com/pytorch/serve/).

# 🚀 Feature request
Provide scripts to automatically generate a [pytorch/serve](https://github.com/pytorch/serve/) custom handler for transformers models to be able to serve models after training. 

Example usage could be:
generate_serving_handler --output-filename transformers_ner_handler.py --handler-type "ner" /path/to/checkpoint_directory 

## Motivation
There is already an increasing need to serve transformers models with pytorch/serve, see for example:
https://github.com/pytorch/serve/issues/249
https://github.com/pytorch/serve/issues/267

My understanding of the goal of transformers is to be able to support NLP users starting with model prototyping, ranging over model training and evaluation all the way into their productive system. Native integration with a serving framework seems like a logical step given this understanding.

There is ongoing work in `pytorch/serve` to showcase how users can write custom handlers for their transformers models. 
However, handler code will have to differ quite significantly for different transformers applications (NER, QA, Summarization, Classification) which will put a heavy burden on the user. 

Technically, this feature could also be integrated on the `pytorch/serve` side, but I'm wondering if it would not be easier to maintain here as one could more easily test that new integrated models can still be served with the handler code. 


## Your contribution
I would be happy to support and submit a PR - for simple sentence classification I have handler code ready, other applications would have to be designed and integrated.

Comments:
That would be awesome – and might be of interest to @mfuntowicz among others.

We'll help you if needed.

Awesome! Thanks for the upvotes and interest, very appreciated
@julien-c, thanks for your support of the idea. 

I will create a PR and get started on a basic first version and cross-reference it here. 
@all: Feel free to chime in on the PR to help shape the interface :)

@MFreidank I was curious if you had an update here! I've implemented a version of a PyTorch serve handler that operates on the  https://huggingface.co/transformers/main_classes/pipelines.html abstraction for a currently unreleased project, as opposed to a distinct model or tokenizer. I think it's a nice abstraction for this use-case, what do you think? 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add option to keep tb_writer open after training is done.
Author: jaymody
Pull request: no
Labels: name: wontfix

Currently, `Trainer` automatically closes the tensorboard writer `tb_writer` after the training loop is done. Often I'll want to add extra observations, metrics, data, etc ... to the tensorboard event that requires the trained model (ie after the training loop is done). For example.

```
tb_writer = SummaryWriter(log_dir="some_dir")

# do some stuff with tb_writer before training

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
    compute_metrics=compute_metrics,
    tb_writer=tb_writer,
)
trainer.train()

# do extra stuff with tb_writer after the model is trained (that requires a trained model)
```

## Your contribution

Submitted a PR: [Adds option to keep tb_writer open after training finishes](https://github.com/huggingface/transformers/pull/5328#issue-440856807)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Which Marian version was used to train the Helsinki-NLP/* checkpoints?
Author: vmedappa
Pull request: no
Labels: name: wontfix

name: marian

Trying to find the **Marian version used to train Marian MT transformers**. This would help me understand the **benchmarks for translation times**. I see that multiple papers mention various benchmarks for CPU translation : 

- https://www.aclweb.org/anthology/D19-5632.pdf - has the benchmarks for the latest update Marian v1.9
- https://www.aclweb.org/anthology/P18-4020.pdf - does not mention the version but I presume it uses versions between  Marian v1.7 and Marian v1.5

**Marian v1.9 suggests considerably faster translation times than Marian v1.7 and Marian v1.5.** 


Finding the version used for the huggingface transformers both in the huggingface and Helsinki NLP documentation hasn't been fruitful. Would be really helpful to have this answered. 

Comments:
Those models were trained by @jorgtied as part of the OPUS project. He might know the answer.

@sshleifer  Do you know how can I fine tune the models with my specific data?

I just started a PR to support that, but it's still a week away. At the moment, your best bet is to modify `summarization/finetune.py`.


> Trying to find the **Marian version used to train Marian MT transformers**. This would help me understand the **benchmarks for translation times**. I see that multiple papers mention various benchmarks for CPU translation :
> 
>     * https://www.aclweb.org/anthology/D19-5632.pdf - has the benchmarks for the latest update Marian v1.9
> 
>     * https://www.aclweb.org/anthology/P18-4020.pdf - does not mention the version but I presume it uses versions between  Marian v1.7 and Marian v1.5
> 
> 
> **Marian v1.9 suggests considerably faster translation times than Marian v1.7 and Marian v1.5.**
> 
> Finding the version used for the huggingface transformers both in the huggingface and Helsinki NLP documentation hasn't been fruitful. Would be really helpful to have this answered.

The version should be marked in the original model files distributed at https://github.com/Helsinki-NLP/Opus-MT. Most of them will be v1.7. I just started recently to use v1.9 for upcoming models.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
XLNet with high CPU usage
Author: yourh
Pull request: no
Labels: name: wontfix

The same issue as these two:

https://github.com/huggingface/transformers/issues/1722#issue-517212978
https://github.com/huggingface/transformers/issues/1529#issue-507567684

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Possibility to use WhitespaceSplit as pre_tokenizer instead of BPE/Sentencepiece?
Author: tonytan48
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I was customizing a tokenizer with vocab and merges from tokenizers module. However when running intended Language Modelling task with the customized tokenizer, the class in transformers repository(PretrainedTokenizer) is not the same. I was trying to run a roberta-for-masked-lm objective with whole word tokenization scheme but seems if I were to  RobertaTokenizer, it cannot support changing the 'pre_tokenizer' scheme. May I know in such case what shall I do


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Specify tensorboard logging in BART finetuning
Author: isabelcachola
Pull request: yes
Labels: name: wontfix

Allows you to specify the directory in which to store tensorboard logs when running the BART finetuning script. 

Issue: #4349


Comments:
You may also need to rebase to fix `check_code_quality`.
Also `run_examples_torch` will only pass if it doesn't add new dependencies.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Cannot reduce n_ctx for distil gpt2 from 1024 to 256
Author: nim17
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): gpt2

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
Hi! Do you mind filling in the template?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Example: PyTorch Lightning returns missing attribute error (Token Classification)
Author: stefan-it
Pull request: no
Labels: name: wontfix

name: Ex: Named Entity Recognition

Hi,

the following error message is currently thrown (version 3.0.0 of Transformers) when running the `run_ner_pl.sh` example from the token classification example:

```bash
Traceback (most recent call last):
  File "run_pl_ner.py", line 198, in <module>
    trainer = generic_train(model, args)
  File "/mnt/transformers-stefan/examples/lightning_base.py", line 330, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 918, in fit
    self.single_gpu_train(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/distrib_parts.py", line 176, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 1093, in run_pretrain_routine
    self.train()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 335, in train
    self.reset_train_dataloader(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/data_loading.py", line 189, in reset_train_dataloader
    self.train_dataloader = self.request_dataloader(model.train_dataloader)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/data_loading.py", line 352, in request_dataloader
    dataloader = dataloader_fx()
  File "/mnt/transformers-stefan/examples/lightning_base.py", line 142, in train_dataloader
    * float(self.hparams.num_train_epochs)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/utilities/parsing.py", line 116, in __getattr__
    raise AttributeError(f'Missing attribute "{key}"')
AttributeError: Missing attribute "n_gpu"
```

It seems that `n_gpu` is set in the `lightning_base.py` file:

https://github.com/huggingface/transformers/blob/7f60e93ac5c73e74b5a00d57126d156be9dbd2b8/examples/lightning_base.py#L139-L142

The example is running when `n_gpu` is changed to `gpus`.

I will prepare for fixing it and hopefully this does not introduce regression bugs for other PL examples 😅

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
PRETRAINED_INIT_CONFIGURATION for local model path
Author: stas00
Pull request: no

Tokenizers have a special dict `PRETRAINED_INIT_CONFIGURATION`, which tells the tokenization_utils_base, which extra args to pass to the tokenizer's `__init__`, except it doesn't work for the local model, as the hash is for online s3 models.

I have:

```
PRETRAINED_INIT_CONFIGURATION = {
    "stas/fsmt-wmt19-ru-en": {
        "langs": ["ru", "en"],
    },
    "stas/fsmt-wmt19-en-ru": {
        "langs": ["en", "ru"],
    },
    "stas/fsmt-wmt19-de-en": {
        "langs": ["de", "en"],
    },
    "stas/fsmt-wmt19-en-de": {
        "langs": ["en", "de"],
    },
}

```

So in my own code I use

```
   if LOCAL:
        path = "/code/huggingface/transformers-fair-wmt/data/fsmt-wmt19-ru-en/"
        mname = path
        mname_tok = f"stas/fsmt-wmt19-{src}-{tgt}"
        tokenizer = FSMTTokenizer.from_pretrained(mname_tok)
        model = FSMTForConditionalGeneration.from_pretrained(mname)
    else:
        # # s3 uploaded model
        mname = f"stas/fsmt-wmt19-{src}-{tgt}"
        tokenizer = FSMTTokenizer.from_pretrained(mname)
        model = FSMTForConditionalGeneration.from_pretrained(mname)
```

So `mname_tok` overrides to look up the dict above, since it fails to find that entry using local path.

This, however, doesn't work in tools that aren't under my control, `run_eval.py` in seq2seq for example.

**edit**, more to it - it doesn't pass `PRETRAINED_VOCAB_FILES_MAP` args either for the same reason - it fails to look up those entries for local path. I need to change them all.

Any suggestions on how to fix this problem?

Comments:
I found a sort of band-aid, I added this code in the model's tokenization code, right after  init of `PRETRAINED_INIT_CONFIGURATION, PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES`:

```
LOCALIZE=1
if LOCALIZE:
    old, new  = ("stas/", "/mnt/nvme1/code/huggingface/transformers-fair-wmt/data/")

    def localize(buf): return buf.replace(old, new)

    for d in [PRETRAINED_INIT_CONFIGURATION, PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES]:
        for k, v in d.copy().items():
            d[localize(k)] = v

    for d in [PRETRAINED_VOCAB_FILES_MAP]:
        for tk, tv in d.items():
            for k, v in tv.copy().items():
                tv[localize(k)] = v
```

It's still not great, since now I can't commit this file to repo as this leads to a problem of committing other changes in this file. But at least I can move forward.

I dug dipper and found how to solve it - needed to create `model_dir/tokenizer_config.json` and put the special init params there. And do several more tweaks to make the vocab files to not include the language names in the filename, but use the generic 'vocab-src.txt', 'vocab-tgt.txt'.
I want to load pre-trained model from file instead of file name
Author: August-us
Pull request: no
Labels: name: wontfix

Thanks for your excellent code. I recently encountered such a probelm, I want to load pretrained model from anthor machine, and this server could not map the path to my code. But I could load this model in a buffer. So I want to use this buffer becoming the args. What should I do ？ tks. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[Quick poll] Give your opinion on the future of 🤗 transformers
Author: thomwolf
Pull request: no
Labels: name: Discussion

name: wontfix

The 🤗 transformers library is at a crossroad 🚏 and could evolve in many directions, from teaching to research & applications.

We made a quick poll to get your opinion.

If you have 2-3 minutes and want to participate in shaping the future of the library 👉 https://docs.google.com/forms/d/e/1FAIpQLSeKWNE1SyaSvqLYxWQxTA_XeRCVm3_ohmr3UXJgpIxzZhSXlg/viewform

(please reply in the above feedback form rather than to this thread)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[gen utils] missing else case
Author: stas00
Pull request: no

This code: https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L361-L369
```
        # current position and vocab size
        if hasattr(self.config, "vocab_size"):
            vocab_size = self.config.vocab_size
        elif (
            self.config.is_encoder_decoder
            and hasattr(self.config, "decoder")
            and hasattr(self.config.decoder, "vocab_size")
        ):
            vocab_size = self.config.decoder.vocab_size
```
1. `else` is missing - I hit that case while porting a model. Probably needs to assert there?
```
raise ValueError("either self.config.vocab_size or self.config.decoder.vocab_size need to be defined")
```

2. also the comment on top seems to be outdated (just `vocab_size` is being set there)


Comments:
sent PR https://github.com/huggingface/transformers/pull/6980