noamaanMulla-03/transformers-issues huggingface dataset:
TFBert runs slower than keras-bert， any plan to speed up？
Author: kismit
Pull request: no
Labels: name: wontfix

classification task run with keras-bert using 7ms, but run with TFBert using 50+ms, both of them runing on GPU, detail arguments:
hidden layers: 6
max_seq_length: 64
cuda: 2080ti

As I see, most of time using cost by bert encoder, average 6 ms per encoder layer, while keras-bert encoder per layer use less than 1ms

do we have any plan to solve this problem？
ths！

Comments:
This is related: https://github.com/huggingface/transformers/pull/6877

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
special token inconsistency for [UNK] token
Author: cceyda
Pull request: no
Labels: name: wontfix

## Environment info
     
- `transformers` version: 3.0.2
- Platform: Linux-4.15.0-108-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9

### Who can help
 tokenizers: @mfuntowicz

## Information

Bert tokenizer treats [UNK] token as **special** during `tokenizer.convert_ids_to_tokens(...,skip_special_tokens=True) ` 
while `tokenizer(...,return_special_tokens_mask=True)` doesn't (for obvious reasons).
I think it would be better to preserve [UNK] tokens in `convert_ids_to_tokens` for consistency of the term "**special token**".

## To reproduce

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
# fast tokenizer also has the same problem
# tokenizer = AutoTokenizer.from_pretrained("bert-large-cased",use_fast=True)
text="sentence 한 셔 word"
print('tokens:',tokenizer.tokenize(text))

tokens=tokenizer(text,return_special_tokens_mask=True)
print("input_ids:",tokens["input_ids"])
print("special_token_mask:",tokens["special_tokens_mask"])
no_special=tokenizer.convert_ids_to_tokens(tokens["input_ids"], skip_special_tokens=True)
special=tokenizer.convert_ids_to_tokens(tokens["input_ids"])
print('tokens from ids (skip special): ',no_special)
print('tokens from ids (skip special): ',special)

print('special tokens',tokenizer.all_special_tokens)
```

## Expected behavior

Also I think keeping [UNK] would be a better behavior, as `convert_ids_to_tokens` is used in inference pipelines and using skip_special_token for getting rid of [CLS][SEP][PAD] tokens leads to unintended loss of [UNK] tokens, which is important.

Related: https://github.com/huggingface/transformers/issues/4391


Comments:
I think that's reasonable, the point of `skip_special_tokens` isn't to skip unknown tokens. cf @mfuntowicz @thomwolf @n1t0 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
the result of translation task on en-zh is not good,especially in short text
Author: barton-wa
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to use Huggingface model for continuous values directly?
Author: monk1337
Pull request: no
Labels: name: wontfix

Hi, I have a dataset which contains continuous values [ batch_size, features ] 

Features look like this :

`[0.49221584, -0.021571456, -0.0920076, -0.14408934, -0.62306774]`

I want to apply transformer model on these values and pass it to the final layer, something like this 

`batch_data ==> Transformer ==> output_layer ==> classification`

Currently, I am using hand-coded multi-head attention and norm with the feed-forward network to pass these values to the transformer block. 

I Gone through hugging face models, but all the models accept tokens and sequences, Is there any way/hack How I can use hugging face transformer models on direct continuous values?

Comments:
you could input the `input_embeds` tokens instead of the `input_ids` to the forward pass and adjust the `hidden_size` accordingly. Or you just tweak the model files yourself and you remove `nn.Embeddings` and replace it by a dense layer. Btw, normally you get much better answers for these kind of questions when you post it on https://discuss.huggingface.co/ . We try to move "non-bug" questions to this forum :-) 

@patrickvonplaten Thank you for the reply, Sure from onwards,  I'll post my doubts there.
If you could provide any quick template to start with continuous values, that’d be helpful.

Hi, @patrickvonplaten  I asked this question on discussion forum but didn't get any update yet. Can you provide any starter code/ Template where I can feed continuous values directly? 

https://discuss.huggingface.co/t/how-to-use-huggingface-model-for-continuous-values-directly/816

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How does relative distance is computed for cross-attention in T5 model?
Author: wasiahmad
Pull request: no
Labels: name: wontfix

Let's assume we have a source sequence of length 7 and a target sequence of length 5. In cross-attention sublayer at each decoder layer, every token in the target sequence attends every token in the input sequence.

In T5 model, we compute the relative distance to compute bias using query-len and key-len as in https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_t5.py#L289.

My question is, how the distance between two tokens are computed if they belong to the source and target sequence. The relative distance matrix (5 x 7) would look like:

```
tensor([[ 0,  1,  2,  3,  4,  5,  6],
        [-1,  0,  1,  2,  3,  4,  5],
        [-2, -1,  0,  1,  2,  3,  4],
        [-3, -2, -1,  0,  1,  2,  3],
        [-4, -3, -2, -1,  0,  1,  2]])
```

Once we put the distances into bucket for the cross-attention, it would look like:

```
tensor([[0, 0, 0, 0, 0, 0, 0],
        [1, 0, 0, 0, 0, 0, 0],
        [2, 1, 0, 0, 0, 0, 0],
        [3, 2, 1, 0, 0, 0, 0],
        [4, 3, 2, 1, 0, 0, 0]])
``` 

Given that cross-attention is a part of the decoder, the `bidirectional` flag is set to False. So, it means while decoding at step `i`, the decoder will treat all the source tokens at position `i, i+1, i+2, ...` having a distance `0` from the target token at position `i`. Is this correct?



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Albert loads model on both CPU and GPU at the same time
Author: vivekatwal
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
MODEL_DEVICE = torch.device('cuda')
MODEL_PATH='./models'
tokenizer = AlbertTokenizer.from_pretrained(MODEL_PATH)
qa_model = AlbertForQuestionAnswering.from_pretrained(MODEL_PATH).to(MODEL_DEVICE)

I am using the above code to load the model.
This model occupies both RAM memory(1.5 GB) and GPU memory(650 MB).
I have specified torch device as Cuda, But still, it doesn't behave as expected.
When "cpu" is specified it works well and doesn't load into GPU. But when cuda is specified it load in CPU and GPU as well.
I  tried , "cuda", "cuda:0" 

Any solution to this ?



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
transformer multitasking
Author: max-yue
Pull request: no
Labels: name: wontfix

Doing multitasking using transformers library such as this one https://github.com/JayYip/bert-multitask-learning. cws|NER|weibo_ner&weibo_cws, one problem will be sampled at each turn, say weibo_ner&weibo_cws, then weibo_ner and weibo_cws will trained for this turn together. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Update README.md
Author: dartrevan
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
[from_pretrained] Allow tokenizer_type ≠ model_type
Author: julien-c
Pull request: yes

For an exemple usage of this PR, see the `tokenizer_class` attribute in this config.json: https://s3.amazonaws.com/models.huggingface.co/bert/julien-c/dummy-diff-tokenizer/config.json

Instead of a class, we could have used a `tokenizer_type` belonging to the set of all `model_type`s, like `"bert"`, etc. but it feels more restrictive, especially in case we start having tokenizer classes that are not obviously linked to a "model", like a potential "TweetTokenizer"

Context: https://github.com/huggingface/transformers/pull/6129

**Update: documented by @sgugger in https://github.com/huggingface/transformers/pull/8152**

Comments:
> Not sure I fully understand the use case, but nothing against the principle of it.

The idea is to prevent combinatorial explosion of "model types" when only the tokenizer is different (e.g. Flaubert, CamemBERT if we wanted to support them today)

In the future we might even want to have a few model-agnostic tokenizer classes like ByteLevelBPETokenizer (basically RobertaTokenizer), as they can be initialized pretty exhaustively from the init args stored in `tokenizer_config.json`





Documented by @sgugger in https://github.com/huggingface/transformers/pull/8152
Fix some tooling for windows
Author: jplu
Pull request: yes

# What does this PR do?
This PR fixes a small issue on the `check_repo.py` script with the encodings. I have also updated the `Makefile` to move the creation of the variables `fork_point_sha`, `joined_dirs` and `modified_py_files` in the `modified_only_fixup` target because they are used only there.

The makefile raises an error on Windows environment when creating the `modified_py_files` variable:
```
'tests' is not recognized as an internal or external command, operable program or batch file
```

ping @stas00 and @sgugger 

Comments:
FutureWarning: This config doesn't use attention memories, a core feature of XLNet even though I'm using mem_len
Author: zainsarwar865
Pull request: no

So every time I try to run xl-net using the run_generation.py script, I get the warning : `FutureWarning: This config doesn't use attention memories, a core feature of XLNet. Consider setting mem_len to a non-zero value, for example xlnet = XLNetLMHeadModel.from_pretrained('xlnet-base-cased'', mem_len=1024), for accurate training performance as well as an order of magnitude faster inference.`

The issue is that I am using mem_len, I'm setting it to 384 (have tried other values as well) but I still get this warning.

Comments:
Any solutions to this?

Thanks!

Hey, how are you setting `mem_len=384`? I am under the impression that the default `run_generation` script doesn't allow you to change the model from the default configuration, which would explain the warning.

Here is how I am setting it
`model = model_class.from_pretrained(args.model_name_or_path,cache_dir=args.cache_dir,mem_len=args.mem_len)`



Ok, I've understood the issue - when using this call, the config first gets created with `mem_len=0` (hence the warning) then `mem_len` gets changed. I'll open a PR to move the warning to the `forward` call. In the meantime, you can suppress the warnings; memories are actually getting used.

Okay great! 
Thanks a lot. Closing the issue.
Which model to choose for seq2seq(generating headers for articles)?
Author: AndreyStille
Pull request: no

Hi
Task is generating headers for articles. I have dataset with articles and correct names.
I think about using T5 for summarization or GPT-2 for seq2seq modeling. I'm not sure what way to choose
Can you please give me advise which direction is more relevant for this task?

Comments:
Hi! Questions like this are welcome on the forum, we try to keep the github issues for bugs/feature requests only :) Thanks!

Forum link: https://discuss.huggingface.co
[fsmt tokenizer] support lowercase tokenizer
Author: stas00
Pull request: yes

This PR:

* [x] implements support for `do_lower_case` in the fsmt tokenizer - requested [here](https://github.com/huggingface/transformers/pull/8374#issuecomment-723419221)
* [x] adds test to validate the new feature
* [x] adds case detector in the converter script

@LysandreJik 

Comments:
Add new token classification example
Author: sgugger
Pull request: yes
Labels: name: model card

# What does this PR do?

This PR adds a new example script leveraging the dataset library, the offsets from the fast tokenizers of the tokenizers library and the Trainer API.

Comments:
Hi @sgugger , thanks for these new features :hugs: 

I have a some comments:

* Only training and validation files can be passed via command line argument. The `do_predict` method is already supported, so a new argument for specifiying the test file would be great to have!
* A new version of the `run.sh` script would also be a great feature to start fine-tuning just be executing a shell script. The current `run.sh` script should also get an "_old" suffix then :)
training T5 on  multiple datasets 
Author: rabeehkarimimahabadi
Pull request: no

Hi
I'd need to train finetune_trainer with T5 on multiple datasets. Could you give me some pointers, how I can form a dataloader with multiple datasets in the very simple form of it?
thanks

Comments:
Hi, these questions are better suited for the forum as we try to keep the github issues for bugs/feature requests only. Could you open an issue on the [forum](https://discuss.huggingface.co) instead?

Hi, the link is broken. thanks

On Mon, Nov 9, 2020 at 3:50 PM Lysandre Debut <notifications@github.com>
wrote:

> Hi, these questions are better suited for the forum as we try to keep the
> github issues for bugs/feature requests only. Could you open an issue on
> the forum <https://hdiscuss.huggingface.co> instead?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/8380#issuecomment-724060649>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ARPXHHZTAWRY5UBXSGMBOVTSO76R7ANCNFSM4TNT3AXA>
> .
>


fixed!
[fsmt convert script] fairseq broke chkpt data - fixing that
Author: stas00
Pull request: yes

This PR's main purpose:

* [x] adjusts for breaking fairseq [changes](https://github.com/pytorch/fairseq/commit/3b27ed7996b0315f471c795cf9b7dfcc18467cbe)

Plus:

* [x] adds support for older `bpecodes` filenames - specifically `code` in iwslt14
* [x] improves reporting
* [x] removes CDN note that will be shortly become irrelevant https://github.com/huggingface/transformers/pull/8324

The break was manifesting during conversion in:
```
Traceback (most recent call last):
  File "src/transformers/convert_fsmt_original_pytorch_checkpoint_to_pytorch.py", line 271, in <module>
    convert_fsmt_checkpoint_to_pytorch(args.fsmt_checkpoint_path, args.pytorch_dump_folder_path)
  File "src/transformers/convert_fsmt_original_pytorch_checkpoint_to_pytorch.py", line 118, in convert_fsmt_checkpoint_to_pytorch
    src_lang = args["source_lang"]
KeyError: 'source_lang'
```

@sshleifer, @LysandreJik 

Comments:
+1 It would be good to document supported/tested versions, but I think it would be useful to try to keep up to date (to the extent possible.) I think most people choose whether to upload to hf **after** they have trained a model, and simply won't upload if their version if it is not supported. (Rather than re-train with a supported `fairseq` version.) 

@stas00 You could also write an integration test that runs only if fairseq is installed (probably in examples), that tests conversion. This wouldn't get run by CI, but might make trouble-shooting easier.

Well, in this case, they haven't changed the output for the model - they changed their hub tools instead (i.e. how the model is loaded) - so no re-training is necessary.  So requiring a fixed fairseq model would probably be sufficient.

The problem is that they stopped making releases last Dec. https://github.com/pytorch/fairseq/tags How can we tell a developer to install version X if it doesn't exist (or check for it)?

I would leave it as is for now, since so far there is barely any interest in fsmt and should it start getting more traction I'll be all over it making it super-resilient. 

Makes sense, feel free to merge @LysandreJik
Bug fix for permutation language modelling
Author: shngt
Pull request: yes

Addresses a bug in permutation language modelling data collator, where `&` is used instead of `|` to compute the non-functional token mask (tokens excluding [PAD], [SEP], [CLS]). For verification, may refer to original XLNet code (https://github.com/zihangdai/xlnet/blob/master/data_utils.py#L602).

Addresses #6812 (further investigation needed, however)
 
@patrickvonplaten @LysandreJik 

Comments:
[docs] remove sshleifer from issue-template :(
Author: sshleifer
Pull request: yes

+ Removes sshleifer from issue-templates :( 
+ For previous @sshleifer stuff, if it's in `src/` I put  @patrickvonplaten, if it's `examples/seq2seq` I put @patil-suraj.
+ @stas00 if you want to take over any thing or feel comfortable being the point person for certain things, feel free to suggest.



Comments:
Thank you for the invitation, @sshleifer. 

My feeling is that a person in charge of any domain should have commit rights to that domain. Since I don't have those I'd be happy to be delegated to. (with the exception of fsmt since I wrote it)
comment correction in modeling_bart.py
Author: jc-hou
Pull request: no

Hello, in here 
https://github.com/huggingface/transformers/blob/07708793f20ec3a949ccab32cc4fe0c7272dcc4c/src/transformers/modeling_bart.py#L588

I think the comment should be 
```
# Convert to Bart output format: (BS, seq_len, model_dim) -> (seq_len, BS, model_dim) 
```
Before transpose , shape of x and encoder_hidden_states are both (BS, seq_len, model_dim) to me.
Thanks.




Comments:
Great catch, you are correct.