noamaanMulla-03/transformers-issues huggingface dataset:
Issue while model sharing and uploading on huggingface
Author: saburbutt
Pull request: no


The model I am using (Bert, XLNet ...): Roberta for questionanswering

I am trying to follow the tutorial given in https://huggingface.co/transformers/model_sharing.html and while I am able to load my model in the local repository, I am unable to save my model and tokenizer using "model.save_pretrained("https://huggingface.co/saburbutt/testing")
tokenizer.save_pretrained("https://huggingface.co/saburbutt/testing")"

If I try to open the link it says "Cannot GET /saburbutt/testing/tokenizer_config.json"

While I try to "echo "hello" >> README.md" or use git functions, It gives me the error "fatal: not a git repository (or any of the parent directories): .git" The task I am working on is SQuAD


<!-- A clear and concise description of what you would expect to happen. -->
I expect the model to be saved in the huggingface/saburbutt/testing repository

Comments:
You cannot save directly remotely like this (though it could be nice to be able to do this in the future, cc @madlag).

You need to first create a repo with `transformers-cli repo create`, or directly on the website.

Then clone it locally and add your files, then push.

Hopefully #8520 makes it clearer?


yes. Thankyou. :)

Hello, I have the same issue with different outputs. I followed all steps in https://huggingface.co/transformers/model_sharing.html 
in order, entered my account with transformers-cli login, created repo, installed lfs, cloned repo, added BERT model via "git add BERTMODEL", committed and pushed but always I got the same error. 

remote: 
remote: -------------------------------------------------------------------------
remote: Your push was rejected because it contains files larger than 10M.
remote: Please use https://git-lfs.github.com/ to store larger files.
remote: -------------------------------------------------------------------------
remote: 
remote: Offending files:
remote:  - BERTMODEL (ref: refs/heads/main)
To https://huggingface.co/Serdar/your-model-name
 ! [remote rejected] main -> main (pre-receive hook declined)

OS: Ubuntu 20.04

I already used "git lfs install" but I could not figure out this problem. I wish someone can help

@serdarakyol Please make sure you read the Getting started guide at https://git-lfs.github.com/ – in your case I think you didn't lfs-track your actual model file

@julien-c Thank you so much. fixed the problem
How to Improve inference time of facebook/mbart many to many model?
Author: Vimal0307
Pull request: no

If we tried to run translation service on facebook mbart many to many on cpu it take 9 secs to translate, how do we reduce the inference time further

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
AttributeError: 'Trainer' object has no attribute 'log_metrics'
Author: MariaFjodorowa
Pull request: no

## Environment info

- `transformers` version: 4.3.3
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.7.1+cu101 (True)
- Tensorflow version (GPU?): 2.4.1 (True)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: False

### Who can help

@sgugger

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)
`examples/language-modeling/run_mlm.py`

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below) 
Just a usual txt file with texts line by line.

Logging on the epoch's end of mlm training fails:

```
Traceback (most recent call last):
  File "examples/language-modeling/run_mlm.py", line 442, in <module>
    main()
  File "examples/language-modeling/run_mlm.py", line 416, in main
    trainer.log_metrics("train", metrics)
AttributeError: 'Trainer' object has no attribute 'log_metrics'
```

## To reproduce

Steps to reproduce the behavior:

```
pip install transformers
cd transformers
python examples/language-modeling/run_mlm.py \
    --model_name_or_path Geotrend/bert-base-ru-cased \
    --train_file <path to train file> \
    --validation_file <path to validation file> \
    --do_train \
    --do_eval \
    --num_train_epochs 1 \
    --output_dir <path to output dir> \
    --save_steps 10000 \
    --line_by_line True
```

## Expected behavior

It works after checkout to the previous commit, [b01483f](https://github.com/huggingface/transformers/commit/b01483faa0cfb57369cbce153c671dbe48cc0638)  :

```
02/28/2021 14:00:45 - INFO - __main__ -   ***** Train results *****
02/28/2021 14:00:45 - INFO - __main__ -     epoch = 1.0
02/28/2021 14:00:45 - INFO - __main__ -     train_runtime = 1091.7453
02/28/2021 14:00:45 - INFO - __main__ -     train_samples_per_second = 70.642
02/28/2021 14:00:45 - INFO - __main__ -   *** Evaluate ***
[INFO|trainer.py:1600] 2021-02-28 14:00:45,719 >> ***** Running Evaluation *****
[INFO|trainer.py:1601] 2021-02-28 14:00:45,719 >>   Num examples = 154244
[INFO|trainer.py:1602] 2021-02-28 14:00:45,719 >>   Batch size = 8
100% 19281/19281 [06:56<00:00, 46.28it/s]
02/28/2021 14:07:42 - INFO - __main__ -   ***** Eval results *****
02/28/2021 14:07:42 - INFO - __main__ -     perplexity = 4.859176983612205
```


Comments:
Hi there. As is mentioned at the very beginning of the [examples README](https://github.com/huggingface/transformers/tree/master/examples#important-note), running the examples requires an install from source.

If you want the examples associated with v4.3.3, you can find them [here](https://github.com/huggingface/transformers/tree/v4.3.3/examples).
About max_length in generation_utils.py
Author: LinjianLi
Pull request: no

In `generation_utils.py`, the docstring of the `beam_search` function shows the example of usage.

```
>>> input_ids = torch.ones((num_beams, 1), device=model.device, dtype=torch.long)
>>> input_ids = input_ids * model.config.decoder_start_token_id

>>> something else that I omit here

>>> outputs = model.beam_search(input_ids, beam_scorer, logits_processor=logits_processor, **model_kwargs)
```

The `beam_search` function uses `while cur_len < max_length` to control the length of generated sequence. But the `cur_len` counts the length including the start token which is a special token. When the user sets `max_length = 1`, does it not mean that the user wants the model to generate one token **while not considering the start token** (I am not sure that is it just me or others may think that way too)? But the `cur_len` will be 1 at the beginning because of the start token and the statement below in the source code.

```
batch_beam_size, cur_len = input_ids.shape
```

The control flow will jump out of the `while` loop and not generate any token.

Maybe `while cur_len < max_length` should be changed to `while cur_len <= max_length`. And maybe other functions should also change the corresponding loop control statement if I am right.

Comments:
Hey @LinjianLi,

note that `max_length` states the maximum length of both generated tokens and input tokens (which is always at least 1). This means that we count the first special token also as an output token (it will be in the final output) and thus should also be included when computing `max_length`

> Hey @LinjianLi,
> 
> note that `max_length` states the maximum length of both generated tokens and input tokens (which is always at least 1). This means that we count the first special token also as an output token (it will be in the final output) and thus should also be included when computing `max_length`

Thanks for your reply!
Question: change location of cache datasets
Author: ioana-blue
Pull request: no

This is not a bug, it's a question. I don't have too much space allowed in my home dir and I have to store most stuff elsewhere. I noticed that the library caches a lot of stuff under ~/.cache/huggingface (in particular under datasets). How do I change the location of the cache dir? I have `export TRANSFORMERS_CACHE=...` in my run script but that doesn't seem to be it. 

I started using some large datasets and whatever caching happens because of these datasets is causing a disk quota issue. I hope it's possible to change the location of the cache dir. 

Thanks!

Comments:
One more thing (maybe this is a bug): why are the datasets stuff cached in ~/.cache/huggingface/datasets when I have both the env var set up AND I specify a cache dir with --cache_dir when I run my scripts? The pretrained models go in the --cache_dir specified but the datasets don't. This is confusing (and maybe buggy?)

I wonder if this has something to do with newer versions of the library. I remember at the last upgrade I had to install the package `datasets` separately, which I didn't have to do before. Is there a way to specify a cache for the `datasets` and it's separate than the model one?


I'll have to look into this more, but I'm doing an educated guess that there is probably a way to send the cache_dir to the datasets api. In any case, I would like to learn how to change the cache dir globally, if at all possible. Thanks!

I'm guessing HF_HOME is what I'm looking for... 

Yes, that worked. Closing.
Bug in Electra Example
Author: david-waterworth
Pull request: no

The description for Electra (https://huggingface.co/google/electra-small-discriminator) contains code the example below. The last line fails, I think instead of predictions.tolist() it should be predictions.squeeze() as predictions is 1xN

Also the example doesn't seem to detect the corrupted tokens.

```
from transformers import ElectraForPreTraining, ElectraTokenizerFast
import torch

discriminator = ElectraForPreTraining.from_pretrained("google/electra-small-discriminator")
tokenizer = ElectraTokenizerFast.from_pretrained("google/electra-small-discriminator")

sentence = "The quick brown fox jumps over the lazy dog"
fake_sentence = "The quick brown fox fake over the lazy dog"

fake_tokens = tokenizer.tokenize(fake_sentence)
fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt")
discriminator_outputs = discriminator(fake_inputs)
predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2)

[print("%7s" % token, end="") for token in fake_tokens]
[print("%7s" % int(prediction), end="") for prediction in predictions.tolist()]

```

Comments:
Ah, good point! I'll edit it now, thanks for letting us know.

Just fixed it in [hf@cf81dc](https://huggingface.co/google/electra-small-discriminator/commit/cf81dc100ac08ff43eb688cb1e3e7d69a822f359), I added you as a co-author too.

Thanks @LysandreJik, out of interest did you get the same results as me - the model doesn't appear to have identified the fake token? I'll do some more investigation and perhaps post in the forum but it seems that no matter what I do it always retuns zeros? Also really the example should demonstrate the model working :)
Bug when combining grouped beam search and constrained prefix decoding
Author: mnschmit
Pull request: no

## Environment info

- `transformers` version: 4.3.3
- Platform: Linux-5.8.0-38-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help

@patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): T5

The problem arises when using: my own modified scripts

## To reproduce

Steps to reproduce the behavior: run this simple script

```python
from transformers import T5TokenizerFast, T5ForConditionalGeneration

tokenizer = T5TokenizerFast.from_pretrained('t5-small')

inp = 'The <extra_id_0> walks in <extra_id_1> park'
enc_inp = tokenizer(inp, return_tensors='pt')
model = T5ForConditionalGeneration.from_pretrained('t5-small')


def prefix_allowed_tokens_fn(batch_id, input_ids):
    return [2]  # dummy value


out = model.generate(
    **enc_inp,
    num_beams=2,
    num_beam_groups=2,
    diversity_penalty=0.2,
    prefix_allowed_tokens_fn=prefix_allowed_tokens_fn
)
```

This produces the following error:
```
Traceback (most recent call last):
  File "debugging/grouped_beam_search.py", line 14, in <module>
    out = model.generate(
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/mounts/Users/student/martin/.local/lib/python3.8/site-packages/transformers/generation_utils.py", line 1041, in generate
    return self.group_beam_search(
  File "/mounts/Users/student/martin/.local/lib/python3.8/site-packages/transformers/generation_utils.py", line 2161, in group_beam_search
    next_token_scores = logits_processor(
  File "/mounts/Users/student/martin/.local/lib/python3.8/site-packages/transformers/generation_logits_process.py", line 89, in __call__
    scores = processor(input_ids, scores)
  File "/mounts/Users/student/martin/.local/lib/python3.8/site-packages/transformers/generation_logits_process.py", line 458, in __call__
    for batch_id, beam_sent in enumerate(input_ids.view(-1, self._num_beams, input_ids.shape[-1])):
RuntimeError: shape '[-1, 2, 1]' is invalid for input of size 1
```

## Expected behavior

No error.

As far as I can tell, the `PrefixConstrainedLogitsProcessor` still receives the original number of beams even when grouped beam search is used. But it should be the number of subbeams. So replacing `num_beams` with `num_beams // num_beam_groups` in the constructor of `PrefixConstrainedLogitsProcessor` in method `_get_logits_processor` in file `generation_utils.py` should fix it.
What do you think?

Comments:
Hey @mnschmit,

thanks for your bug report! Yes, you're right -> I think we should indeed replace `num_beams` by `num_beams // num_beam_groups`. Do you want to open a PR to fix it? :-) Otherwise, I can do it as well
Extracting word representations from BPE-tokenization-based models (GPT-2, RoBERTa, etc.)
Author: kanishkamisra
Pull request: no

Hi! I am trying to extract representations from models (GPT-2 and RoBERTa) when the words are segmented into pieces. For this issue, let's assume I'm trying to extract the representation of `afoot` in the sentence: `the game is afoot !`

For this I first load my libraries and instantiate the RoBERTa tokenizer:

```py
import torch
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('roberta-base')
```
My inputs are in the form of `(sentence, idx)` where sentence is the context in which the desired word occurs in and idx which the index of the word (`afoot`) in the space-segmented form of the sentence `['the', 'game', 'is', 'afoot', '!']`: 

```py
sentence = ("The game is afoot !", 3)
```

The problem occurs when I compare the encoded token_ids for the word when it appears individually: 

```py
tokenizer.encode_plus('afoot', add_special_tokens = False)['input_ids']
#> [2001, 9210]
```

to the token_ids for the same word when it appears in a sentence:

```py
tokenizer.encode_plus('the game is afoot !', add_special_tokens = False)['input_ids']
#> [627, 177, 16, 10, 2917, 27785]
```

**Why do I even need the individually split token_ids?** 

Because I want to know which indices correspond to the desired word so that I can query the model when I pass the sentence to it as an input and extract the representations for the wordpieces/BPEs and then average them to represent the word's vector.

This is not a problem when I use BERT, where individual vs words as part of a sentence have the same segmentation.

Any ideas how I can solve this issue?

Environment:
```
python = 3.8.3
transformers = 3.1
torch = 1.6
```

Comments:
SOLVED! Prefixing spaces before the word seems to do the trick for now. But I will wait to close the issue for people who have more elegant solutions to this problem.

> SOLVED! Prefixing spaces before the word seems to do the trick for now. But I will wait to close the issue for people who have more elegant solutions to this problem.

But how do you deal with the case that there are no spaces between two words?  For example
```
tokenizer.tokenize("I have a dog, which loves eating meat!")
#>  [['I', 'Ġhave', 'Ġa', 'Ġdog', ',', 'Ġwhich', 'Ġloves', 'Ġeat', 'ing', 'Ġmeat', '!']]
```

There is no space before ',' and '!', but both are words. How do you distinguish them from 'ing'? THX for your reply!
The size of CoNLL-2003 is not consistant with the official release.
Author: h-peng17
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: -
- Python version: -
- PyTorch version (GPU?): -
- Tensorflow version (GPU?): -
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: -

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->@sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: CoNLL-2003

## To reproduce

Steps to reproduce the behavior:

1. just run the code

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

The official release of CoNLL-2003 is: 
# train 14987
# dev 3466
# test 3684
While CoNLL-2003 in datasets is:
# train 14041
# dev 3250
# test 3453

Wish for your reply~ Thank you!


Comments:
Hi there, 

The training scripts load the dataset using the `datasets` library, so this issue is related to the `datasets` lib, you can open it in that repo https://github.com/huggingface/datasets.

Thank you!
(Sorry I can not visit the forum) BORT question: pre-training-using-knowledge-distillation is better than pre-training-only for downstream tasks?
Author: guotong1988
Pull request: no

The paper shows the MLM accuarcy comparision.

![image](https://user-images.githubusercontent.com/4702353/109590213-87ba6900-7b46-11eb-9fea-f046d23abeb2.png)


What is the **downstream tasks'** performance comparison for pre-training-using-knowledge-distillation and pre-training-only to the end?

Thank you very much.

Comments:
https://github.com/alexa/bort/issues/10
Return cross-attention weights in generation function
Author: Mehrad0711
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

With v4.2.0 release, generation can now return encoder and decoder self-attention weights but it still doesn't return cross-attention weights. These weights are already computed and returned in model `forward` method, and just need to be returned in `generate` method.
## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
Visualizing cross-attention weights is useful for many applications such as token-alignment.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->
I can submit a PR.


Comments:
Hi @patrickvonplaten!
Please let me know what you think about the feature. I can send a PR for it once you confirm.

Yes, this would be a nice addition indeed :-)

Happy to help you in your PR!
Fix typos
Author: WybeKoper
Pull request: yes

# What does this PR do?

I fixed a couple of typos in comments.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

Comments:
BART : How can I train and evaluate BART on CNN/DM dataset
Author: astariul
Pull request: no

# ❓ Questions & Help

@sshleifer

I found examples for text summarization on CNN/DM dataset using BERT, but I couldn't find any example using BART.

**Are you going to add it later, or update the existing example to add BART, or it's not scheduled ?** 

Comments:
BART has only just been implemented and is not part of release code yet. I'm sure that examples will be added later on, you just need a bit more patience.

How is this going? I would like to train BART for text summarization with my own dataset and I have no idea of what preprocessing is needed or what inputs the model needs. I would appreciate the help @sshleifer.

No training code yet, but we have an eval example. Its very new obviously, so make an issue and tag me if it breaks :)

https://github.com/huggingface/transformers/blob/b3e0a1cf0573a909f1769b5f1c2b7273faf36cf4/examples/summarization/bart/evaluate_cnn.py

The associated PR I opened has the training code. Just in case you want to test it out and run some experiments/give feedback. I based it on [this colab](https://colab.research.google.com/drive/1C4jEf0fnLiz6Xdx4TDz1OoO4BRCjCx1m) that I wrote. 

I couldn't reach the accuracy in the paper with CNNDM dataset using the pre-trained model (facebook/bart-large-cnn).  Can anybody reproduce the accuracy properly?