noamaanMulla-03/transformers-issues huggingface dataset:
[Feature] Add a new tiny feature for self-attention analysis
Author: YNNEKUW
Pull request: yes

# What does this PR do?
To implement the integral in [this paper](https://arxiv.org/abs/2004.11207), users need to take two items out: Output and Attention weights. However, the "Output" here is somewhat special; the attention weights of each layer should be multiplied by a scalar "alpha". I try to put this "alpha" in and set default to 1.0 (which is the same as the original BERT).

@LysandreJik



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Multiple token prediction with MLM
Author: jannisborn
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

It would be great if the `fill_mask` interface would be able to predict multiple tokens at a time.

## Motivation
I didn't find any related issue. Using transformer for chemical data, queries like `fill_mask('CCCO<mask>C')` work fine. But writing `fill_mask('CC<mask>CO<mask>C')` I obtain: 
```
~/miniconda3/envs/paccmann/lib/python3.7/site-packages/transformers/pipelines.py in __call__(self, *args, **kwargs)
    553                 values, predictions = topk.values.numpy(), topk.indices.numpy()
    554             else:
--> 555                 masked_index = (input_ids == self.tokenizer.mask_token_id).nonzero().item()
    556                 logits = outputs[i, masked_index, :]
    557                 probs = logits.softmax(dim=0)

ValueError: only one element tensors can be converted to Python scalars
```

## Your contribution
The user can easily implement an auto-regressive solution and given that `fill_mask` returns the `topk` tokens, one could even select between greedy search, beam search or a probabilistic sampling. But a one-shot approach would be preferable since it minimizes probabilities to aggregate errors as in auto-regression. Rather than outsourcing this to the user, I'd prefer a solution integrated into the package.

As minimum request, I would like to propose to raise a proper error message instead of the obtained `ValueError`.



Comments:
An error message would be great, want to submit a PR?

For multiple masked token support, I'm not entirely sure. The sampling might be very use case-specific.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I'm facing a situation where I've to fetch probabilities from BERT MLM for multiple words in a single sentence.

    Original : "Mountain Dew is an energetic drink"
    Masked : "[MASK] is an energetic drink"

But BERT MLM task doesn't consider two tokens at a time for the MASK. 

@stephen-pilli  

Here's how to mask multiple tokens.

```
import torch

sentence = "The capital of France <mask> contains the Eiffel <mask>."

token_ids = tokenizer.encode(sentence, return_tensors='pt')

# print(token_ids)

token_ids_tk = tokenizer.tokenize(sentence, return_tensors='pt')

print(token_ids_tk)

masked_position = (token_ids.squeeze() == tokenizer.mask_token_id).nonzero()

masked_pos = [mask.item() for mask in masked_position ]

print (masked_pos)

with torch.no_grad():

    output = model(token_ids)

last_hidden_state = output[0].squeeze()

print ("\n\n")

print ("sentence : ",sentence)

print ("\n")

list_of_list =[]

for mask_index in masked_pos:

    mask_hidden_state = last_hidden_state[mask_index]

    idx = torch.topk(mask_hidden_state, k=100, dim=0)[1]

    words = [tokenizer.decode(i.item()).strip() for i in idx]

    list_of_list.append(words)

    print (words)


best_guess = ""

for j in list_of_list:

    best_guess = best_guess+" "+j[0]
```
[examples] fix sys.path in conftest.py
Author: stas00
Pull request: yes

the pt/tf reshuffle broke the examples tests setup. This PR fixes it, by 

* fixing `sys.path` setting now that it's one level down 

@sgugger 

Comments:
That can't work since `conftest` now believes the module "tensorflow" is the folder in the examples with the same name, which is the reason why I moved it.

Path in conftest need to be adapted instead.

ah, OK, then I will make copies of it. But you can't move it since sys.path was relying on it being in a top-level subfolder - so will have to fix that or make the code more flexible. It's currently broken.

I will take care of it.

thank you for letting me know tf didn't like it.

ok, 

1. put `conftest.py` back and fixed it to set up `sys.path` correctly
2. made 2 copies of it for legacy/research.

> The fix in itself LGTM but why add a conftest to legacy and research-projects? Both are not supposed to be tested.

I'm adding deepspeed tests for wav2vec2 - it's been complicated to make it work (some parts still don't work). So I want to make sure it doesn't break again.

I don't think it's a good idea to put these tests under common tests.

So I can't run the tests w/o this conftest on dev box.

I really don't think it's a good idea to add this to `research-projects`: all research projects have a pinned version of Transformers and this would use the master install to test them, which is incompatible with what is advertised.

Maybe add it specifically to the wav2vec2 research project? Or maybe wav2vec2 should move to a maintained example @patrickvonplaten ?

> I really don't think it's a good idea to add this to `research-projects`: all research projects have a pinned version of Transformers and this would use the master install to test them, which is incompatible with what is advertised.

Good point! 

OK, will recall the 2 other files for now.

> Maybe add it specifically to the wav2vec2 research project? Or maybe wav2vec2 should move to a maintained example @patrickvonplaten ?

Yes, totally agree!

`pathlib` is easier to use for finding out older ancestors than doing `dirname` 4 times:
```
import sys
from pathlib import Path
git_repo_path = Path(__file__).resolve().parents[3] / "src"
sys.path.insert(1, str(git_repo_path))
```
Supporting `config_path` for `AutoModel`
Author: hwijeen
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. --> 
Creating a model like this:
```python
AutoModel.from_pretrained('https://myserver/model/pytorch_model.bin', config='path/to/local/config.json') 
```
## Motivation

For the moment, instantiating a model like above is possible with `PretrainedModel` but not with `AutoModel`. i.e.
```python
BertModel.from_pretrained('https://myserver/model/pytorch_model.bin', config='path/to/local/config.json') # possible
AutoModel.from_pretrained('https://myserver/model/pytorch_model.bin', config='path/to/local/config.json') # not possible
```

To elaborate, the optional `config` argument passed to **`AutoModel.from_pretrained`** method should be `PretrainedConfig`, while it could be either `PretrainedConfig` or 'a string or path valid as input to `PretrainedConfig.from_pretrained`' in the case of **`PretrainedModel.from_pretrained`**.

The difference comes from the lack of [this line](https://github.com/huggingface/transformers/blob/8780caa388c7b2aa937454ed96bcdd3f097f851d/src/transformers/modeling_utils.py#L974) in `AutoModel`.



<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution
I would like to submit a PR.
<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
Hey @hwijeen! Would the `AutoModel.from_config()` method work for you?

Thank you @LysandreJik for your quick reply! I checked out the `AutoModel.from_config()` method. It is a convenient method but does not fit my use case, as **it does NOT load weights**.

Hi @LysandreJik , I update this issue to clarify. I would appreciate if you could give it a pass! I am sorry to nudge you twice but I think this issue and related PR(#10981) could be useful for those who try to load pretrained weights from private server!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
GPT Neo, Print Most Probable Next Word: String Indices Must Be Integers
Author: BigSalmon2
Pull request: no


This code is supposed to generate the next most probable word. However, the following problem arises.

```
!pip install git+https://github.com/huggingface/transformers.git
import torch
from transformers import GPTNeoForCausalLM, AutoTokenizer
model = GPTNeoForCausalLM.from_pretrained("EleutherAI/gpt-neo-1.3B")
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-1.3B")

prompt = """In the"""
prompt = prompt.strip()
text = tokenizer.encode(prompt)
myinput, past = torch.tensor([text]), None
logits, past = model(myinput, past_key_values = past)
logits = logits[0,-1]
probabilities = torch.nn.functional.softmax(logits)
best_logits, best_indices = logits.topk(10)
best_words = [tokenizer.decode([idx.item()]) for idx in best_indices]
text.append(best_indices[0].item())
best_probabilities = probabilities[best_indices].tolist()
words = []
for i in range(10):
    m = (best_words[i])
    print(m)
```

`TypeError: string indices must be integers`

Comments:
You're doing something wrong here:
```py
logits, past = model(myinput, past_key_values = past)
```

The model returns a dict. Your `logits` and `past` are the keys of the dicts.

If you want the values, then either do:
```py
output = model(myinput, past_key_values = past)
logits = output.logits
past = output.past_key_values
```
or
```py
logits, past = model(myinput, past_key_values = past, return_dict=False)
```

This code must have worked with versions <=3. Please read the migration guide relative to switching to version 4 [here](https://huggingface.co/transformers/migration.html#switching-the-return-dict-argument-to-true-by-default)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
FineTune XLSR-Wav2Vec2 on New Langauge WER still 1 
Author: edwin-19
Pull request: no

Hi, can i ask is there anyone facing issues fine tuning wav2vec2 for languages not in the common dataset.

I am trying to finetune for a language not within in the common datasets, but i get a WER of 1 not matter how many steps i tried to finetune,  I have a similar issue here #10884 

You can check my code in the repo here: (Note i did make some small changes that weren't in the original notebook by the huggingface team to fit the training notebook they provided)
https://github.com/edwin-19/wave2vec2-hf-sprint/blob/master/Train.ipynb

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Implement fast tokenizer for Big Bird models
Author: tanmaylaud
Pull request: no

This issue is similar to #10498 


## Your contribution
I can help with the implementation. Can someone provide me the high-level steps ?
@stefan-it 


Comments:
The way I would approach is to start from a copy-paste of a very similar fast tokenizer (it's based on SentencePiece, so ALBERT is an example of these), and adapt to Big Bird.

Feel free to open a PR in early stages and let us know if you run in any issues!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

> This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
> 
> Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

It is in progress in this [PR](https://github.com/huggingface/transformers/pull/11075)
[RAG] ModuleNotFoundError: No module named 'git' when finetuning the model
Author: giobin
Pull request: no

## Environment info
- `transformers` version: 4.6.0.dev0
- Platform: Linux-3.10.0-1127.10.1.el7.x86_64-x86_64-with-debian-buster-sid
- Python version: 3.6.13
- PyTorch version (GPU?): 1.8.1+cu102 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
- rag: @patrickvonplaten, @lhoestq

## Information
I get an import error ModuleNotFoundError: No module named 'git' when running the RAG finetuning script.
I am not sure if the module `git` is somewhere in the project or must be installed. I am using a virtual environment built with `pip install -e '[.dev]'` from `/transformers` and `pip install -r requirements.txt` from /rag. 

The problem arises when using:
[RAG] finetune_rag.py
I'm using the original `finetune_rag.sh` script (here below) but without GPU and less iterations. I also modified the `finetune_ray.py` path in the command below from `examples/rag/finetune_rag.py` to just `finetune_rag.py` since there is no `examples/rag` directory in the repo. I call the script from the `transformers/examples/research_projects/rag/` directory. 

export PYTHONPATH="../":"${PYTHONPATH}"
python finetune_rag.py \
    --data_dir mydatadir \
    --output_dir myoutdir \
    --model_name_or_path facebook/rag-sequence-base \
    --model_type rag_sequence \
    --fp16 \
    --profile \
    --do_train \
    --do_predict \
    --n_val -1 \
    --train_batch_size 8 \
    --eval_batch_size 1 \
    --max_source_length 128 \
    --max_target_length 25 \
    --val_max_target_length 25 \
    --test_max_target_length 25 \
    --label_smoothing 0.1 \
    --dropout 0.1 \
    --attention_dropout 0.1 \
    --weight_decay 0.001 \
    --adam_epsilon 1e-08 \
    --max_grad_norm 0.1 \
    --lr_scheduler polynomial \
    --learning_rate 3e-05 \
    --num_train_epochs 10 \
    --warmup_steps 500 \
    --gradient_accumulation_steps 1 \`

The tasks I am working on is:
just to run the simplest finetuning exercise possible to get familiar with the haggingface library.

## To reproduce
Steps to reproduce the behavior:

1. `cd transformers/examples/research_projects/rag/`
2. `bash finetune_rag.sh` (the same code shown above)

Error:
`Traceback (most recent call last):
  File "finetune_rag.py", line 40, in <module>
    from callbacks_rag import (  # noqa: E402 # isort:skipq
  File "/nlu/users/giovanni_bonetta/transformers/examples/research_projects/rag/callbacks_rag.py", line 11, in <module>
    from utils_rag import save_json
  File "/nlu/users/giovanni_bonetta/transformers/examples/research_projects/rag/utils_rag.py", line 14, in <module>
    import git
ModuleNotFoundError: No module named 'git'
`

## Expected behavior
have the finetuning script recognising `git` and running.


Comments:
Indeed you need to install `pip install GitPython` to make it work
I created a PR to add it to the requirements.txt of the RAG example

Thanks @lhoestq . I confirm that it works after installing GitPython.
DeBERTa xlarge v2 throwing runtime error
Author: roshan-k-patel
Pull request: no

- `transformers` version: 4.4.2
- Platform: Linux-3.10.0-1127.el7.x86_64-x86_64-with-redhat-7.8-Maipo
- Python version: 3.6.13
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script: yes

```
RuntimeError: Error(s) in loading state_dict for DebertaForSequenceClassification:
        size mismatch for deberta.encoder.rel_embeddings.weight: copying a param with shape torch.Size([512, 1536]) from checkpoint, the shape in current model is torch.Size([1024, 1536]).
```

I've seen a previous post made about this error and i believe it is a known issue. On the thread i found it was mentioned that a fix was due to come out a month ago. Has the fix come out?

[Downloaded from here](https://huggingface.co/microsoft/deberta-v2-xlarge)

Comments:
@roshan-k-patel this model has been trained using DebertaV2 and not Deberta, hence the size mismatch. Use DebertaV2ForSequenceClassification instead of DebertaForSequenceClassification 😃 .

Hi @frankhart2018, do you mean the [mnli variant](https://huggingface.co/microsoft/deberta-xlarge-mnli)?

No, @roshan-k-patel, I am referring to the model that you mentioned in the issue `deberta-v2-xlarge`. This is trained on DebertaV2 instead of Deberta architecture and hence requires instantiating `DebertaV2ForSequenceClassification`.

I see @frankhart2018, i used deberta base and large to for my task and it worked fine. For the x-large one i pulled the files from hugging face and scp to my server. I'm using the simpletransformers library to initialise and load the model. Are you familiar with the library and know how i might go about instantiating `DebertaV2ForSequenceClassification` instead? Also do you know the performance differences between the normal variants and mnli variants? Do the mnli ones tend to perform better on tasks. 

From the github repo [here](https://github.com/microsoft/DeBERTa), on the benchmarks it shows scores like **87.5/88.4**. Do you know if the second score after the '/' is the mnli version score? Thanks

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
OSError: Can't load config for '/content/wav2vec2-large-xlsr-asr-demo'. Make sure that:
Author: Kowsher
Pull request: no

I'm using
pip install transformers==4.4.2

After completing the training process of ASR I can not read the trained file from my local storage. Although the path is right. But can read  from hugging face 
model = Wav2Vec2ForCTC.from_pretrained("/content/wav2vec2-large-xlsr-asr-demo").to("cuda")

The error:


OSError: Can't load config for '/content/wav2vec2-large-xlsr-asr-demo'. Make sure that:

- '/content/wav2vec2-large-xlsr-asr-demo' is a correct model identifier listed on 'https://huggingface.co/models'

- or '/content/wav2vec2-large-xlsr-asr-demo' is the correct path to a directory containing a config.json file

Comments:
Can you `ls` what's in `/content/wav2vec2-large-xlsr-asr-demo`?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
about .py file
Author: rabrabrab
Pull request: no

i can't download the file" convert_tf_checkpoint_to_pytorch.py" and three other .py file that you set a hyperlink on it. if i click on the link it notice me: 404. Where can i get them? thank you!

Comments:
Hi, could you provide the location of the wrong links? Without additional information we cannot help you.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
"Converting Tensorflow Checkpoints" meets  ('Pointer shape torch.Size([312]) and array shape (128,) mismatched', torch.Size([312]), (128,))
Author: LivinLuo1993
Pull request: no

when "Converting Tensorflow Checkpoints", I see this "Pointer shape {pointer.shape} and array shape {array.shape} mismatched"
AssertionError: ('Pointer shape torch.Size([312]) and array shape (128,) mismatched', torch.Size([312]), (128,)), and the pretrainmodel comes from https://github.com/ZhuiyiTechnology/pretrained-models


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.