noamaanMulla-03/transformers-issues huggingface dataset:
is it possible to extract the attention weights on test inputs when the pretrained model is fine-tuned on custom data?
Author: hittle2015
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help
I am wondering if it's possible to  look into the attention weights on test data when the fine-tuned model is running. I tried to look for some docs for help but could not find useful guidance. Any pointers will be appreciated.  Thanks a lot in advance.
<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
cannot load pytorch_model.bin / pytorch version ?
Author: woong97
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
torch version 1.4.0
I execute run_language_modeling.py and save the model. However, when I load the saved model, "OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a Pytorch model from a TF 2.0 checkpoint, please set from_tf=True" occurs. 

If I install torch==1.6.0, it is successful to load model. However, I have to use torch version 1.4.0 and torchvision 0.5.0. How can I load pytorch_model.bin in torch 1.4.0???

++ I tried to train run_language_modeling.py in torch version 1.4.0, but  it cannot import "torch.optim.lr_scheduler" thus the train code cannot be executed.

Thus my question is
[1] How can I load pytorch_model.bin in torch version 1.4.0   / or 
[2] How can I train run_language_modeling.py in torch version 1.4.0?
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
Hi! Could you provide the code you're using, as well as all the environment information?

@LysandreJik should we update the issue template for this last option `Questions & Help`?

I feel like our first question to everybody is always `Could you provide the code you're using, as well as all the environment information`

You're right, we should!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Configuration initialized from checkpoint does not keep the checkpoint identifier in its attributes
Author: LysandreJik
Pull request: no
Labels: name: wontfix

Since version v3.4.0, initializing a model using the `from_pretrained` method adds a `name_or_path` attribute to the configuration, referencing the checkpoint used for initialization:

```py
from transformers import BertModel

model = BertModel.from_pretrained(model_name)
print(model.config.name_or_path)
# model_name
```

However, initializing the configuration on its own with the `from_pretrained` method does not yield the same attribute:

```py
from transformers import BertConfig

config = BertConfig.from_pretrained(model_name)
# config has no `name_or_path` attribute
```

This means that the configuration object initialized is not the same in both cases, whereas it probably should be.

Comments:
Good point.

Maybe what we could do is:
- initializing the configuration with `from_pretrained` initializes the `name_or_path` attribute of the config as you mention, and
- using the configuration in a *model* `from_pretrained` method override the `name_or_path` attribute with the one of the model so that it's in priority linked to the weights path.

Another option would be to have two attributes in the configuration:
- `configuration_name_or_path`
- `weights_name_or_path`

respectively populated by the config `from_pretrained` and the model `from_pretrained`. Maybe with a property linking to one in priority.

but I'm wondering if it's worth so many attributes... 🤔

I would go for the first version you proposed: having `name_or_path` for the configuration initialized if used alongside `from_pretrained`, which gets overridden by the model `from_pretrained`.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Reformer implementation in Tensorflow
Author: gcuder
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Since there is an implementation of the Reformer in Pytorch, my question is if there will be an implementation for Tensorflow too? 


Comments:
This would be cool! I don't believe it's on our roadmap currently (cc @patrickvonplaten), but should be part of a general TF overhaul we'll be doing in the coming months.

No date for that yet, but I think you can expect it in the future.

Could be a "Good First Issue" :D Yeah, I think this is not really on our roadmap because `Reformer` is a pretty complicated model (It tweaks the backprop pass) and does not really have pretrained weights :-/ 

Sounds like a good first project to deep dive in TensorFlow and rip a few hair out in the process :) 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Use pipeline on fine tuned model
Author: prince14322
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->
## Details
I have fine tuned 'roberta-large' model according to my dataset. It is a sequence classification task
```
MODEL_NAME = 'roberta-large'
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) 
model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME)

# Prediction function
def predict(sent):
    sequence = tokenizer.encode_plus(sent, return_tensors="pt")['input_ids'].to(device)
    logits = model(sequence)[0]
```
The above works fine but now I would like to use this model in pipeline like we have one for question-answering
```
nlp = pipeline('question-answering', model='distilbert-base-cased-distilled-squad', tokenizer='bert-base-cased')
```
Any example would help.
Thank You.

<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Some weights of AlbertForPreTraining were not initialized from the model checkpoint at albert-base-v2 and are newly initialized: ['sop_classifier.classifier.weight', 'sop_classifier.classifier.bias']
Author: L-Zhe
Pull request: no

Albert leverage sentence coherence predict loss(SOP) to optimizer its parameters, and I wanna employ it to score the coherence between two sentences. But when I use AlbertForPreTraining to load albert-xxlarge-v2 checkpoint, it reminds me that:
_Some weights of AlbertForPreTraining were not initialized from the model checkpoint at albert-base-v2 and are newly initialized: ['sop_classifier.classifier.weight', 'sop_classifier.classifier.bias'] 
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference._ 

I try many times and find the output of each time is different, this means that the final linear layer for classification has not been loaded and just initial by random. I wanna know how to leverage pretrain SOP classification without fine-tuning?

Hope to get response rapidly.

Comments:
I'm not sure we have a completely fine-tuned SOP classification model. My best advice is to try out different models and see which model not randomly allocates the weights for those layers.
[ray] add maintainers for Ray / Tune
Author: richardliaw
Pull request: yes

# What does this PR do?

Adds maintainers for Ray / Raytune integration!

cc @sgugger 
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->

Comments:
Reformat the TF serving outputs
Author: jplu
Pull request: yes

# What does this PR do?

This PR properly reformat the `serving_output` methods.


Comments:
Merged PR to unblock TF Bart - Split PR. However merge to master made tf templates test fail, see: https://github.com/huggingface/transformers/runs/1676878391 . @jplu, I think they need some updating.
Can not load a saved tokenizer using AutoTokenizer
Author: hadifar
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform: ubuntu 18.04
- Python version: 3.8
- PyTorch version (GPU?): No
- Tensorflow version (GPU?): No
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

@mfuntowicz
@patrickvonplaten

## Information

I'm using following code to save and load t5 tokenizer:

```
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('t5-small')
tokenizer.add_tokens(['<sep>', '<hl>'])
tokenizer.save_pretrained('./t5-tokenizer-test/')

tokenizer2 = AutoTokenizer.from_pretrained('./t5-tokenizer-test/')

```

But it throws the following exception:

During handling of the above exception, another exception occurred:
```

Traceback (most recent call last):
  File "/home/amir/PycharmProjects/question_generation/testifier.py", line 19, in <module>
    tokenizer2 = AutoTokenizer.from_pretrained('./t5-tokenizer-test/')
  File "/home/amir/PycharmProjects/question_generation/venv/lib/python3.7/site-packages/transformers/models/auto/tokenization_auto.py", line 345, in from_pretrained
    config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
  File "/home/amir/PycharmProjects/question_generation/venv/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 349, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/amir/PycharmProjects/question_generation/venv/lib/python3.7/site-packages/transformers/configuration_utils.py", line 418, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load config for './t5-tokenizer-test/'. Make sure that:

- './t5-tokenizer-test/' is a correct model identifier listed on 'https://huggingface.co/models'

- or './t5-tokenizer-test/' is the correct path to a directory containing a config.json file
```

If I replace Autotokenizer with T5Tokenizer the issue will be fixed:
```

from transformers import T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained('t5-small')
tokenizer.add_tokens(['<sep>', '<hl>'])
tokenizer.save_pretrained('./t5-tokenizer-test/')

tokenizer2 = T5Tokenizer.from_pretrained('./t5-tokenizer-test/')
```





Comments:
Hi @hadifar 

The `AutoTokenizer` needs to know the model type to load the correct `Tokenizer` class, and that information is stored in the `config` file, so if `config.json` is not present it can not load the correct class. And `config.json` is saved when saving the model using `.save_pretrained` method. To load a separately saved tokenizer you should use the respective tokenizer class
cannot import name 'RobertaConfig
Author: wahlforss
Pull request: no

## 🐛 Bug

When I run run_glue.py with the roberta model I get an ImportError: cannot import name 'RobertaConfig'

I can't run the run_glue.py with any model since it cannot import the RobertaConfig on line 34.

Any ideas why?

Comments:
Needed to update the pip package... 

which pip package did you have to update? Same problem here...


@stellaywu what is your current `transformers` or `pytorch-transformers` version?

@LysandreJik  it's transformer 2.0.0 

Thatś weird, transformers 2.0.0 works on a clean install in my environment. Could you please double check that the python code you´re running is in the same environment? Something like this:

```py
import transformers
print(transformers.__version__)
# ´2.0.0´

print(tranformers.RobertaConfig)
# Does it crash with ´AttributeError: module ´transformers´  has no attribute RobertaConfig´ ?
```

you are right, it doesn't. I have probably mixed up environment. Thanks!

> @stellaywu what is your current `transformers` or `pytorch-transformers` version?

Dear, I have an error:
`ImportError: cannot import name 'RobertaForQuestionAnswering' from 'pytorch_transformers'`

Actually, I have installed the pytorch_transformers by:
`pip install pytorch-transformers`

however, the error is occurred.

Any idea for this?
 

You should upgrade your transformers version, `RobertaForQuestionAnswering` was probably not present in this early a version:

```
!pip install transformers torch
from transformers import RobertaForQuestionAnswering
```

> You should upgrade your transformers version, `RobertaForQuestionAnswering` was probably not present in this early a version:
> 
> ```
> !pip install transformers torch
> from transformers import RobertaForQuestionAnswering
> ```

Actually, I use pytorch_transformers not transformers. Could you have any suggests?

Installing version v1.1.0 or v1.2.0 of `pytorch-transformers`, I can also import `RobertaConfig`. RoBERTa was added in v1.1.0, so any version earlier than that will not have it.

Is there a reason you're not using `transformers`? Most models are in `transformers`, as are most features, and a lot of bugs have been solved since `pytorch-transformers`.
Masking in Pooling Layer from BERT Output?
Author: datistiquo
Pull request: no
Labels: name: wontfix

In Keras when the embedding layers using masking it is propagate to layer afterwards like Pooling or RNN-Layers. I wonder if this holds for using transformers Bert Models? I.e. in the following are the attention masks used in the following Pooling as masking too? So, the averages do not include padding tokens?

    
```
    id_ = Input((MAX_SEQUENCE_LENGTH,), dtype=tf.int32)
    mask_ = Input((MAX_SEQUENCE_LENGTH,), dtype=tf.int32)
    atn_ = Input((MAX_SEQUENCE_LENGTH,), dtype=tf.int32)
    

    bert_model = TFBertModel.from_pretrained('bert-base-uncased')
    
    embedding = bert_model(id_, attention_mask=mask_, token_type_ids=atn_)[0]
    
    x = GlobalAveragePooling1D()(embedding)    #are here attention_mask used as mask?
    x = Dropout(0.2)(x)
    out = Dense(3, activation='softmax')(x)

    model = Model(inputs=[id_, mask_, atn_], outputs=out)
    model.compile(loss='sparse_categorical_crossentropy', optimizer=opt)
```


https://discuss.huggingface.co/t/bert-output-for-padding-tokens/1550

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to perform model.predict loop with TFRobertaForSequenceClassification?
Author: MiriamFarber
Pull request: no
Labels: name: wontfix

I'd like to perform inference loop for the following roberta model:

```
model = TFRobertaForSequenceClassification.from_pretrained('roberta-base',return_dict=True,num_labels=2)
```

on a large set of pairs of sentences (couple of hundred thousands). I wanted to use `model.predict` and specify batch size, but there is no way to pass the below inputs (encoded_data is tokenization of the input data) to `model.predict`

```
attention_mask=encoded_data['attention_mask'], 
token_type_ids=encoded_data['token_type_ids']
```

So what is the alternative way to do that?

Comments:
Hi, this [Kaggle notebook](https://www.kaggle.com/xhlulu/jigsaw-tpu-xlm-roberta) shows a very concise way to efficiently train/predict Huggingface's `XLMRoberta` (which is the same format as `Roberta`) . Hope it help!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
EncoderDecoderModel: tie weights between different classes of models
Author: alexyalunin
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Tie weights between different classes of models, tie embedding matrices, update tutorial.

## Motivation

I have been following the Longformer2Roberta tutorial https://github.com/huggingface/transformers/blob/master/model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md and it seems like the crucial part of tying weights is missing.
The EncoderDecoderModel is initialized with "allenai/longformer-base-4096" and "roberta-base", and "allenai/longformer-base-4096" in its turn was initialized from "roberta-base". It seems natural to be able to tie their attention and FFNN weights, although it might be problematic to deal with positional embedding. Anyway, I think one feature that definitely should be implemented is the tying embedding matrices. 

## Your contribution

For now I solve the issue with
```
model.encoder.embeddings.word_embeddings.weight = model.decoder.roberta.embeddings.word_embeddings.weight
```
@patrickvonplaten 

Comments:
Yeah I think you have a good point here! Was discussing this with @ibeltagy  and I think we should add a `_tie_encoder_decoder_word_embeddings(...)` that does exactly what you suggested. We should probably run this method when initializing an Encoder-Decoder model and if word embeddings are of the same size. We can provide a `tie_encoder_decoder_word_embeds` config params that defaults to True. 

@alexyalunin do you want to try to make a PR for this ? :-)

> Yeah I think you have a good point here! Was discussing this with @ibeltagy and I think we should add a `_tie_encoder_decoder_word_embeddings(...)` that does exactly what you suggested. We should probably run this method when initializing an Encoder-Decoder model and if word embeddings are of the same size. We can provide a `tie_encoder_decoder_word_embeds` config params that defaults to True.
> 
> @alexyalunin do you want to try to make a PR for this ? :-)

Ok, let me try this. I will put you in the reviewers. 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
T5 (probably BART) issues with the `tf.saved_model.save` API and the `output_xxx` configuration attributes.
Author: LysandreJik
Pull request: no
Labels: name: wontfix

The TensorFlow implementation of the T5 (and very probably the BART) model has an issue with using the tf.saved_model.save API alongside the `output_attentions=True` and the `output_hidden_states=True` configuration attributes.

The tests are skipped currently due to this issue.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.