noamaanMulla-03/transformers-issues huggingface dataset:
GPT Neo for Sequence Classification
Author: saichandrapandraju
Pull request: no
Labels: name: Good First Issue

Hi,
Is there a way to use GPT Neo for classification tasks like BoolQ ? 
As 'OpenAI GPT2' integration of HF has 'GPT2ForSequenceClassification', is there a similar one for GPT Neo?

Comments:
@patil-suraj this may be a good first issue? Feel free to open a PR!

Thanks @NielsRogge .

Hi @patil-suraj ,

Is there any workaround to make it work in my local?

We could for sure add `GPTNeoForSequenceClassification`. 

It would be as easy as 
- just copying the `GPT2ForSequenceClassification` module and replacing the `GPT2` with `GPTNeo`
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/src/transformers/models/gpt2/modeling_gpt2.py#L1225-L1231
- `config.hidden_size` instead of  config.n_embd
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/src/transformers/models/gpt2/modeling_gpt2.py#L1232
- remove the model_parallel logic
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/src/transformers/models/gpt2/modeling_gpt2.py#L1236-L1238
- add a test in `tests/test_modeling_gpt_neo.py`  similar to
https://github.com/huggingface/transformers/blob/afe479adb5474250215438fe27db9dc9dbbbde09/tests/test_modeling_gpt2.py#L357

Marking this as "Good First Issue".

Feel free to take a stab if you want, I would be happy to help.

Hi Guys,
is anyone working on this?
I can make PR for this. I might also need to use it in the future.


Hi @bhadreshpsavani, Feel free to open a PR :)
Added Sequence Classification class in GPTNeo
Author: bhadreshpsavani
Pull request: yes

# Added Sequence Classification Class in GPT Neo Model

Fixes #11811

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #11811
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@patrickvonplaten @patil-suraj @sgugger 


Comments:
Want to use bert-base-uncased model without internet connection
Author: IamSparky
Pull request: no

I want to use the bert-base-uncased model in offline , for that I need the bert tokenizer and bert model have there packages saved in my local . **I am unable to understand how should I achieve it in my local without any internet connection** ?

import transformers
transformers.BertTokenizer.from_pretrained("bert-base-uncased", do_lower_case=True)
transformers.BertModel.from_pretrained("bert-base-uncased")

currently getting the error
![image](https://user-images.githubusercontent.com/42636586/119517715-7c698900-bd95-11eb-9ab6-6d647e095204.png)

What file should I add inplace for ("bert-base-uncased") so that it can work corectly in offline?
here is the link to my [notebook](https://www.kaggle.com/soumochatterjee/inference-commonlit-readability) 

Comments:
This would be better suited on the Forum, but I would suggest doing (with git-lfs installed)

```bash
git clone https://huggingface.co/bert-base-uncased
```

and then
```python
import transformers
transformers.BertTokenizer.from_pretrained("./bert-base-uncased", do_lower_case=True)
transformers.BertModel.from_pretrained("./bert-base-uncased")
```

> 
> 
> This would be better suited on the Forum, but I would suggest doing (with git-lfs installed)
> 
> ```shell
> git clone https://huggingface.co/bert-base-uncased
> ```
> 
> and then
> 
> ```python
> import transformers
> transformers.BertTokenizer.from_pretrained("./bert-base-uncased", do_lower_case=True)
> transformers.BertModel.from_pretrained("./bert-base-uncased")
> ```

@julien-c 
Clone is failing with below errors

```
C:\Users\Karthik\Desktop>git clone https://huggingface.co/bert-base-uncased
Cloning into 'bert-base-uncased'...
remote: Enumerating objects: 52, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (50/50), done.
remote: Total 52 (delta 19), reused 0 (delta 0)
Unpacking objects: 100% (52/52), 304.24 KiB | 61.00 KiB/s, done.
Updating files: 100% (10/10), done.
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'


Exiting because of "interrupt" signal.
```

@julien-c I also got the same error.

Can you add `GIT_CURL_VERBOSE=1 GIT_TRACE=1` to your command to get more info?

And also paste your `git --version` and `git lfs --version`


I was able to resolve this problem with the help of code :
[LINK](https://www.kaggle.com/abhishek/bert-base-uncased)
```
BERT_MODEL_PATH = 'PATH FOR THE DATASET YOU SAVED IN YOUR LOCAL THROUGH THE LINK'

TOKENIZER = transformers.BertTokenizer.from_pretrained(BERT_MODEL_PATH, do_lower_case=True, local_files_only=True)
model = BertModel.from_pretrained("BERT_MODEL_PATH")
```

Posted my solution to my asked question here in the issue
How get sentenses embbedings from TFBertForMaskedLM
Author: resquilleur
Pull request: no

Good afternoon!

I am solving a text clustering problem by fine-tuning a trained BERT model. After seeing a number of articles on the subject, I decided to use the masking problem and the TFBertForMaskedLM model for fine-tuning. I was able to fine-tune the network on my set, and now I want to use the embbedings of this model to transform my set and feed into the clustering algorithm.

The question and problem is that the output of `bert_model.layers[0]` is `[None, max_len, emb_size]`, I get emb of each word, and I need emb of a document or sequence, any way out?


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!

Resolved
ProphetNetForConditionalGeneration model isn't returning all objects properly
Author: pranonrahman
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.1
- Platform: Google Colab
- Python version: 
- PyTorch version (GPU?): GPU
- Tensorflow version (GPU?): 
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
- text generation: @patrickvonplaten



## Information

The model I am using Prophetnet:

The problem arises when using:
* my own modified scripts: [My notebook](https://colab.research.google.com/drive/1rmZyTXsdEDDpx8tbX-Gt6Uj9NuQi92VK?usp=sharing)

The tasks I am working on is:
*  an official SQUaD task: Question Generation

## To reproduce

Steps to reproduce the behavior:

1. Just run the notebook 
2. After running a single inference I am only getting 4 objects while I should get loss and other objects.


## Expected behavior

After running a single inference I am only getting 4 objects while I should get loss and other objects.  @patrickvonplaten



Comments:
What are all the objects you expect to get? The `loss` is only returned if you pass the labels to the model - otherwise it cannot compute any loss. Please check out the [return statement of ProphetNetForConditionalGeneration's forward method for more information](https://huggingface.co/transformers/model_doc/prophetnet.html#transformers.ProphetNetForConditionalGeneration.forward). 

Thank you, it worked @LysandreJik
Inference for pinned model keeps loading
Author: Kvit
Pull request: no

I have pinned the enterprise model: `ligolab/DxRoberta`
This model is pinned for instant start. 
When I try the Inference API `fill-mask` task, it responds instantly 
However, when I try API call to embedding pipeline https://api-inference.huggingface.co/pipeline/feature-extraction/ligolab/DxRoberta , I keep getting the message: `{'error': 'Model ligolab/DxRoberta is currently loading', 'estimated_time': 20}` and status does not change with time. 
API call to embedding pipeline was working yesterday when I tested it. 



Comments:
after I have unpinned the model embeddings pipeline started working again (unless you did something on the back end)

I have repeated the experiment:  pinned model - embedding pipeline API returns "loading" status; unpinned model - embedding pipeline returns valid results (after model 'warm up'). Look like inference API to embedding pipeline stops working if the model is pinned for instant inference access. 

Tagging @Narsil for visibility (API support issues are best handled over email if possible!)

Thanks for tagging me, the `ligolab/DxRoberta` is defined a `fill-mask` by default, so that was what was being pinned down leading the issues you were encountering. You can override that by changing the `pipeline_tag` in the model card (if you want).

There is currently no way to specify the `task` when pinning, so I did it manually for now ! You should be good to go !

If we change `pipeline_tag `do we still need to use API endpoint `/pipeline/feature-extraction/ ` ?

No if you change the default tag, then the regular route /models/{MODEL} will work ! 

@Narsil , could you chare snippet of using `pipeline_tag` in the card? I don't recall seeing this option it in the documentation https://github.com/huggingface/model_card.

Just `pipeline_tag: xxx`, see https://huggingface.co/docs#how-is-a-models-type-of-inference-api-and-widget-determined
Distillation of Pegasus using Pseudo labeling
Author: ibrahim-elsawy
Pull request: no


### Who can help
@sgugger 
@patrickvonplaten 

Models:

- Distillation of Pegasus

## Information

The model I am using (google/pegasus-xsum):

The problem:
- Trying to implement Pegasus Distillation using [Pseudo Labeling](https://github.com/huggingface/transformers/blob/master/examples/research_projects/seq2seq-distillation/precomputed_pseudo_labels.md) in [PRE-TRAINED SUMMARIZATION DISTILLATION](https://arxiv.org/pdf/2010.13002v2.pdf)
- By copying layers from the Teacher model, freezing the positional, token embeddings, all Encoder layers 
- The model trained for two epochs on Xsum-Dataset using cross-entropy loss function between logits of student and output 
generated from the teacher model 
- Generating outputs from the student model gives repeated words and poor generation, although the losses function decreases from 8 to 0.7647 in training and 0.5424 in validation 

```python
[have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have have wing wing wing wing wing wing wing wing wing wing wing wing wing']
```
How can I improve the generation of the model 


Comments:
fine tuning encoder decoder for custom language translation
Author: YanSoares
Pull request: no

Hello everyone,
I would like to know if you can train a BERT2GPT model (or other models) for translation of customized languages ​​(from scratch). I need to translate gloss signals from ASL to English.

I have already looked for tutorials on the internet, but most of them are for the task of generating text, I cannot find tutorials for translating text.

I read about EncoderDecoder, I think it's possible, I just don't know how to make a notebook to perform training from scratch using the hugging face models.

Could you help me? Has anyone done something like that?

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?


You might also find [this notebook](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/BERT2BERT_for_CNN_Dailymail.ipynb) showing how to train an encoder-decoder model interesting.

Thanks!

Hello
Thanks for reply.
I opened a question in the forum, I am waiting for some help. 
Thank you again!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
'Tensor' object has no attribute 'size'
Author: waqarkaleemkhan
Pull request: no

I am trying to implement a transformer for language classification using TensorFlow below is the model code I used but it throws an error of tensor object has no attribute size please help for the notebook you can visit the below link

https://github.com/waqarkaleemkhan/Transformer_for_Language_classification/blob/master/Transformer_for_language_classification/model.ipynb


from transformers import AutoModel
bert = AutoModel.from_pretrained('bert-base-cased')
input_ids = tf.keras.layers.Input(shape=(SEQ_LEN,), name='input_ids', dtype='int32')
mask = tf.keras.layers.Input(shape=(SEQ_LEN,), name='attention_mask', dtype='int32')
embeddings = bert(input_ids, attention_mask=mask)
X = tf.keras.layers.LSTM(64)(embeddings)
X = tf.keras.layers.BatchNormalization()(X)
X = tf.keras.layers.Dense(64, activation='relu')(X)
X = tf.keras.layers.Dropout(0.1)(X)
y = tf.keras.layers.Dense(3, activation='softmax', name='outputs')(X)


Comments:
Hi! You're using `AutoModel`, which is a pytorch model, with TensorFlow instructions. Change to `TFAutoModel`!

Hi @LysandreJik Thanks for your response when I try to import TFAutoModel from TensorFlow it gives an error that cannot import name 'TFAutomodel' from 'transformers' (unknown location)
my environment 
python =3.7
TensorFlow= 2.0
can you please guide me further what to do

What's your `transformers` version? Can you install a more recent version of `tensorflow` and see if it fixes your issue?

I have updated my TensorFlow and transformer but still I have the same issue 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Warpping model.generate before exporting into tensorflow savedmodel format
Author: techsachinkr
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
I had been using BART model and there i can use model.generate if i am working with checkpoints. However model.generate is not exported when i export the model as savedmodel ".pb" format. So is there a way to wrap model.generate while exporting savedmodel, so that it allows using beam search and generation of summary_ids to generate summary

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
Will allow summary generation with  savedmodel format for BART like models that uses  model.generate
Related to https://github.com/huggingface/transformers/issues/5443 which has been marked closed.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
tf generate compatible with tf.function
Author: kev1876
Pull request: no

because the tf generate function is not compatible with tf.function，so i can not use this function in tf server


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Bug in GPT2ForSequenceClassification
Author: abiolaTresor
Pull request: no

## Environment info
google collab

##Problem
When I run the example code provided on huggingface site for GPT2ForSequenceClassification, an error is raised sayin that 'microsoft/dialogrpt' is not a checkpoint. See the bug bellow:
![bug1](https://user-images.githubusercontent.com/48957493/115378658-10619700-a1d1-11eb-949b-67c46735534c.png)

Then , I replace 'microsoft/dialogrpt' by 'gpt2' but when I run the code twice, the logits have diffetent values at each run. After i've dug deeper, I've seen that the problem occurs when the top linear layer is being built. Its weights seem to be set randomly, so they differ from a run to another. Do you have any way to get rid of this problem? Thanks!



Comments:
Ah, indeed, I think the identifier is incorrect. It needs to be updated to `microsoft/DialogRPT-updown`. Do you want to open a PR to fix this?

The issue you mention regarding the random weights is because the `gpt2` checkpoint doesn't have a sequence classification head. This is not the case for the checkpoint mentioned above (`microsoft/DialogRPT-updown`) which does have a sequence classification head, so it will not be reinitialized every different run.

@LysandreJik Thanks a lot! Indeed, the identifier `microsoft/DialogRPT-updown` works! But, can I change the code example on hugging face site through a pull request? Since I won't really change some source code, is it a change that must be made in the Transformers repo? The (wrong) code example is as following
![bug](https://user-images.githubusercontent.com/48957493/115460610-c3111400-a228-11eb-8ad0-3db5e9d1ac44.png)



Yes, the example is created here: https://github.com/huggingface/transformers/blob/5e04d7086803ae4a3892f4082f2835a756592c2c/src/transformers/models/gpt2/modeling_gpt2.py#L1239

If you can update this, then it will update it in the docs as soon as we merge it.

All the docs visible on the website are in the repository :)

Ok, thanks! I'll deal with it then :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
batch_encode_plus set a sort parameter
Author: yuys0602
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
When use `tokenizer.batch_encode_plus`, I hope get the sorted results, so hope have the sort parameter.
Such as: `tokenizer.batch_encode_plus(..., sorted=True)`

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
Sorted by length?
How this helps you?

> Sorted by length?
> How this helps you?

for next lstm layer

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
`ci/circleci: run_tests_torch` reaches 10 min. time limit
Author: PhilipMay
Pull request: no

`ci/circleci: run_tests_torch` reaches 10 min. time limit - see here:
https://app.circleci.com/pipelines/github/huggingface/transformers/23426/workflows/349bd527-b66a-46ed-a168-365794da6856/jobs/211948

Comments:
An other one here: https://app.circleci.com/pipelines/github/huggingface/transformers/23647/workflows/b0de5fa6-3f1d-446f-8ce9-11461ff1fb10/jobs/214869

@sgugger are you aware of this issue?

Yes we are aware. This is something we will work on in the next weeks, we're just wrapping another project first.

Seems to be fixed now. Closing.
Output probability from `model.generate` for TF models
Author: gcuder
Pull request: no

# 🚀 Feature request

Since pytorch models already have the option to output probablities when using `model.generate(...)`, I wanted to ask if there's any chance if this will also be implemented for TensorFlow models?



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
ERRORS: run_mlm_performer.py
Author: luoda888
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: tfs4.5.1
- Platform: ubuntu18.04
- Python version: 3.8
- PyTorch version (GPU?): 1.7
- Tensorflow version (GPU?): 2.3.0
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

I try to use the run_mlm_performer.py

`TOKENIZERS_PARALLELISM=false python run_mlm_performer.py --model_name_or_path ../cache_model/bert-base-chinese/ --tokenizer_name ../cache_model/bert-base-chinese/ --train_file ../data/wikicorpus_zh_one_article_per_line-jianti.txt --do_train --fp16 --output_dir ./test-mlm --max_seq_length 512 --per_device_train_batch_size 256 --reinitialize --overwrite_output_dir True --preprocessing_num_workers 8`

And
 raise AttributeError(
jax._src.traceback_util.FilteredStackTrace: AttributeError: "FlaxBertSelfAttention" object has no attribute "dropout_rate"

You will find that FlaxBertSelfAttention no defined dropout_rate...

Comments:
I tried to use self.config.attention_probs_dropout_prob instead of config.dropout_rate. It seems worked.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.