noamaanMulla-03/transformers-issues huggingface dataset:
convert_graph_to_onnx.py quantization still has relative imports which break when running as a script.
Author: nbertagnolli
Pull request: no

It's my first issue so please let me know if I missed something I was supposed to include : ).

## Environment info
- `transformers` version: 4.12.3
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.8
- PyTorch version (GPU?): 1.10.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No



### Who can help
@LysandreJik
@sgugger

I didn't quite know who to tag but I saw you two on a PR related to this in the past so figured that might be the right place to start.  Sorry if I got it wrong.

## Information

I'm converting a fine-tuned distillbert model to onnx.  Everything works fine when I don't run quantization, but if I try to quantize with the script.  For example by running:

`python transformers/convert_graph_to_onnx.py --framework pt --model distilbert-base-uncased --tokenizer=distilbert-base-uncased --quantize onnx/distilbert-toxic-uncased.onnx
`

I get:

`Error while converting the model: attempted relative import beyond top-level package`

I believe the problem occurs in lines 23-26 of transformers/onnx/convert.py.  When I change these relative imports to the main library transformers like you did in this PR [#10857](https://github.com/huggingface/transformers/pull/10857) everything seems to work : ).  I'd love to open a PR with this fix if you think that would be a good idea.


The problem arises when using:
* [ x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

1. cd Transformers/src
2. Run `python transformers/convert_graph_to_onnx.py --framework pt --model distilbert-base-uncased --tokenizer=distilbert-base-uncased --quantize onnx/distilbert-base-uncased.onnx`
3. You should get the error `Error while converting the model: attempted relative import beyond top-level package`


## Expected behavior

I expect that quantization should complete and generate the `distilbert-base-uncased-quantized.onnx` file


Comments:
I have the fix on my fork and would love to open a PR if this is the right thing to do : ).  https://github.com/nbertagnolli/transformers/tree/bugfix/onnx-convert-script 

Thanks for flagging this. This looks like the right fix, so feel free to open a PR! :-)
LEDForSequenceClassification example throws a ValueError on missing decoder_input_ids/embeds
Author: alceausu
Pull request: no

## Environment info

- `transformers` version: 4.9.0
- Python version: 3.8.10
- PyTorch version (GPU?): 1.9.0


### Who can help
@patrickvonplaten @beltagy 


## Information

LEDForSequenceClassification example throws an error on line 207 of modeling_led.py:
```
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
```

## To reproduce

Run the code from the documentation of [LEDForSequenceClassification](https://huggingface.co/transformers/model_doc/led.html#ledforsequenceclassification)
```
>>> from transformers import LEDTokenizer, LEDForSequenceClassification
>>> import torch

>>> tokenizer = LEDTokenizer.from_pretrained('allenai/led-base-16384')
>>> model = LEDForSequenceClassification.from_pretrained('allenai/led-base-16384')

>>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
>>> labels = torch.tensor([1]).unsqueeze(0)  # Batch size 1
>>> outputs = model(**inputs, labels=labels)
>>> loss = outputs.loss
>>> logits = outputs.logits
```

Comments:
The same example in [BartForSequenceClassification](https://huggingface.co/transformers/model_doc/bart.html#bartforsequenceclassification) works just fine.

Yes, however this is simply because BART can automatically generate `decoder_input_ids` for training: https://arxiv.org/abs/1910.13461 this would never be used for a downstream task even for BART. E.g. See this comment: https://github.com/huggingface/transformers/blob/a3ded170e22b37027dab456a12ff2f523c99d998/src/transformers/models/bart/modeling_bart.py#L594. Now LED was never pretrained from scratch and therefore there is no use case where the `decoder_input_ids` should **not** be provided. See comment here: https://github.com/huggingface/transformers/blob/a3ded170e22b37027dab456a12ff2f523c99d998/src/transformers/models/led/modeling_led.py#L1510 for comparison. => In LED we should always provide the `decoder_input_ids` in the forward pass
layoutlmv2 input tensor shape
Author: gauravlochab
Pull request: no

![question_hugging_face](https://user-images.githubusercontent.com/20882775/140917182-f88fc896-a3fd-40c5-988d-d588110aac45.JPG)
Hi, 
AFAIK encoder in layoutlmv2 only takes 512 tokes but this snippet from hugging face documentation attached above contradicts this.
could anyone help me understand this?


Comments:
Hi,

LayoutLMv2 (and LayoutXLM) take both an `image` input and the regular `input_ids`. The `input_ids` indeed have a length of max 512 tokens (if you pad them up to the max length), however the image input is turned into a feature map, which is concatenated with the text tokens. This feature map is then flattened into a list of "image tokens", which are then concatenated with the text tokens.


@NielsRogge As you mentioned, the feature map (7*7) is flattened into a list of image tokens, which in this case is 49 (documentation), which is then concatenated with 512 to make the entire length of input 561, which is then sent into the encoder, which only accepts 512 tokens as input. What is going on with the "image tokens"? Aint they're going into the encoder? 
but as written "these are then concatenated with the text tokens, and send through the Transformer encoder"
What exactly am I missing here?

I'm not sure what you mean. You just need to give a resized `image` (224x224) as well as `input_ids` (and `token_type_ids`, `bbox`) to the model. The model will internally create the image tokens using its visual backbone.

![layoutlmV2](https://user-images.githubusercontent.com/20882775/140938612-035f3e36-f64b-482a-b880-af2a17054d7c.jpg)
are these 49 tokens fed into the transformer encoder after concatenation?


Yes, the text and image tokens are first concatenated, before being fed to the Transformer encoder.

 layoutlm paper says "We initialize the weight of LayoutLM model with the pre-trained BERT base model."  with is trained with 512 tokens only.
how come are we able to pass 561 tokens into the encoder in layoutlmv2 which is initialized with the pre-trained BERT model?

As stated in the [LayoutLMv2 paper](https://arxiv.org/abs/2012.14740):

> The model is initialized from the existing pre-trained model checkpoints. For the encoder along
with the text embedding layer, LayoutLMv2 uses the same architecture as UniLMv2 (Bao et al.,
2020), thus it is initialized from UniLMv2. For the ResNeXt-FPN part in the visual embedding
layer, the backbone of a Mask-RCNN (He et al., 2017) model trained on PubLayNet is leveraged. 
The rest of the parameters in the model are randomly initialized.

thanks for the clarification @NielsRogge
Add notebook INC quantization for text classification tasks
Author: echarlaix
Pull request: yes

Add a notebook showing how to apply Intel Neural Compressor (INC) quantization (dynamic, post-training and aware training approach) for text classification tasks.


Comments:
Should we add an **Optimum** section? :) 

Great addition!
I have a couple of comments on the notebook. Could you share a Colab version where I could put them?

(For instance the notebook does not run to completion :-/)

I shared the notebook (which now runs to completion) 

Thank you for your comments @sgugger, the notebook was updated accordingly.

Thanks for the detailed comments @LysandreJik. I updated the notebook accordingly and we do not set `use_fast` to `True` anymore as it's not related to quantization and could indeed be misleading. I also rephrased the ambiguous comment and added an introduction as well as a model sizes comparison between the full-precision and the quantized model. Finally concerning the loading of the quantized model warnings, it's a good point and we will modify our `IncQuantizedModel` class in order to have something clearer for the user.

Then LGTM :)
Fix list index out of range when padding nested empty lists
Author: qqaatw
Pull request: yes

# What does this PR do?

This PR fixes `list index out of range` error when nested empty lists are supplied.

```
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
padded = tokenizer.pad({"input_ids": [[], [], []] })
print(padded)
``` 

```
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    padded = tokenizer.pad({"input_ids": [[], [], []] })
  File "/src/transformers/tokenization_utils_base.py", line 2730, in pad
    while len(required_input[index]) == 0:
IndexError: list index out of range
```


@LysandreJik

Comments:
What is the use-case of passing nested empty lists like this?

> What is the use-case of passing nested empty lists like this?

I'm using `tokenizer.pad` to pad question answering labels like the below snippet, 

```python
# labels may look like this, where it should be padded to (batch_size, max_answers), 
# and sometimes the entire batch might have no answer being like this: `[[], [], []]`, so I proposed this PR.
start_position = [
    [3, 5, 10], # 3 answers
    [4], # 1 answer
    [], # no answer
]
end_position = [
    [4, 7, 14],
    [8],
    [],
]
```

Also, could you checkout this issue #13879 and see if that makes sense to you? (similar use-case as this PR.)

Thanks a lot!

I would appreciate it greatly if you could take a look at this when you're back @SaulLu!
[testing] solve the port conflict
Author: stas00
Pull request: yes

This is a fix for https://github.com/huggingface/transformers/pull/14331 which accidentally collapsed 2 distinct ports and the deepspeed tests started to fail.

Since `torch.dist` never releases the port, those tests that run emulated launcher env never release the port (but can re-use it). But then it prevents other tests that run with `python -m torch.distributed` to succeed and ending up with `Address already in use` error.

I'm going to merge this to unblock the CIs (ours and Deepspeed's) since tomorrow is a holiday, but would be happy to do a follow up PR if something can be improved.

@sgugger 

Comments:
Thanks for fixing!
overflow error when initialising transformerXL model (transfo-xl-wt103)
Author: blrtvs
Pull request: no

## Information 

I don't if I'm allowed to post this here as I use pytorch-pretrained-bert, but maybe someone knows this error. I want to run the code below: 

```
import torch
from pytorch_pretrained_bert import TransfoXLTokenizer, TransfoXLModel, TransfoXLLMHeadModel
import logging


logging.basicConfig(level=logging.INFO)

model = TransfoXLModel.from_pretrained('transfo-xl-wt103')
```

But I get the following error: 

```
Traceback (most recent call last):
  File "load_transfo.py", line 7, in <module>
    model = TransfoXLModel.from_pretrained('transfo-xl-wt103')
  File "/home/user123/anaconda3/envs/test_transfo372/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling_transfo_xl.py", line 939, in from_pretrained
    model = cls(config, *inputs, **kwargs)
  File "/home/user123/anaconda3/envs/test_transfo372/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling_transfo_xl.py", line 1033, in __init__
    div_val=config.div_val)
  File "/home/user123/anaconda3/envs/test_transfo372/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling_transfo_xl.py", line 780, in __init__
    self.emb_layers.append(nn.Embedding(r_idx-l_idx, d_emb_i))
  File "/home/user123/anaconda3/envs/test_transfo372/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 100, in __init__
    self.weight = Parameter(torch.Tensor(num_embeddings, embedding_dim))
RuntimeError: $ Torch: invalid memory size -- maybe an overflow? at /pytorch/aten/src/TH/THGeneral.cpp:188

```

Has somebody an idea how to get this work? When I use newer versions of torch and pytorch-pretrained-bert I just get a different RuntimeError (RuntimeError: Trying to create tensor with negative dimension -200001: [-200001, 16])

## Environment info 
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: I use pytorch-pretrained-bert=0.6.1 
- Platform: Debian GNU/Linux 9.13 (stretch)
- Python version: 3.7
- PyTorch version (GPU?): 1.0.1
- Tensorflow version (GPU?):not installed

Models:
 @patrickvonplaten


Comments:
Hey @blrtvs,

Could you maybe try to use a `transformers` version > 2.0 instead?

@patrickvonplaten Yes, I tried it and it works so far. But I'm trying to get the LAMA code from facebookResearch working and it uses pytorch-pretrained-bert. When I use transformers instead, I get many other exception due to slightly different syntax and things like that. But I already postet my issue on the LAMA Repo, maybe someone will answer. But the invalid memory/ overflow exception from above is solved, thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
there is not more toolkit and model  architecture about tabular format in hugginggface, tabular text in NLP = language + layout
Author: franztao
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
last few years, many industry work of deep learning about tabular text boom, such as "CascadeTabNet","tablenet". many old work about  tabular text  developed from web html table, but there is no united toolkit and mode architecture to accelerate the tabluar text research in NLP.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->


## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
We have support for the TAPAS model in `transformers` - would it help your use-case?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
fix loading flax bf16 weights in pt
Author: patil-suraj
Pull request: yes

# What does this PR do?

#13098 now enables saving flax weights in bf16. But converting flax bf16 weight to pt fails because `torch.from_numpy` can not handle `bfloat16` and `bfloat16` is also not fully supported by PT.

This PR fixes this by casting flax `bf16` weights to `fp32` when converting flax weights to PT.

Comments:
[wav2vec2] fix --gradient_checkpointing
Author: stas00
Pull request: yes

This PR fixes `--gradient_checkpointing` in wav2vec2 examples.

@sgugger 

Comments:
it was failing deepspeed tests for wav2vec2 - which I still run manually to check that Deepspeed didn't break something. wav2vec2 is a complex model with several manual fixes to make DS work.

but I hear you about the pinned version! This makes sense. I was running those with master, so should probably add a skip rule to match the pinned version. Which would make it much more difficult for me to run the tests. 

@patrickvonplaten, do we merge this or close the PR? Thank you!
Structuring Dataset for Fine Tuning GPT-2 with Song Lyrics?
Author: 1337-Pete
Pull request: no
Labels: name: wontfix

I'm looking to fine tune a GPT-2 model with several songs from the same artist. However, I'm not exactly sure how I should be feeding nearly 300 songs into the model to learn from.

Currently, I'm breaking the lyrics into blocks of 100 and feeding them into my model:

```
lyrics = open('/JayZ_lyrics.txt', "r", encoding="utf-8").read()
lyrics = lyrics.split('\n')
lyrics = ''.join(lyrics)

tokenizer = GPT2Tokenizer.from_pretrained('distilgpt2')
tokenized_text = tokenizer.encode(lyrics)

examples = []
block_size = 100
for i in range(0, len(tokenized_text)-block_size+1, block_size):
    examples.append(tokenized_text[i:i+block_size])

inputs, labels = [], []
for ex in examples:
    inputs.append(ex[:-1])
    labels.append(ex[1:])
```


Then, I feed my dataset into the model and train on it.

```
dataset= tf.data.Dataset.from_tensor_slices((inputs,labels))
BATCH_SIZE = 16
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)

optimizer = tf.keras.optimizers.Adam(epsilon=1e-08, clipnorm=1.0) 
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model = TFGPT2LMHeadModel.from_pretrained('distilgpt2')
model.compile(optimizer=optimizer, loss=[loss, None, None, None, None, None, None], metrics=[metric])
model.fit(dataset, epochs=4, callbacks=[LRFinder()])
```

Is this the right approach? Or, should I turn every song into a vector, find the len of longest song, pad the other songs to keep uniformity amongst vector lengths, feed in dataset.

Also, should I include a BOS and/or EOS token `
` for each block that I create?

Comments:
By the way, did this approach work for you? I tried a similar one and I couldn't stop getting an error about incompatible output shapes when calculating sparse categorical accuracy.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Any solution or suggestions yet? @1337-Pete
Sequential constraints?
Author: kcarnold
Pull request: no
Labels: name: WIP

fairseq has an implementation of unordered and ordered multi-token constraints: see their [PR](https://github.com/pytorch/fairseq/pull/2402) and [example](https://github.com/pytorch/fairseq/tree/1bba712622b8ae4efb3eb793a8a40da386fe11d0/examples/constrained_decoding).

This is more advanced than the single-token constraints that have been occasionally [requested here](https://github.com/huggingface/transformers/issues/10485) mainly due to the bookkeeping involved; see the papers referenced in the fairseq PR.

Has anyone looked into porting this feature? fairseq's [constraint tracking logic](https://github.com/pytorch/fairseq/blob/master/fairseq/token_generation_constraints.py) looks to be well-factored and could probably be adopted verbatim, license permitting. The beam search modifications ([fairseq implementation](https://github.com/pytorch/fairseq/blob/ee0d5a0f65a25e5f5372776402aac5cb9c4adbf1/fairseq/search.py#L210)) may be able to be implemented as a `LogitsProcessor`, or maybe even just a `prefix_allowed_tokens_fn`, but the papers propose some additional logic around making sure that a partial constraint stays in the beam; I'm not sure whether those hooks are sufficient to implement that logic (or how essential it is to the functionality).

(I've found one [related issue](https://github.com/huggingface/transformers/issues/1163).)

Comments:
Has anyone working on this yet?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

If no one else beats me to it, I may tackle this mid-summer. No promises. 

@kcarnold any update? 

There is another recent paper on constrained decoding with complex constraints (fairseq only has positive constraints): [NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
](https://arxiv.org/abs/2010.12884)
Use `AlbertConverter` for FNet instead of using FNet's own converter
Author: qqaatw
Pull request: yes

# What does this PR do?

**Edited: Finally using `AlbertConverter` directly because the slow tokenizers between `Albert` and `FNet` are basically identical. (FNet doesn't have `bos_token` and `eos_token` while Albert does, but this difference doesn't matter.)**

---

This PR adds normalizer to `FNetConverter`, making `do_lower_case` and `keep_accents` options of `FNetTokenizerFast` work.

This normalizer is copied from that of `Albert` as FNetTokenizerFast is adepted from `Albert`.


https://github.com/huggingface/transformers/blob/3ea15d27832d47d44ad046c3a776c7b582b0984b/src/transformers/models/fnet/tokenization_fnet_fast.py#L57-L61

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik @SaulLu @sgugger 

Comments:
@SaulLu if this looks good to you, feel free to merge
How to export BertForMaskedLM to onxx using onxx export script?
Author: Oxi84
Pull request: no

Where would I add parameter that I want to convert BertForMaskedLM to onxx. I found the code bellow, but it probably only exports just bert model and not the one with MaskedLM head.

    from pathlib import Path
    from transformers.convert_graph_to_onnx import convert
    convert(framework="pt", model="bert-base-cased", output=Path("onnx/bert-base-cased.onnx"), opset=11)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.