noamaanMulla-03/transformers-issues huggingface dataset:
Longformer TF int32 vs int64
Author: ichenjia
Pull request: no
Labels: name: bug

### System Info

Transformers Version: 4.20.0.dev0
Ubuntu 20
Python 3.8

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hi, 

@ibeltagy 
 
I am trying an example of fine-tuning longformer and got the error of

`TypeError: Input 'updates' of 'TensorScatterUpdate' Op has type int32 that does not match type int64 of argument 'tensor'.
`

Not sure what's going on. Here is my code example. Any help would be great:

`
from transformers import LongformerTokenizer, TFLongformerForSequenceClassification
from datasets import Dataset

import tensorflow as tf
import pickle
import numpy as np

tokenizer = LongformerTokenizerFast.from_pretrained('allenai/longformer-base-4096')
model = TFLongformerForSequenceClassification.from_pretrained('allenai/longformer-base-4096', num_labels=2, gradient_checkpointing=True)
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=tf.metrics.SparseCategoricalAccuracy(),
)


my_dict = {'text': ["random text 1", "random text 2", "random text 3"],
            'label': np.array([0, 0, 1], dtype=np.int64)}



dataset = Dataset.from_dict(my_dict)



tokenizer = LongformerTokenizer.from_pretrained('allenai/longformer-base-4096')

def tokenize_function(examples):
   return tokenizer(examples["text"], padding="max_length", truncation=True)



tokenized_datasets = dataset.map(tokenize_function, batched=True)



small_train_dataset = tokenized_datasets.shuffle(seed=42)

from transformers import DefaultDataCollator

data_collator = DefaultDataCollator(return_tensors="tf")


tf_train_dataset = small_train_dataset.to_tf_dataset(
    columns=["attention_mask", "input_ids", "token_type_ids"],
    label_cols=["labels"],
    shuffle=True,
    collate_fn=data_collator,
    batch_size=8,
)



model.fit(tf_train_dataset, batch_size=1)

`

### Expected behavior

Not giving the error

Comments:
I believe you'll find the solution in [#13632](https://github.com/huggingface/transformers/issues/13632)



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Can't find ‘romanian_postprocessing.md’ file
Author: sriatz
Pull request: no
Labels: name: bug

### System Info

In this model card: https://huggingface.co/facebook/mbart-large-en-ro, it says ' Instructions in romanian_postprocessing.md'. But I cannot find romanian_postprocessing.md. 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

N/A

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[Doctest] Add `configuration_roberta.py`
Author: daspartho
Pull request: yes

Add `configuration_roberta.py` to `utils/documentation_tests.txt` for doctest.

Based on issue #19487

@ydshieh could you take a look at it?
Thanks :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

haha, no worries 😄
I think group_by_length feature can be faster using a namely length list
Author: sorryhyun
Pull request: no

### Feature request

Current group_by_length requires ```lengths``` list that can be fully-loaded from dataset:
```
class Trainer:
...

    def _get_train_sampler(self) -> Optional[torch.utils.data.Sampler]:
    ...

        if self.args.group_by_length:
            if is_datasets_available() and isinstance(self.train_dataset, datasets.Dataset):
                lengths = (
                    self.train_dataset[self.args.length_column_name]
                    if self.args.length_column_name in self.train_dataset.column_names
                    else None
                )
```
For example, If I want to make lengths list with 'attention_mask' column, ```lengths``` list would be look like:
```
[[1,1,1,1,1],[1,1,1,1,1,1,1,1,1],[1,1,1] ...]
```
But If I want to use length information only, why can't I make this list like:
```
[5,9,3...] # length of each row
```
If I make this 'length file' before training with group_by_length feature, then the ```lengths``` list can be easily made by :

```
def appender(length):
    temp = []
    temp.append([1 for _ in range(length)])
    return temp

class customized_trainer(Trainer):
    def handle_group_by_length(self, filepath=''):
        with open(filepath,'r') as f:
            lengthsfile=json.load(f) # json or whatsoever

        import multiprocessing as mp
        pool = mp.Pool(6)
        lengths = pool.map(appender, lengthsfile)
        lengths = [ent for sublist in lengths for ent in sublist]

        self.lengths = lengths

```

```
trainer.handle_group_by_length('file.json') # this would load the length file
```

This reduced the loading time into about 1/10, 30 mins into 3 mins for me. 

### Motivation

While I tried to train with ~25GB text dataset, this code required about 30 minutes to load ```lengths``` list.
I found that loading full column in the group_by_length feature is the problem.
So I changed the code to load much smaller dataset - column length list - and It worked.


### Your contribution

This would require users to make "row length list file' before training. So I'm not that assertive to this proposal. 

Comments:
cc @sgugger 

As explained in the documentation, you should provide the `lengths` column in the dataset. If you use the `Dataset.map` method to build it, you won't need to implement any multiprocessing since it will do it for you. It will also cache the result so you only need to the computation once.
Add Table Transformer
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds [Table Transformer](https://github.com/microsoft/table-transformer) by Microsoft, which are DETR-compatible models for table detection and table structure recognition tasks in unstructured documents.

Note: I'm making some updates to the original DETR implementation, however these are justified by the fact that the original DETR implementation by Facebook AI also includes these things, which I didn't add when first porting DETR. Hence, our DETR implementation is now more aligned with the original one.

To do:

- [ ] transfer checkpoints to the Microsoft organization
- [ ] add link to notebook



Comments:
_The documentation is not available anymore as the PR was closed or merged._

I'm very much not in favor of adding a new config parameter that controls where the layernorm is applied. I'm not surprised the original code has it, as Facebook AI usually codes models in a modular way, but not Transformers. We had the same thing with BART and friends, and they are coded as distinct models in the library.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing this PR in favor of #19614
Mutli target classification
Author: baniasbaabe
Pull request: no

### Feature request

Is there a way to do multi target clasification e.g for text classification? 
for example:
Input: text
Output 1: Male/Female
Output 2: Happy/Angry

### Motivation

It‘s annoying to embed the outputs of the model into a custom model

### Your contribution

Unfortunately not

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!
[Doctest] Add `configuration_bigbird_pegasus.py` and `configuration_big_bird.py`
Author: xlahuerta
Pull request: yes

Add `configuration_bigbird_pegasus.py` and `configuration_big_bird.py` to `utils/documentation_tests.txt` for doctest.

Based on issue https://github.com/huggingface/transformers/issues/19487

@ydshieh could you please check it?
Thank you :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for the amazing work you guys do <3
[Time Series Transformer] Add doc tests
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR improves the code snippets in the docs of Time Series Transformer and makes sure they are tested.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

LGTM! thanks!

@ydshieh always better to learn more than earn more

@NielsRogge for generation you can also use the other test batch if you like
fix BLOOM ONNX config
Author: nmntz
Pull request: yes

Fixes dynamic axes for BloomOnnxConfig. After this PR https://github.com/huggingface/transformers/pull/18344, if use_past is used
* past/present keys should have the dynamic axes `{0: 'batch', 1: 'past_sequence + sequence'}`
* past/present values should have the dynamic axes `{0: 'batch', 2: 'past_sequence + sequence'}`


Should also fix failing tests for BLOOM's ONNX export.
(tested using `RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -k "bloom" -s -x`)
cc @lewtun @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix `test_tf_encode_plus_sent_to_model` for `TAPAS`
Author: ydshieh
Pull request: yes

# What does this PR do?

`TapasTokenizer.encode_plus` requires a `table` argument. Currently, this test calls `TokenizerTesterMixin.test_tf_encode_plus_sent_to_model`, therefore it didn't provide this argument and fails.

This PR completely overwrites this test in `TapasTokenizationTest`, just like the pytorch one `test_torch_encode_plus_sent_to_model`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Allow TFBertTokenizer to use Tensorflow text BertTokenizer (and not FastBertTokenizer) to make it servable by TF Serving
Author: piEsposito
Pull request: no

### Feature request

I would like to serve a bundle of Tokenizer + Model on TF Serving, but can't do it because TF Serving still have no support for TF FastBertTokenizer annd FastBertNormalize operations (https://github.com/tensorflow/serving/issues/2064).

It would be good if we could let [TFBertTokenizer ](https://github.com/huggingface/transformers/blob/4ed0fa3676ad8900eaa982a6c5c2ad6b75c8ea46/src/transformers/models/bert/tokenization_bert_tf.py) give the user an option not to use Tensorflow FastBertTokenizer when creating a TFBertTokenizer, so that it is servable on TFServing. 

It would consist of moving (or creating an option to change) this

https://github.com/huggingface/transformers/blob/4ed0fa3676ad8900eaa982a6c5c2ad6b75c8ea46/src/transformers/models/bert/tokenization_bert_tf.py#L67-L69

To this:
```python
# to avoid naming collision with transformers BertTokenizer
from tensorflow_text import BertTokenizer as TFBertTokenizerLayer

lookup_table = tf.lookup.StaticVocabularyTable(
    tf.lookup.KeyValueTensorInitializer(
        keys=vocab_list,
        key_dtype=tf.string,
        values=tf.range(
            tf.size(vocab_list, out_type=tf.int64), dtype=tf.int64),
            value_dtype=tf.int64
        ),
        num_oov_buckets=1
)
        
self.tf_tokenizer = TFBertTokenizerLayer(
    lookup_table, token_out_type=tf.int64, lower_case=do_lower_case
)
```

### Motivation

I would like to serve a bundle of Tokenizer + Model on TF Serving, but can't do it because TF Serving still have no support for TF FastBertTokenizer annd FastBertNormalize operations (https://github.com/tensorflow/serving/issues/2064).

As this lib is much faster to solve this kind of thing than TF Serving, I thought it was worth it trying to solve it from here. 

### Your contribution

I can definitely submit a PR with that if you approve the idea. 

EDIT: I've created https://github.com/huggingface/transformers/pull/19590 to showcase the idea. 

Comments:
Fix `ImageToTextPipelineTests.test_small_model_tf`
Author: ydshieh
Pull request: yes

# What does this PR do?

`ImageToTextPipelineTests::test_[small/large]_model_[pt/tf]` were all skipped before. I believe it was enabled after #19366 (or its child commit - BTW, thank you @sgugger !).

We have to update the wrong expected values.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
fix flaubert tokenizer
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix an issue from #19330, see the comments in the changes.

Current test failure [here](https://github.com/huggingface/transformers/actions/runs/3231701081/jobs/5291558259)
```bash
E               if self.do_lowercase:
E           AttributeError: 'FlaubertTokenizer' object has no attribute 'do_lowercase'
```



Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add `configuration_xlm_roberta_xl.py`
Author: ztjhz
Pull request: yes

Add configuration_xlm_roberta_xl.py to utils/documentation_tests.txt for doctest.

Based on issue #19487 

@sgugger @ydshieh  could you please check it?
Thanks :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Whisper] Fix gradient checkpointing (again!)
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/19537#issuecomment-1276629836


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Collator only gets keys from the dataset which are inputs to the model
Author: julian-tonita
Pull request: no

### System Info

- `transformers` version: 4.21.3
- Platform: macOS-12.1-arm64-arm-64bit
- Python version: 3.9.10
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.10.2 (False)
- Tensorflow version (GPU?): 2.7.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Steps to reproduce:
- Use a dataset with certain keys expected by a custom collator but not the model inputs
- Pass the dataset and custom collator to the huggingface trainer
- Run training and the collator will not be passed the correct keys

### Expected behavior

Huggingface Trainer automatically removes keys from the dataset which aren't needed by the model, but doesn't allow for the possibility that the collator might take different inputs than the model. This is unexpected as the collator is passed the data prior to the model so when stripping unused keys it should be done after the collator not prior.

Comments:
You can just the option `remove_unused_keys=False` from your training arguments in this case.

Okay, thanks! I looked through before and didn't see anything, actually CTRL-F'd `key` but there was nothing. I believe it has actually been renamed `remove_unused_columns` which is why I didn't find it. Thanks for the help again! 

Ah yes, sorry about the wrong name!

No worries, thanks for the help.
[Doctest] Add `configuration_resnet.py`
Author: daspartho
Pull request: yes

Add `configuration_resnet.py` to `utils/documentation_tests.txt` for doctest.

Based on issue #19487

@ydshieh could you take a look at it?
Thanks =)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Could you run `make style` to see what's wrong 🙏 ?
Fix whisper doc
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes whisper doc-test. I used `add_code_sample_docstrings` but didn't properly check that it does not support whisper model (which has to be given `input_ids`) 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks, I missed that 😅 

> Thanks, I missed that 😅

Me too, I am bad guy!
Sharding fails in TF when absolute scope was modified if `.` in layer name
Author: ArthurZucker
Pull request: yes
Labels: name: TensorFlow

name: Core: Modeling

# What does this PR do?

Fixes #18776, by taking care of the particular case of absolute scope modifications


## Who can review?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Do you think we should add a special test for that @sgugger ? 

On it! 🤗

Just added a test, RAG works, I think I'll try to test most of our models just to be sure that there is no other strange pattern. 

Tested with `"openai/clip-vit-large-patch14",  "xlm-roberta-base"` in addition, works nicely.
Beam search indices calculation issue
Author: woshizouguo
Pull request: no

### System Info

In the generator, the final [beam_indices](https://github.com/huggingface/transformers/blob/bd469c40659ce76c81f69c7726759d249b4aef49/src/transformers/generation_utils.py#L2376) calculated here may not be the `beam_indices` with the best score.

For example, assume a generated text ended at time step T, the output length = T, and it has best score in the `beam_hypo` in the end.  The beam indices length is T.

When in `T+1` step, the `beam_indices` still keeps adding next beam index from TopK, for example, the TopK returns a text with `T+1` length, the beam_indices will add this Top 1 beam idx in `T+1` even though it is not the best score. `beam_indices` length becomes `T+1`.  So in the end, the `beam_indices` represents a longer sequence T+1, but the `generated_outputs.sequence` is a short sequence T with best score, and its beam indices are not stored in `beam_hypo`

### Who can help?

@patrickvonplaten

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The logic is shown in the code.

### Expected behavior

`beam_indices` should be consistent with and representing the best sequence.

Comments:
cc @gante 

Look like it was fixed in the latest version. The bug exists in Version 4.18

Hi @woshizouguo 👋 Glad to hear it is fixed in the most recent versions! Feel free to reopen this issue if you believe you have further queries (related to the latest version, as we can't change the past :) )
Tokenizer from_pretrained should not use local files named like token…
Author: sgugger
Pull request: yes

…izer files

This fixes the issue reported in #19488.
Basically, if a user has a local file in the working directory named like any of the files the tokenizer is looking for in `from_pretrained`, for instance `tokenizer.json`, that file is going to be used instead of the file in the repo/folder passed along. The added test fails on current main and is fixed by the PR.

Fixes #19488


Comments:
_The documentation is not available anymore as the PR was closed or merged._