noamaanMulla-03/transformers-issues huggingface dataset:
[doc] :class: hunt
Author: stas00
Pull request: yes

part 2 Doc clean up continued from https://github.com/huggingface/transformers/pull/14954

- `:meth:`, `:func` - all is good
- `:class:`

```
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's#:class: ?`([^`]+)`#[`$1`]#g' {} \;
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's#:class: ?\*([^\*]+)\*#[`$1`]#g' {} \;
git checkout examples utils/check_repo.py
```

let me know if there are any other left-over rst tags to hunt down.

@sgugger 


Comments:
Unable to see total_flos in ViT training logs
Author: minh-h-ng
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.14.1
- Platform: Ubuntu 20.04.1
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): n/a
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: DDP

### Who can help

Models:
- TAPAS, LayoutLM, LayoutLMv2, LUKE, ViT, BEiT, DEiT, DETR, CANINE: @NielsRogge

Library:
- Trainer: @sgugger

Examples:
- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): ViT

The problem arises when using:
* [ o] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ o] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. download transformers with git clone and pip install
2. run example script in transformers/examples/pytorch/image-classification/run_image_classification.py
3. the resulting logs always give "total_flos=0.0"

```
"log_history": [
  {
  ...
  "step": 17,
  "total_flos: 0.0,
  "train_loss": 0.54699146...,
  ... 
```

## Expected behavior

Being able to see total floating points operations of the whole training (total_flos) in the logs


Comments:
Could you update your Transformers to the last version? I think this was fixed recently.

I updated Transformers to the latest (master) version and I can see total_flos now. Thank you for your help.
[OPTIMIZATION] remove a redundant condition statement for empty str judgement in `whitespace_tokenize`
Author: jacklanda
Pull request: yes

It seems a redundant conditional judgement here for spliting a str object, because it still works when the str object is None or empty string. Maybe I just don't know what the `if` conditional judgement here used to represent some special meaning.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add on option to output a checkpoint every x minutes
Author: wulu473
Pull request: no

# 🚀 Feature request

Extending `transformers.trainer_utils.IntervalStrategy` by an option `MINUTES` to be able to save a checkpoint based on training time.

## Motivation

For hyper-parameter searches over large batch sizes (where gradient accumulation is necessary) the time between steps can vary by orders of magnitude between different runs. This could lead to very different number of written checkpoints for different runs. In this case it could be useful to specify to have checkpoints output every x hours rather than every x steps. 

## Contribution

Happy to submit a PR for that.


Comments:
cc @sgugger 

This sounds like a useful feature, feel free to suggest a PR to add that!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Missing parameters when iterating over `module.parameters()`
Author: jankrepl
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: `4.15.0`
- Python
  - Implementation: CPython
  - Version       : 3.8.12
- Platform
  - OS          : Darwin
  - Release     : 20.6.0
  - Machine     : arm64
  - Architecture: 64bit
- PyTorch version (GPU?): `torch==1.10.1` no GPU
- Tensorflow version (GPU?): None
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten, @LysandreJik

## Information
For some reason, the parameter count of an entire `GPT2LMHeadModel` is not considering the `lm_head`. This behavior might not be only specific to this class, however, this is the one I am working with.

## To reproduce
```python
from transformers import AutoModelForCausalLM


def count_params(module):
    return sum(p.numel() for p in module.parameters())


name = "gpt2"

model = AutoModelForCausalLM.from_pretrained(name)

print(f"Whole model: {count_params(model)}")
print(f"Transformer: {count_params(model.transformer)}")
print(f"Head: {count_params(model.lm_head)}")
```
The standard output:
```bash
Whole model: 124439808
Transformer: 124439808
Head: 38597376
```

The core of the problem is that the `lm_head.weight` is not inside of `model.named_parameters()`.

## Expected behavior
The expected behavior is for the total number of parameters to be equal to the parameters inside of the `transformer` backbone PLUS the number of parameters inside of `lm_head`.

See relevant code here: 
https://github.com/huggingface/transformers/blob/1c121916f3adee769eb43d4656b621be60427bbd/src/transformers/models/gpt2/modeling_gpt2.py#L951-L952



Comments:
Hey @jankrepl,

`self.lm_head` is identical to `self.transformer.wte` since those weights are tied (input and output word embeddings are the same for a lot of Transformer models) which is why `self.lm_head` doesn't show up here I think 

```python
from transformers import AutoModelForCausalLM


def count_params(module):
    return sum(p.numel() for p in module.parameters())


name = "gpt2"

model = AutoModelForCausalLM.from_pretrained(name, tie_word_embeddings=False)

print(f"Whole model: {count_params(model)}")
print(f"Transformer: {count_params(model.transformer)}")
print(f"Head: {count_params(model.lm_head)}")
```

should give

```
Whole model: 163037184
Transformer: 124439808
Head: 38597376
```

Oh wow! What a nice trick to reduce the number of parameters!

Thank you for the quick help:)
[examples/translation] support mBART-50 and M2M100 fine-tuning
Author: patil-suraj
Pull request: yes

# What does this PR do?

the `run_translation.py` does not support fine-tuning mBART-50 and M2M100 as we need to set `src_lang` and `tgt_lang` attributes, but the script only checks for `MBartTokenizer`, this PR

- adds the `MULTILINGUAL_TOKENIZERS` list where we can add all tokenizers which require setting `src_lang` and `target_lang` attributes, this avoids having multiple if/else statements (Thanks Sylvain!)
- adds the `--forced_bos_token` argument which is used to set the `config.forced_bos_token_id` attribute which is required for mBART-50 and M2M100 during generation to be able to force the target language token as the first generated token. We could use the `--target_language` argument to set this, but this attribute shouldn't be set auto-magically as generations change completely depending upon the forced id, so IMO it's better to ask the user to explicitly provide it.



Comments:
@patil-suraj , can you please help in suggesting how to finetune m2m100 on more than one-pair.I am able to finetune for one lang pair using below script:

CUDA_VISIBLE_DEVICES=0,1,2,3,6 python -m torch.distributed.run --nproc_per_node=5 run_translation.py --model_name_or_path=m2m100_418M_new_token --do_train --do_eval --source_lang ja --target_lang en --fp16=True --evaluation_strategy epoch --output_dir bigfrall --per_device_train_batch_size=48 --per_device_eval_batch_size=48 --overwrite_output_dir --forced_bos_token "en" --train_file orig_manga/orig/train_exp_frame_50k.json --validation_file orig_manga/orig/valid_exp_frame_50k.json --tokenizer_name tokenizer_new_token  --num_train_epochs 50 --save_total_limit=5 --save_strategy=epoch --load_best_model_at_end=True --predict_with_generate

But, now I want to finetune it on ja-en and ja-zh pairs. How to pass these both languages?

Hi @nikhiljaiswal !
It would be nice if you ask this question on the [forum ](https://discuss.huggingface.co/). PR comments won't be a good place to discuss this. Thanks!
DistilBertForSequenceClassification returning nans
Author: drisspg
Pull request: no

## DistilBertForSequenceClassification returning NaNs
<!-- A clear and concise description of the question. -->
DistilBertForSequenceClassification using the distilbert-base-uncased is returning Nans for both the logits and loss. Has anyone encountered this issue? 

Comments:
I'm facing this issue. How did you resolve this?

Me too, and how did you resolve this problem?
Replace assertion with exception
Author: jaketae
Pull request: yes

# What does this PR do?

Replaces `assert` with `ValueError` as per #12789.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger @patil-suraj 

Comments:
How to instantiate custom T5ForConditionalGeneration
Author: Tomiinek
Pull request: no

## Information

I have problems using `T5ForConditionalGeneration`.

I want to evaluate performance of the model without any information from the encoder, i.e., figure out what is the lower bound of decoder performance. To do so, I use a model as follows during both training and validation:
```python
bs = labels.shape[0]
outputs = model(   
   input_ids=torch.zeros(bs, 1).long(),
   attention_mask=torch.ones(bs, 1).float(),
   labels=labels
)
```
where labels look like this:
``` python
[[ 103,   25,  214,  125,   34, 1416,  114,    3,   58,    1],
 [  11,  125,  103,   62,  217,    3,    5,    1, -100, -100],
 [ 131,  600,  631,   21,   80,  568,    3,    5,    1, -100]]
```
so the sequences apparently end with `
` and have paddings replaced with `-100`.

I take the loss from outputs as `outputs.loss` and get the most probable tokens as `tokens = outputs.logits.argmax(-1)` to calculate token accuracy and get a better notion about what is happening.

**I do not want to use pre-trained weights**, so I instantiate the model as follows:
``` python
config = AutoConfig.from_pretrained('t5-small')
model = T5ForConditionalGeneration(config)
```
At the beginning of the training, everything seems to be ok. The token accuracy is **~41%** and output distributions make sense (the token distribution on the first token in labels should roughly reflect frequency of the first tokens in the dataset, the decoder does not have any information from the encoder). However, after a while of training (1k steps), the model improves rapidly, train and val losses drop, and the model achieves accuracy hitting **80%** which is not expected at all. It is absolutely certain about the first token and predicts it correctly (which is not possible!), so I suspect that the input tokens somehow leak into the predictions or that the masking in the decoder is not causal.

**The issue does not occur when using the pre-trained weights**, i.e., instantiating the model as:
``` python
model = T5ForConditionalGeneration.from_pretrained('t5-small')
```
And the token accuracy is constantly around **49%** (verified after 15k steps).

Any ideas why or what am I doing wrong?   @patrickvonplaten

## Environment info

- `transformers` version: 4.9.2
- Python version: 3.7.0
- PyTorch version (GPU?): 1.10.1+cu113 (True)


Comments:
Fixed a bug for num_return_sequences don't take effect in Text2TextGenerationPipeline.
Author: nursery42
Pull request: yes

# Fixed a bug for num_return_sequences don't take effect in Text2TextGenerationPipeline.
The previous postprocess function always returns one result. This leads to num_return_sequences doesn't take effect in Text2TextGenerationPipeline and only can get one text result, no matter how we change the parameters.
We can get multiple text results in the postprocess method and get multiple results from Text2TextGenerationPipeline.



<!-- Remove if not applicable -->

Fixes https://github.com/huggingface/transformers/issues/13027#issuecomment-969899988


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?







Comments:
Hi @enze5088 ,

Do you mind adding a test ?
I think extending tests/test_pipelines_text2text_generation.py::Text2TextGenerationPipelineTests::test_small_model_pt would be enough (just extend with `num_return_sequences=3` and check that the returned results returns 3 strings (ideally they should be different, but given it's a random model, it might not be)

> Hi @enze5088,
> 
> Do you mind adding a test? I think extending tests/test_pipelines_text2text_generation.py::Text2TextGenerationPipelineTests::test_small_model_pt would be enough (just extend with `num_return_sequences=3` and check that the returned results returns 3 strings (ideally they should be different, but given it's a random model, it might not be)

Thank you for your reply, I extend the tests/test_pipelines_text2text_generation.py file. But I don't know what it means when check_code_quality fails and displays "test/test_pipelines_text2text_generation.py will be reformatted"


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Closing back in favor of https://github.com/huggingface/transformers/pull/14988 (Same commits with code quality)
JAVA Predict
Author: qianyiwei
Pull request: no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- how save pertained BERT model in .pb, and load model by java and predict???
- this is model saved:
-   model = TFBertForSequenceClassification.from_pretrained(model_path, num_labels=num_classes)
    model.summary()
    optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate, epsilon=1e-08, clipnorm=1)
    loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    model.compile(optimizer=optimizer, loss=loss, metrics=[tf.keras.metrics.SparseCategoricalAccuracy(name='accruacy')])
    # fit model
    bert_history = model.fit(ds_train_encoded, epochs=number_of_epochs, validation_data=ds_val_encoded,use_multiprocessing=True)

    tf.saved_model.save(model, saved_model)

ValueError: Exception encountered when calling layer "tf_bert_for_sequence_classification" (type TFBertForSequenceClassification).

- `transformers` version:4.12.5
- Platform:java
- Python version:3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?):2.4.1
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
where can I find the dataset bert-base-chinese is pretrained on?
Author: BoomSky0416
Pull request: no


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fine-tune Integer Bert for question answering task
Author: mamoon115
Pull request: no

How we can fine-tune the integer for the question answering task? Any available script

Comments:
@sgugger ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How do I preserve HTML structure when putting data through transformers
Author: brianSERP-Ai
Pull request: no

I am trying to create a bulk paraphrasing tool. I have managed to get everything working fine with normal text but now I would like to also keep the <html> structure intact.  Can anyone give me some tips on how to achieve this? When I put html through the models it is removed in the output.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Loading from checkpoint without skipping requires the same number of GPUs
Author: vitalyshalumov
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.12.5
- Platform: Linux-4.4.0-210-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.9.0a0+2ecb2c7 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

When changing the number of GPUs available, the trainer fails to load from the checkpoint that was trained with a different number of GPUs checkpoint. 
Note that this does not happen when the flag ignore_data_skip is used.

The "per_device_train_batch_size" remains unchanged between runs.

I use Trainer API 


The error looks like this:

 File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1298, in train
    self._load_rng_state(resume_from_checkpoint)
  File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1529, in _load_rng_state
    torch.cuda.random.set_rng_state_all(checkpoint_rng_state["cuda"])
  File "/opt/conda/lib/python3.8/site-packages/torch/cuda/random.py", line 73, in set_rng_state_all
    set_rng_state(state, i)
  File "/opt/conda/lib/python3.8/site-packages/torch/cuda/random.py", line 64, in set_rng_state
    _lazy_call(cb)
  File "/opt/conda/lib/python3.8/site-packages/torch/cuda/__init__.py", line 114, in _lazy_call
    callable()
  File "/opt/conda/lib/python3.8/site-packages/torch/cuda/random.py", line 61, in cb
    default_generator = torch.cuda.default_generators[idx]
IndexError: tuple index out of range


Thank you



Comments:
Yes, in general the API to resume training requires you to have the exact same setup. You won't get to the same results otherwise.

> Yes, in general the API to resume training requires you to have the exact same setup. You won't get to the same results otherwise.

My aim is not to reproduce the results. 
From time to time I have to switch between different hardware setups and I wish to continue from a checkpoint. Is it at all possible or I cannot work around this error?

This would require some digging into the Trainer to fix the error you get and see if there are others that arise. It's fixable but not high priority on our side so it might take a little bit of time.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale, will whip something around soon to avoid the error.

> 

Thank you!
bidirectional training for GPT2
Author: lytum
Pull request: no

# 🚀 Feature request

May I ask, if there is possible to fine-tune GPT2 in bidirectional training and bidirectional training together? 
Like the UniLM: https://github.com/microsoft/unilm/tree/master/unilm-v1


## Motivation

GPT2 is widely used in text generation in unidirectional training (left-to-right). Could we use GPT2 to process context in bidirectional, while generating text in unidirectional. 

Thanks for any feedback and really appreciate for any tips.

Comments:
Hi,

I've answered your question on the [forum](https://discuss.huggingface.co/t/fine-tuning-gpt2-for-bidirectional-training/12155/4).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.