noamaanMulla-03/transformers-issues huggingface dataset:
how to fine tune with the gpt2?what's the dataset format with my own data
Author: ucas010
Pull request: no

### System Info

transformers-cli env

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.26.0.dev0
- Platform: Linux-3.10.0-1160.81.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@sgugger @Rocketknight1 
@gmftbyGMFTBY 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-generation
python run_generation.py \
    --model_type=gpt2 \
    --model_name_or_path=gpt2
    ???
   --train_file ?
   --valid_file?

### Expected behavior

fine tuned gpt2 will generate chinese well 

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions as we keep the issues for bugs in the library and feature requests only.

@ucas010  The script you linked is not for training/fine-tuing however. It's only for generation with a provided prompt.

@ydshieh year，I have tried with Chinese，but the result is not good , you see 
02/14/2023 09:52:08 - INFO - __main__ - Namespace(model_type='gpt2', model_name_or_path='gpt2', prompt='', length=20, stop_token=None, temperature=1.0, repetition_penalty=1.0, k=0, p=0.9, prefix='', padding_text='', xlm_language='', seed=42, no_cuda=False, num_return_sequences=1, fp16=False, device=device(type='cuda'), n_gpu=4)
Model prompt >>> 刑事责任，民事责任，刑事诉讼案件，简易程序，人民法院
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
=== GENERATED SEQUENCE 1 ===
刑事责任，民事责任，刑事诉讼案件，简易程序，人民法院，傳，文愛民要也�

@ucas010 , as sgugger mentioned, this kind of question is for [Hugging Face Forums](https://discuss.huggingface.co/).
The GitHub repository  is only for issues, bugs or features in the library.

I am going to close this issue. But FYI: GPT-2 is only trained on English corpus, and you can't use it for other languages (not even with just fine-tuning).
Enable `requires_grad` on input embedding to train on top of frozen layers
Author: younesbelkada
Pull request: yes

# What does this PR do?

## Motivation

In the context of `peft`, users currently needs to manually add a forward hook that enables gradient computation to the input after computing the embedding, for e.g. for `t5` one needs to call:
```python
def make_inputs_require_grad(module, input, output):
    output.requires_grad_(True)

model.get_input_embeddings().register_forward_hook(make_inputs_require_grad)
```


This PR makes the life easy for the users by wrapping this protocol in a single method `enable_input_require_grads` | Related: https://github.com/huggingface/peft/issues/80

cc @pacman100 

Wdyt @sgugger ? Maybe there is a better solution but not sure here, would love some guidance!

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks everyone! Merging!
Fix TF CTC tests
Author: gante
Pull request: yes

# What does this PR do?

(see title)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh before @amyeroberts work in #21502 these tests had no overwrite, correct. However, see the note in Amy's PR -- the labels were not correctly handled, which meant that `test_dataset_conversion` was probably terminating early [here](https://github.com/huggingface/transformers/blob/edc1e734bfc01109b8c66881d950ebbda032a6d2/tests/test_modeling_tf_common.py#L1815), which would explain the ausence of crashes prior to the PR :)

Thanks for the fix @gante ! 

@ydshieh Yes, Joao's correct. There's three different cases where the state of the tests changed: 
* For `test_loss_computation`, the previous test was being skipped. [The given reason](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/tests/models/hubert/test_modeling_tf_hubert.py#L327) was incorrect shapes - however, at the time of adding #21502 the returned loss was actually `None`.
* Some tests were being skipped by overloading [with an empty test](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/tests/models/hubert/test_modeling_tf_hubert.py#L313) rather than `unittest.skip` and so previously showed as passing.
* The branch calculating loss wasn't being touched when fitting the model - `input_values` passed in as a dictionary - as `labels` was [taken from keyword arguments, rather than `outputs['labels']`](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/src/transformers/models/hubert/modeling_tf_hubert.py#L1640). This meant some tests e.g. [`test_keras_fit` in TFModelTesterMixin](https://github.com/huggingface/transformers/blob/6f79d264422245d88c7a34032c1a8254a0c65752/tests/test_modeling_tf_common.py#L1526) previously passed as the memory intensive operation was skipped.
Flaky feature extraction tests for Donut
Author: amyeroberts
Pull request: no
Labels: name: WIP

### System Info

Note: I have observed with my own setup, but most recently this was seen in a CI environment  

- `transformers` version: 4.26.0.dev0
- Platform: macOS-13.0.1-arm64-arm-64bit
- Python version: 3.9.15
- Huggingface_hub version: 0.11.0
- PyTorch version (GPU?): 1.14.0.dev20221118 (False)
- Tensorflow version (GPU?): 2.10.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.5.3 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: Noe
- Using distributed or parallel set-up in script?: No

### Who can help?

@amyeroberts 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

pytest tests/models/donut/test_feature_extraction_donut.py::DonutFeatureExtractionTest

### Expected behavior

Tests don't randomly fail e.g. like in [this CI run](https://app.circleci.com/pipelines/github/huggingface/transformers/53575/workflows/40f5b896-c941-4a9f-8946-26f54bb14505/jobs/644204).

There is an issue occurring when the amount to pad is being calculated - some dimensions are having negative padding found. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This is not yet resolved. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: filter encoder inputs when its signature does not accept wildcards
Author: gante
Pull request: yes

# What does this PR do?

Now that we are moving towards any-to-text modalities, `generate` should be strengthened to work as-is without throwing exceptions just because a user has designed a slightly different architecture.

This PR enables the case where the model has some kwargs for operations between the encoder and the decoder -- i.e. for kwargs that can't be used in the encoder, but are also not decoder inputs. Normally, when a kwarg is not an encoder input, a `decoder_` prefix is added to its name, which is not the right argument naming in this case. Fortunately, the solution is simple :D 

This PR is a soft requirement to integrate [MM-CoT](https://github.com/amazon-science/mm-cot/tree/main), the alternative being the incorrect renaming of a few arguments to `decoder_(...)`.



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Tests for the feature added 👍 

The failing CI tests are two known failures, merging.
Seq2SeqTrainer with predict_with_generate prints config every step
Author: eyalmazuz
Pull request: no

### System Info

Transformers version 4.26.1
Python version 3.8.12

### Who can help?

@sgugger

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I'm working on a Hebrew-Arabic machine translation task by training the T5 model from scratch

The issue occurs with the Translation code from the tutorial in the huggingface's course:
https://huggingface.co/course/chapter7/4?fw=pt

I have a slightly modified version of it (extra metrics and a slightly different dataset)

here's a link to my code:
https://pastebin.com/LaCAPsCF

``python3 train_model.py --dataset_path ./data/HF_HE_AR_Dataset.json --tokenizer_path ./T5Tokenizer/ --max_length=128 --batch_size=16 --logging_steps 100 --save_steps 100 --model t5-base``

this is was the command that I ran but it doesn't matter much since the problem is with ``predict_with_generate=True,`` in the Seq2SeqTrainingArguments. If it's false the problem do not occurs


This is what happens when it reaches the evaluation loop

![image](https://user-images.githubusercontent.com/34383384/218470931-e9b694ea-7dc7-464e-8a3a-37bf28c3898a.png)



### Expected behavior

Only the TQDM bar is printed

Comments:
cc @gante 

Hey @eyalmazuz 👋 There is a chance that the issue is sorted already (see [here](https://github.com/huggingface/transformers/pull/21385)). To try it out, install `transformers` from `main` -- let me know if it works!

`pip install --upgrade git+https://github.com/huggingface/transformers.git`

Hi @gante there's no extra printing when using transformers-main I'll close the issue
thank you for your help
Donut base-sized model, pre-trained only for a new language tutorial
Author: Wyzix33
Pull request: no

### Feature request

I'm trying to generate a new pre-training for Donut model using Romanian language documents.

I have about 100k scanned documents and want to create a pre-training for Romanian language to be able to fine-tune different types of documents for parsing and classification on it.

Can anyone share or create a tutorial on how to generate new Donut pre-train model from scratch?

Thanks

### Motivation

Donut only have a a few language models

### Your contribution

I can share the pretrained model i create for Romanian language

Comments:
Those questions are most suited to the [forums](https://discuss.huggingface.co/) where the whole community will be able to help. We keep issues for bugs and feature requests only.

OK, will do that! 
Thanks
from_pretrained() breaks with empty state_dict and model path as None
Author: harubaru
Pull request: no

### System Info

- `transformers` version: 4.26.1
- Platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.12.1+cu116 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker @youne

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

To reproduce the behavior, you can run this snippet of code:

```py
from transformers import GPTJConfig, AutoModelForCausalLM
from collections import OrderedDict

config = GPTJConfig(
    n_positions=128,
    n_embd=16,
    n_layer=2,
    n_head=2
)

AutoModelForCausalLM.from_pretrained(
    None, config=config, state_dict=OrderedDict()
)
```

### Expected behavior

The expected behavior from this is to allow a model to be initialized from scratch with an empty `state_dict` and `None` as the pretrained model. There is a [tool that I am working with](https://github.com/coreweave/tensorizer) that is broken due to a regression that was introduced around version 4.23.1. From the trace of running the reproduction code, a PR that handles the case when `resolved_archive_file` and the pretrained model path are `None` would fix this issue.

It also seems that this issue is related: https://github.com/huggingface/transformers/issues/21526

Comments:
This was fixed by #21542. You will need to install from source while the fix makes it way to the next release.

The PR didn't seem to fix the issue, I get a different error this time but the reproduction code still throws an error with the latest changes from source

Indeed, the fix was incomplete for generative models. The PR linked above should fully fix it.
Fix generation config for empty state dict
Author: sgugger
Pull request: yes

# What does this PR do?

This PR follows up on #21542 which didn't fix the problem for generative models. For those, the generation config can't be generated properly and returns another type of error than the one intercepted in `from_pretrained`. The fix is thus easy.

To make sure to catch the problem on all models, the test added in #21542 is graduated as a common test.

Fixes #21610

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix typo in documentation.
Author: mmcdermott
Pull request: yes

# What does this PR do?

Replaces "the value used to module the next token probabilities" with "the value used to modulate the next token probabilities", which I think is what was originally meant.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
fix: Race Condition when using Sagemaker Checkpointing and Model Repository
Author: DougTrajano
Pull request: yes

# What does this PR do?

Fixes #21586

With the following changes:

- Added `_add_sm_patterns_to_gitignore()` as a helper method in the Trainer class that will add the patterns used by the SageMaker Checkpointing feature in the .gitignore file when initializing the Model Repository.
- A condition in the `init_git_repo()` to check if we have an important SageMaker environment variable 

It also includes a fix in the huggingface_hub library in order to consider excluding patterns when defining large files: https://github.com/huggingface/huggingface_hub/pull/1339

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@sgugger as we discussed this issue in the https://github.com/huggingface/transformers/issues/21586

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Perfect, you just need to run `make style` on your branch with our quality tools installed and we should be good to merge!

> Perfect, you just need to run `make style` on your branch with our quality tools installed and we should be good to merge!

ok, let me do that so.

@sgugger sorry for the delay, some meetings here :/

it's done
Remove extra "`max_length` is reached." from InfNaNLogitsProcessor documentation
Author: mmcdermott
Pull request: yes

# What does this PR do?

Remove extra "`max_length` is reached." from InfNaNLogitsProcessor documentation


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
_The documentation is not available anymore as the PR was closed or merged._
BLIP-2 batch generate error
Author: LiJunnan1992
Pull request: no

### System Info

`transformers` version: 4.27.0.dev0

### Who can help?

@NielsRogge  

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The following error happens when running BLIP-2 model's generate() with num_beams>1 and input batch_size>1

<img width="892" alt="Screenshot 2023-02-13 at 6 53 28 pm" src="https://user-images.githubusercontent.com/13638455/218443029-8bc45f00-4884-4785-95a9-228aa08d1266.png">


### Expected behavior

The model should be able to do batch generate.

Comments:
Can confirm that:
1. The issue only happens with `batch_size>1`
2. The issue happens with both `num_beams>1` and `num_return_sequences>1` (they both rely on input replication, which is my suspicion)
3. https://github.com/huggingface/transformers/pull/21580, which addresses some BLIP2 `.generate()` issues does not fix this issue

I think that [this](https://github.com/salesforce/LAVIS/blob/3ac397aa075c3e60b9521b012dda3660e3e35f1e/lavis/models/blip2_models/blip2_opt.py#L213) is not incorporated in our current implementation. It seems the authors only defined this for OPT but not for T5.

@NielsRogge yeah, that's the fix 👍 However, I'm generalizing the PR to correctly expand any model input, as it is a bit limited at the moment (it is expanding tensors with specific names, as opposed to all tensors that might be used as model input).

FYI, I have tried to do repeat_interleave for the inputs_embeds, but that results in another error.
T5 does not seem need this because of the encoder-decoder architecture.

@LiJunnan1992 that's correct, only non-encoder inputs need the expansion. In a nutshell, in the generation loop, we have a 1:1 input-output row correspondence, so we need to expand the model inputs before the loop accordingly.

T5-BLIP2 sends `inputs_embeds` to the (text) encoder, whereas OPT-BLIP2 has no (text) encoder at all. In the former, the encoder outputs need to be expanded, whereas in the latter the `inputs_embeds` need the expansion treatment.

#21624 takes care of all those cases for any model input in `model_kwargs`, which should future-proof `.generate()`

@LiJunnan1992 if you install `transformers` from `main`, it should be working 🙏 

@gante I can verify that this is working now, thanks! When will I be able to pip install this version?

@LiJunnan1992 we aim at monthly releases, so 1-2 weeks from now :)
Fix Blip-2 CI again
Author: ydshieh
Pull request: yes

# What does this PR do?

The fix added in #21566 have to be applied to a later merged commit #21624

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@gante Don't worry. You added 2 new tests, and we just need to use FP16 (for those 2 new tests) to avoid GPU OOM. The original 2 tests are not undo by your PR.

Oh, I see! haha that makes more sense now :)
Skip wav2vec2 hubert high mem tests
Author: amyeroberts
Pull request: yes

# What does this PR do?

Skips some more troublesome Hubert and Wav2Vec2 tests. `dataset_conversion` for Hubert is now occasionally causing errors - see [this comment](https://github.com/huggingface/transformers/pull/20725#issuecomment-1430442813) until resolved. Skipping all tests that I've seen hit high memory and have changed because of #21502. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._