noamaanMulla-03/transformers-issues huggingface dataset:
Log the right train_batch_size if using auto_find_batch_size and also log the adjusted value seperately.
Author: muellerzr
Pull request: yes
Labels: name: trainer

# What does this PR do?

This PR will log the `train_batch_size` that is reduced when using the auto-batch-finder utility, and will also seperatly log what the reduced batch size is on a debug level.


Fixes # (issue)
Solves #23762 
Solves #21950 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Enable code-specific revision for code on the Hub
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a new `code_revision` argument to all auto classes `from_pretrained` (and the auto models `from_config`) to allow for a specific revision for code on the Hub. Since code can now live in a different repo than the weights, the `revision` argument can't be used directly for the code files and we need a new argument.

This PR also makes `code_revision` default to `revision` when the repo contains both the code and the model weights.

Fixes #23745

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[WIP] CI/CD Testing
Author: AdnaneKhan
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
no_cuda does not take effect in non distributed environment
Author: sywangyi
Pull request: yes

Fixes # (issue)
no_cuda does not take effect in non distributed case. gpu is still selected.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

- trainer: @sgugger



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Sentences tokenized by LLaMA's tokenizer have `bos` tokens but do not have `eos` tokens.
Author: yqy2001
Pull request: no

### System Info

transformers version: main
Platform: Linux-5.4.0-113-generic-x86_64-with-glibc2.10
Python version: 3.8.16
Huggingface_hub version: 0.11.1
PyTorch version (GPU?): 1.12.1 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: Not

### Who can help?

@ArthurZucker @sgug

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I find that the batches tokenized by llama's tokenizer have `bos` tokens but do not have `eos` tokens, leading to my finetuned llama do not stop properly during inference. Is it a bug, or are there some reasons for this practice?

https://github.com/huggingface/transformers/blob/b8648290d2d97e7c7dbccd2d4a6a4f44e70d3b63/src/transformers/models/llama/tokenization_llama.py#L72

### Expected behavior

An explanation to my confusion.

Comments:
Same issue

Hey! Sorry for the late reply, and thanks for opening an issue 🤗 
This is expected, because the official repository's default behaviour is the same. That is because during inference you don't need it to be added. You should initialise the tokenizer with the argument set to `True`. Tell me if this does not adresses your issue 

Oh, I got it. Thanks for your reply! @ArthurZucker
Pytorch 2 compile + fsdp + transformers crash
Author: agemagician
Pull request: no

### System Info

- `transformers` version: 4.28.0.dev0
- Platform: Linux-5.10.147+-x86_64-with-glibc2.31
- Python version: 3.9.16
- Huggingface_hub version: 0.13.3
- PyTorch version (GPU?): 2.0.0+cu117 (False)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.6.7 (cpu)
- Jax version: 0.4.6
- JaxLib version: 0.4.6
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

text models: @ArthurZucker and @younesbelkada
trainer: @sgugger
PyTorch: @sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Training the official "run_clm.py" script works on TPU only when:
1. base training.
2. base training + PyTorch compile.
3. base training + FSDP.

But it doesn't work when I combine both FSDP + PyTorch compile.

I have created an example here to reproduce the problem:
https://colab.research.google.com/drive/1RmarhGBIjeWHIngO7fAp239eqt5Za8bZ?usp=sharing


### Expected behavior

The script should work using both FSDP + PyTorch compile.

Comments:
I'm not sure PyTorch XLA supports torch.compile + FSDP yet.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@agemagician, did you resolve this issue? If so, could you share the details with me?

No, it is not supported yet .
Question about Bloom pretrain
Author: ZeyuTeng96
Pull request: no

### Feature request

Hi, a question about bloom pretrain

In pretrain phase, I prepared set of unlabeled text in a .txt file. Each line is a paper or a paragraph in a paper. Each line should be independnet. So, the next line or next text is not relevant to previous one. The run_clm.py script can read those text line by line and concatenate all texts from our dataset and generate blocks by user-defined block_size param or a default value(which is 1024). 

I have a question about the concatenation. If each line (or text) in my .txt file describe different thing (means each text or paraphs are independent), then the concatenation will merge them all without an explicit 'end of text/end of paper' mark. How the Bloom model predicts next token based on previous context. How the model can predict the first token in the new paragraph by seeing previous context (the previous context describe different context).

I tried to make one block only contain one paragraph or text, but they do not have same length and get an error. If I use the concatenation mechanism, I feel like it is totally wrong. Can anyone help me to figure out these. 

### Motivation

Trying to make each line of text in .txt file as an individual block.

### Your contribution

I am not sure about which one is right

Comments:
Hi, @ZeyuTeng96 thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
bnb_4bit for Flan-T5-XL/XXL?  Can't load on Colab T4...
Author: i-am-neo
Pull request: no

### System Info

- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.15.107+-x86_64-with-glibc2.31
- Python version: 3.10.11
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.9 (gpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.10
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@sgugger (I think)

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Not sure this is the best place to raise the issue - please feel free to redirect.

Following your great [Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA blog](https://huggingface.co/blog/4bit-transformers-bitsandbytes), I used your example [Basic usage Google Colab notebook](https://colab.research.google.com/drive/1ge2F1QSK8Q7h0hn3YKuBCOAS0bK8E0wf?usp=sharing), modified the model_id to "google/flan-t5-xl" (the 3B model) as follows.  

However, the Colab session crashes (free T4 GPU).
According to the blog, T5 is supported, but I may be missing something.  Please point me the right direction?

```
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from transformers import T5ForConditionalGeneration

model_id = "google/flan-t5-xl"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16import torch

)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model_4bit = T5ForConditionalGeneration.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
# crashes - runs out of memory
```

### Expected behavior

I had expected the model to load and use test it.  I'd like to finetune Flan-T5 using LoRA.  In case the settings for T5 needs to be something specific, it would be helpful to know what they are.  Thanks.

Comments:
Can load using b16 sharded version.
tf run_clm.py calls model.get_input_embeddings().weight which does not exist
Author: wesboyt
Pull request: no

### System Info

seems to happen on mac, linux, and windows, both python 3.9 and 3.11. currently using stable from pip.

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

if you run run_clm.py from scratch it will produce an error saying that embeddings.weight doesn't exist.

https://github.com/huggingface/transformers/blob/e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6/examples/tensorflow/language-modeling/run_clm.py#L486

Looks like it details a temporary workaround, I'm guessing the get_input_embeddings function's contract has changed.

### Expected behavior

I would expect the weight variable to exist.

Comments:
cc @Rocketknight1 

I was able to fix it locally by calling embeddings.hidden_size. Not sure if that is correct.

Hi @wesboyt, can you paste the exact command you called `run_clm.py` with?

--overwrite_output_dir --do_train --model_type gpt2 --tokenizer_name gpt2-it --train_file encoded.txt --do_eval --dataloader_num_workers 1 --output_dir out --block_size 256 --save_steps 100000 --validation_file features.txt --learning_rate 0.001 --num_train_epochs 1 --optim adamw_torch --per_device_train_batch_size 8 --config_overrides num_hidden_layers=14,n_head=16,vocab_size=13,hidden_size=1024

from what i see in the debugger the type of the actual embeddings variable is TFSharedEmbeddings

I was able to fix it by using hidden_size it was succesfully training after that change


the actual hidden size variable in the debugger did not align with the config overrides hidden_size parameter, it was like 728 or 768 or something along those lines.

Hi @wesboyt, I can't reproduce the issue here - I was able to train `gpt2` using the TF `run_clm.py` script and didn't encounter these errors. Can you try the command I used and confirm that there isn't some environment issue on your machine?
```python run_clm.py --model_name_or_path gpt2 --output_dir output --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 --block_size 128```

Yea that runs for me, maybe its related to some of the other parameters. Its all good i fixed it locally for me and if people encounter it in the future they should be able to find this.

I will close this, sorry to distract the team. I believe it happened because i used torch adamw inside of tf. Funny how it still trained  after my hiddensize fix.
transformers-cli serve  doesn't support  muti-workers
Author: CaoYiwei
Pull request: no

### System Info

transformers version=4.28.1

error:
You must pass the application as an import string to enable "reload" or "workers"

transformers-cli serve uses fastapi with uvicorn, the application adds routes in a class rather than decorators. In this way, uvicorn run can not import the applications. I don't have a solution yet. Does anyone have ideas to fix this ?

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

transformers-cli serve --workers 2  

error:
You must pass the application as an import string to enable "reload" or "workers"

### Expected behavior

cli command supports muti-workers

Comments:
This is not an API we still maintain FYI.
BertTokenizer and encode_plus()
Author: SimplyLucKey
Pull request: no

I see that from version 2.4.0 I was able to use `encode_plus()` with `BertTokenizer`

However it seems like that is not the case anymore. 
`AttributeError: 'BertTokenizer' object has no attribute 'encoder_plus'`

Is there a replacement to `encode_plus`?

Comments:
No it’s still there and still identical. It’s just that you made a typo and typed `encoder_plus` instead of `encode_plus` for what I can tell.

Though we recommand using just the `__call__` method now which is a shortcut wrapping all the encode method in a single API. You can read more details on the additional features that have been added in v3 and v4 in the doc if you want to simplify your preprocessing.

Here: https://huggingface.co/transformers/preprocessing.html

> No it’s still there and still identical. It’s just that you made a typo and typed `encoder_plus` instead of `encode_plus` for what I can tell.
> 
> Though we recommand using just the `__call__` method now which is a shortcut wrapping all the encode method in a single API. You can read more details on the additional features that have been added in v3 and v4 in the doc if you want to simplify your preprocessing.

Oops sorry I completely missed that. Thank you!

long_text = "This is a very very long text. " * 300
tokenizer = BertTokenizer.from_pretrained("bert-large-uncased")
# tokenize without truncation
inputs_no_trunc = tokenizer.encode_plus(long_text, add_special_tokens=False, return_tensors='pt')

I get the following error:

AttributeError: 'BertTokenizer' object has no attribute 'encode_plus'

Is their a substitution for this?
[Time-Series] Autoformer model
Author: elisim
Pull request: yes

# What does this PR do?

Adding Time Series Autoformer model https://arxiv.org/abs/2106.13008

Related issue: #21890 

@kashif :) 

## Differences between the vanilla transformer

<img width="484" alt="image" src="https://user-images.githubusercontent.com/17675462/229438415-6a5bba78-c3bf-47b1-966a-f3664a0921e0.png">

* Introduced Series Decomposition in encoder & decoder --- done, waiting for review
* Replaced canonical self-attention with autocorrelation block --- done, waiting for review
* Added seasonal and trend inputs for the decoder --- added todo places in the code
* trend seasonal pseudo code:
```
mean_data = mean(enc_input)
zeros = zeros() # size: x_dec.size(0), prediction_length, x_dec.size(2)
seasonal_init, trend_init = decomp_layer(enc_input)

trend_init = concat(trend_init, mean_data)
seasonal_init = concat(seasonal_init, zeros)
```


Comments:
One small open issue left, is adding the series decomposition to the decoder with the trend input. Will do after the initial review :)

_The documentation is not available anymore as the PR was closed or merged._

some of the TF tests are failing and I believe they are unrelated

PR is green 

thank you @amyeroberts will get it fixed!

@amyeroberts, thank you for the comprehensive CR! I sincerely appreciate the effort and time you dedicated to thoroughly assessing this pull request. 

Will be fixed!

CR changes I did:

* Added `layer_norm_eps` 
* Model layers are now take the config, except the `AutoformerAttention` which I wasn't sure about
* Better variables names 
* Addressed questions I could answer

`fix-copies` is falling because diffs with the time-series-transformer. It's about to decide if to change time-series-transformer here, or to remove "copied from..."

@kashif @amyeroberts
add type hint in pipeline model argument
Author: y3sar
Pull request: yes

# What does this PR do?

This PR adds type hints for model argument in pipeline function

## Who can review?
@Narsil 
@amyeroberts 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think I should add PretrainedModel and TFPretrainedModel in string form just like PreTrainedTokenizerFast was given in tokenizer argument. TYPE_CHECKING is false by default

LGTM.
Failed to import transformers.models.bart.modeling_tf_bart because no module named 'keras'
Author: jybsuper
Pull request: no

The following line of code causes an error: `ModuleNotFoundError: No module named 'keras'` whenever I try to initialize a Bart model:
https://github.com/huggingface/transformers/blob/0a632f076d6b275690176b79c64c5559e1240b05/src/transformers/modeling_tf_utils.py#L39
Replaced it with `from tensorflow.python.keras.saving.hdf5_format import save_attributes_to_hdf5_group` and the error was gone.
Is this a bug or I didn't install all necessary packages?

Comments:
Just had this exact same issue. Thank you for the fix!

Shouldn't this be open until gets fixed?

Confirming the issue still exists when using latest TF (2.11). Downgrading TF to 2.9 fix the issue.

https://stackoverflow.com/questions/74586892/no-module-named-keras-saving-hdf5-format

working for me on TF==2.9