noamaanMulla-03/transformers-issues huggingface dataset:
I used a trainer to pretraining a BertForMaskedLM model, but the training loss always be zero
Author: ryanforhoon
Pull request: no

### System Info

> transformers==4.28.0


### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I used a trainer to pretraining a BertForMaskedLM model, but the training loss always be zero
```

config = BertConfig(vocab_size=40000,num_hidden_layers=6,)
model = BertForMaskedLM(config)
print('Number of parameters: ', model.num_parameters())


pretrained_models_path = "my path"

training_args = TrainingArguments(
        output_dir=pretrained_models_path,
        overwrite_output_dir=True,
        per_device_train_batch_size=32,
        num_train_epochs=10,
        save_steps=10000,
        save_total_limit=2,
        prediction_loss_only = True,
        fp16=True,
    )
trainer = Trainer(
        args=training_args,
        train_dataset=train_dataset,
        data_collator=data_collator,
        model=model,
    )

trainer.train()
```

when I finish the training, the results show these:


```
Step | Training Loss
-- | --
500 | 0.000000
1000 | 0.000000
1500 | 0.000000
2000 | 0.000000
2500 | 0.000000
3000 | 0.000000

TrainOutput(global_step=3130, training_loss=0.0, metrics={'train_runtime': 367.6023, 'train_samples_per_second': 272.033, 'train_steps_per_second': 8.515, 'total_flos': 1779771658266624.0, 'train_loss': 0.0, 'epoch': 10.0})
```


### Expected behavior

How can I modify the code to resolve this issue?

Comments:
Please use the [forums](https://discuss.huggingface.co/) to debug such issues in your code :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Trainer logs to wrong wandb project
Author: david-waterworth
Pull request: no

### System Info

- `transformers` version: 4.30.2
- Platform: Linux-5.15.0-76-generic-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I've written my own `train.py` based on [sequence_classification](https://huggingface.co/docs/transformers/tasks/sequence_classification). I see the same issue for official scripts.

before running I initialise wandb logging (`wandb init`). This creates `wandb/settings` containing the project name I chose for my model

```
[default]
entity = my-username
project = my-project
base_url = https://api.wandb.ai
```

But the Trainer logs everything to the project `huggingface`, i.e. it's ignoring/overriding the project name I've configured.

### Expected behavior

Don' over-ride the configured wandb project with a default.

Comments:
cc @muellerzr 

This actually isn't possible currently, you need to set environmental variables such as `export WANDB_PROJECT my-project` 

@david-waterworth You can use the os module to set the environment variable or use what @muellerzr suggested
```
import os
os.environ["WANDB_PROJECT"] = "my-project"
```

Thanks @muellerzr but I'm not sure what you mean by not possible? It is possible in general, the steps I follow are:

1. Create a new project

``` bash
mkdir my_project
cd my_project
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
```

3. Initialise wandb

``` bash
pip install wandb
wandb init
```

In the last step I create a new project name (say test), this creates `wandb/settings`

``` bash
cat wandb/settings
```

```
[default]
entity = my-user
project = test
base_url = https://api.wandb.ai
```

4. Create a script

``` python
import wandb
wandb.init() # Don't pass a project name!

print(wandb.run.project_name()) # correctly picked up setting from wandb/settings
```

I'm assuming that what the trainer does is check the env variable, and if its not set, explicitly passes "huggingface" as the project - i.e.

```
wandb.init("huggingface")
```

As a workaround, I can parse the settings file myself and set the env variable

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: add missing logits processors docs
Author: gante
Pull request: yes

# What does this PR do?

Some logits processor classes were missing on the docs. This PR corrects it by:
1 - Adding all missing logits processors to the docs
2 - Adding missing top-level imports (to stay consistent with previously existing logits processors)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I've confirmed that the previously missing docs are now properly rendered in the doc preview 👍 

@ArthurZucker re tests -- fully on board. If you're okay with it, I'd like to do it on a separate PR :) Ditto for the side bar on the left, it is in need of an update!

@ArthurZucker ready for a re-review! 

To recap, the previously reviewed version:
- Added missing logits processor docs

This version also:
- Updates the left-handed TOC for this page (splits the classes by framework, so it's easier to navigate; I've decided against enumerating the logits processors in the docs since the list is quite big)
- Sorts the logits processors in the docs by alphabetical order
- Adds missing output classes to the output class docs section

In parallel, other concerns raised in the PR are being tackled in another places:
- #25692 Adds logits processors to the doctests

(preview of the left-handed side TOC)
<img width="471" alt="Screenshot 2023-08-24 at 14 21 45" src="https://github.com/huggingface/transformers/assets/12240844/997a2a9c-38ee-467f-a8b4-13b42a73f1c6">
Generate: logits processors are doctested and fix broken doctests
Author: gante
Pull request: yes

# What does this PR do?

This PR:
- Adds `logits_processors.py` to the doctests
- Fixes the broken tests (while fixing the tests, also improved the examples with better practices and shorter examples)
- Updates sample-related examples to much shorter examples (the examples were too long and not very representative of the processor)

Comments:
It's nice to see the red cross 🔥 

_The documentation is not available anymore as the PR was closed or merged._

ooh boy 🤦 

Super glad to see the full power of the `tests_pr_documentation_tests` on display 🚀 

@ArthurZucker ready for a review! I've updated the PR header with a summary of the changes :)
[CLAP] Fix logit scales dtype for fp16
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

On some hardware, taking `torch.log` of a tensor in float16 on the CPU fails:
```python
in __init__(self, config)
   1956         audio_config = config.audio_config
   1957 
-> 1958         self.logit_scale_a = nn.Parameter(torch.log(torch.tensor(config.logit_scale_init_value)))
   1959         self.logit_scale_t = nn.Parameter(torch.log(torch.tensor(config.logit_scale_init_value)))
   1960 

RuntimeError: "log_vml_cpu" not implemented for 'Half'
```
Note that this only failed for me on a Colab T4, but not on a Titan RTX (used to test #25682).

Let's take `math.log` **then** convert it to a tensor - this will respect the dtype of the model but not take `torch.log` of a float16 CPU param.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yep try with this:
```python
import torch

torch.tensor(0).half().dtype()
```
Gives:
```
tensor(0., dtype=torch.float16)
```

Is used when we load `diffusers` pipelines in fp16 (load state dict in fp16 on cpu then move to cuda)
[DOCS] Add example for HammingDiversityLogitsProcessor
Author: jessthebp
Pull request: yes

# What does this PR do?

Adds an example to the docstring of HammingDiversityLogitsProcessor class definition in [this file](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/logits_process.py).

Part of the docs work on [Generate: have an example on each logits processor class docstring ](https://github.com/huggingface/transformers/issues/24783)

Changes -

-   Added example
-   Added some more info about beam search and how hamming diversity works with it
-   Added warning/tip about resources

Happy to edit it all down a bit if it's too much stuff!

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.

## Who can review?

@gante 

Comments:
@gante -- hey! Currently, the checks are failing because of some changes in the docs, but not sure if I can/should fix those-- using black, the files I've changed seem to be formatted correctly? Is there something I need to change in the docs to have this merge? 

@gante Fixes made! :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25481). All of your documentation changes will be reflected on that endpoint.

@jesspeck thank for you iterating and thank you for the contribution 🤗
fix encoder hook
Author: SunMarc
Pull request: yes

# What does this PR do ? 
Fixes #23385. This PR makes sure that we indeed have a hook with `io_same_device=True` on the encoder which was not the case previously. If we don't have that, the generation will fail as the output of the encoder won't be on the same device as `unfinished_sequences = torch.ones(input_ids.shape[0], dtype=torch.long, device=input_ids.device)` and this will lead to an error here: `next_tokens = next_tokens * unfinished_sequences + pad_token_id * (1 - unfinished_sequences)`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Docs: fix indentation in `HammingDiversityLogitsProcessor`
Author: gante
Pull request: yes

# What does this PR do?

Fixes indentation, which is causing the doc builder to crash. Also fixes the example, which was incorrect (thus causing the newly added doctests to fail)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

(took the liberty to merge to unblock others, but I'm open to post-merge PR reviews)
Correct attention mask dtype for Flax GPT2
Author: liutianlin0121
Pull request: yes

# What does this PR do?


Fixes #25634


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. Link: https://github.com/huggingface/transformers/issues/25634
- [N/A] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sanchit-gandhi 
@ArthurZucker

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25636). All of your documentation changes will be reflected on that endpoint.

@ArthurZucker Sure! I added a test :-)


>would be great to make the test a fast test by defining a tester function in the model tester, and then executing it in the model test

@sanchit-gandhi Good point! Done. Let me know if you have further suggestions. :-)

cc @sanchit-gandhi feel free to merge if it's alright with you!

@sanchit-gandhi Hey thanks! I change to assertTrue.

No problem! Feel free to merge it (it seems that I can't).
[idefics] small fixes
Author: stas00
Pull request: yes

- fix the example doc - where there was a wrong copy-n-paste model class used
- remove `IdeficsGatedCrossAttentionLayer` from init imports as this is not really needed

thank you, @ydshieh for noticing these

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25764). All of your documentation changes will be reflected on that endpoint.
No inf checks were recorded for this optimizer
Author: lanqianlong
Pull request: no

### System Info

transformers 4.30.0.dev0 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`trainer = transformers.Trainer(
    model = model,
    train_dataset=data,
    args =training_args,
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),   
)

trainer.train()
`

### Expected behavior

```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/tmp/ipykernel_1639/4032920361.py in <cell line: 1>()
----> 1 trainer.train()

~/.local/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1659             self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size
   1660         )
-> 1661         return inner_training_loop(
   1662             args=args,
   1663             resume_from_checkpoint=resume_from_checkpoint,

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   2013                         optimizer_was_run = scale_before <= scale_after
   2014                     else:
-> 2015                         self.optimizer.step()
   2016 
   2017                     if optimizer_was_run:

~/.local/lib/python3.8/site-packages/accelerate/optimizer.py in step(self, closure)
    132             elif self.scaler is not None:
    133                 scale_before = self.scaler.get_scale()
--> 134                 self.scaler.step(self.optimizer, closure)
    135                 self.scaler.update()
    136                 scale_after = self.scaler.get_scale()

~/.local/lib/python3.8/site-packages/torch/cuda/amp/grad_scaler.py in step(self, optimizer, *args, **kwargs)
    370             self.unscale_(optimizer)
    371 
--> 372         assert len(optimizer_state["found_inf_per_device"]) > 0, "No inf checks were recorded for this optimizer."
    373 
    374         retval = self._maybe_opt_step(optimizer, optimizer_state, *args, **kwargs)

AssertionError: No inf checks were recorded for this optimizer.


```

Comments:
Miss configuration LoraConfig caused the issue

@lanqianlong what was the misconfiguration?
Enable use of best epoch in Trial, with early stopping, during hyperparameter search
Author: antonioalegria
Pull request: no
Labels: name: trainer

name: Feature request

### Feature request

When running a `Trainer.hyperparameter_search`, each trial's value is calculated from the last epoch's chosen metric. However, especially when using early stopping and `load_best_model_at_end`, it would be useful to use the best model instead.

This could be a parameter of `Trainer.hyperparameter_search` or a an overridable function getting the best value, or some callback.

### Motivation

Often, we use early stopping and take the best model from a particular run because it's possible for models to start overfitting and dropping off after a certain number of epochs. This phenomenon can also appear during hyper parameter search and, as such, we'd like to be able to use the best epoch's value to compare trials.

Without this we may get results that are not fully representative.

### Your contribution

Happy to help testing or in other ways I can. Not sure where to start but if there is a clear place to do it I'd be open to help.

Comments:
cc @sgugger 

Yes this is not currently supported. Could be nice to add, but this is not high-priority on our side, so it would have to be a contribution :-) Happy to review a PR!

Hi, I was looking to contribute here so I dug into the code. @antonioalegria , what hyperparameter search backend are you searching? Based on my reading of the code, it seems this is already supported for the `RAY` backend. Passing in `ray_scope=all` to `TrainingArguments` should work. 
  * Code pointer on [huggingface](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/integration_utils.py#L360) end.
  * https://docs.ray.io/en/latest/tune/api/doc/ray.tune.ExperimentAnalysis.get_best_trial.html#ray-tune-experimentanalysis-get-best-trial

I can look to contribute this for the wandb and optuna backends. However, the sigopt backend seems rather opaque.
`clean_up_tokenization_spaces=False` not working for decoding single token
Author: devymex
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-5.15.0-73-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.2
- Accelerate version: 0.21.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
    - distributed_type: MULTI_GPU
    - mixed_precision: fp16
    - use_cpu: False
    - num_processes: 8
    - machine_rank: 0
    - num_machines: 1
    - gpu_ids: all
    - rdzv_backend: static
    - same_network: True
    - main_training_function: main
    - downcast_bf16: no
    - tpu_use_cluster: False
    - tpu_use_sudo: False
    - tpu_env: []
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: N

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```py
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b-hf', use_fast=False)
tokenizer.decode([29892,1235], skip_special_tokens=False, clean_up_tokenization_spaces=False)
tokenizer.decode([1235], skip_special_tokens=False, clean_up_tokenization_spaces=False)
```

Actual outputs:

```
, let
let
```

### Expected behavior

```
, let
 let
```

The token 1235 is ` let`, notice the second line of the output, there should be a space before the word 'let', but it not.

The option `clean_up_tokenization_spaces=False` not working when single token input.

Comments:
This is expected, `clean_up_tokenization_spaces` is not related to spaces that are outputed by the model but spaces around `special tokens`. The output is stripped because when you encode you also add a space. If you want a one to one decoding of a token use `tokenizer.convert_ids_to_tokens`
Amazon Bedrock model as HfAgent
Author: austinmw
Pull request: no

### Feature request

I'd like to be able to use Amazon Bedrock available models, for example Claude, as the HfAgent model.

### Motivation

Expanded model support

### Your contribution

Not sure currently.

Comments:
Hi @austinmw Thank you for this feature request.

I am not sure however, cc my colleague @sgugger who knows much better on the Agent topic!



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
VS Code Pylance does not highlight transformers imports
Author: mmlynarik
Pull request: no

### System Info

When Pylance is used as language server for VSCode, it does not highlight `transformers` imports even though library is correctly installed. Classes imports and library name itself are gray instead of yellow, see the enclosed figure. I'm not sure if it relates to Pylance itself, but it would be nice if this behavior was fixed.

![image](https://github.com/huggingface/transformers/assets/44208384/b39dbf5d-cf3a-4be3-bbf5-b32ddd7817dc)


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Create and activate new virtual environment
2. Install transformers 4.30.2
3. Write into a new python module: `from transformers import BatchEncoding`

### Expected behavior

The import statement should be highlighted the same way as other libraries imports.

Comments:
Sounds like an issue for VsCode or Pylance no?

This is probably the result of lazy module loading, or even the simple absence of a `__all__` (as far as I can see). This may not be something fixable within `transformers` but it does result from a design choice. https://github.com/huggingface/transformers/blob/main/src/transformers/__init__.py#L7139

The `__all__` is defined in the `_LazyModule` as you can see [here](https://github.com/huggingface/transformers/blob/6c57ce15587810968d64fb4b5700b63726397194/src/transformers/utils/import_utils.py#L1048).

> Sounds like an issue for VsCode or Pylance no?

Might be, but it’s surprising that tokenizers and datasets which are two most closely related libriaries from the HF ecosystem are functioning correctly. 

Pretty sure this is fixed now, per the above issue on the Pylance repo.

Fix repros on my side; if anyone else would care to confirm, we can probably close this issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.