noamaanMulla-03/transformers-issues huggingface dataset:
run run_language_modeling got bug
Author: ucas010
Pull request: no

### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-3.10.0-1160.81.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@vanpelt @pvl @arfon @xeb @kashif @sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

link:https://github.com/huggingface/transformers/blob/main/examples/legacy/run_language_modeling.py
```
export TRAIN_FILE=/path/to/dataset/wiki.train.raw
export TEST_FILE=/path/to/dataset/wiki.test.raw

python run_language_modeling.py \
    --output_dir=output \
    --model_type=gpt2 \
    --model_name_or_path=gpt2 \
    --do_train \
    --train_data_file=$TRAIN_FILE \
    --do_eval \
    --eval_data_file=$TEST_FILE
```
error:
Traceback (most recent call last):
  File "/data/transformers/examples/legacy/run_language_modeling.py", line 375, in <module>
    main()
  File "/data/transformers/examples/legacy/run_language_modeling.py", line 291, in main
    data_args.block_size = tokenizer.max_len
AttributeError: 'GPT2TokenizerFast' object has no attribute 'max_len'

### Expected behavior

looking forward to kind reply
and solve the problem

Comments:
This is an unmaintained example that won't work with the last version of transformers.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Rework a bit the LLaMA conversion script
Author: sgugger
Pull request: yes

# What does this PR do?

This PR makes sure the LLaMA conversion script stays up to date with `save_pretrained` by having the checkpoint being loaded in an actual model then saved via that method. This avoids a lot of hard-coded values in JSON files.

It keeps the old logic and merely re-loads the result in a Transformer model (after cleaning anything to make sure we never go above the model size in CPU RAM). It also changes a bit the API to put everything in the output folder like we usually have in repos on huggingface.

cc @zphang so you are aware of this.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I don't see how you can reduce the memory requirement since the files provided by Meta each contain a part of all weights, so you need to have them all loaded to reconstruct just one of the weights. That's why I didn't bother implementing sharding on the fly.

Indeed, just realised you have to `cat` them 😞 my bad!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22236). All of your documentation changes will be reflected on that endpoint.
Improve error message
Author: Mahrkeenerh
Pull request: yes

# What does this PR do?

Add specific numbers to error message.

@amyeroberts 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@Mahrkeenerh for the `check_repository_consistency` tests, running `make fix-copies` in the top level of the repo should make all the necessary code updates and resolve these. It might be necessary to run `make style` as well to fix any resulting formatting issues. 

@amyeroberts ran the commands, and it seems like `make style` changed 2 unrelated files as well (combined multiline definition into single line), do I add them, or ignore those changes?

@Mahrkeenerh OK, thanks for the update. Two things to try: 
* Rebase from main to include most recent changes
* Make sure the most recent style settings and libraries are in the environment `pip install -e .[quality]`
* Run `make style` again

If they're still being added, push them and I'll re-review to double check the diff's OK.



@Mahrkeenerh All looks good to me. Thanks again for this addition! 

@amyeroberts all green :green_circle:
TensorFlow: pin maximum version to 2.12
Author: gante
Pull request: yes

# What does this PR do?

TF Text 2.12 has been released (a few hours after TF 2.12), so that problem got sorted by itself. 

Adding `cmake` to install onnx from source gets rid of the remaining problems :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Error while using CLIP embeddings with VisualBERT.
Author: nityanandmathur
Pull request: no

### System Info

- `transformers` version: 4.26.0
- Platform: Linux-4.18.0-348.2.1.el8_5.x86_64-x86_64-with-glibc2.28
- Python version: 3.10.8
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.12.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am trying to use CLIP embeddings with VisualBERT for multimodal image classification.

1. Generating CLIP embeddings for text(j[1]) and images(j[0]) for a batch in dataloader.
2. Providing these embeddings to the VisualBERT model.
3. Calculating cross entropy loss.
```python
model.train()

for epoch in range(EPOCH):
    for j in tqdm(trainloader):
        # Features
        text_tokens = clip.tokenize(j[1]).to(DEVICE)
        j[0] = j[0].to(DEVICE)
        with torch.no_grad():
            text_features = clip_model.encode_text(text_tokens).to(DEVICE)
            image_features = clip_model.encode_image(j[0]).to(DEVICE)
        
        print(text_features.shape)
        print(image_features.shape)
        visualbert_inputs = {
            "inputs_embeds": text_features.to(DEVICE),
            "visual_embeds": image_features.to(DEVICE),
        }

        # Forward Pass
        output = model(**visualbert_inputs)
        loss = loss_fn(output,j[2]).to(DEVICE)

        #Backpropagation
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        print(f"EPOCH:{epoch}, LOSS:{loss.item()}")
```

Error:
![image](https://user-images.githubusercontent.com/77379835/227358978-275e1afc-2372-4094-80c4-92188fe8e925.png)



### Expected behavior

The VisualBERT model requires input embeddings of dimension `inputs_embeds (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)`. 
How to convert the CLIP encodings to the input embeddings of VisualBERT?

Comments:
cc @ArthurZucker and @amyeroberts
Enable bf16 option for XLA devices
Author: jeffhataws
Pull request: yes

# What does this PR do?

XLA devices like TPU and NeuronCore supports BF16 natively. This PR enables --bf16 option to work for XLA devices.

Since BF16 doesn't require gradient scaling, gradient scaling path is disabled for XLA devices.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> I don't see how the gradient scaling path is disabled, could you share more information on that? A scaler is still defined at line 591.

Line 568 disables the code section that enables grad scaler when XLA device is detected (is_torch_tpu_available()).

This PR accidentally disabled gradient scaling when using FP16 on XLA devices. 

Indeed. Do you want to make a PR with a fix @Lokiiiiii ?
added biogpt token classification
Author: upjabir
Pull request: yes

# What does this PR do?

It add the class for BioGptForTokenClassification based on BioGpt model

Fixes #21786


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @younesbelkada @NielsRogge @sgugger



Comments:
_The documentation is not available anymore as the PR was closed or merged._

@NielsRogge @sgugger can you please look into it ?
Is biogpt's tokenizer bugged?
Author: fedshyvana
Pull request: no

### System Info

- `transformers` version: 4.27.1
- Platform: Linux-5.15.0-67-generic-x86_64-with-glibc2.17
- Python version: 3.8.13
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1 (True)

### Who can help?

@ArthurZucker and @younesbelkada could you please confirm this behavior is intended? Sorry if I mistagged. Thanks!

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer
tokenizer_name = "microsoft/BioGPT-Large"
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)
print('bos token: ', tokenizer.bos_token, 'id: ', tokenizer.bos_token_id)
print('eos token: ', tokenizer.eos_token, 'id: ', tokenizer.eos_token_id)
print('token ids: ', tokenizer("this is a test")['input_ids'])
print('tokens: ', tokenizer.decode(tokenizer("this is a test")['input_ids']))
```

Output:
```
bos token:  
 id:  0
eos token:  
 id:  2
token ids:  [2, 54, 34, 21, 229]
tokens:  
this is a test
```

### Expected behavior

I would expect the tokenizer to prepend the BOS token (i.e. 0) and append the EOS token (i.e. 2) while currently the tokenizer prepends the EOS token, and does not add a special token to the end of the sequence of tokens. 

Comments:
@fedshyvana I believe this is how biogpt is trained on fairseq . For more information , you check into official repo of BioGpt.

@upjabir thanks for pointing it out! I am looking at  https://github.com/microsoft/BioGPT/blob/main/src/language_model_prompt_dataset.py which I believe is the code you're referring to. If I understand correctly, they use:
[EOS] token_1, ..., token_n as input
and 
token_1, ..., token_n [EOS] as target

i.e. it seems like they just don't use a separate BOS token at all. 
But in the HF BioGPT model config it says: 
  "bos_token_id": 0
  "eos_token_id": 2
  
 Should we change it to:
  "bos_token_id": 2
  "eos_token_id": 2
  
  Or would it not make any difference at all? Thank you!

@fedshyvana bos_token_id , eos_token_id is added to vocabulary as we always do for every tokenizer.But during building inputs with special tokens, we are only considering eos_token_id . Although we are not using bos_token during handling special token , i believe it will helpful in some rare case
[safetensors] don't use in `torch<1.10`
Author: stas00
Pull request: yes

`safetensors` only seems to work with pt>=1.10. This PR fixes this breakage:

```
python -c 'import sys; from transformers import AutoModel; AutoModel.from_pretrained(sys.argv[1])'  "bigscience/bigscience-small-testing"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/auto/auto_factory.py", line 471, in from_pretrained
    return model_class.from_pretrained(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/modeling_utils.py", line 2424, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/modeling_utils.py", line 413, in load_state_dict
    return safe_load_file(checkpoint_file)
  File "/home/stas/anaconda3/envs/py38-pt19/lib/python3.8/site-packages/safetensors/torch.py", line 101, in load_file
    result[k] = f.get_tensor(k)
AttributeError: module 'torch' has no attribute 'frombuffer'
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I need a PR merged to test my fix for the torch and tf run which is broken on main (but the fix itself does not trigger any tests). This one seems a good candidate so merging now and will keep an eye on the test :-) 

Shouldn't those be fixed in safetensors instead? ie here https://github.com/huggingface/safetensors/blob/5c1d366813e46c6f9f2c71aa8b89e0c916a92b2f/bindings/python/setup.py#L23 ?

Can be both :)

Also, I left torch version out on purpose at the time, as I wasn't sure about the support policy and wether or not supporting older version was worth the effort (since they require a lot more handling from safetensors itself).
Resnet flax
Author: Shubhamai
Pull request: yes

# Flax Implementation of `microsoft/resnet-50`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@sanchit-gandhi  I guess :sweat_smile: 

## Status

Last Updated - Sunday, 12 February 2023

### TODO
 - [x] Blocked on merge of [this PR](https://github.com/huggingface/transformers/pull/21581) to add support for BatchNorm layers. 
 - [ ] Uploading [Shubhamai/resnet-50](https://huggingface.co/Shubhamai/resnet-50) flax weights to [microsoft/resnet-50](https://huggingface.co/microsoft/resnet-50).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Very cool! Sorry to only reply here now - looks like you've made a really solid start to this PR! Let's get the Batch Norm PR merged ASAP and then go full send on Flax ResNet! 🚀

Feel free to ping me with any questions / queries! More than happy to help with the integration!

@sanchit-gandhi the PR is also now ready for your review, thanks a lot for your time. 

Thanks for the review, I will make the changes soon. Many of the reviews here also apply to the [Flax Convnext PR](https://github.com/huggingface/transformers/pull/21485) so will make corresponding changes there too. 

Hey @Shubhamai! Really nice work on this PR - just a few small changes to go now! Feel free to ping me once you're happy with the last bits and I'll get you a final review. Taking a look at Flax RegNet in the meantime!

Made the request changes and apologies for the late reply, had a busy schedule lately.

Thanks for the review @amyeroberts 🙌 Feel free to propagate the changes forward to [Flax RegNet](https://github.com/huggingface/transformers/pull/21867) @Shubhamai  and we can get a final review there!

We can merge when the CI is green! 🟢

Thank you so much for the review & merge @sanchit-gandhi @amyeroberts , really appreciate your work & time on this ❤️
Changed world_size() to get_world_size() bugfix
Author: Charlie-Bell
Pull request: yes

Edited one line in src/transormers/generation/utils.py. Changed dist.….world_size() to dist.get_world_size() since world_size() doesn't exist in pytorch.dist.

# What does this PR do?

Fixes # Pytorch 2 generation/utils.py , 'torch.distributed' has no attribute 'world_size' #22375
https://github.com/huggingface/transformers/issues/22375


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Library:

- generate: @gante



Comments:
_The documentation is not available anymore as the PR was closed or merged._

BTW, there is a CI error due to this branch being created from an older version of `main` -- you should rebase with `main` to make our CI green 

> BTW, there is a CI error due to this branch being created from an older version of `main` -- you should rebase with `main` to make our CI green

Funny, GitHub is telling me that "This branch is 1 commit ahead of huggingface:main." and also tells me the fork is already synced when I try syncing.
Also when I fetch upstream and rebase as in the contribution guidlines I am told "Current branch changed-world-size-to-get-world-size-in-generation-utils is up to date."

Maybe I missed something, but it seems the only difference in codebase is the 1 line change. Maybe it's worth it to re-run the ci/circleci: tests_torch_and_tf?

@Charlie-Bell my apologies, there is indeed a problem in `main` I've found after writing the comment above! #22383 will fix it -- apologies for the confusion 🙏
Translated documentation in italian
Author: nickprock
Pull request: yes

## What does this PR do?

Italian translation of doc related to the preprocessing of :hugs: Transformers.
* updated _toctree.yml
* added perf_infer_tpu.mdx
* added perf_infer_special.mdx
* added perf_train_tpu.mdx
* added perf_train_special.mdx

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

See issue: [[#17459](https://www.linkedin.com/feed/hashtag/?keywords=%2317459)](https://github.com/huggingface/transformers/issues/17459)


@sgugger, @stevhliu, @MKhalusova and @omarespejel 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Adapt find_tied_parameters to handle breaking change in Accelerate
Author: sgugger
Pull request: yes

# What does this PR do?

In the upcoming version of Accelerate, `find_tied_parameters` returns a list of list instead of dictionary. While there is a hack in place to make sure the code in Transformers keeps working, it is a hack so it would be best to change the way we handle the result of `find_tied_parameters`.

This PR does just that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Returning n-best hypotheses from Wav2Vec2ProcessorWithLM decoder
Author: vsokolovskii
Pull request: no

### Feature request

Currently, the [Wav2Vec2ProcessorWithLM](https://github.com/huggingface/transformers/blob/ae54e3c3b18bac0832ad62ea9b896dfd52a09850/src/transformers/models/wav2vec2_with_lm/processing_wav2vec2_with_lm.py#L67) decode function returns [only the best hypothesis](https://github.com/huggingface/transformers/blob/ae54e3c3b18bac0832ad62ea9b896dfd52a09850/src/transformers/models/wav2vec2_with_lm/processing_wav2vec2_with_lm.py#L572). Shall we extend its functionality and make it return n-best hypotheses, logit_scores, lm_scores, word_offsets so that people could rescore these hypotheses with a larger LM. 

For example, take a look at [NeMo article](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/asr_language_modeling.html#neural-rescoring) regarding the rescoring of n-best hypotheses.

### Motivation

 I suppose many people use n-gram models during the shallow fusion stage, the n-grams models are a good fit during the beam search because they are fast. People perform the rescoring of the n-best hypotheses with a larger LM (using them during the decoding is too slow so it makes sense to apply them during the rescoring of n-best hypotheses that come out of the ASR system). They fuse the score which comes out of the ASR with the perplexity-like score from the LM. If this external model is trained on the domain data it will drastically improve the WER of the resulting model.

### Your contribution

If it sounds like a good feature to you, that can be potentially adopted let me know and I'll prepare the PR 😃 

Comments:
cc @sanchit-gandhi 

> 

please, tell me your opinion on this feature :)