noamaanMulla-03/transformers-issues huggingface dataset:
v4.30-release run_ner  gives datasets.builder.DatasetGenerationError
Author: santoshcoder23
Pull request: no

### System Info

 transformers==v4.30-release
pytorch latest
datasets latest
Ubuntu-20.0



### Who can help?

@sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

python run_ner.py \
  --model_name_or_path bert-base-uncased \
  --dataset_name conll2003 \
  --output_dir /tmp/test-ner \
  --do_train \
  --do_eval

Error: _raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.builder.DatasetGenerationError: An error occurred while generating the dataset_


### Expected behavior

Evaluation should run on pre-trained model on conll2003 dataset.

Comments:
That sounds like an issue for `datasets`, not Transformers :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
is_torch_bf16_gpu_available does not check for AMD GPUs
Author: cjekel
Pull request: no

### System Info

transformer 4.30.2
python 3.9.13.1
torch 2.0.1
rocm 5.4.2
AMD mi250x

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

AMD GPUs like the mi250 and mi250x support bf16. See https://www.amd.com/en/products/server-accelerators/instinct-mi250x 

`transformers.utils.import_utils.is_torch_bf16_gpu_available` returns `False` with mi250x. Additional inspection shows that the function does not check AMD gpus at all. The problem occurs because `torch.version.cuda=None` when using hip.

Steps to reproduce:
1. have AMD GPU that supports bf16
2. The problem arises when calling `transformers.TrainingArguments` and using something like `--bf16 True`.  You'll see something like this...
```python
File lib/python3.9/site-packages/transformers/training_args.py:1297, in TrainingArguments.__post_init__(self)
   1294             raise ValueError("Your setup doesn't support bf16/(cpu, tpu, neuroncore). You need torch>=1.10")
   1295         elif not self.no_cuda and torch.cuda.is_available() and not is_torch_bf16_gpu_available():
   1296             # gpu
-> 1297             raise ValueError(
   1298                 "Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0"
   1299             )
   1301 if self.fp16 and self.bf16:
   1302     raise ValueError("At most one of fp16 and bf16 can be True, but not both")

ValueError: Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0
```
3. It's a bit easier to call the function directly.
```python
import transformers
transformers.utils.import_utils.is_torch_bf16_gpu_available()
```
which returns `False` instead of `True`.

### Expected behavior

`transformers.utils.import_utils.is_torch_bf16_gpu_available()` should check more than just Cuda to check if bf16 is available.  

A quick work around is to add
```python
    if torch.version.hip is not None:
        return True
```
to src/transformers/utils/import_utils.py. However, I don't know which AMD GPUs actually support bf16. It looks like mi200 does as well.

Comments:
We don't officially support AMD GPUs yet. This is coming soon as we get AMD GPUs to run our CI and check everything runs smoothly, so stay tuned!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Cannot load BART model
Author: Onkar-2803
Pull request: no

Trying to load the BART model as specified on the [website](https://huggingface.co/docs/transformers/model_doc/bart#mask-filling:~:text=Mask%20Filling-,The%20facebook,-/bart%2Dbase%20and) with the following code:
`model = BartForConditionalGeneration.from_pretrained("facebook/bart-large", forced_bos_token_id=0`

Error: facebook/bart-large is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

@patrickvonplaten 

Comments:
It seems like Facebook just disappeared from HuggingFace. Still waiting for something back...

Yeah, no models & datasets visible.

https://huggingface.co/facebook/bart-large is back up now 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
TypeError: 'BertTokenizer' object is not callable
Author: axhiao
Pull request: no

```
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
sequence_a = "HuggingFace is based in NYC"
sequence_b = "Where is HuggingFace based?"
encoded_dict = tokenizer(sequence_a, sequence_b)
```
This will produce error 'BertTokenizer' object is not callable.  Maybe I should call `tokenizer.tokenize()`? But I think the [doc](https://huggingface.co/transformers/glossary.html#token-type-ids) should also be updated!

Comments:
Hello! It seems you're running on an older `transformers` version. The `__call__` was implemented only in version v3.0.0+. Do you mind pasting your environment information here?

You should have a dropdown in the documentation with the different versions, so that you can select the documentation that works for your version.

yes, you are right. I got the old version because I had the [mmf ](https://github.com/facebookresearch/mmf) framework depending on transformers==2.0.3. So when I run pip install transformers, it just did nothing due to the existence of the old version. Thank you!

Thank you a lot!

So we have to update the transformer to the latest version, and it would work? Is that the case
add pathname and line number to logging formatter in debug mode
Author: ranchlai
Pull request: yes

# What does this PR do?
This PR adds pathname and line number to logging formatter in debug mode
It make the debug much easier when setting `export TRANSFORMERS_VERBOSITY=debug`. 
It has no effect in other logging levels(info, warning, etc ). 

Hope this is acceptable. It's ok if not. 

before, there is no way to know where the logging comes from. 
```
loading file vocab.json
loading file merges.txt
loading file tokenizer.json
loading file added_tokens.json
loading file special_tokens_map.json
loading file tokenizer_config.json
```
after:
```
[INFO|/home/ranch/models/transformers/src/transformers/tokenization_utils_base.py:1842] 2023-07-31 17:59:45,311 >> loading file vocab.json
[INFO|/home/ranch/models/transformers/src/transformers/tokenization_utils_base.py:1842] 2023-07-31 17:59:45,311 >> loading file merges.txt
[INFO|/home/ranch/models/transformers/src/transformers/tokenization_utils_base.py:1842] 2023-07-31 17:59:45,311 >> loading file tokenizer.json
[INFO|/home/ranch/models/transformers/src/transformers/tokenization_utils_base.py:1842] 2023-07-31 17:59:45,311 >> loading file added_tokens.json
[INFO|/home/ranch/models/transformers/src/transformers/tokenization_utils_base.py:1842] 2023-07-31 17:59:45,311 >> loading file special_tokens_map.json
[INFO|/home/ranch/models/transformers/src/transformers/tokenization_utils_base.py:1842] 2023-07-31 17:59:45,311 >> loading file tokenizer_config.json
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._

hi, @amyeroberts thanks very much for commenting! I have added a "detail" level. It's the same as debug but will also print the pathname and line-number for easy debugging. I don't know if that looks good ?  Thanks!
Add `token` arugment in example scripts
Author: ydshieh
Pull request: yes

# What does this PR do?

If the change is good, I will apply the same to other files.

Let me know if you have opinion on the `False` vs `None` thing here.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger  Ready to go 🚀 

Thank you for the review, changed it to

```python
    token: str = field(
        default=None,
        metadata={
            "help": (
                "The token to use as HTTP bearer authorization for remote files. If not specified, will use the token "
                "generated when running `huggingface-cli login` (stored in `~/.huggingface`)."
            )
        },
    )
    use_auth_token: bool = field(
        default=None,
        metadata={
            "help": "The `use_auth_token` argument is deprecated and will be removed in v4.34. Please use `token`."
        },
    )
```
RuntimeError: Cross Attention in GPTBigCodeModel
Author: dawnik17
Pull request: no

### System Info

Code: https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/models/gpt_bigcode/modeling_gpt_bigcode.py

In the case of cross attention, if `self.c_attn` is initialised to give output of dimension `2 * self.embed_dim` **(Line 112 & 227)**

The `key_value` split in **line 246**,

`key, value = key_value.split((self.head_dim, self.head_dim), dim=-1)`

* would raise an exception _(when self.embeded_dim != self.head_dim)_

`RuntimeError: split_with_sizes expects split_sizes to sum exactly to 2*self.embeded_dim`

_PS - I could be mistaken, but it would be worth having a look (and correct me if I am wrong!)._

### Who can help?

 @ArthurZucker @younesbelkada @sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
inp = torch.rand((2, 4, 16))
g = GPTBigCodeAttention(config, is_cross_attention=True)
g(inp, encoder_hidden_states=inp)
```

`RuntimeError: split_with_sizes expects split_sizes to sum exactly to 32 (input tensor's size at dimension -1), but got split_sizes=[4, 4]`

### Expected behavior

The forward method should return the attention output of the shape [2, 4, 16].

Comments:
Hey! Would you mind sharing a full reproduced? (ex the config you are using)? 
This might just be a raise Value error missing and a check on the shapes! 

Sure @ArthurZucker 

Reproducible Code:
```
import torch
from transformers import GPTBigCodeConfig

config = GPTBigCodeConfig(multi_query=False, n_embd=16, n_head=2)
attn = GPTBigCodeAttention(config, is_cross_attention=True)

inp = torch.rand((2, 4, 16))
attn(inp, encoder_hidden_states=inp)
```
Thanks in advance :)

Bump :) 
@ArthurZucker

Okay! Thanks for bumping. 
Few things here: 
- `GPTBigCodeAttention` is not in the `__init__` so not on the surface of transformers: we don't really support using it outside in such a way. Having a working snipper which uses a `GPTBigCodeModel` is better. 
- You are not properly setting the arguments that you want to set: `attn.embed_dim` will show `768`. To change the number of heads and the `attn.embed_dim`, use `config = GPTBigCodeConfig(multi_query=False,hidden_size=16, num_attention_heads=2)`
- I don't recommend trying to use it this way, as it is not intended. I tried a few things to see if a quick fix was possible, but it seems that during integration all edge cases were not tested. Especially `add_cross_attention` which is not part of the configuration. 


Got it. 
Thanks @ArthurZucker!
Update rescale tests - cast to float after rescaling to reflect #25229
Author: amyeroberts
Pull request: yes

# What does this PR do?

In #25229 - the casting to float was moved back to after rescaling. This wasn't reflected in the specific rescaling tests for EfficientNet and ViVit, resulting in failing tests. 

This PR resolves this. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[DOCS] Add example and modified docs of EtaLogitsWarper
Author: agno-nymous
Pull request: yes

# What does this PR do?

See #24783
Added example to EtaLogitsWarper and also modifed its docstrings to make it more understandable


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante 

Comments:
@ashishthomaschempolil thank you for iterating 🤗 

_The documentation is not available anymore as the PR was closed or merged._
Remove `pytest_options={"rA": None}` in CI
Author: ydshieh
Pull request: yes

# What does this PR do?

This option causes the (TF/Flax) jobs to spend 6-8 minutes (for a full set run) to prepare something for reporting after the actual tests are finished.

Taking [this TF job (nightly run)](https://app.circleci.com/pipelines/github/huggingface/transformers/69562/workflows/8fd9db08-9730-4d57-90b5-660c8a48a55c/jobs/872686/steps) for example, we can see the situation in the following screenshot

<img width="1044" alt="Screenshot 2023-08-02 132209" src="https://github.com/huggingface/transformers/assets/2521628/67e6bc89-d0d3-4d6a-9090-f3e1042be639">


Note that the torch job doesn't have this option, as it is removed ~ 3 years ago by Stas in #7995. Also, we still have all the reports we need in the artifact tab. (I don't remember the details about `-rA` though - Stas is the expert of this)

Comments:
_The documentation is not available anymore as the PR was closed or merged._


> For reference, I think `-rA` generates a [detailed summary report for all groups](https://docs.pytest.org/en/6.2.x/usage.html#detailed-summary-report).

Oh yes, my memory mixed the `--make-reports` and `-rA` things. Thanks!


> As it was removed for the torch job a long time ago, I'm happy for it to be removed here :)

If you were not happy, we will have to spend more🤑 on CircleCI credits 💸 😆 (and for nothing)
🌐 [i18n-KO] Translated `perf_infer_gpu_many.md` to Korean
Author: heuristicwave
Pull request: yes

# What does this PR do?

Translated the `perf_infer_gpu_many.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [X] Check for missing / redundant translations (번역 누락/중복 검사)
- [X] Grammar Check (맞춤법 검사)
- [X] Review or Add new terms to glossary (용어 확인 및 추가)
- [X] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [X] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

May you please review this PR? @nuatmochoi, @bolizabeth, @hyunhp, @mjk0618, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

May you please review this PR? @sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._
'NoneType' object has no attribute 'flush'
Author: 17Reset
Pull request: no

When I use PyInstaller to package transformers program, if I choose windowless mode, I get the following error, but I don't want console mode, I want to build the program based on windowless mode.
`  File "transformers\utils\import_utils.py", line 37, in <module>
    logger = logging.get_logger(__name__)  # pylint: disable=invalid-name
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\utils\logging.py", line 124, in get_logger
    _configure_library_root_logger()
  File "transformers\utils\logging.py", line 88, in _configure_library_root_logger
    _default_handler.flush = sys.stderr.flush
                             ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'flush'`

Comments:
cc @LysandreJik 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
transformers.LlamaTokenizer.from_pretrained does not work after transformers are packaged with Pyinstaller
Author: 17Reset
Pull request: no

When I use the following code in my python project, the model will load and work properly, but when I package my project using Pyinstaller, I found by debugging that this code to load the model, does not work properly: tokenizer = transformers.LlamaTokenizer.from_ pretrained(LLM_CHECKPOINT). This complete function is shown below, where LLM_CHECKPOINT = 'D:/LLM/alpaca-llama-7b-fp16' and device = torch.device('cpu'):
import torch
import transformers

def llm_initial(LLM_CHECKPOINT, device):
    tokenizer = transformers.LlamaTokenizer.from_pretrained(LLM_CHECKPOINT)
    model = transformers.LlamaForCausalLM.from_pretrained(LLM_CHECKPOINT).to(device)
    model.eval()

    generation_config = transformers.GenerationConfig(
        # max_new_tokens: This is the maximum number of tokens to generate.
        # The generated sequence will not exceed this length.
        max_new_tokens=256,
        # temperature: This is a parameter for controlling the randomness of predictions
        # by scaling the logits before applying softmax. Higher values (greater than 1)
        # increase randomness, while lower values make the model more deterministic.
        temperature=1,
        # top_k: This parameter controls the number of highest probability predictions
        # to consider for the next token. It's used to introduce some randomness and
        # creativity into the model's outputs.
        top_k=40,
        # top_p: This parameter is also known as nucleus sampling and is used to ensure
        # that the cumulative probability of the considered tokens is at least top_p.
        # This parameter also introduces randomness into the model's outputs.
        top_p=0.9,
        # repetition_penalty: This parameter is used to control for repetitive behavior
        # in the model's outputs. If the value is greater than 1.0, the model is
        # penalized for generating the same token repeatedly.
        repetition_penalty=1.15
    )
    return [device, tokenizer, model, generation_config]


def llm_response(device, tokenizer, model, generation_config, prompt):
    prompt = generate_prompt(prompt)
    input_ids = tokenizer.encode(prompt, return_tensors='pt').to(device)

    with torch.enable_grad():
        output_ids = model.generate(input_ids=input_ids, generation_config=generation_config)

    LLM_RESPONSE = tokenizer.decode(output_ids[0], skip_special_tokens=True)
    return LLM_RESPONSE.replace(prompt, '').strip()

Comments:
The Pypi package does not have access to your local drive where `LLM_CHECKPOINT` points to, that's why you get different behavior.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: deprecation timeline for parameterization though the model config
Author: gante
Pull request: yes

# What does this PR do?

We had a warning about deprecating the parameterization of `.generate()` through the model config, but there was no specific date. This PR makes it clear it will go away in `v4.32`.

The extra parameters and code to allow the old and the new way of parameterizing `.generate()` to live together are causing a myriad of issues, so let's get rid of them :)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24457). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.