noamaanMulla-03/transformers-issues huggingface dataset:
huggingface transformer running on CPU behind celery/redis doens't work (but works by itself)
Author: HodorTheCoder
Pull request: no

Hello,

I am actually creating this for posterity because it took me a day to figure it out and if anybody else has this issue, hopefully this helps.

I am running a Bert2Bert EncoderDecoderModel inside a docker container, running behind celery that is getting jobs through redis.  This is in a production test environment on a machine w/o a GPU, so yes, it's slow, but it's not a deal breaker.

Anyways-- testing and everything works great when it's by itself.  However, when I put it behind celery within a task, it would load the model and then get to generate some text and just hang. I couldn't figure out what the problem was until I found this thread:

https://github.com/celery/celery/issues/4113

The issue is how the CPU version of the model does forking-- the default celery configuration breaks unless you add the following to your celery config when running celery:

--pool=solo

Setting this fixes the concurrency issues with forking and everything works.  So, it's a configuration issue.  

Go forth and prosper.

Comments:
Anwer is included in the main post.  Thanks.

Another way that "fixed" the problem for me was setting the number of torch threads to 1: torch.set_num_threads(1) BEFORE loading the model in the worker.

Thanks, man, it took me a day to find your comment as well. 😆
remove `return_dict_in_generate` condition on storing scores.
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes an issue in `generate` where the `output_scores` (or `output_attentions` or `output_hidden_states` ) cannot be obtained unless `return_dict_in_generate`  is set to `True`. This is problematic because it's not what we want when we have a flag for each of these outputs. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19373). All of your documentation changes will be reflected on that endpoint.
Making `ConvBert Tokenizer` independent from `bert Tokenizer`
Author: IMvision12
Pull request: yes

# What does this PR do?

Fixes #19303

Added `BertTokenizer` class in tokenization_convbert.py and `BertTokenizerFast` in tokenization_convbert_fast.py


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Done! @sgugger  do i need to do same for convbert_fast?

@sgugger Thanks for quick feeback,
For tokenization_convbert.py i have change the comment to `# Copied from transformers.models.bert.tokenization_bert.BertTokenizer with ConvBertTokenizer->BertTokenizer`

and for tokenization_convbert_fast.py i have change the comment to `# Copied from transformers.models.bert.tokenization_bert_fast.BertTokenizerFast with ConvBertTokenizerFast->ConvBertTokenizer
`

for convbert_fast `make repo-consistency` gives error : - src/transformers\models\convbert\tokenization_convbert_fast.py: copy does not match models.bert.tokenization_bert_fast.BertTokenizerFast at line 55

You'll need to add broader patterns than just the full name of the tokenizer as BERT is used in the docstrings for instance. To see what the copy utils wants to modify, you can run `make fix-copies` locally :-)

After running `make fix-copies` it changes `slow_tokenizer_class = BertTokenizer` but it should be `ConvBertTokenizer` in tokenization_convbert_fast.py

Yes, that's why I made the suggestions above.

So, do i need to copy `BertTokenizer` and `BertTokenizerFast` class to tokenization_convbert_fast.py as after adding those classes it passes all tests

Just accept the suggestions above.

Done @sgugger are there any more changes?
Fix gather for metrics
Author: muellerzr
Pull request: yes
Labels: name: Examples

name: bug

# What does this PR do?

This PR fixes a failing test in the CI due to `gather_for_metrics` not receiving a tuple. This will soon be redundant with a change in Accelerate, but good to have this fix in now anyways with the right version imo

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix gather for metrics in summarization example
Author: muellerzr
Pull request: yes
Labels: name: Examples

name: bug

# What does this PR do?

Fixes the failing test on the summarization no_trainer for now, eventually this API will be doable but it's not there yet :) 

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Removes `ProphetNet` config dependency from `XLM-ProphetNet` config
Author: srhrshr
Pull request: yes

@sgugger , 

Per the issue #19303, the `ProphetNet` config dependency is removed from `XLMProphetNetConfig` and it now directly inherits from `PretrainedConfig`.  


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
removed dependency on bart tokenizer(slow/fast) LED
Author: Infrared1029
Pull request: yes

# What does this PR do?

removes the dependency between LED tokenizer and Bart's (slow version)

Fixes # (issue)

follows hugging's face philosophy of Do Repeat Yourself


## Who can review?
@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot for the feedback! ill fix everything and open a brand new one.
Added type hints for TF: TransfoXL
Author: thliang01
Pull request: yes

Based on Issue #16059 
I have added type hints for Tensorflow TransfoXL Model.



Comments:
_The documentation is not available anymore as the PR was closed or merged._

I removed these Optional types as your suggested.
Holp you may check it and then merge my request.
@Rocketknight1 

Looks good to me, thank you!
Exporting TensorFlow models to ONNX exports with a static batch size of 2 and sequence length of 8
Author: dwyatte
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.0
- Platform: Darwin-21.6.0-x86_64-i386-64bit
- Python version: 3.7.12
- Huggingface_hub version: 0.10.0
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.8.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NA
- Using distributed or parallel set-up in script?: NA


### Who can help?

@Rocketknight1

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

With `tensorflow` and `onnx` installed, run 
`python -m transformers.onnx --model=bert-base-uncased --feature=sequence-classification onnx/`

along with a simple script to test the exported model

```py
import numpy as np
from transformers import AutoTokenizer
from onnxruntime import InferenceSession


tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
session = InferenceSession("./onnx/model.onnx")

inputs = tokenizer(["A string", "A longer string", "An even longer string"], return_tensors="np", padding=True)
inputs = {k: v.astype(np.int32) for k, v in inputs.items()}

print(session.run(output_names=["logits"], input_feed=dict(inputs)))
```

This raises the error:
```
InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: token_type_ids for the following indices
 index: 0 Got: 3 Expected: 2
 index: 1 Got: 6 Expected: 8
 Please fix either the inputs or the model.
```

because we use the TensorSpec from the dummy inputs here https://github.com/huggingface/transformers/blob/0fc68a7e14b1e6450829e7be76f74abbc84f051e/src/transformers/onnx/convert.py#L265

### Expected behavior

This issue is related to https://github.com/huggingface/transformers/issues/16885 which was a feature request for setting the batch size and sequence size, but I'm not sure we want this to be configurable. Arguably, it should be at parity with the PyTorch -> onnx capability which configures dynamic batch size and and sequence length set by the model. Therefore I'm marking this as a bug instead of a feature request

Expected behavior is that the exported onnx model runs with any batch size and up to the model's max sequence length

Comments:
@dwyatte Can you please look into #19231? Somehow the sequence length is capped to 5 tokens in the generated tflite. tflite files are used to run models on mobile devices. 

Since your issue also has a fixed sequence length, the cause may be related. I tried converting BERT to tflite and the input size was 1x5, as with other models.

Please look into this if you can. Thanks for your time! :)
PyTorch >= 1.7.0 and TensorFlow >= 2.4.0
Author: sgugger
Pull request: yes

# What does this PR do?

As discussed in #18817 we have decided that Transformers will support versions of PyTorch and TensorFlow for two years after their releases. As a result, Transformers can now assume a minimum version of 1.7.0 for PyTorch and 2.4.0 for TensorFlow. This PR enforces this in the setup and then simplifies a lot of code that was written to support older PyTorch versions.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger curious what the reasoning was for blacklisting torch 1.12.0? I looked in  https://github.com/huggingface/transformers/issues/18817, but there's no info there for 1.12

That particular release broke weightnorm, and thus all audio models.
SSLError
Author: LaddieTJC
Pull request: no
Labels: name: bug



Version:
Python 3.8
transformer 4.21.0

I am trying to use DistilBertTokenizer using the following line:
`tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased", do_lower_case=True)`

but SSLerror arise from the line above
`requests.exceptions.SSLError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /distilbert-base-uncased/resolve/main/vocab.txt (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))`


Does anyone manage to solve this? I tried some methods online but still got the same error.

Comments:
It seems there is an issue with your SSL module and `requests`, not with `transformers`; I would head to Stack Overflow or another forum focused on these issues, we're unfortunately unlikely to be able to help you out here.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Support mixed precision FP16 in TF Segformer | Nan loss
Author: joihn
Pull request: no
Labels: name: bug

### Description

I'm trying to bring mixed precision training (FP16) support to [TF Segformer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/segformer/modeling_tf_segformer.py) .
I had to do a very small modification to the inital code for supporting Fp16 ([my commit ](https://github.com/huggingface/transformers/commit/41d2bd145111a743a577f15955bac58a74871b33)).

Everytime, the net converge fine for the first 10-15 epoch, and then the loss suddenly goes to Nan.
Any idea ? 

I'm using 
`policy = mixed_precision.Policy("mixed_float16")` ([TF doc here ](https://www.tensorflow.org/guide/mixed_precision))


### System
Tensorflow version 2.8.2
Cuda 11.6
Nvidia titan X

### Who can help?

@sayakpaul @NielsRogge 


### Reproduction

1) Modify 1 line of code as in ([my commit ](https://github.com/huggingface/transformers/commit/41d2bd145111a743a577f15955bac58a74871b33)).
2) Launch a training with mixed precision FP16
3) wait 10~15 epoch

### Expected behavior

The net shoudl train fine with FP16 mixed precision

Comments:
Thanks! Are you casting the softmaxed outputs or the logits to FP32 before loss computation as recommended [here](https://www.tensorflow.org/guide/mixed_precision#building_the_model)? 

Nit: You didn't provide any training code. I guess it's always better to provide a self-contained Colab Notebook or a notebook that anyone can spin up to debug the issue further. I also acknowledge that Colab doesn't always provide a GPU that has support for Tensor cores but I hope you got the point. 

thanks for the quick answer, 
Even though my commit above doesn't include it, yes I tried casting the logits output to FP32, ans sadly it didn't solve it :/ 

Good point about the small code example for debugging, I will do one today :) 

Then we need to inspect the layer-wise activations and the weight distributions that are probably impacted by the casting. Maybe try using the TensorBoard callback and inspect if that's the case. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Make attention_mask axes dynamic when exporting onnx
Author: unbuilt
Pull request: yes

# What does this PR do?

Make `attention_mask` axes dynamic when exporting onnx


## Who can review?
[@fatcat-z](https://github.com/fatcat-z)


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17711). All of your documentation changes will be reflected on that endpoint.

cc @michaelbenayoun @lewtun 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @unbuilt just checking if you were able to test that the script runs correctly with your change and the default settings? If yes, this looks good to merge IMO :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Remove dependency of Bert from Squeezebert tokenizer
Author: rchan26
Pull request: yes

Hi @sgugger,

Fixes #19303, the BertTokenizer dependency has been removed from `SqueezeBertTokenizer` and the BertTokenizerFast dependency has been removed from `SqueezeBertTokenizerFast`.

I ran `pytest tests/models/squeezebert/test_tokenization_squeezebert.py`, which passed.

Thanks for reviewing this! :)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I see that my code currently fails the style and code consistency checks. When I run `make style`, it seems to change a lot of files and ones that I did not touch. Is this normal?

I'm also trying to run `make repo-consistency` but keep getting
```
python utils/check_copies.py --fix_and_overwrite
make: python: No such file or directory
make: *** [fix-copies] Error 1
```
which is strange as I am running this from the root directory...

Hi @sgugger, many thanks for the quick replies! I have made the changes you mentioned above, and regarding `make repo-consistency` and `make-style`, it seems like `pip install -e ."[quality]"` did the trick!
update attention mask handling
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes error when using whisper with inference API.

Working script : 
```python 
>>> from transformers import WhisperForConditionalGeneration, WhisperProcessor, AutomaticSpeechRecognitionPipeline, , load_dataset, AutoModel
>>> processor = WhisperProcessor.from_pretrained("openai/whisper-large")
>>> model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large")

>>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")

>>> model.config.forced_decoder_ids = processor.get_decoder_prompt_ids(task="transcribe", language = "en")
>>> model.config.max_length = 224
>>> pipeline = AutomaticSpeechRecognitionPipeline(
    model = model, 
    tokenizer = processor.tokenizer, 
    feature_extractor = processor.feature_extractor)
>>> print(pipeline(ds[0]["audio"]["array"]))
{'text': ' Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.'}
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Using the `pipeline` wrapper  also works : `pipe = pipeline("automatic-speech-recognition", model="openai/whisper-medium.en", device=0)`.
Remove Dependency between Bart and LED (slow/fast)
Author: Infrared1029
Pull request: yes

# What does this PR do?

Removes the dependency between LED and Bart

## Who can review?
@sgugger 

Comments:
@sgugger hopefully this does it?

_The documentation is not available anymore as the PR was closed or merged._

Just tried locally your PR and found the reason why we can't have a global copied from on the tokenizer (sorry I didn't spot it earlier): the `_pad` method is overwritten.
So instead we need to applied the copied from on each method (except `_pad`) and not copy the whole class.

Sorry about that!

> Just tried locally your PR and found the reason why we can't have a global copied from on the tokenizer (sorry I didn't spot it earlier): the `_pad` method is overwritten. So instead we need to applied the copied from on each method (except `_pad`) and not copy the whole class.
> 
> Sorry about that!

so basically, write those copy comments again on both the slow and fast tokenizers?

Yeah, sorry

> Yeah, sorry

Oh dont be, on it now :D, thanks for the quick reply!

@sgugger anything left?

btw, the `mask_token` method in the fast tokenizer, there is a method and a setter , but i have the same comment on both (Copied from `BartTokenizerFast.mask_token)` is that the right way or am i supposed to keep the one above the method only and ignore the setter's one?

thanks a lot for the quick replies, what else is left?

Should be good now, just waiting for all tests to pass :-)

oh finally, the green light
Removed `Bert` interdependency in `tokenization_electra.py`
Author: OtherHorizon
Pull request: yes

# What does this PR do?


Related to #19303 
Removes `bert` dependency from `tokenization_electra.py`


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
 @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks a lot for working on this one! It's missing a few copied froms, I left suggestions. Once you're done, be sure to run `make style` on your branch (for code-formatting) and we should be good to merge!

`make style` changes a lot of files.

Make sure you install the specific versions of black that we use with `pip install -e .[quality]`, it's probably because you don't ahve the same version as the one we use.