noamaanMulla-03/transformers-issues huggingface dataset:
Possibility of disabling add_pooling_layer that works for all models
Author: ZhaofengWu
Pull request: no

# 🚀 Feature request

A way to ensure that no additional pooling layers are added that works for all model types.

## Motivation

I just want a barebone transformer and I need the per-token representation. Hence the pooling layer that is added in BERT-based models is not necessary for me, but it still occupies memory and forward pass time. Now, I think this can be disabled via `AutoModel.from_pretrained(name, add_pooling_layer=False)` (right?), but I don't think this would work for all models since some don't have this flag (e.g., GPT-2).

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Commenting for activity

The GPT-2 doesn't have a pooling layer, so what would you like to disable?

The issue is that if I pass `add_pooling_layer=False` to GPT-2, it will break. But I want a model-agnostic way to create a base model and ensure that there is no pooling layer added.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Commenting for activity

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[CI] move env print to util, add pt, nccl versions
Author: stas00
Pull request: yes

This PR:

- moves multiple python one liners to print pt env into a script - this is both faster and easier to maintain
- adds torch version 
- adds NCCL version (could remove if it's useless info, i just thought it might be useful in certain situations)

@LysandreJik 

Comments:
[RFC] Amphere/tf32 defaults for transformers
Author: stas00
Pull request: no
Labels: name: Performance

## Background

It's possible to use the new TF32 format automatically when doing fp32 processing for a ~3x speed up, without doing any changes to the code, other than flipping the switch on. But the speed up may come at a cost of accuracy. You can see the differences between the formats in the following image:

![tf32-Mantissa-chart-hi-res-FINAL-400x255](https://user-images.githubusercontent.com/10676103/142496315-d762a485-22d9-459e-8e22-8acf845f64db.png)

You can see that both TF32 and FP32 have the same dynamic range (the magnitude of numbers), but the former has a much lower precision, which depending on a situation may or may not impact the final outcome.

## Emerging Need

As Amphere hardware is emerging and TF32 automatic enabling seems to be going in the direction of being disabled by default (probably starting from pt-1.11?) as discussed here: https://github.com/pytorch/pytorch/issues/67384, we need to communicate to our users how to turn it on/off and what are the impacts on speed and accuracy might be.

Having it on could bring a ~3x speed improvement, and most likely according to the NVIDIA engineers the training quality shouldn't be impacted. But we don't have our first hand experiences yet to provide pragmatic recommendations.

## Available Guides

The on/off machinery is explained here: https://pytorch.org/docs/master/notes/cuda.html#tf32-on-ampere

The other crucial document is: https://pytorch.org/docs/master/notes/numerical_accuracy.html

TF32 Educational blog post: https://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precision-format/

## Plan of action

So this issue is both an RFC and is also documenting the need to update: https://huggingface.co/transformers/performance.html

I trust that the commentary will emerge once you start experimenting with the new hardware.

@sgugger, @LysandreJik, @patil-suraj, @patrickvonplaten 


Comments:
[trainer] add tf32-mode control
Author: stas00
Pull request: yes

This PR adds tr32-mode control support for HF Trainer for Ampere cards. RFC: https://github.com/huggingface/transformers/issues/14450

pytorch had this mode on by default since pt-1.7, but are discussing to turn it off in the coming new release. https://github.com/pytorch/pytorch/issues/67384

Here is the proposed logic: 
1. by default HF Trainer will set it to Enabled,  This is marked as experimental should we discover that this is not a safe default down the road.
2. and `--tf32 0` will disable it. 
3. If the setup uses a wrong gpu or too low torch version - it will silently do nothing as it's irrelevant.

The PR adds:
- `is_torch_tf32_available` and `require_torch_tf32` helper utils
- adds basic test

Fixes: https://github.com/huggingface/transformers/issues/14450

@sgugger, @LysandreJik 


Comments:
The benchmarks are terrible: https://github.com/huggingface/transformers/issues/14608
going to ask for advice from the pytorch experts
Added an error when `temperature` is defined but `do_sample` is False.
Author: JulesGM
Pull request: yes

# What does this PR do?

This is to prevent the possibility that people think that setting a temperature is enough to get sampling behavior.
I found this confusing at first.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case): No, but is kind of a similar idea, in that it makes things more clear.
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section? Quickly I must admit.
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? No.
- [N/A] Did you make sure to update the documentation with your changes? N/A, this is to make implicit behavior more clear.
- [ ] Did you write any new necessary tests?: N/A


## Who can review?
@patrickvonplaten

Comments:
@patrickvonplaten Added as an edit, not sure if it pinged you.

The test that fails has nothing to do with this change

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to train bilingual models?
Author: puraminy
Pull request: no

In [language_modeling](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling) examples, there is an script to train a T5 model for a language. I would like to know how can I use or change this script to train a bilingual language for X and Y languages? Is it possible to do if via this code? 

Also I would like to know if it's possible to initialize training from a multilingual model like `mt5`, and then use training data for one or two languages  for further training?

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
OnnxRuntime error " The input tensor cannot be reshaped to the requested shape."
Author: husnain-ali21
Pull request: no

Hello! I have quantised the bart model for abstractive summarisation using transformers.onnx as suggested at https://huggingface.co/transformers/serialization.html.When I run the session with inputs for getting outputs  back, I get a shape mismatch error. I have checked that the input shapes  of my input text are in accordance with 
https://github.com/huggingface/transformers/blob/master/src/transformers/models/bart/configuration_bart.py

 Here is the error message I am receiving.

`onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. Name:'Reshape_109' Status Message: /Users/runner/work/1/s/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:42 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape &, std::vector<int64_t> &, bool) gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{2}, requested shape:{1,1}`






## Environment info

- `transformers` version: 4.11.3
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.9.6
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

Models:

@patil-suraj
@patrickvonplaten


## Information

Model I am using (Bart, i.e  facebook/bart-large-cnn ...):





Comments:
Not sure about this error, but we have added an experimental bart onnx generation example [here](https://github.com/huggingface/transformers/tree/master/examples/onnx/pytorch/translation), maybe that could help :) 

Thank you, That would be of great help!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Intel OpenVINO inference backend
Author: dkurt
Pull request: no

# 🚀 Feature request

Hi all! We (Intel team) would like to contribute in Hugging Face by integration of optimized deep learning inference framework OpenVINO. According to the contribution guidelines, we should discuss it with community and developers to match the expectations. Let me share what we have as this moment:

* New format for the [models hub](https://huggingface.co/) - a pair of `ov_model.xml` and `ov_model.bin` which is OpenVINO IR
* New class `OVAutoModel` which downloads `ov_model.xml/.bin` by default or framework specific checkpoint (PyTorch, TensorFlow) with on-the-fly conversion. Example:
  ```python
  from transformers import OVAutoModel
  model = OVAutoModel.from_pretrained("dkurt/test_openvino")
  ```
  (see https://huggingface.co/dkurt/test_openvino)
  or
  ```python
  # Downloads PyTorch weights and converts to OpenVINO in runtime
  model = OVAutoModel.from_pretrained("albert-base-v2", from_pt=True)
  ```
  or
  ```python
  # Downloads TensorFlow weights and converts to OpenVINO in runtime
  model = OVAutoModel.from_pretrained("distilbert-base-uncased", from_tf=True)
  ```

## Motivation

Expected efficiency improvement as well as utilization of Intel hardware (CPU/iGPU/VPU and others supported)

## Your contribution

Will do our best to complete this feature request. Looking forward for feedback from developers and experienced users.


Comments:
Hi, @mfuntowicz and @LysandreJik. Just wanted to let you know that there is a PR with my proposal at https://github.com/huggingface/transformers/pull/14203. I would be very happy to get any feedback from you and understand if it goes in a right direction.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Vision Transformer (ViT) feature vector example (not classification)
Author: raulcarlomagno
Pull request: no

# 🚀 Feature request


## Motivation

I would like to see an example of using Vision Transformer just for feature extraction, get the feature vector, before the classification part of the neural network as can be done with tensorflow hub
https://www.tensorflow.org/hub/common_signatures/images?hl=en#image_feature_vector


Comments:
In HuggingFace Transformers, models typically output a dictionary. You can access the feature vector by getting the `pooler_output` key of that dictionary (assuming you're using `ViTModel`). It's a tensor of shape `(batch_size, hidden_size)`, so in case you're only providing a single image, and you're using the base-sized model, this will be a tensor of shape `(1, 768)`. 

Here's an example:

```
from transformers import ViTFeatureExtractor, ViTModel
from PIL import Image
import requests

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = ViTFeatureExtractor.from_pretrained('google/vit-base-patch16-224-in21k')
model = ViTModel.from_pretrained('google/vit-base-patch16-224-in21k')

inputs = feature_extractor(images=image, return_tensors="pt")
outputs = model(**inputs)
feature_vector = outputs.pooler_output
```

thank you, that is what i was looking for !
thank you

@NielsRogge is it always `pooler_output` key that contains the feature vector for all image transformers (such as CLIP, DeiT, VisualBERT, DETR)?
Add BartForTokenClassification
Author: ZIZUN
Pull request: yes

# What does this PR do?
+ Add BartForTokenClassification class for NER task

## Before submitting
- [x] Did you read the [contributor guideline]
- [x] Did you make sure to update the documentation with your changes?
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patrickvonplaten, @patil-suraj @sgugger 

Comments:
Finished ci test and passed all.

Hey @ZIZUN, 

Thanks for the PR! Do we have a fine-tuned checkpoint for `BartForTokenClassification`? I'm wondering how useful this architecture is

Hello @patrickvonplaten 
Yes, I tried to finetune this architecture for the Korean NER dataset. The results are as follows.

|                                                                  | Slot F1 (%) |
| ---------------------------------------------------- | ----------- |
| [CNN-BiLSTM-CRF](https://github.com/monologg/korean-ner-pytorch)           | 74.57       |
| **[KoBART(ours)](https://huggingface.co/hyunwoongko/kobart)**                                                 | **84.17** |
| Bert-Multilingual                                                 | 84.20       |
| [KoBERT](https://github.com/monologg/KoBERT-NER)                         | 86.11       |
|  RoBERTa                                                             | 87.58       |

This structure is an auto-regressive architecture, so the score is lower than bi-directional model such as BERT.
But the score is resonable, so I think this architecture is a valueable thing for research.

Thank you for leaving comments! @sgugger @patil-suraj 
I think you are all right. It's sad but the results are a little disappointing. If I make a notebook, I will share it with you.
Have a nice day.
AttributeError: 'str' object has no attribute 'squeeze'
Author: eracle
Pull request: no

Hello guys, I am new to Hugging Face. I was running it on my Ubuntu 18, in Jupyter notebook.
I was running the example case (tutorial) for the https://huggingface.co/neuraly/bert-base-italian-cased-sentiment model. But I get this error:
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_29169/2345971003.py in <module>
      1 # Remove the fake batch dimension
----> 2 logits = logits.squeeze(0)
      3 
      4 # The model was trained with a Log Likelyhood + Softmax combined loss, hence to extract probabilities we need a softmax on top of the logits tensor
      5 proba = nn.functional.softmax(logits, dim=0)

AttributeError: 'str' object has no attribute 'squeeze'
```
When I inspect the value of the logits variable, its content is the string "logits".


## Environment info
- `transformers` version: 4.11.3
- Platform: Linux-5.4.0-89-generic-x86_64-with-glibc2.27
- Python version: 3.9.7
- PyTorch version (GPU?): 1.10.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Nope
- Using distributed or parallel set-up in script?: Nope


### Who can help


Models:
- neuraly/bert-base-italian-cased-sentiment: @gianpy15

Library:
- Tokenizers: @LysandreJik

Documentation: @sgugger
 -->

## Information
The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behaviour:

1. Run the example in the tutorial.



Comments:
That code can't work since Transformers v4
```
# Call the model and get the logits
logits, = model(tensor)
```
you need to do
```
# Call the model and get the logits
logits = model(tensor).logits
```


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Can we save tokenized datasets?
Author: onjas-6
Pull request: no

It takes a lot of time to tokenize my dataset, is there a way to save it and load it?

Let's say I'm using the IMDB toy dataset, How to save the `inputs` object?
```
from datasets import load_dataset
raw_datasets = load_dataset("imdb")
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
inputs = tokenizer(sentences, padding="max_length", truncation=True)



Comments:
Sure, that's possible.

You can tokenize your entire dataset using the tokenizer, then save it to disk using  the `save_to_disk` method as explained in the [docs](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=save_to_disk#datasets.Dataset.save_to_disk).

Small example:

```
from datasets import load_dataset
from transformers import AutoTokenizer

datasets = load_dataset("imdb")
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")

encoded_datasets = datasets.map(lambda examples: tokenizer(examples['text']), batched=True)
encoded_datasets.save_to_disk('.')
```

You can later load it back in using the `load_from_disk` method as explained [here](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=save_to_disk#datasets.Dataset.load_from_disk).

> Sure, that's possible.
> 
> You can tokenize your entire dataset using the tokenizer, then save it to disk using the `save_to_disk` method as explained in the [docs](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=save_to_disk#datasets.Dataset.save_to_disk).
> 
> Small example:
> 
> ```
> from datasets import load_dataset
> from transformers import AutoTokenizer
> 
> datasets = load_dataset("imdb")
> tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
> 
> encoded_datasets = datasets.map(lambda examples: tokenizer(examples['text']), batched=True)
> encoded_datasets.save_to_disk('.')
> ```
> 
> You can later load it back in using the `load_from_disk` method as explained [here](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=save_to_disk#datasets.Dataset.load_from_disk).

Thank you Niels! My problem is a bit more complex, I created a custom dataset using method here: https://github.com/huggingface/notebooks/blob/master/transformers_doc/custom_datasets.ipynb, but the imdbDataset object created doesn't has `save_to_disk` method.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
one of the variables needed for gradient computation has been modified by an inplace operation
Author: kebinC
Pull request: no

It occured when i forward a bert model twice and backward loss.
```
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [1, 20]] is at version 4; expected version 3 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!


  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 511, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/mnt/xxx/xxx/pytorch_train/image_clas/models/text_relevance_model.py", line 100, in forward
    bert_output = self.bert_model(**bert_input)
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 989, in forward
    past_key_values_length=past_key_values_length,
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 220, in forward
    position_embeddings = self.position_embeddings(position_ids)
  File "/share/xxx/env/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
```

it related to https://github.com/huggingface/transformers/issues/11941, how can i solve it ?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.