noamaanMulla-03/transformers-issues huggingface dataset:
Fix problem_type to match with the applied loss function for distillbert sequence classification 
Author: sidhantls
Pull request: yes

# What does this PR do?
The problem_type in config is not correct with the loss function applied. Can be seen [here](https://github.com/huggingface/transformers/blob/242ec31aa59b358e631d981b545fd08330584ea8/src/transformers/models/distilbert/modeling_distilbert.py#L649). 

This PR fixes this so that the applied loss is consistent with the problem type: `BCEWithLogitsLoss` is applied for the `problem_type` of `single_label_classification`, and `CrossEntropyLoss` is applied for the problem type `multi_label_classification`

Fixes #12014

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@LysandreJik

Comments:
cc @abhi1thakur @sgugger 

No, this is incorrect. What `single_label_classification` means each sample can only have one label (but there could be multiple classes) so the loss to use is cross entropy. `multi_label_classification` means each sample can have zero or several labels, so in this case we use bce (because there can't be a softmax).
Distributed multi-node support for CPU cluster
Author: ddkalamk
Pull request: no

# 🚀 Feature request

## Motivation
Current distributed run supports only multi-GPU/TPU run. This feature requests adding support for distributed CPU runs using MPI/GLOO or recently added custom backend e.g. intel oneCCL backend using [torch-ccl](https://github.com/intel/torch-ccl) plugin.

## Your contribution
An example use for Intel oneCCL backend (using Intel torch-ccl) can be found at https://github.com/ddkalamk/transformers/blob/pcl-v4.0.0/examples/question-answering/run_squad.py#L746

Assumption is we launch the application using MPI (similar to horovod) and based on environment variable initialize distributed backend.

Here is another and more comprehensive use case from Facebook DLRM workload:
https://github.com/facebookresearch/dlrm/blob/master/extend_distributed.py#L59



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Compute probability of target sentences given an input
Author: fferlito
Pull request: no

I need to test the probabilities that the model would produce certain outputs. Let me give you an example:
I have a source sentence X, and several possible target sentences Y1, Y2, Y3, Y4 , ... 

I want to know if I can compute the probability that the model would give to each of the translation Y, given X. 
Is there a function to compute these values?

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fine-Tuning TFGPT2LMHeadModel / What to pass to fit
Author: demongolem-biz
Pull request: no

I am trying to fine-tune this pretrained model on my own data and I can't seem to get the format correct for what the model would like to see as input.

I am using TFGPT2LMHeadModel, GPT2Config and GPT2TokenizerFast.

When I do `model.fit(x,y, epochs=EPOCHS)`,

- If x and y are the outputs of tokenizing on GPT2TokenizerFast (i.e.  `tokenized = tokenizer(data_list, return_tensors='tf', add_special_tokens = True, truncation=True, padding = 'longest')`), I get: `ValueError: Unsupported value type BatchEncoding returned by IteratorSpec, serialize`.  I tried this because of what I saw on [this example code snippet](http://huggingface.co/transformers/model_doc/gpt2.html#tfgpt2lmheadmodel)
- If instead I choose x as `np.asarray(df['input_ids'].tolist()).astype('int32')` (and y is the corresponding thing for my label data) I get `InvalidArgumentError: Incompatible shapes: [32,154] vs. [2,32,16,154]` which looks a lot closer.

It seems like I have to choose the correct portions of the tokenizer output to feed to the fit function, but I am not choosing correctly.  Could you please clarify this for me?

I am using tensorflow 2.4.1 and transformers 4.5.1.

Comments:
When I run code supplied by another user from [another issue](https://github.com/huggingface/transformers/issues/2439), which supposedly worked at one point in time, I get a similar dimension mismatch.  Is there a golden combination of tf and transformers I am supposed to be using?

Ah, the 16 above is the batch size, which must be 16.  If I create a dataset with this batch size, then my model with train with the fit function.  Like

`dataset = tf.data.Dataset.from_tensor_slices((x, y)) ` 
`dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)`

Hi, Tensorflow maintainer here! Can you paste me a minimal example that reproduces the problem? You don't have to share your data or anything, just give me a few lines with a made-up tiny dataset that I can run here to recreate the problem - it'll make it much easier for me to track it down. Alternatively, if you're loading your data from HF datasets, that's fine too.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
encoder decoder in transformers
Author: lytum
Pull request: no

Thanks for your contribution for EncoderDecoderModel

I want to ask a question about the pool_layer of encoder.

Generally the default 'add_pooling_layer=True' in encoder, while the output of encoder in EncoderDecoder is without pool_layer. Is my understanding correct?

Now i want to add a classification layer in encoder, how should i do now?

Thanks in advance

Comments:
Hi there!

It would be nice if you ask such questions on the [forum](https://discuss.huggingface.co/). Use issues to report bugs and for feature requests or anything else that can't be discussed on the forum. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Penalise n-gram repetition in generated sequences
Author: KMFODA
Pull request: no

# 🚀 Feature request

As per this [forum post](https://discuss.huggingface.co/t/force-decoder-to-avoid-repetition-between-generated-sentences/625) sometimes it's helpful to have a parameter that can increase the diversity amongst different generated sentences. This can be a penalty on the number of repeated n-grams between each generated sentence.

## Motivation

If a generation model is being used and `num_returned_sequences` is greater than `1` there are a number of use cases that make it beneficial to be able to have a parameter that can increase the diversity amongst generated sentences or at least avoid exact replicas. For example when trying to create different paraphrases of the same sentence or question.

Example:
Original Question ::
What is the expected close date of the opportunity

Paraphrased Questions Generated by T5::
0: What will be the expected close date of the opportunity?
1: What is the expected closing date for the opportunity that you are considering?
2: What is the expected close date of the opportunity?
3: What is the expected close date on the opportunity?
4: When would be the expected close date of the opportunity?

## Your contribution

I'm happy to submit a PR to work on this if it makes sense to add this as a feature. I would just appreciate a steer on where the best place would be to add this penalty.

@patil-suraj @patrickvonplaten 

Comments:
Hi @KMFODA 

The `generate` method already supports penalizing n-gram repetition, this can be done by passing the argument `no_repeat_ngram_size` , if it's passed it will ensure that all n-grams of the given size will only occur once.

This however does not mean that the different `num_returned_sequences` sequences will be diverse since AFAIK in beam search usually the sequences are very close to each other. 

You could try using beam sampling bypassing `do_sample=True`  which will use sampling in each beam, which could introduce some diversity.

There's also an option of using [diverse beam search](https://arxiv.org/abs/1610.02424) which can be enabled using the option `diversity_penalty`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fixes bug that appears when using QA bert and distilation.
Author: madlag
Pull request: yes

This is a fix for
https://github.com/huggingface/transformers/issues/11626
and this is a somewhat related to:
https://github.com/huggingface/transformers/issues/11941


During backward pass Pytorch complains with:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
This happens because the QA model code modifies the start_positions and end_positions input tensors, using clamp_ function: as a consequence the teacher and the student both modifies the inputs, and backward pass fails.

From a quick check it looks like this is used in at least all QA code, like in:
```
cd transformers/src/transformers/models
grep -nr '[a-z]_(' . | grep clamp 

...
./xlnet/modeling_xlnet.py:1877:            start_positions.clamp_(0, ignored_index)
...
```

(and maybe for other models use.)
This may intended, but it's quite hard for the end user to track down these bugs as the issues reveal.
(And maybe PyTorch changed something and made this apparent)


Comments:
It looks like I should also modify a bunch of other models (utils/check_copies.py fails) to match the same change I did in BERT, I will wait to hear from you what is the process to do so.


It should be ok now, only non related tests are failing in run_tests_torch.
NN_pruning module for Question Answering
Author: fdlci
Pull request: no

Hi!

I am trying to run the launch_qa_sparse_single.py file from the question answering example from your nn_pruning library (https://github.com/huggingface/nn_pruning). I haven't changed anything from the original code and I get this error:

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
***** Running training *****
  Num examples = 131754
  Num Epochs = 20
  Instantaneous batch size per device = 16
  Total train batch size (w. parallel, distributed & accumulation) = 16
  Gradient Accumulation steps = 1
  Total optimization steps = 164700
  0%|                                                                                       | 0/164700 [00:00<?, ?it/s]Traceback (most recent call last):
  File "question_answering/launch_qa_sparse_single.py", line 33, in <module>
    main()
  File "question_answering/launch_qa_sparse_single.py", line 23, in main
    qa.run()
  File "./question_answering/xp.py", line 324, in run
    self.train()
  File "./question_answering/xp.py", line 312, in train
    model_path= model_path
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/transformers/trainer.py", line 1120, in train
    tr_loss += self.training_step(model, inputs)
  File "/home/ines/NN_pruning/nn_pruning/nn_pruning/sparse_trainer.py", line 86, in training_step
    return super().training_step(*args, **kwargs)
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/transformers/trainer.py", line 1542, in training_step
    loss.backward()
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/ines/NN_pruning/venv_nn_prun/lib/python3.7/site-packages/torch/autograd/__init__.py", line 147, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [16]] is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

I found several solutions to this problem on the internet but all the solutions I came accross with tell me to change something in the architecture of the model. Unfortunately here, we are using a Trainer from the transformers library so I don't really know how to fix this issue. Thank you for your help.

I am running this code with torch==1.8.1 and cuda=11.1.

Comments:
Hi there, would you mind opening this issue in `nn_pruning` repo? Thanks.

This was bypassed in nn_pruning then fixed in transformers in https://github.com/huggingface/transformers/pull/12026 .
adds metric prefix.
Author: riklopfer
Pull request: yes

# What does this PR do?

Adds metric prefix to metrics dict. This is needed for  `metric_for_best_model` to function properly. See https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L1516

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:


- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
You just need to tweak the test of this script in examples/pytorch/test_examples.py, since it's looking for f1 instead of eval_f1. Same for exact.

@sgugger, the example tests are fixed now, this other failure is a mystery to me. I suspect that failure is caused by another change. Let me know if you think the `run_torch_tests` failure is related to this PR and I'll look into it. 

No this failure is independent and currently being investigated, so we can merge this PR safely. Thanks again!
ImportError: cannot ipmort name 'TFAutoModel'
Author: Holy-Shine
Pull request: no

transformers: 4.6.1
tensorflow-gpu: 2.0.0

when I wrote code below in my jupyter-notebook:
`from transformers import TFAutoModel`
I got an ImportError:

> **ImportError: cannot ipmort name 'TFAutoModel'**

I wonder what's wrong with my code or dev environment.

Comments:
Fix tapas issue
Author: NielsRogge
Pull request: yes

# What does this PR do?

Fixes #12060

However, the (slow) integration tests of TAPAS that use relative position embeddings are failing for me locally, most likely due to the new version of the [torch-scatter](https://github.com/rusty1s/pytorch_scatter) dependency. I'll look into that.

Update: just tested the models in Google Colab (which has `torch 1.8.1+cu101`). Everything seems to work fine there. However, when running locally on `torch 1.8.1+cu111`, I'm getting entirely different logits/hidden states. Both are using `torch-scatter 2.7.0`.

Comments:
How to update the GPT2 with loss which are provided from another separate module?
Author: yanan1116
Pull request: no

Suppose I have N prompts(sentences) for generation. They are fed into GPT2 and get the corresponding synthesis sentences. 
And I have a separate black box which can return loss given these synthesis samples. The black box is just another component. 
It is natural for every batch that GPT2 generate samples and get the loss, repeatedly.
What I want to do is use the loss from the black box to update the parameters of GPT2, at each batch.
The generation of GPT2 is quite simple, but how can I implement the idea of updating it with the loss?
Is there any example for doing this ?
Please give some thoughts, thanks.

Comments:
I guess you can do it like so:
```
from transformers import GPT2Tokenizer, GPT2LMHeadModel
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')

model.train()
# when generating, we will use the logits of right-most token to predict the next token
# so the padding should be on the left
tokenizer.padding_side = "left" 
tokenizer.pad_token = tokenizer.eos_token # to avoid an error

prompts = ["Hello, my dog is a little", "Hello, my dog is"]
inputs = tokenizer(prompts, padding=True, return_tensors="pt")

output_sequences = model.generate(
    input_ids=inputs['input_ids'],
    attention_mask=inputs['attention_mask']
)

loss = black_box(output_sequences)
loss.backward()
```

Please note that the [forum](https://discuss.huggingface.co/) is a better place to ask questions, Github issues are mostly for bugs/feature requests.

Thanks.

> I guess you can do it like so:
> 
> ```
> from transformers import GPT2Tokenizer, GPT2LMHeadModel
> tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
> model = GPT2LMHeadModel.from_pretrained('gpt2')
> 
> model.train()
> # when generating, we will use the logits of right-most token to predict the next token
> # so the padding should be on the left
> tokenizer.padding_side = "left" 
> tokenizer.pad_token = tokenizer.eos_token # to avoid an error
> 
> prompts = ["Hello, my dog is a little", "Hello, my dog is"]
> inputs = tokenizer(prompts, padding=True, return_tensors="pt")
> 
> output_sequences = model.generate(
>     input_ids=inputs['input_ids'],
>     attention_mask=inputs['attention_mask']
> )
> 
> loss = black_box(output_sequences)
> loss.backward()
> ```
> 
> Please note that the [forum](https://discuss.huggingface.co/) is a better place to ask questions, Github issues are mostly for bugs/feature requests.
> 
> Thanks.

thanks, very helpful.
Siamese Multi-depth Transformer-based Hierarchical Encoder
Author: lalitpagaria
Pull request: no
Labels: name: New model

name: Feature request

# 🌟 New model addition

## Model description
Recently Google is published paper titled ["Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching"](https://arxiv.org/abs/2004.12297). And according to paper for long-form document matching SMITH model outperforms the previous state-of-the-art models including hierarchical attention, multi-depth attention-based hierarchical recurrent neural network, and BERT.

I feel it is will add value to already awesome transformers models collection :slightly_smiling_face:

<!-- Important information -->

## Open source status

* [X] the model implementation is available: https://github.com/google-research/google-research/tree/master/smith
* [X] the model weights are available: [SMITH-WP+SP model checkpoint](http://storage.googleapis.com/gresearch/smith_gwikimatch/smith_wsp_pretrain_ckpt_opensource.zip) and [GWikiMatch data](http://storage.googleapis.com/gresearch/smith_gwikimatch/gwikimatch_open_source.zip)
* [X] who are the authors: https://github.com/yangliuy, https://github.com/eladeban


Comments:
Linking Haystack issue https://github.com/deepset-ai/haystack/issues/719

Frequent user of hugging face here, I'm a fan of this new publication and would love to see it implemented. Commenting here for the GitHub algorithm to ++

Hi all, rather than waiting for the implementation in huggingface. Is there a simple way to utilize the pretrained model from the smith repo on our own dataset (to generate document embedding)?