noamaanMulla-03/transformers-issues huggingface dataset:
Wrap RoFormer integration test forward passes with torch.no_grad()
Author: daspartho
Pull request: yes

# What does this PR do?
As proposed in issue #14642, this PR wraps forward passes in RoFormer integration tests with torch.no_grad(). This way, no unnecessary gradients are computed during inference.


## Before submitting
- [ ] This PR fixes a typo or improves the docs.
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?
- [ ] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests?


## Who can review?
@LysandreJik could you please take a look at it?
Thanks :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Wrap VisualBERT integration test forward passes with torch.no_grad()
Author: daspartho
Pull request: yes

# What does this PR do?
As proposed in issue #14642, this PR wraps forward passes in VisualBERT integration tests with torch.no_grad(). This way, no unnecessary gradients are computed during inference.


## Before submitting
- [ ] This PR fixes a typo or improves the docs.
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?
- [ ] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests?


## Who can review?
@LysandreJik could you please take a look at it?
Thanks :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add DPT
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds [DPT](https://arxiv.org/abs/2103.13413), Dense Prediction Transformers, to the library. It's some very nice work from Intel Labs that applies Transformers for dense prediction tasks such as semantic segmentation and depth estimation.

Feel free to play around with the notebook [here](https://colab.research.google.com/drive/177o79Qm8qsjGDQk5TDSS4tawcFNSZsy8?usp=sharing). 

I've defined 3 models:

* `DPTModel`
* `DPTForDepthEstimation`
* `DPTForSemanticSegmentation`.

DPTModel is the backbone only (ViT in this case). The head models use a neck (DPTNeck) combined with a task-specific head (either `DPTDepthEstimationHead` or `DPTSemanticSegmentationHead`).

Important here:
* a neck is an nn.Module that takes a list of tensors and produces another list of tensors.
* a head takes a list of tensors and return `logits`.

To do:

- [x] make sure heads take a list of tensors as input
- [x] add tests for `DPTFeatureExtractor`
- [x] discuss `out_indices` and `in_index` names
- [x] transfer weights to `Intel` organization

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for your reviews, addressed most comments. Main thing to update is:
* rename `out_indices` (which features to use from the backbone)
* rename `in_index` (which features to use in the head)

Ideally we have names that are going to be used by all vision models.

stop with all the emails I’m getting mad



Sent from my iPhone

> On Mar 11, 2022, at 8:37 AM, NielsRogge ***@***.***> wrote:
> 
> ﻿
> @NielsRogge commented on this pull request.
> 
> In src/transformers/models/dpt/modeling_dpt.py:
> 
> > +            if output_attentions:
> +                all_self_attentions = all_self_attentions + (layer_outputs[1],)
> +
> +        if output_hidden_states:
> +            all_hidden_states = all_hidden_states + (hidden_states,)
> +
> +        if not return_dict:
> +            return tuple(v for v in [hidden_states, all_hidden_states, all_self_attentions] if v is not None)
> +        return BaseModelOutput(
> +            last_hidden_state=hidden_states,
> +            hidden_states=all_hidden_states,
> +            attentions=all_self_attentions,
> +        )
> +
> +
> +class DPTReassembleBlocks(nn.Module):
> Renamed to DPTReassembleStage
> 
> —
> Reply to this email directly, view it on GitHub, or unsubscribe.
> Triage notifications on the go with GitHub Mobile for iOS or Android. 
> You are receiving this because you are subscribed to this thread.
Allow custom signature while saving TF models
Author: dimitreOliveira
Pull request: no

### Feature request

Currently, when we use the [save_pretrained](https://github.com/huggingface/transformers/blob/ca485e562b675341409e3e27724072fb11e10af7/src/transformers/modeling_tf_utils.py#L2085) function from this library the model signature used to save the model is the default one that only calls the model on the inputs, I would like to be able to provide a custom signature while using the `save_pretrained` function.

### Motivation

Persisting models with custom signatures is quite important for models that target production setups, especially if they are going to be served with TF Serving.

I might be wrong but it seems that currently, the only way to save a `Transformer` model with a custom signature is by saving it using functions from the TF library, it would be very nice if the HF ecosystem could also support this feature.

### Your contribution

I think this might be simple to implement and I would be happy to draft a PR if you think this could be a helpful feature.

Comments:
cc @Rocketknight1 @gante

Hi @dimitreOliveira, that sounds like a great feature, and we'd be happy to accept that PR! We've been working on making our default signatures more general and usable, but this sounds like a good idea too. Are you planning to add a `signatures` argument that's passed through to `model.save()` when `saved_model=True`?

Hey @Rocketknight1 I am glad you liked the feature, I am happy to collaborate with the TF side of the lib.

Yes, my idea is to just add  a `signature` parameters to the [signatures](https://github.com/huggingface/transformers/blob/ca485e562b675341409e3e27724072fb11e10af7/src/transformers/modeling_tf_utils.py#L2085) function, that parameter would default to `None` and if that is the case we would just use `self.serving` as we already do, this way there would not be any relevant side-effect, and users could just create their custom signatures and pass it while saving. Looking at the code design it seems that this change would be compatible with all TF transformers models ; )
I have not looked yet to see if that would generate any issues with the tests, but if the plan is good I will work on the code during the weekend.

For context, the idea for this feature came to me while I was working on [this repository](https://github.com/dimitreOliveira/hf_tf_serving_examples), that also have a collection of custom signatures that range from text classification to text generation.
Maybe this feature also works for the vision and speech models but I do not have a lot of experience with those, maybe later I could also take a look there.

@Rocketknight1 @gante you can find the draft PR above, let me know if it looks good, then I can finish the work, if needed, I can provide some examples of cool use cases using custom signatures with the models.
Enabling custom TF signature draft
Author: dimitreOliveira
Pull request: yes

# What does this PR do?
This PR is related to the discussion on issue #19094 , which is related to `transformers` enabling users to provide custom signatures for models while saving them with [save_pretrained](https://github.com/huggingface/transformers/blob/ca485e562b675341409e3e27724072fb11e10af7/src/transformers/modeling_tf_utils.py#L2085).

Fixes #19094 


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Adding a test (perhaps inside [this test class](https://github.com/huggingface/transformers/blob/f3d2f7a6e08efe18debf59512325f02128394b43/tests/test_modeling_tf_common.py#L1913)) would be great!

Hey there, I think that this code is finished and ready to be merged, I fixed the code and added the tests, let me know if the tests look too hardcoded or if I could check something else.

It seems that the failed tests are not related to my changes.

cc @gante @Rocketknight1 @sgugger 

@dimitreOliveira it seems ready to merge.

There is an apparently unrelated failure, but let's rule it out. Can I request you to rebase and commit? i.e. ensure your fork is up to date with the latest main and then run
```bash
git checkout main
git pull
git checkout custom_tf_signature
git rebase origin/main
git push origin custom_tf_signature -f
```

@gante should be good now, thanks!
Extend `nested_XXX` functions to mappings/dicts.
Author: Guillem96
Pull request: yes

Extended `nested_XXX` trainer pt utility functions to work with mappings (dict, OrderedDict, etc.)

Some classes that are modeling the models' outputs, inherit from `ModelOutput` which at turn is an `OrderedDict`. Currently, when applying these `nested_XXX` functions to these models ouputs the code fails with an error.

Extending this nested utilities to work with dictionaries fix this issue.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for your feedback @sgugger . 
Suggestions and style applied!

Thanks a lot!
CLI: add import protection to datasets
Author: gante
Pull request: yes

# What does this PR do?

Add import protection to datasets -- I've seen two or three recent issues where the authors can't post their env due to `datasets` failing to import on `pt-to-tf` ([example](https://github.com/huggingface/transformers/issues/19445))

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix doctests for `DeiT` and `TFGroupViT`
Author: ydshieh
Pull request: yes

# What does this PR do?

For `DeiT`: The parameter initialization is changed for some models in #19341. This can affect some tests where a checkpoint without head is used (so randomly initialized head). cc @alaradirik 

For `TFGroupViT`: We loved PyTorch too much (or not enough?) and want to use it in TensorFlow --> It's still too early :-)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update `WhisperModelIntegrationTests.test_large_batched_generation`
Author: ydshieh
Pull request: yes

# What does this PR do?

Update the expected values for `WhisperModelIntegrationTests::test_large_batched_generation`.

It is probably due to the different GPUs used.


See currently failing test [here](https://github.com/huggingface/transformers/actions/runs/3212658214/jobs/5251735170).



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix whisper for `pipeline`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
After the merge of #19378 , the feature extractor does not work with the `pipeline` function. This PR is the same as #19385. 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Before we merge here, let's try to have the following tests working:

- Automatic pipeline tests for dummy whisper model (as discussed offline) 
- 2 slow pipeline tests (one for speech recognition, one for speech translation here: [https://github.com/huggingface/transformers/blob/10100979ed0594d4cfe1982cdfac9642a[…]/tests/pipelines/test_pipelines_automatic_speech_recognition.py](https://github.com/huggingface/transformers/blob/10100979ed0594d4cfe1982cdfac9642a68e473e/tests/pipelines/test_pipelines_automatic_speech_recognition.py#L54)

We were missing a `_CHECKPOINT_FOR_DOC`, so I added a warning when the tests are skipped. 
It seems a little bit problematic as if it is unused, `quality` will fail (and in our case, I had to change the code to use it 😃 ) 
Other models that are not tested : 
`SpeechEncoderDecoderModel, Speech2TextForConditionalGeneration`, which are also just missing the `_CHECKPOINT_FOR_DOC`(see [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/speech_to_text/modeling_speech_to_text.py#L41) )
TF: TFBart embedding initialization
Author: gante
Pull request: yes

# What does this PR do?

### Context
We were initializing the embeddings as `TFSharedEmbeddings(config.vocab_size, config.d_model, config.pad_token_id, name="model.shared")`. Notice the 3rd argument, the pad token id, which is the 3rd argument in `nn.Embedding`. However, for `TFSharedEmbeddings`, [the 3rd argument is the initializer range](https://github.com/huggingface/transformers/blob/4c962d5e790d06c142af35aad165c74c0bcf861a/src/transformers/modeling_tf_utils.py#L2837). This means that some models, like TFMarian, were initializing the embeddings with very [large values](https://github.com/huggingface/transformers/blob/4c962d5e790d06c142af35aad165c74c0bcf861a/src/transformers/models/marian/configuration_marian.py#L135) (stddev=58100).

### Changes
This PR correctly sets the embedding initialization according to the configuration parameter related to weight initialization range. It includes proper weight initialization when the embeddings are resized.

This PR will be used as a reference for embedding weight initialization, regarding the embedding update that is happening in the codebase at the moment.

### Discussion for the future
PT sets the weights in a top-down fashion, with `_init_weights` conveniently fetching information from the `config` and then initializing the weights. On TF, weight initialization is defined in a bottom-up fashion. This means that if we want to replicate the PT initialization for all TF weights, we need to pass the `config` all the way down to the individual layers (= verbose and needs manual changes in many places for all models) 😅 Alternativelly, we can replicate the `_init_weights` logic in TF, and manually set the weights after initializing the model. IMO that would be much cleaner, despite not being the way Keras expects weights to be set.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix `XGLMModelLanguageGenerationTest.test_batched_nan_fp16`
Author: ydshieh
Pull request: yes

# What does this PR do?

#18057 added this test to test running with fp16.

However, `from_pretrained(model_name, torch_dtype=torch.float16` seems **not able to change the dtype** for weights registered below:
https://github.com/huggingface/transformers/blob/a7bc4221c0c09857b30ac467e7de86d3f5a7c482/src/transformers/models/xglm/modeling_xglm.py#L168-L176
and `hidden_states` becomes again `float32` (because `position` is) at
https://github.com/huggingface/transformers/blob/a7bc4221c0c09857b30ac467e7de86d3f5a7c482/src/transformers/models/xglm/modeling_xglm.py#L715
and finally failed at `hidden_states = self.self_attn_layer_norm(hidden_states)` with
```bash
RuntimeError: expected scalar type Float but found Half
```


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger OK, let me check if I can do something for (not a real) weights defined by

```python
self.register_buffer("weights", emb_weights)

```
Decouples `XLMProphet` model from `Prophet` 
Author: srhrshr
Pull request: yes

@sgugger , 

Per the issue #19303, the `Prophet` model dependency is removed from `XLMProphet` and it now directly inherits from `PretrainedModel`.  

- [As discussed in a different PR review](https://github.com/huggingface/transformers/pull/19346#discussion_r988069210) , I've moved some of the docstring examples to inside the corresponding docs/source location.  Let me know if there are some tests you want me to run. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Thanks for reviewing the PR!

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger  - the requested changes have all been done. Thanks for your review!
Can't disable INTEGRATION_TO_CALLBACK
Author: LZY-the-boys
Pull request: no
Labels: name: bug

### System Info

I use transformers.4.20.0




### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When using the huggingface Trainer,  I set `report_to=['none'] ` in training_args to disable wandb logging as the doc says but a value error will be raised. I notice it's because the problem in the following code of `transformers/integrations.py`:

```python
def get_reporting_integration_callbacks(report_to):
    for integration in report_to:
        if integration not in INTEGRATION_TO_CALLBACK:
            raise ValueError(
                f"{integration} is not supported, only {', '.join(INTEGRATION_TO_CALLBACK.keys())} are supported."
            )
    return [INTEGRATION_TO_CALLBACK[integration] for integration in report_to]
```
No 'none' logits is dealed with, nor any other disable methods?  

### Expected behavior

I don't know if it's expected, but it's confusing me and causing inconvenience. So i would like to get the answer :)

Comments:
cc @sgugger

No, this code is not called when you set `report_to="none"` or `report_to=["none"]` since it is filtered out [here](https://github.com/huggingface/transformers/blob/6678350c01629b848aa9c41e169da5d6b8d9e7e9/src/transformers/training_args.py#L1155).

Setting `report_to=["none"]` in `TrainingArugments` works perfectly on my end, please make sure you are using the latest Transformers version and if the issue persists, please give us a code reproducer.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
we want to submit a PR about Source-Free compression training on huggingface NLP model，Where would you suggest submitting it?
Author: leiqing1
Pull request: no

### Feature request

Hello, we have done a Source-Free compression training function, and the benefits of bert-base-cased are as follows. I want to submit a PR, is it ok?  and which repo, huggingface/transformers or huggingface/optimum, is more suitable?
![image](https://user-images.githubusercontent.com/54695910/188194251-ef204386-e1fc-40a3-9796-36a2fa5b0a56.png)
![image](https://user-images.githubusercontent.com/54695910/188194560-171f1c31-1828-4272-9cc6-df241b2fb45d.png)


### Motivation

Improve the inference speed of NLP models

### Your contribution

we want to submit a pr for huggingface～

Comments:
Hey @leiqing1 ! Thanks a lot for being motivated to contribute 🤗 ! You mention this is about training, right? In this case, I would say `huggingface/transformers` is the place to go. But let's wait the opinions from my colleagues @sgugger @LysandreJik and @michaelbenayoun

As the training (for PyTorch models) is done by using the [Trainer class](https://github.com/huggingface/transformers/blob/ecdf9b06bc03af272ceb8d6951e30e677fdfd35c/src/transformers/trainer.py#L223), the PR is likely to involve the integration your compression method into that class.


Let's maybe start with a simple training script that you could add as a new research project in the repo?

Thanks for the reply. @ydshieh Yes, the compression process  is about training.   You mean we can prepare a simple training script to submit to https://github.com/huggingface/transformers/tree/main/examples/research_projects this directory？
@sgugger 

Yes, alongside with the extra modules you might need, all in the same folder.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Update TF whisper doc tests
Author: amyeroberts
Pull request: yes

# What does this PR do?

Fixes doctests for the TF whisper model. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._