fix jit trace error for model forward sequence is not aligned with jit.trace tuple input sequence, update related doc
Author: sywangyi
Pull request: yes

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

@sgugger


Comments:
@liangan1 @jianan-gu @yao-matrix please help review

_The documentation is not available anymore as the PR was closed or merged._

> I still do not understand the main problem: it looks like JIT does not support dictionary inputs which are used in every model in Transformers. Classification models are not the only ones using the `labels` key, all task-specific models do... and a model that a user wants to evaluate will very likely have a dataset with labels. The proposed workaround to use label smoothing makes no sense for an evaluation.
> 
> It looks like this integration has maybe be merged too quickly and doesn't actually work or are there models that can be evaluated with it?

yes. all the cases containing "labels" will fail in jit.trace, while other case like QnA could pass. it's pytorch limitation for jit.trace which only support tuple input now, Intel has commited a PR(https://github.com/pytorch/pytorch/pull/81623) for this and expected to be released in pytorch 1.14 (I also added it in doc).

If we would like to jit.trace successfully for such case, the other option is to modify the model like below, making forward input sequence like tuple input sequence..., 

```py
--- a/src/transformers/models/distilbert/modeling_distilbert.py
+++ b/src/transformers/models/distilbert/modeling_distilbert.py
@@ -731,11 +731,11 @@ class DistilBertForSequenceClassification(DistilBertPreTrainedModel):
     )
     def forward(
         self,
+        labels: Optional[torch.LongTensor] = None,
         input_ids: Optional[torch.Tensor] = None,
         attention_mask: Optional[torch.Tensor] = None,
         head_mask: Optional[torch.Tensor] = None,
         inputs_embeds: Optional[torch.Tensor] = None,
-        labels: Optional[torch.LongTensor] = None,
         output_attentions: Optional[bool] = None,
         output_hidden_states: Optional[bool] = None,
         return_dict: Optional[bool] = None,
```
 
"label smoothing" is just a smart way to walk around the jit.trace failure, since it happens to pop the labels from the input


We are not going to make a breaking change in the parameter order of every model. So basically the jit eval functionality added in #17753 does not work and has never worked for any model which contain labels, can you confirm?

Since it is an **evaluation** function, I fail to see the point of having it in Transformers until PyTorch supports it.

The key point of the jit error cases met here is that jit cannot well handle the case that the dictionary forward parameter order does not match the dataset input order, not specific to whether there are "labels" or not. And  to improve PyTorch jit ability to solve this issue, we landed  https://github.com/pytorch/pytorch/pull/81623 in PyTorch;

For the usage of model inference with jit, for now, there could be many cases that natively get the benefits, like models running question-answering example mentioned above;

For these failed model inferences with jit cases, we are capturing this with the exception here to make it fallback and use logging to notify users; Meanwhile, these failed cases shall work when PyTorch release contains this [feature](https://github.com/pytorch/pytorch/pull/81623), (expect in next release);

Besides, bringing "label smoothing" here with jit is not that reasonable since it would be confusing for users.



Hi, @sgugger to make it a clear, I file a issue to record the issue I meet https://github.com/huggingface/transformers/issues/19973. also I agree that "label smoothing" is a training skill and I have removed it in inference part. This PR could fix the error listed in https://github.com/huggingface/transformers/issues/19973

> You are a bit beating around the bush here: are there any models with a head where this feature can be used right now without hacks? I understand support in PyTorch is coming in the next version for dictionaries, but I think this feature was just added to early. Can the doc explicitly mention that the feature requires a nightly install?

Hi, sgugger 
for pytorch >= 1.14.0 (nightly version is 1.14.0). jit could benefit any models for predict and eval.
for pytorch < 1.14.0.  jit could benefit models like "Question and Answer"， whose forward parameter order matches the tuple input order in jit.trace. If we meet case like "text classification"，whose forward parameter order does not matches the tuple input order in jit.trace in evaluation, jit trace will fail and we are capturing this with the exception here to make it fallback and use logging to notify users

> Thanks for the precision. Could you add all of this to the documentation? Also have one last question on the actual code.

which document would you recommend to add this, since it's not cpu specific.

> which document would you recommend to add this

Every time the jit eval is mentioned.