New serving
Author: jplu
Pull request: yes

# What does this PR do?

This PR proposes a new way to create a saved model to be properly served via TF Serving. The logic behind is to create a `serving` that will be used to create the expected saved model with a proper input signature. Currently the saved models are very limited:

- input sequence length limited to exactly 5 tokens
- input parameters limited to have only `input_ids`
- When `output_attentions` or `output_hidden_states` was set to True, the saved model output contained as many outputs as the number of attentions or hidden state

This PR fixes these 3 issues. A new behavior is also introduced, when doing `model.save_pretrained(...)` a saved model version is also created in same time than the `.h5` weights file.

The proposed logic allows anybody to create its own input signature simply by overwriting the new `serving` method. For example, the default inputs for BERT are now `input_ids`, `attention_mask` and `token_type_ids`, if one wants to replace `input_ids` by `inputs_embeds`, a new model has to be created overwriting the `serving` method like:

```
class CustomBertModel(TFBertModel):
    @tf.function(
        input_signature=[
            {
                "inputs_embeds": tf.TensorSpec((None, None, 768), tf.float32, name="inputs_embeds"),
                "attention_mask": tf.TensorSpec((None, None), tf.int32, name="attention_mask"),
                "token_type_ids": tf.TensorSpec((None, None), tf.int32, name="token_type_ids"),
            }
        ]
    )
    def serving(self, inputs):
        output = self.call(inputs)

        return self.serving_output(output)

model = CustomBertModel.from_pretrained("bert-base-cased")
model.save_pretrained("saving_path")
```
Slow/quick tests are passing.

EDIT: ping @sgugger @patrickvonplaten and @LysandreJik for review.

Comments:
Humm looks like the quick test I have added to make sure that a saved model can be properly created is a bit too long for at least one of the models. @LysandreJik any idea how I can figure out for which model?

Ok, the test `test_saved_model_creation` is skipped if it needs more than 30sec to be executed for a model. For now these models are skipped:
BART
BlenderBot
Funnel
Longformer
Lxmert
Marian
MBart
Mobilebert
Pegasus
T5

Let's see if the test becomes faster once I will optimise these model like I did for BERT. LGTM!

Cool, maybe @sgugger can take a look as well :-) 

Oops forgot to rebase and then the changes for the LED model is missing, and also the changes in the Seq2Seq template. Please wait my next push before merging.

I should have addressed all the comments. The saved model creation tests are silent for the Seq2Seq models until I find a proper fix.

Great should we merge @jplu  @LysandreJik @? - it's blocking the TF-Bart Split PR a bit. 

For me it is good to merge if there are no other comments :)

Cool merging then

I had some more comments actually! With the short names, most of the outputs of the serving methods fit on one line now. black does not put things back on the same line once it has split on several, so it's not fixed by the quality scripts.

I also think it would make future maintenance easier to add the # Copied from comments for dupe code.

They are all on one line (when it is possible, which means not too many characters to fit in).

I will open a PR to take care of adding the `#copied from` comments once I finish to fix the S2S models.

See comment above, and this is just one example, most of those now fit in one line with your last changes.