pipelines: Tentative fix for AutoModel for PegasusConfig.
Author: Narsil
Pull request: yes

# What does this PR do?

Original error lies in `pipeline(task='summarization',
model='google/pegasus-xsum')`.
 - Code fails while trying to infer framework from model_name (str).
 - It attempts to determine framework by running `AutoModel.from_pretrained(..)` then  `TFAutoModel.from_pretrained(...)` and decides by seeing whichever works first.


Proposed fix by:
-  implementing `AutoModel.from_pretrained('google/pegasus-xsum')` that is a
`PegasusConfig`. and returning a `PegasusForConditionalGeneration`.

Not sure if that's desirable as we are loading a
`ForConditionalGeneration` model by default (but it's the only available
anyway).

Other options that are available:

- load `BartModel` (Pegasus inherits from BartForConditionalGeneration)
from `PegasusConfig`, but unsure about side effects and odd to load
`Bart` from `Pegasus`.
- Change `get_framework` function from pipeline. That was my initial
choice but it seems understanding if a config is for a TF or Pytorch
model would require replicating some of `AutoModel` logic anyway
so doing that would lead to a discrepancy between the 2 code paths just
for Pegasus (and maybe BartConfig which also suffers some issues, but
that will be in a follow-up PR).



<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik
@sshleifer

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
After thinking about it a bit, I don't think the `PegasusForConditionalGeneration` should be going there. The `MODEL_MAPPING` is a mapping to all the headless models, i.e., the models that output hidden states without processing them in a head.

Introducing `PegasusForConditionalGeneration` would result in a mismatch between every single other models defined in that mapping and this newly added model. 

Adding `BartModel` would fail because of the configuration, as you've said, so imo the best thing to do here is to create a `PegasusModel` that inherits from `BartModel`, and use this in the `AutoModel`.

Why is summarization pipeline using AutoModel?  Shouldn't it require a model with a head?

The pipeline is using `AutoModel` to load the weights to see if they load. It follows the current installed platform (PT or TF), but if both are installed, it first tries to load the checkpoint in `AutoModel`, and if it fails (wrongly formatted weights), it tries to load it in `TFAutoModel`.

This does mean that the model is loaded twice (once in `AutoModel` and another time in the appropriate auto model), which may not be the best performance-wise.

The easy fix here is to add a base model for Pegasus (and all models should have base models imo), the somewhat more robust fix is to load the checkpoint directly in the appropriate auto model.

> This does mean that the model is loaded twice (once in AutoModel and another time in the appropriate auto model), which may not be the best performance-wise.

Yes this is not ideal. If there was simpler way do determine appropriate framework from config that would be much better. Or attempt the AutoModel way but without going the full way (stopping at checking filenames).

> The easy fix here is to add a base model for Pegasus (and all models should have base models imo), the somewhat more robust fix is to load the checkpoint directly in the appropriate auto model.

That seems probably like the best solution. (at least in the short term)

> and all models should have base models imo

MarianMT, Pegasus, Blenderbot are all only published/trained/used for one task, why should they have base models?

What ever happened to `config.architectures`? Would that help?

Some configs (old ones maybe) don't have `architectures` defined.

> The pipeline is using `AutoModel` to load the weights to see if they load. It follows the current installed platform (PT or TF), but if both are installed, it first tries to load the checkpoint in `AutoModel`, and if it fails (wrongly formatted weights), it tries to load it in `TFAutoModel`.

Just a note that I'm not 100% sure that our design goal with Pipelines is to be able to load a model automatically in PT/TF without any user input (e.g. in case the model is only TF)

Besides, in most cases you would have access to the huggingface.co model list API so you would know if model has PT/TF files.

> Some configs (old ones maybe) don't have `architectures` defined.

Just a note that we can always backport architectures into the hosted config files (will be easier with the new model versioning system)

Obsolete.