Add Visual Question Answering (VQA) pipeline
Author: sijunhe
Pull request: yes

# What does this PR do?

Add Visual Question Answering (VQA) pipeline, as described in #17208. The pipeline currently defaults to [ViLT](https://huggingface.co/docs/transformers/model_doc/vilt), which is also the only model it supports for now. 

It also adds all the necessary class such as `AutoModelForVisualQuestionAnswering`.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@NielsRogge @LysandreJik @Narsil @mishig25

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi,

Can you first make sure that your branch is up to date with the main branch? You can achieve that as follows:

```
git remote add upstream https://github.com/huggingface/transformers.git
git fetch upstream
git rebase upstream/main
```

Hi @sijunhe ,

Thanks a lot, I think for the last test (consistency), usually `make fixup` (make sure you have the right versions with `pip install -e .[dev]` if you want. It works for me most fo the time (don't include files you didn't touch I think).

> Looks like a solid PR ! Thank you for this.
> 
> The main thing I would think is actually remove some of the flexibility introduced here (modeling QA pipeline I think). As it exists mostly as legacy and is usually preventing some modifications later on because we can't do backward compatibilty breaking.
> 
> Aiming for a pure simple `pipe(image=image, question=question)` and `pipe({"image": image, "question": question})` should be enough and not need a full blown class to support.
> 
> The main reason for `{"image": image, "question": question}` is to support datasets which return single item and the reason for `pipe(image=image, question=question)` is just simpler for simple Python code (more natural than the dict let's say :)).
> 
> This will natively support `List[{"image": image, "question": question}]` which is handled by the parent class already (and as already mentionned datasets).
> 
> The code could look like something like
> 
> ```python
> def __call__(self, image, question=None, **kwargs):
>     if isinstance(image, (PIL.Image...)) and question is not None:
>         # Nice pure Python support.
>         inputs_ = {"image": image, "question": question}
>     else:
>         # Suppose this is correct dict or list.
>         inputs_ = image
> ```
> 
> What do you think ?

Thanks for the elaborate reply! I agree that the class is not necessary. This is my first PR so I wasn't sure what kind of inputs needed to be handled and I was mostly following the QA pipeline. :)
Hopefully all the tests pass now and we can land this soon!

This PR is fine for me !

I will let a core maintainer approve this.

@LysandreJik kindly pinging you for a final review

Hmm seems like the CI is flaky and it is failing now after I committed the trivial docstring change suggested by @NielsRogge 

> Hmm seems like the CI is flaky and it is failing now after I committed the trivial docstring change suggested by @NielsRogge

Don't hesitate to rebase on `main` too, might have been fixed within the code itself since .

Hey folks, seems like the last remaining issue was the use of community models as the default in the pipeline and the conclusion was to "specify a revision". However, I am not sure how to specify a version of the model. Any suggestions here? @patrickvonplaten @LysandreJik 

> issue was the use of community models as the default in the pipeline and the conclusion was to "specify a revision". However, I am not sure how to specify a version of the model. Any suggestions here? @patrickvonplaten @LysandreJik

Hey @sijunhe,

Very sorry to keep you waiting here! 

I'm in favor of merging this PR as is and to open a follow-up PR that specifies a revision for all default pipeline models (happy to take care of this early next week) 

@LysandreJik @sgugger would this be ok for you? 

@NielsRogge also good for you?

Thanks folks. I think we are ready to merge!

Thanks again for all you work on this!

@sijunhe great work! I will start the work on the widgets 👍 

@sijunhe the widgets are live on the hub! https://huggingface.co/dandelin/vilt-b32-finetuned-vqa
<img width="639" alt="Screenshot 2022-07-29 at 15 49 21" src="https://user-images.githubusercontent.com/11827707/182323267-e7eab74e-5d88-46e2-8ce6-b3409d21926d.png">