Layoutlmv2 port with testing
Author: raguiar2
Pull request: yes

# What does this PR do?

Trying to open up my own PR to figure out what's wrong with the install on https://github.com/huggingface/transformers/pull/11933, as it seems to be working locally but not on CircleCI



<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)
#11932

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.  [microsoft/unilm#325](https://github.com/microsoft/unilm/issues/325)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation). Not yet
- [x] Did you write any new necessary tests?


## Who can review?

@LysandreJik 


@inproceedings{Xu2020LayoutLMv2MP,
title = {LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding},
author = {Yang Xu and Yiheng Xu and Tengchao Lv and Lei Cui and Furu Wei and Guoxin Wang and Yijuan Lu and Dinei Florencio and Cha Zhang and Wanxiang Che and Min Zhang and Lidong Zhou},
booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL) 2021},
year = {2021},
month = {August},
}

Comments:
Looking good! Feel free to ping me when you want a review :)

@LysandreJik so, the logic in this PR is ready to be reviewed (mostly from https://github.com/microsoft/unilm with a few bug fixes). The one problem I'm having with this PR is that the detectron2 library requires torch to be **already installed** to build the wheel - and doesn't list it as a dependency!

 I have searched through a bunch of python documentation, but I'm still not sure how we can force the torch install to occur before the detectron2 one in setup.py, so any help here would be appreciated if you've seen something like this before. I have also filed an issue (https://github.com/facebookresearch/detectron2/issues/3124) but I'm not sure it's in the scope of the library to support install without torch already on the system. 
 
I could edit the CI instead to add torch before running setup.py, but that seems like it would be error-prone down the road if people are trying to install without pytorch on their system and transformers fails to build. What are your thoughts on how to best solve this issue?

Copying the parts of the detectron library I needed to make layoutlmv2 work was something else I considered, but it is a substantial chunk of the detectron2 code, so I think it's better to just use the library.

@NielsRogge, you have played with LayoutLM in the past, do you want to give this PR a look?

If the install needs to be done as a two-step process (first all deps, then `detection2`) then I would advocate for not putting it in the `setup.py`, and thoroughly document the behavior, both in the documentation and in the code with the appropriate errors raised when a detectron-less install is detected.

Any idea when this PR is going to be merged? I am working on TF version of layoutlm 2 and I'd like for this to be merged before I create a branch for layoutlm 2 in TF

Hi @atahmasb, the author of this PR did not reply yet regarding my review, but maybe I could work on this in a new PR.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.