LayoutLMv3Model output shape is different
Author: pocca2048
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.20.1
- Platform: Linux-4.4.0-62-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- Huggingface_hub version: 0.2.1
- PyTorch version (GPU?): 1.10.0+cu102 (True)
- Tensorflow version (GPU?): 2.7.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

@NielsRogge 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoProcessor, AutoModelForTokenClassification, AutoModel
from datasets import load_dataset

processor = AutoProcessor.from_pretrained("microsoft/layoutlmv3-base", apply_ocr=False)
model = AutoModel.from_pretrained("microsoft/layoutlmv3-base", num_labels=7)

dataset = load_dataset("nielsr/funsd-layoutlmv3", split="train")
example = dataset[0]
image = example["image"]
words = example["tokens"]
boxes = example["bboxes"]
word_labels = example["ner_tags"]

encoding = processor(image, words, boxes=boxes, return_tensors="pt")

outputs = model(**encoding)
encoding.input_ids.shape, outputs.last_hidden_state.shape
```
outputs
```
(torch.Size([1, 208]), torch.Size([1, 405, 768]))
```

### Expected behavior
```
(torch.Size([1, 208]), torch.Size([1, 208, 768]))
```

Hi! Thank you very much for contributing layoutlmv3 model to huggingface.

While using the model, I think I found out that the model has some different parts from the specs.

https://github.com/huggingface/transformers/blob/v4.20.1/src/transformers/models/layoutlmv3/modeling_layoutlmv3.py#L1043
https://github.com/microsoft/unilm/blob/master/layoutlmv3/layoutlmft/models/layoutlmv3/modeling_layoutlmv3.py#L1070

This huggingface implementation has different output shape than original implementation.
In the [documentation](https://huggingface.co/docs/transformers/v4.20.1/en/model_doc/layoutlmv3#transformers.LayoutLMv3Model), it says last_hidden_state has shape of `(torch.FloatTensor of shape (batch_size, sequence_length, hidden_size))` but it does not. (original implementation has, but huggingface implementation does not)
Sequence length includes 
(Presumably because of that) It makes different training result on FUNSD dataset.

In summary,
- `LayoutLMv3Model` outputs different shape (sequence length) than that is written in documentation.
- and that is different from the original implementation.

Thank you.



Comments:
Hi,

The sequence length of the last hidden states of LayoutLMv3 equals the number of text tokens + image tokens.

If you have a text of 208 tokens (as is the case in the code example above), LayoutLMv3 also appends 197 image tokens to it. There are 197 image tokens because `LayoutLMv3Processor` resizes images to 224x224, which, at a patch resolution of 16x16 gives (224/16)**2 = 196 tokens, and one adds one for the CLS token. 

This is also what is done in the original implementation. 

The docstrings of the last hidden states can be improved, definitely. Feel free to open a PR regarding this.

Oops my bad. I confirmed that both have the same shapes.
I thought two would have different shape because of these:
- transformers
https://github.com/huggingface/transformers/blob/d0acc9537829e7d067edbb791473bbceb2ecf056/src/transformers/models/layoutlmv3/modeling_layoutlmv3.py#L1041-L1043

- original
https://github.com/microsoft/unilm/blob/4301ebe1a832b7bcb33be0ab3a460306d467a912/layoutlmv3/layoutlmft/models/layoutlmv3/modeling_layoutlmv3.py#L1070-L1073
```python
        sequence_output = outputs[0]
        sequence_output = self.dropout(sequence_output)
        logits = self.classifier(sequence_output)
```

I will try to work on updating the documentation. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.