Implement multiple span support for DocumentQuestionAnswering
Author: ankrgyl
Pull request: yes

# What does this PR do?

Extends DocumentQuestionAnswering pipeline to include support for multiple spans.

Fixes #18414


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@NielsRogge @sgugger @Narsil 

Comments:
I still need to extend the tests to set the `max_seq_len` to be low (e.g. 10) and make sure that they work.

@NielsRogge another thing we could do in this change, or a separate one if y'all prefer, is to support multiple pages. The [DocQuery pipeline](https://github.com/impira/docquery/blob/main/src/docquery/ext/pipeline_document_question_answering.py#L227) already does this, because it's common to have multi-page documents and quite useful to be able to search across them. It's important to have this in the pipeline itself, so that you can properly handle the "no answer" score and top-k across pages. The main consideration is how we change the input shape of the pipeline to accommodate it, since we currently accept a (single) image and (optional) list of word_boxes. The options I can think of are:

- Image can also be a list of images, and then `word_boxes` can be a list of list of word boxes
- Combine image and `word_boxes` into one list of `(image, [(word, box)])` tuples
- Add a new optional argument, called pages, and use that instead of the image/word_box arguments if it's set.


_The documentation is not available anymore as the PR was closed or merged._

gentle nudge @NielsRogge 

Hey @ankrgyl ,

Thanks for this PR !

Do you mind adding a few tests showcasing the switch to `ChunkPipeline` ? (Ideally a test that wouldn't work before and would work after this PR).

No need to add tests for ALL supported models, but at least one would go a long way.

Absolutely. Apologies for the delay -- I will update the PR within the next few days.

@Narsil apologies again for the delay. I wanted to make sure I had some time to test it thoroughly. I added tests for the two flavors of LayoutLM models, but not for Donut, since there are no spans for Donut (i.e. there is no parameter or argument I can provide that hits two different code paths with the Chunk Pipeline, at least as far as I can tell).

I found several small bugs along the way and fixed them too.

Hi @ankrgyl Thank you for this PR. We need some help for CI failure regarding this new addition 🙏 

After this PR, we have 5 CI failures
```bash
FAILED tests/pipelines/test_pipelines_document_question_answering.py::DocumentQuestionAnsweringPipelineTests::test_large_model_pt_chunk - AssertionError: Lists differ: [{'score': 0.9974, 'answer': '1110212019', 'start': 23, [69 chars] 16}] != [{'score': 0.9967, 'answer': '1102/2019', 'start': 22, '[67 chars] 15}]
FAILED tests/pipelines/test_pipelines_document_question_answering.py::DocumentQuestionAnsweringPipelineTests::test_large_model_pt_donut - TypeError: forward() got an unexpected keyword argument 'page'
FAILED tests/pipelines/test_pipelines_document_question_answering.py::DocumentQuestionAnsweringPipelineTests::test_large_model_pt_layoutlm_chunk - AssertionError: Lists differ: [{'sc[39 chars]t': 16, 'end': 16}, {'score': 0.9998, 'answer'[31 chars] 16}] != [{'sc[39 chars]t': 15, 'end': 15}, {'score': 0.9924, 'answer'[3...
FAILED tests/pipelines/test_pipelines_document_question_answering.py::DocumentQuestionAnsweringPipelineTests::test_pt_LayoutLMConfig_LayoutLMForQuestionAnswering_LayoutLMTokenizerFast_nofeature_extractor - IndexError: tuple index out of range
FAILED tests/pipelines/test_pipelines_document_question_answering.py::DocumentQuestionAnsweringPipelineTests::test_pt_LayoutLMConfig_LayoutLMForQuestionAnswering_LayoutLMTokenizer_nofeature_extractor - IndexError: tuple index out of range
```
  - (For `test_large_model_pt_layoutlm_chunk`, I got different error from our CI where it is just output not equal expected value)
  - The 2 tests with `_chunk` at the end are new tests. The other 3 tests worked well in previous commits before this PR
  - We need to install `tokenizers==0.13.1` (as it has some new fix on its own)
 
Could you take a look, please 🙏? You can find more detailed information [here](https://github.com/huggingface/transformers/actions/runs/3231701081/jobs/5291537526)



I'm so sorry about that! I will take a look ASAP.