Add splinter
Author: oriram
Pull request: yes

# What does this PR do?

[Splinter](https://arxiv.org/abs/2101.00438) implementation
@patil-suraj @LysandreJik @patrickvonplaten 

Comments:
Thanks a lot for the PR! Think we can merge this soon :-) Some points that I think will be important to adapt before merging are:

- Simplify the logic of `splinter_qass` vs `new_splinter_qass`. IMO there should only be one `splinter_class` class attribute and if this has to be reinitialized or set to 0 we could instead add a `reinit` function. I don't really understand why we need to identical `splinter_qass` and `new_spliter_qass` modules
- Make sure we don't have hardcoded id's such as 102 in the model
- Add a QA integration test to make sure the model works as expected

@patil-suraj @patrickvonplaten @sgugger 
Any idea why this exception is raised when calling `make quality`?
```
Traceback (most recent call last):
  File "/mnt/c/Program Files/JetBrains/PyCharm 2019.3.2/plugins/python/helpers/pydev/pydevd.py", line 1434, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/mnt/c/Program Files/JetBrains/PyCharm 2019.3.2/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/mnt/c/Users/ori/Desktop/Ori/Research/transformers_splinter/utils/check_copies.py", line 353, in <module>
    check_copies(args.fix_and_overwrite)
  File "/mnt/c/Users/ori/Desktop/Ori/Research/transformers_splinter/utils/check_copies.py", line 186, in check_copies
    new_diffs = is_copy_consistent(filename, overwrite)
  File "/mnt/c/Users/ori/Desktop/Ori/Research/transformers_splinter/utils/check_copies.py", line 164, in is_copy_consistent
    theoretical_code = blackify(lines[start_index - 1] + theoretical_code)
  File "/mnt/c/Users/ori/Desktop/Ori/Research/transformers_splinter/utils/check_copies.py", line 104, in blackify
    result = black.format_str(code, mode=black.FileMode([black.TargetVersion.PY35], line_length=119))
  File "/home/oriram/venv/transformers_splinter/lib/python3.7/site-packages/black/__init__.py", line 1063, in format_str
    src_node = lib2to3_parse(src_contents.lstrip(), mode.target_versions)
  File "/home/oriram/venv/transformers_splinter/lib/python3.7/site-packages/black/__init__.py", line 1171, in lib2to3_parse
    raise exc from None
black.InvalidInput: Cannot parse: 15:4:     def __init__(self, config, add_pooling_layer=True):
```
Didn't happen before. Tried to debug it, but couldn't understand the cause.
Also, it seems like it's not in any file related to Splinter, as I don't have the argument `add_pooling_layer`.

@sgugger 
One idea I had in mind regarding the `self.splinter_qass` and `self.new_splinter_qass` was just to create two more checkpoints (so overall there will be 4 rather than 2): 
```
tau/splinter-base-with-qass 
tau/splinter-base
tau/splinter-large-with-qass 
tau/splinter-large
```
In the ones without qass, I'll drop the weights from the state_dict at `pytorch_model.bin`. 
This will also keep the message `All the weights of SplinterForQuestionAnswering were initialized from the model checkpoint at X` correct.
Does that sound OK?

I imagine there will be an associated config parameter to determine which layer use?
It looks like a good idea.

As a result of #13023 , you will need to rebase your PR on master and solve the merge conflicts (basically, you will just need to re-add the models/tokenizers and config in the auto-mappings as strings). Let us know if you need any help with that.

Hi @sgugger @patil-suraj @patrickvonplaten,
Sorry to bother you, but I couldn't find the reason as to why the `run_tests_tf` (and the two others) fail.
CircleCI doesn't provide much info..
Other than that and the rebase, I think everything is set to do the PR, took care of:

- Removing abstractions from Bert classes
- Separating the models into `splinter-base` and `splinter-base-qass` (and same for `large`), as well as removing all references to `config.initialize_new_qass` from the code
- Added an integration test
- Removed all unnecessary classes (`SplinterForMaskedLM` etc.)
- Fixed `Copy from` issues
etc.

Thanks!! 

Thanks @sgugger for your quick response!
I think the tests are now failing due to the rebase issue, as the errors don't seem related to Splinter.
Any chance you can help with the rebase? Don't have any experience with that..
Also, I noticed that you made some changes to the structure of the `auto_..` classes..
Would really appreciate it :)

Hello @oriram, I just took care of the merge and the auto-classes

Many thanks @LysandreJik @sgugger!!
Are we ready to merge then?

Hi @LysandreJik :)

- Changed copyrights in Splinter's 4 files
- Regarding your comments on `SplinterTokenizer` - The difference stems from dealing with the special `[QUESTION]` token which is used for building question representations

Many thanks!

Great @LysandreJik!
Let's merge? :)

@patil-suraj @sgugger @LysandreJik @patrickvonplaten 
Just wanted to say many thanks again for all your effort in this PR!!

@oriram - thanks a mille for your great PR! Let's try to promote Splinter so that people see its power for QA :-)