Add BERT Loses Patience (Patience-based Early Exit)
Author: JetRunner
Pull request: yes
Labels: name: Ex: Sequence Classification

Add BERT Loses Patience (Patience-based Early Exit) based on the paper https://arxiv.org/abs/2006.04152 and the official implementation https://github.com/JetRunner/PABEE

It's impossible to make PABEE's ALBERT and BERT compatible with the standard API (e.g., `run_glue.py` so I keep the modeling files in a separate directory, under `example/bert_loses_patience`, instead of putting them alongside `modeling_bert.py`)

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=h1) Report
> Merging [#5078](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e4aaa4580515446cd5a2972ab42fec0b95819c84&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5078/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5078   +/-   ##
=======================================
  Coverage   77.26%   77.26%           
=======================================
  Files         133      133           
  Lines       22146    22146           
=======================================
+ Hits        17110    17111    +1     
+ Misses       5036     5035    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5078/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `75.40% <0.00%> (+0.40%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=footer). Last update [e4aaa45...8e0cf02](https://codecov.io/gh/huggingface/transformers/pull/5078?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hi @LysandreJik , my worry is that PABEE is not a standard inference method. It cannot deal with batch inference. Also, it can only support classification & regression (no tagging, no summarization, etc.). 

From another aspect, as a researcher, when I try to do a little tweak with ALBERT, I won't be happy if someone adds some new stuff into the model and it'll add unnecessary burdens to the researchers. They'll definitely hate it. So IMO, it's better to implement them separately. Also, as @sshleifer suggested, I refactored the model to inherit `AlbertTransformer` and `AlbertModel`, etc.

> Hi @LysandreJik , my worry is that PABEE is not a standard inference method. It cannot deal with batch inference. Also, it can only support classification & regression (no tagging, no summarization, etc.).
> 
> From another aspect, as a researcher, when I try to do a little tweak with ALBERT, I won't be happy if someone adds some new stuff into the model and it'll add unnecessary burdens to the researchers. They'll definitely hate it. So IMO, it's better to implement them separately. Also, as @sshleifer suggested, I refactored the model to inherit `AlbertTransformer` and `AlbertModel`, etc.

Since I refactored the code with inheritance, I figure it is okay to use `adaptive_forward` since I won't have to overwrite the standard `forward` (which would be confusing since it's hard to tell which part is modified for the users). Also, it's better to preserve the standard `forward` so we can easily compare `adaptive_forward` to the standard `forward`.

@sshleifer I copy most stuff from the original ALBERT & BERT modeling code so I think it also does not make sense if I refactor the parts I copied.

Re. trainer, maybe we can refactor the code later? I think it's quite optional here but it requires a lot of changes on `run_glue_with_pabee.py`. Some part of me wonders if it is worthy.

The code you copied has gotten cleaned up since you copied it, hence the suggestions.

> LGTM pending suggestions, test.

Okay, I’ll add a test