Output hidden states
Author: drjosephliu
Pull request: yes
Labels: name: Core: Modeling

Attempts to close issue #3879 by refactoring all models to take in an extra argument `output_hidden_states` in the `forward()` method. 

@patrickvonplaten 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=h1) Report
> Merging [#4978](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/68e19f1c228c92d5d800533f558faff24b57127a&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `93.85%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4978/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4978      +/-   ##
==========================================
+ Coverage   77.93%   77.94%   +0.01%     
==========================================
  Files         137      137              
  Lines       23475    23511      +36     
==========================================
+ Hits        18295    18326      +31     
- Misses       5180     5185       +5     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_mmbt.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tbWJ0LnB5) | `22.11% <ø> (ø)` | |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `94.78% <ø> (ø)` | |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `25.82% <0.00%> (+0.16%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `74.48% <ø> (ø)` | |
| [src/transformers/modeling\_tf\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG5ldC5weQ==) | `79.85% <64.28%> (-0.19%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `93.20% <85.71%> (-0.12%)` | :arrow_down: |
| [src/transformers/modeling\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19hbGJlcnQucHk=) | `80.43% <100.00%> (-0.05%)` | :arrow_down: |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `96.23% <100.00%> (-0.02%)` | :arrow_down: |
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `88.21% <100.00%> (ø)` | |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `99.14% <100.00%> (ø)` | |
| ... and [24 more](https://codecov.io/gh/huggingface/transformers/pull/4978/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=footer). Last update [68e19f1...ddaeb44](https://codecov.io/gh/huggingface/transformers/pull/4978?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@drjosephliu - this is really great work! I added two additional tests for output hidden states that have to pass for all models. Can you fix the remaining models? :-) I think after that we are good to merge!

This is great, looking forward to having this in master:)

Just pushed the fixes. Fingers crossed for a hole in one here !

Looks like one more merge conflict :(. Otherwise LGTM, epic contribution! Also patrick has left at least one unresolved comment w.r.t Pipfile

I added the refactorisation for mobilebert, so the relevant tests should be passing now. However, there are a bunch of tokenisation tests which are now failing instead and I can't figure out why as I'm pretty certain the changes I made didn't affect the tokenisers whatsoever. FWIW, those tests were already failing on master, so I hope they're unrelated to the changes I've made. Anyways, let me know if there are any additional edits that need to be made.

P.S. what a coincidence to bump into you here @sshleifer ! Glad to see you settling in here.

Haha small world!

I think there is a git issue causing it to appear that 188 files are changed in the PR.

I don't think it's horribly damaging, (the LHS looks wrong, the RHS looks correct  in the diff viewer), but if you have an easy way to resolve it that would be nice.


Well crap, it seems like it's gotten worse. I'm out of ideas here, because I've `git fetch upstream` and `git merge upstream/master`, so it's telling me everything's up to date. It seems like it's comparing against an old version of master that's a couple commits ago. Perhaps I can try deleting my local master, creating and pulling a new master, merging output_hidden_states into master and then push and make a new PR. I'm wondering if that will work? Unless you have any other ideas.

Ok, so with a bit of git ninja'ing, it's no longer showing ~200 files changed. Hoping this works now.

@drjosephliu - Amazing work! Thank's a lot for this. Really helps the library to become more flexible :-) 

The PR is good to merge for me.

Pinging @LysandreJik to verify since it's a big one. Should be merged today though I hope :-) 

Awesome, no problem and great working with y'all !