Blenderbot
Author: sshleifer
Pull request: yes

Continued from https://github.com/huggingface/transformers/pull/4803 
Co-authored by @mariamabarham 

New models: `facebook/blenderbot-3B` and `facebook/blenderbot-90M`.

They produce similar, but not always identical outputs to their facebook counterparts, with the differences due to length penalty implementations.

They are identical to bart, besides one layernorm change for the blenderbot 90M checkpoint

```
if self.do_blenderbot_90_layernorm:
    x = self.layernorm_embedding(x)
    x += positions
else:
    x += positions
    x = self.layernorm_embedding(x)
```

I also wrote a [gist](https://gist.github.com/sshleifer/cb245b8739420724a32fc0c22344aee0) explaining the various layernorm sequences. Will update it once this is finalized.

The blenderbot 3b tests can run on 1 GPU, but are ridiculously slow on CPU.
Additionally, `test_feedforward_chunking` and `test_model_outputs_equivalence` were flaky locally, and are currently skipped.

#### Done
[x] forward pass in one file
[x] passing integration tests

#### TODO:

- [ ] `blenderbot.rst`
- [ ] model cards




### Ways to avoid new if statement
- Don't port bbot-90m. 
- separate `Blenderbot90Model`.
- There are also solutions where we parametrize out EncoderLayer/DecoderLayer, but these seem more confusing/harder to understand/less consistent to me.

Comments:
I don't really understand why we don't completely separate Blenderbot from Bart here. I thought we kind of agreed on not adding any if statements to existing models (also if it's only one) to make them work with new models.

With @sgugger's recent PRs that completely separate model files from each other (Roberta, Longformer, Electra from BERT), I don't see why we would not do the same here?

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=h1) Report
> Merging [#7418](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/7296fea1d689f47de69fd45e438e42d65ca5a393?el=desc) will **increase** coverage by `1.89%`.
> The diff coverage is `96.66%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7418/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7418      +/-   ##
==========================================
+ Coverage   76.45%   78.35%   +1.89%     
==========================================
  Files         181      184       +3     
  Lines       35781    35928     +147     
==========================================
+ Hits        27355    28150     +795     
+ Misses       8426     7778     -648     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_blenderbot.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmxlbmRlcmJvdC5weQ==) | `95.83% <95.83%> (ø)` | |
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.39% <100.00%> (+0.01%)` | :arrow_up: |
| [src/transformers/configuration\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2F1dG8ucHk=) | `96.34% <100.00%> (+0.04%)` | :arrow_up: |
| [src/transformers/configuration\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2JhcnQucHk=) | `94.11% <100.00%> (+0.11%)` | :arrow_up: |
| [src/transformers/configuration\_blenderbot.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2JsZW5kZXJib3QucHk=) | `100.00% <100.00%> (ø)` | |
| [src/transformers/modeling\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19hdXRvLnB5) | `86.08% <100.00%> (-0.97%)` | :arrow_down: |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `94.98% <100.00%> (+0.87%)` | :arrow_up: |
| [src/transformers/modeling\_blenderbot.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ibGVuZGVyYm90LnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/modeling\_mbart.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tYmFydC5weQ==) | `100.00% <100.00%> (ø)` | |
| [src/transformers/tokenization\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYXV0by5weQ==) | `92.64% <100.00%> (+0.10%)` | :arrow_up: |
| ... and [23 more](https://codecov.io/gh/huggingface/transformers/pull/7418/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=footer). Last update [7296fea...978290a](https://codecov.io/gh/huggingface/transformers/pull/7418?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Spent 2H getting Blender+Bart docs more consistent.

`blenderbot.rst` just points there instead of repeating.

![image](https://user-images.githubusercontent.com/6045025/94772210-23249380-0387-11eb-964a-a4d0337eae99.png)


Thanks for all the help @sgugger and @LysandreJik and sorry for being difficult.

Thanks for cleaning the docstrings and making all nice and shiny to go with the rest of the docs!
Hopefully it's going to be easier now that the templates have been updated.

<img src="https://media.giphy.com/media/osjgQPWRx3cac/giphy.gif"/>

@sshleifer @stephenroller is there a particular reason why the 9.4B one wasn't ported over?

I know it was mentioned in the paper that the 9.4B wasn't statistically any better than the 2.7B one in human evaluations, but it'd still be a useful release IMO.

I think it was just a matter of prioritization? I didn't directly work on it.

I can only speak for myself, not HF, but I would welcome a PR adding support for the 9.4. It should only be a configuration change compared the 2.7, I would think.

@stephenroller I see, can you point me to the 9.4B model artifact? I'll see if I can load that using the HF class with some tweaks.

If you manage to load it, we'd love to host it on https://huggingface.co/facebook 

cc @patrickvonplaten and others

Think @patil-suraj is working on it :-)

Hey @patil-suraj are you actively working on this and have an ETA? I want to start playing with the 9.4B within HF asap.

The files model files are in this tarfile:

https://dl.fbaipublicfiles.com/parlai/_models/blender/BST9B.tgz

Compared to the 2.7B, the following hyperparameters are expected to change (2.7B setting -> 9.4B setting):
- embedding size: 2560 -> 4096
- hidden state size (ffn size): 10240 -> 16384
- number of encoder layers: 2 -> 4
- number of decoder layers: 24 -> 32
- number of heads: 32 -> 32 (unchanged, but one might expect it to)

The dictionary and formatting is _exactly_ the same as the 2.7B model.

Hi @g-karthik , I'm working on it, it should be on hub by the end of next week

@patil-suraj can you please share the PR so I can take a look? I do not see the model on the model hub.