Adding Finnish BERT.
Author: haamis
Pull request: yes

We have trained BERT-base on Finnish text and wish to have it included in the library. Both cased and uncased models are available. You can see the paper [here](https://arxiv.org/abs/1912.07076) and a website for the model can be found [here](http://turkunlp.org/FinBERT/).

These changes passed all the relevant tests (\*auto\*, \*common\*, \*_bert_test\*) including `test_model_from_pretrained` with the Finnish models.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=h1) Report
> Merging [#2190](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e92bcb7eb6c5b9b6ed313cc74abaab50b3dc674f?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2190/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #2190   +/-   ##
=======================================
  Coverage   81.35%   81.35%           
=======================================
  Files         120      120           
  Lines       18254    18254           
=======================================
  Hits        14851    14851           
  Misses       3403     3403
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/tokenization\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2190/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rva2VuaXphdGlvbl9iZXJ0LnB5) | `96.38% <ø> (ø)` | :arrow_up: |
| [transformers/configuration\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2190/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2NvbmZpZ3VyYXRpb25fYmVydC5weQ==) | `87.09% <ø> (ø)` | :arrow_up: |
| [transformers/modeling\_tf\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2190/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3RmX2JlcnQucHk=) | `96.31% <ø> (ø)` | :arrow_up: |
| [transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2190/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2JlcnQucHk=) | `87.75% <ø> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=footer). Last update [e92bcb7...3c1aede](https://codecov.io/gh/huggingface/transformers/pull/2190?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This looks really awesome, thanks for sharing!

**Uskomatonta**!! (yes this is the only Finnish word I know)

The new recommended way of uploading the files is inside folders, that way you'll be able to 
do `AutoModel.from_pretrained("TurkuNLP/bert-base-finnish-[un]cased-v1")` out-of-the-box, without even having to modify the lib's code (though we may want to add "bert-base-finnish-[un]cased-v1" as a shortcut name anyways).

(We just [updated the documentation](https://github.com/huggingface/transformers/commit/855ff0e91d8b3bd75a3b1c1316e2efd814373764#commitcomment-36452545) this morning so this is very new)

Do you want to re-upload the files inside folders? Or I can do it on our side too.

Also the ArXiv link in the PR's post seems broken, is the paper not public yet?

Anyways, thanks for sharing, this is awesome!

> (We just updated the documentation this morning so this is very new)

Ah, just missed it then.

> Do you want to re-upload the files inside folders? Or I can do it on our side too.

Would be great if you could do it, thanks.

> Also the ArXiv link in the PR's post seems broken, is the paper not public yet?

The paper is scheduled to be announced on arxiv at 8pm EST so I'll fix the link tomorrow (in 12 hours or so).

Looks like a URL was incorrect in the original PR, fixed it.

Merging now, thank you again!

By the way, would you guys be interesting in beta-testing a new way of pre-training a tokenizer on a corpus, @haamis?

@n1t0 is working on something that might be of interest to you.

Also @haamis we're rolling out model and contributor pages: e.g. https://huggingface.co/TurkuNLP

Anything you would like to see added to this page? How can it be most helpful? Thanks!

> By the way, would you guys be interesting in beta-testing a new way of pre-training a tokenizer on a corpus, @haamis?
> 
> @n1t0 is working on something that might be of interest to you.

I saw there was a faster implementation of the tokenizer in the works, improved speed would be nice.

> Also @haamis we're rolling out model and contributor pages: e.g. https://huggingface.co/TurkuNLP
> 
> Anything you would like to see added to this page? How can it be most helpful? Thanks!

That looks good to me.

Sorry about the late reply.