add qa and result
Author: pohanchi
Pull request: yes


Comments:
this is a template for albert qa(squad). all modification is on the directory "transformers/new_template ",    "examples/qa_albert/template"  the true tree structure you can refer to https://github.com/pohanchi/huggingface_albert, test squad 1.1 on albert_base and albert_xlarge is very close to the original paper  

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=h1) Report
> Merging [#1716](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/8a62835577a2a93642546858b21372e43c1a1ff8?src=pr&el=desc) will **decrease** coverage by `5.09%`.
> The diff coverage is `0%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1716/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master    #1716     +/-   ##
=========================================
- Coverage   85.14%   80.04%   -5.1%     
=========================================
  Files          94       99      +5     
  Lines       13920    14806    +886     
=========================================
  Hits        11852    11852             
- Misses       2068     2954    +886
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/new\_template/tokenization\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/1716/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL25ld190ZW1wbGF0ZS90b2tlbml6YXRpb25fYWxiZXJ0LnB5) | `0% <0%> (ø)` | |
| [transformers/new\_template/modeling\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/1716/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL25ld190ZW1wbGF0ZS9tb2RlbGluZ19hbGJlcnQucHk=) | `0% <0%> (ø)` | |
| [transformers/new\_template/optimization\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/1716/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL25ld190ZW1wbGF0ZS9vcHRpbWl6YXRpb25fYWxiZXJ0LnB5) | `0% <0%> (ø)` | |
| [transformers/new\_template/configuration\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/1716/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL25ld190ZW1wbGF0ZS9jb25maWd1cmF0aW9uX2FsYmVydC5weQ==) | `0% <0%> (ø)` | |
| [transformers/new\_template/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/1716/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL25ld190ZW1wbGF0ZS9fX2luaXRfXy5weQ==) | `0% <0%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=footer). Last update [8a62835...38bba06](https://codecov.io/gh/huggingface/transformers/pull/1716?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hi @pohanchi, great work on your ALBERT implementation! We're in the process of merging #1683 which already implements ALBERT, and would love your opinion on the implementation.

Because I don’t know how to upload the file to your team server(
https://amazonS3....), so something roughly method I use is that using
drive to store some big file (pytorch model state dict)and let people
download by themselves from drive. I just want to know that whether to do
that. By the way, I am a beginner of coderev .what is the converge mean ?
And the bigger is good or other XD.

On Tue, Nov 5, 2019 at 00:44 Lysandre Debut <notifications@github.com>
wrote:

> Hi @pohanchi <https://github.com/pohanchi>, great work on your ALBERT
> implementation! We're in the process of merging #1683
> <https://github.com/huggingface/transformers/pull/1683> which already
> implements ALBERT, and would love your opinion on the implementation.
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/1716?email_source=notifications&email_token=AIEAE4FDQIOBGB3ZCSXYFJ3QSBGOTA5CNFSM4JIPCM6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC75BDQ#issuecomment-549441678>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AIEAE4EGLCYTJRVDCKNDRRDQSBGOTANCNFSM4JIPCM6A>
> .
>


My Repository is using lamb optimizer to train albert model. If you use lamb optimizer, make sure adam_epsilon set up to 1e-12 and weight decay set to 0.1 this will help model become stable in training

also the learning rate need to small than 5e-5 
if large, sometimes performance will drop. 


That's nice to know, thanks @pohanchi 

Closing this since ALBERT is now in the library.