get_linear_schedule_with_warmup Scheduler
Author: h56cho
Pull request: no

Hello,

When I try to execute the line of code below, Python gives me an import error:

```js
from pytorch_transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
                                  AdamW, get_linear_schedule_with_warmup)

ImportError: cannot import name 'get_linear_schedule_with_warmup' from 'pytorch_transformers' (/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/pytorch_transformers/__init__.py)
```

What should I then import to use the linear scheduler with warm up?

Thank you,

Comments:
You might be using an old version of the library, try updating it to v2.2.0

You're trying to import a method only available in more recent Transformers versions from a (very old) Transformers version called _pytorch-transformers_.

With Transformers 2.1.1 (the second recent one) and the new version 2.2.0, you can import correctly the `get_linear_schedule_with_warmup`. In fact, Transformers modifies its source code for what concern the optimization process (e.g. learning rate). You can see the changes [here](https://github.com/huggingface/transformers/commit/022525b0031bcdbbb62d1223f75919983f2ac426).

> Hello,
> 
> When I try to execute the line of code below, Python gives me an import error:
> 
> ```js
> from pytorch_transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
>                                   AdamW, get_linear_schedule_with_warmup)
> 
> ImportError: cannot import name 'get_linear_schedule_with_warmup' from 'pytorch_transformers' (/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/pytorch_transformers/__init__.py)
> ```
> 
> What should I then import to use the linear scheduler with warm up?
> 
> Thank you,

You should use the `transformers` library instead of the `pytorch_transformers`. The `get_linear_schedule_with_warmup` is only defined in the former, in its latest version.

Thank you all,

Hello,

So I installed transformers 2.2.0,
```
pip install transformers
```
and tried to import the same things:
```js
from transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
                                  AdamW, get_linear_schedule_with_warmup) 
```

and it's still giving me the same error:
```
from transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
                                  AdamW, get_linear_schedule_with_warmup)
2019-11-27 08:40:15.940560: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-11-27 08:40:15.954925: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe20d2f5a50 executing computations on platform Host. Devices:
2019-11-27 08:40:15.954938: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
Traceback (most recent call last):

  File "<ipython-input-1-99af21631e15>", line 1, in <module>
    from transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,

ImportError: cannot import name 'get_linear_schedule_with_warmup' from 'transformers' (/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/__init__.py)
```

What should I do to be able to use the linear warm up scheduler?

Thank you,

It's very strange.. In my environment works as expected the import statement.
```
> import transformers
> transformers.__version__
>>> '2.2.0'
> from transformers import get_linear_schedule_with_warmup
> ...
```
Please, share with us your **OS** and your **Python version**.

> Hello,
> 
> So I installed transformers 2.2.0,
> 
> ```
> pip install transformers
> ```
> 
> and tried to import the same things:
> 
> ```js
> from transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
>                                   AdamW, get_linear_schedule_with_warmup) 
> ```
> 
> and it's still giving me the same error:
> 
> ```
> from transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
>                                   AdamW, get_linear_schedule_with_warmup)
> 2019-11-27 08:40:15.940560: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
> 2019-11-27 08:40:15.954925: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe20d2f5a50 executing computations on platform Host. Devices:
> 2019-11-27 08:40:15.954938: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
> Traceback (most recent call last):
> 
>   File "<ipython-input-1-99af21631e15>", line 1, in <module>
>     from transformers import (GPT2Config, GPT2LMHeadModel, GPT2DoubleHeadsModel,
> 
> ImportError: cannot import name 'get_linear_schedule_with_warmup' from 'transformers' (/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/__init__.py)
> ```
> 
> What should I do to be able to use the linear warm up scheduler?
> 
> Thank you,

Hello,

I tried uninstalling transformers and install the module again and it works now.

Thank you for all your help,