comet_ml init weirdness
Author: stas00
Pull request: yes

Using a slightly bogus invocation of the following
```
cd examples/seq2seq
PYTHONPATH="../../src" BS=2 python finetune.py --data_dir cnn_dm --do_predict --do_train --eval_batch_size $BS --fp16 --fp16_opt_level O1--freeze_embeds --freeze_encoder --gpus 1 --gradient_accumulation_steps 1 --learning_rate 3e-5 --max_target_length 142 --model_name_or_path sshleifer/student_cnn_12_6 --n_val 500 --num_train_epochs 2 --output_dir distilbart-cnn-12-6 --tokenizer_name facebook/bart-large --train_batch_size $BS --val_check_interval 0.25 --val_max_target_length 142 --warmup_steps 500
```
I get:
```
Traceback (most recent call last):
  File "finetune.py", line 18, in <module>
    from callbacks import Seq2SeqLoggingCallback, get_checkpoint_callback, get_early_stopping_callback
  File "/mnt/nvme1/code/huggingface/transformers-comet_ml/examples/seq2seq/callbacks.py", line 11, in <module>
    from utils import save_json
  File "/mnt/nvme1/code/huggingface/transformers-comet_ml/examples/seq2seq/utils.py", line 22, in <module>
    from transformers import BartTokenizer, EvalPrediction, PreTrainedTokenizer, T5Tokenizer
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/__init__.py", line 22, in <module>
    from .integrations import (  # isort:skip
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/integrations.py", line 17, in <module>
    if comet_ml.config.get_config("comet.api_key"):
AttributeError: module 'comet_ml' has no attribute 'config'
```
no idea why this happens. this PR adds a band-aid - and then once I apply it - the real error shows up `$BS` wasn't defined - i.e. my args had an issue. But I need to see the real error and not some totally unrelated `comet_ml` error.

perhaps something else needs to be fixed.

While we are at it, once again I ended up with `comet_ml` w/o explicitly installing it.

So I again get to enjoy the incessant:
```
comet_ml is installed but `COMET_API_KEY` is not set.
```
:(

```
pipdeptree --reverse --packages comet_ml
```
doesn't give me the parent who pulled it in which is very odd, since it works for other packages.

Is there a different way to trace what pulled in a certain package?

If I install/update it explicitly it appears I already have the latest version:
```
pip install comet_ml -U
Requirement already up-to-date: comet_ml in /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages (3.2.5)
```

@sgugger, @dsblank 


Comments:
Thanks for the report... I'll take a look at this.

Again, if you find out how comet_ml got installed, that would be helpful.

> Again, if you find out how comet_ml got installed, that would be helpful.

Yay, I found at least one of them `pytorch-lightning`. https://github.com/PyTorchLightning/pytorch-lightning/blob/master/environment.yml#L49

I have a feeling `pipdeptree` misses packages not installed directly via pypi deps, but via a local `pip install -e .[dev]`

I probably should just go ahead and create the key - though I have no use for it - please let me know whether it'd serve better if I didn't and continued reporting any related problems.

Thanks for tracking down pytorch's dependencies. I see that that is for conda. What is weird though is that I can't figure out how `comet_ml.config` wouldn't be defined. In any event, this PR seems fine. (I'd like to get to the bottom of this at some point).

Remember that you don't have to set a `COMET_API_KEY` (unless you really want to log stuff). You can also set `COMET_MODE="DISABLED"`. 

I think it could be some fragile error handling. As I mentioned the crash happened when I mistakenly provided a bogus value to one of the pytorch-lightening clargs - so it was supposed to fail telling me that the argument was wrong, but instead failed with the error posted in OP. Once the band-aid was added it reported the error properly w/o crashing. 

I still have it re-producable w/o rebasing to this fix:

```
cd examples/seq2seq
BS=2; PYTHONPATH="../../src" python finetune.py --data_dir cnn_dm --do_predict --do_train --eval_batch_size $BS --fp16 --fp16_opt_level O1 --freeze_embeds --freeze_encoder --gpus 1 --gradient_accumulation_steps 1 --learning_rate 3e-5 --max_target_length 142 --model_name_or_path sshleifer/student_cnn_12_6 --n_val 500 --num_train_epochs 2 --output_dir distilbart-cnn-12-6 --tokenizer_name facebook/bart-large --train_batch_size $BS --val_check_interval 0.25 --val_max_target_length 142 --warmup_steps 500
```
giving:
```
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/util.py:36: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import namedtuple, Mapping, Sequence
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/vendor/graphql-core-1.1/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Traceback (most recent call last):
  File "finetune.py", line 18, in <module>
    from callbacks import Seq2SeqLoggingCallback, get_checkpoint_callback, get_early_stopping_callback
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/examples/seq2seq/callbacks.py", line 11, in <module>
    from utils import save_json
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/examples/seq2seq/utils.py", line 22, in <module>
    from transformers import BartTokenizer, EvalPrediction, PreTrainedTokenizer, T5Tokenizer
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/src/transformers/__init__.py", line 22, in <module>
    from .integrations import (  # isort:skip
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/src/transformers/integrations.py", line 17, in <module>
    if comet_ml.config.get_config("comet.api_key"):
AttributeError: module 'comet_ml' has no attribute 'config'
```
See if you get it yourself (pre-this PR)? or if you want me to try something let me know 


Hmm, actually the crash described in the OP happens all the time pre this PR. I even uninstalled and reinstalled `comet_ml` via `pip`.

I figured it out:

```
python -c "import comet_ml; print(comet_ml.config)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/__init__.py", line 34, in <module>
    from .api import API, APIExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/api.py", line 28, in <module>
    from .experiment import CommonExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/experiment.py", line 97, in <module>
    from .gpu_logging import (
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/gpu_logging.py", line 27, in <module>
    import pynvml
ModuleNotFoundError: No module named 'pynvml'
```
If I install `pynvml` the error disappers.

So basically `comet_ml` fails to load and the try: block ignores the exception.

The bug is somewhere in seq2seq utils (**edit**: doesn't seem to be the case)

Following the trace in https://github.com/huggingface/transformers/pull/8410#issuecomment-724281982

This does the right thing:
```
cd examples/seq2seq
PYTHONPATH="../../src" python -c "import utils"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/examples/seq2seq/utils.py", line 22, in <module>
    from transformers import BartTokenizer, EvalPrediction, PreTrainedTokenizer, T5Tokenizer
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/src/transformers/__init__.py", line 22, in <module>
    from .integrations import (  # isort:skip
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/src/transformers/integrations.py", line 16, in <module>
    import comet_ml  # noqa: F401
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/__init__.py", line 34, in <module>
    from .api import API, APIExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/api.py", line 28, in <module>
    from .experiment import CommonExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/experiment.py", line 97, in <module>
    from .gpu_logging import (
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/gpu_logging.py", line 27, in <module>
    import pynvml
ModuleNotFoundError: No module named 'pynvml'
```
but one above that imports `utils` eats the exception:
```
PYTHONPATH="../../src" python -c "import callbacks"
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/util.py:36: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import namedtuple, Mapping, Sequence
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/vendor/graphql-core-1.1/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2020-11-09 14:15:06.573247: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
/mnt/nvme1/code/github/00nlp/fairseq/fairseq/optim/adam.py:8: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Collection
(main-38) /mnt/nvme1/code/huggingface/transformers-master-stas00/examples/seq2seq [stas00/transformers|patch-3|+1?8]> PYTHONPATH="../../src" python -c "import callbacks"
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/util.py:36: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import namedtuple, Mapping, Sequence
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/vendor/graphql-core-1.1/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2020-11-09 14:15:31.366275: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
/mnt/nvme1/code/github/00nlp/fairseq/fairseq/optim/adam.py:8: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Collection
```


Oh, great find! Thanks for this... we'll take a further look on our end and get something out soon.

This is very odd, since I don't see any `try` blocks around this sequence of imports - except at the end inside `integrations.py` and even if I remove it there, the exception is still suppressed. `import utils` catches the error, but `import callbacks` which imports `utils` suppresses it (see https://github.com/huggingface/transformers/pull/8410#issuecomment-724312036).

Could this somehow be related to this:

```
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
```

Do you by chance mess something up with the importer there? it feels like something overrides `import` because suddenly it ignores import errors. (**edit**: ruled that out too - see the next comment)

It's possibly the doings of `PL`, observe this:
```
PYTHONPATH="../../src" python -c "import utils"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/examples/seq2seq/utils.py", line 22, in <module>
    from transformers import BartTokenizer, EvalPrediction, PreTrainedTokenizer, T5Tokenizer
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/src/transformers/__init__.py", line 22, in <module>
    from .integrations import (  # isort:skip
  File "/mnt/nvme1/code/huggingface/transformers-master-stas00/src/transformers/integrations.py", line 16, in <module>
    import comet_ml  # noqa: F401
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/__init__.py", line 34, in <module>
    from .api import API, APIExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/api.py", line 28, in <module>
    from .experiment import CommonExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/experiment.py", line 97, in <module>
    from .gpu_logging import (
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/gpu_logging.py", line 27, in <module>
    import pynvml
ModuleNotFoundError: No module named 'pynvml'
```
and now let's add `import pytorch_lightning` first:
```
PYTHONPATH="../../src" python -c "import pytorch_lightning; import utils"
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/util.py:36: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import namedtuple, Mapping, Sequence
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/wandb/vendor/graphql-core-1.1/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2020-11-09 14:35:24.485463: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
/mnt/nvme1/code/github/00nlp/fairseq/fairseq/optim/adam.py:8: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Collection
```

The exception was suppressed - so most likely it's PL's doing - most likely `import` functionality gets modified incorrectly.

Do you have enough @dsblank to proceed from here?  this is no longer `transformers`-related.

Yes, I'll take it from here. We have a fix, and I'll test some more tomorrow and probably have a new comet_ml release out shortly. Thanks again!

Oh, and I forgot to give the fully isolated command that reproduces the problem:
```
python -c "import pytorch_lightning; import comet_ml"
```
doesn't fail and it should

whereas:
```
python -c "import comet_ml"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/__init__.py", line 34, in <module>
    from .api import API, APIExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/api.py", line 28, in <module>
    from .experiment import CommonExperiment
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/experiment.py", line 97, in <module>
    from .gpu_logging import (
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/gpu_logging.py", line 27, in <module>
    import pynvml
ModuleNotFoundError: No module named 'pynvml'
```
as it should.