Bug: Unable to set remote path as default_root_dir
🐛 Bug

To Reproduce
Steps to reproduce the behavior:

Initialise Trainer with a remote defualt_root_dir "gs://bucket/test"
Call Trainer.fit()
See error

Traceback (most recent call last):
File "train_segmentation.py", line 40, in 
main()
File "train_segmentation.py", line 29, in main
trainer.fit(model)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/states.py", line 48, in wrapped_fn
result = fn(self, *args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1074, in fit
results = self.accelerator_backend.train(model)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/accelerators/gpu_backend.py", line 51, in train
results = self.trainer.run_pretrain_routine(model)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1170, in run_pretrain_routine
self.logger.save()
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py", line 27, in wrapped_fn
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 211, in save
save_hparams_to_yaml(hparams_file, self.hparams)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/core/saving.py", line 367, in save_hparams_to_yaml
with open(config_yaml, 'w', newline='') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'gs://bucket/test/lightning_logs/version_0/hparams.yaml'
After changing the open call to cloud_open repeating the above steps yields:
Traceback (most recent call last):
File "train_segmentation.py", line 40, in 
main()
File "train_segmentation.py", line 29, in main
trainer.fit(model)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/states.py", line 48, in wrapped_fn
result = fn(self, *args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1016, in fit
results = self.accelerator_backend.train()
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/accelerators/gpu_backend.py", line 54, in train
results = self.trainer.run_pretrain_routine(model)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1165, in run_pretrain_routine
self.logger.log_hyperparams(ref_model.hparams)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py", line 27, in wrapped_fn
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 175, in log_hyperparams
self.log_metrics(metrics, 0)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py", line 27, in wrapped_fn
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 189, in log_metrics
self.experiment.add_scalar(k, v, step)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/base.py", line 450, in experiment
return get_experiment() or DummyExperiment()
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py", line 27, in wrapped_fn
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/base.py", line 449, in get_experiment
return fn(self)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 141, in experiment
self._experiment = SummaryWriter(log_dir=self.log_dir, **self.kwargs)
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 115, in log_dir
version = self.version if isinstance(self.version, str) else f"version{self.version}"
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 230, in version
self._version = self._get_next_version()
File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 243, in get_next_version
existing_versions.append(int(d.split("")[1]))
ValueError: invalid literal for int() with base 10: '0/'
Code sample
Expected behavior
Default log saving behaviour is observed in the remote bucket
Environment
Docker Container:
FROM pytorch/pytorch:latest
RUN apt-get update --fix-missing && 
DEBIAN_FRONTEND="noninteractive" apt-get install -y git libglib2.0-0 && 
apt-get clean
RUN pip install git+https://github.com/XDynames/pytorch-lightning.git@master --upgrade

PyTorch Version 1.6:
OS (e.g., Linux): Linux
How you installed PyTorch: pip install git+https://github.com/XDynames/pytorch-lightning.git@master --upgrade
Python version: 3
CUDA/cuDNN version: 10.1/7
GPU models and configuration: 1xV100
Any other relevant information: Full tensorflow install to enable tensorboard cloud support

Additional context