Using MLflow on google colaboratory with github to build cosy environment: on VS code
Tags: vscode, googlecloud, python
Published: 2022-03-19 03:38:15

# Introduction
In some previous articles, I built a cosy environment to perform machine learning experiments with
- google colaboratory (to perform experiments)
- github (to manage source codes and information of each experiment)
- ngrok (to connect to mlflow window)

It was not bad, but how cosy the environment is small a little bit. There are two reasons:
1. The text editor on google colab is not cosy.
2. It is troublesome to run some cells when committing and pushing every time.

We can use a terminal if we buy an account of google colab pro [[official post on twitter](https://twitter.com/googlecolab/status/1336698772760379392?lang=en)], but I do not need a pro account for now except for those troubles. Then, what should I do? One of the answers is vscode with google colab.

Note that, I usually use windows 10, the following discussion is for windows.

# Process
I'll suppose that vs code is installed on the computer.

## Downloading cloudflare
1.Download the execution file from [[cloudflare zero trust](https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/installation/#windows)]

2.Rename the file name to `cloudflare.exe`

## Installing `remote-ssh` extension on vs code
1.Launch vs code

2.Install `remote-ssh` extension

Pressing `Ctrl + Shift + X`, the extensions tab is opened. Inputting `remote ssh` in the search box like below, the extension would appear.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/brao1ymw3fehcbml6xrz.png)

There it is! It is installed when the green Install button is clicked though there is no green Install button on the extension in the picture because I have already got it. 

3.Setup ssh config

There is the `config` file in the home directory, like `~/.ssh/config`. The file should be the following one.
```config
Host *.trycloudflare.com
    HostName %h
    User root
    Port 22
    ProxyCommand <absolute path to cloudflare.exe> access ssh --hostname %h
```

In my case, I put `cloudflare.exe` in C directory, so my ProxyCommand line is `ProxyCommand C:\\cloudflared.exe access ssh --hostname %h`.

## Preparing config data
1.Create `config` directory on the google drive

2.Create `general_config.yaml` and upload it to the `config` directory

The `general_config.yaml` must have the following information.
```yaml
github:
  username: username
  email: email
  token: access token
cloudflare:
  password: password_you_decided
```

`token` in `github` block cloud be got by following [[Creating a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)].


## Creating a new google colab notebook
1.Create a new google colab notebook to access google colab from vs code

2.Run the following codes

```python
# Prepare environment
!pip install colab_ssh --upgrade

# Import necessary modules
import os
from colab_ssh import launch_ssh_cloudflared
from google.colab import drive
import yaml

# Mount my google drive
drive_path = "/content/gdrive"
drive.mount(drive_path)

# Load general config
config_path = os.path.join(drive_path, "MyDrive", "config", "general_config.yaml")
with open(config_path, 'r') as yml:
  config = yaml.safe_load(yml)

# Set git config
config_github = config["github"]
!git config --global user.email {config_github["email"]}
!git config --global user.name {config_github["username"]}

# Create symbolic link
!ln -sfn /content/gdrive/MyDrive/ /root/workspace

# Launch ssh cloudflare
launch_ssh_cloudflared(password=config["cloudflare"]["password"])
```

The outputted cell has the following information.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nkiqf22z55wvc7a2ajvb.png)

3.Copy the url in `VSCode Remote SSH` block

## Accessing to google colab from vs code
1.Open command palette on vs code by pressing `Ctrl + Shift + P`

2.Input `Remote-SSH: Connect to Host...` in the box and press enter

3.Input the copied url in the box and press enter

4.Input `Continue` in the box and press enter

5.Input the password written in `general_config.yaml` in the box and press enter

There we go! The vs code has already been accessed to google colab. Press `Ctrl + Shift + @` and then, input `python -V` or `pip list`. We could see the same output as one when we run the same codes on a google colab notebook.

# Conclusion
Now, we could edit any file on vs code and also commit and push on the terminal of vs code. It is quite cosy for me. Furthermore, we cloud use mlflow that I introduced a little bit in the previous article. Just run `mlflow ui` in the terminal and open the outputted url in any browser.

I would appreciate it if someone shares tips in the discussion box below.