Setup Your Own Deep Learning Sandbox: Quick Guide | by Thomas Morris | Towards Data Science

Like many data science enthusiasts I dove into this field by filling every spare minute learning the DS eco-system –namely Python, Jupyter, NumPy, Pandas, Scikit-learn, Matplotlib, and a dash of Seaborn. Together with the all powerful Unix tools –vi, less, sort, cut, grep, cat, tail, head, etc.– you have an incredibly productive framework to source, clean and analyze datasets, engineer features, run regression models, and build professional visuals. However, once your curiosity (or job) pulls you towards the “deep learning” end of the “machine learning” world, you will quickly discover –as I did– not even a high-end Macbook Pro will have enough muscle to run even the most basic convolutional neural net models (CNN for short). I figured this out quickly working through the fast.ai course by Jeremy Howard and Rachel Thomas. If you are like me and your data science training funding comes from your own pocket then I have an inexpensive, simple, yet powerful solution for you.
Firstly, I admit I like running everything I can on my laptop. However, for some things my laptop just doesn’t cut it e.g. for my music composition hobby, I have a beefed up Intel i7 box in my basement that runs a music library server. But for training DL models an i7 and lots of RAM won’t do much for you. You need GPUs. And GPUs are expensive. But they’re not expense to rent. With the buffet table of cloud options now available in the marketplace my new MO has become, don’t buy, rent.
Through a bit of Google’ing, and trial-and-error I setup a Jupyter based client-server sandbox using my Macbook Pro and Visual Studio Code as the client, and Jupyter server running on a high-end Linux instance on Google Cloud. The results? My DL model training now happens in seconds. On my Macbook Pro? Well, after 30 minutes running and the fan blazing away I killed the process. You get the picture?
Note: This approach uses a full Linux server in the Google cloud. There are other paths that cost less e.g. Google Colab, and Kaggle Kernels. However, with this approach you gain configuration flexibility, a $300 credit (if used sparingly will last a while), and some experience with the Google Cloud Platform and using their command line interface.
First step is to create an account on the Google Cloud Platform (GCP). I’m going to assume you’ve figured that out and are logged into to the GCP console. Next, check to ensure you got your $300 credit. In the top left corner click on the menu and select Billing → Overview. In the bottom right corner of the Overview you should see the Promotional credits balance. And it should be $300.
GCP now has a service for provisioning a VM server configured specifically for running Jupyter notebooks. Click on the GCP menu (top left corner) and scroll down to the ARTIFICIAL INTELLIGENCE section. Then select AI Platform → Notebooks.
From the Notebook Instances screen select NEW INSTANCE at the top. This menu has many options including some beta and experimental choices. I am going with the PyTorch 1.4 option with one NVIDIA Tesla T4 GPU.
Next screen presents options. For this exercise I am going with the defaults. The Customize button allows you to add more disk space to your boot drive, add GPUs RAM, etc. The default costs $0.382 per hour. Google E2 instances are billed by the second after the first minute of usage. This default configuration will run 38.2 cents per hour. That’s 785 hours of free usage.
Once you click on Create, Google will take a couple minutes to provision your VM instance. When it’s complete you will see a screen like the following.
You now have a fully functioning Jupyter Notebook with a powerful GPU to support DL model building. Click on OPEN JUPYTERLAB next to your Instance Name. A new tab will open in your browser and a Jupyter notebook will be opened. The interface is very straight forward. You can browse the file system, create a Git repository, or startup up a terminal session.
At this stage you have a fully functioning, powerful DL sandbox. However, this is not a cost effective way to work. Your new VM instance is eating away at that $300 credit every second you have the instance running. I prefer to do all my data gathering, feature engineering, and initial regression work locally on my macbook. Then when I have my dataset cleaned and prepped and I am at the stage to try and fit a DL model I crank up my VM instance, connect, run my model, get my results, then shutdown the instance. To execute this workflow efficiently there’s a few more steps I will now walk through.
Go back to the Notebooks console and shutdown your instance.
Tip: Go to the Billing section and under Budgets & alerts create a Budget that will email you when you have reached $1 of usage.
To open up remote access to your VM instance you need to do two things.
Step 1. Open up HTTP access to your VM instance. From the main console menu, under the COMPUTE section, select Compute Engine → VM instances.
Then click on your instance from the list. The details of your instance will be displayed. We need to make two changes here. Click on EDIT at the top, scroll down and check the two HTTP boxes in the Firewalls section, then scroll on to the bottom and click Save.
Step 2. Follow the menu below and then click CREATE FIREWALL RULE at the top of the Firewall console.
Input the noted fields in the following two screens.
Your firewall rule should now be active. If it isn’t you will know in the next steps.
Step 3. Stop your VM Instance. Go back to the Notebooks console click the check box next to your instance and click on STOP.
Step 1. First step is to install the SDK on you local computer. Here’s the Google instructions for downloading and installing their SDK for MacOS, Windows, and Linux. Make sure you follow all steps including putting the SDK in your PATH.
Step 2. Initialize the SDK by running the following command from a terminal.
> gcloud init...> You must log in to continue. Would you like to log in (Y/n)?
You will be prompted to log in. When you press Y and Return you will be redirected to a Google website to log into your account. Once you successfully log in you should be returned to your terminal session where you will be prompted to select a compute region.
> Do you want to configure a default Compute Region and Zone? (Y/n)?
Select Y. In one of the earlier screen shots I suggested noting your instance name and the region in which it was created. Find that region in the list that is presented and enter its index number. Once that completes run the following command.
> gcloud config list
You should see something like the following in your terminal window.
Step 3. Before we left our VM instance we shut it down. Let’s now start it back up using the SDK. Here’s the command. Replace inst_name with the name of your instance.
> gcloud compute instances start inst_name
Now let’s practice stopping the instance. Here’s the command.
> gcloud compute instances stop inst_name
This can take a couple minutes so just let it run. After the little spinner thing completes you should get this as a result.
Once you see the ...done then startup your instance again and let’s move on.
Step 4. Time to connect to the instance and finish the server side setup. Enter the following command and replace inst_name with your instance.
> gcloud compute ssh inst_name
You will be prompted for a password. This is the account password you created initially. Next you will see a screen like the following.
Okay, we are almost there. Just a few configuration changes to Jupyter and then we will be able to connect with Visual Studio Code and run a model. Yeah! 😎
Step 1. Create a folder for notebooks. This is where Jupyter will store notebook files. I also recommend creating two subfolder whose purpose will become clear when we run the sample model below.
> mkdir jnotebooks> mkdir jnotebooks/data> mkdir jnotebooks/pics
Step 2. Create Jupyter configuration. This creates a .jupyter folder off your root directory.
> jupyter notebook --generate-config
Now for a little tricky part. We need to configure a few Jupyter parameters in the file .jupyter/jupyter_notebook_config.py. I recommend using vim. Before editing the file note your root path by executing pwd. Here’s the parameters. All these parameters are already in the file. However, they are commented out. They need to be uncommented and modified.
c.NotebookApp.ip = '*'c.NotebookApp.port = 8888c.NotebookApp.notebook_dir = '/home/username/jnotebooks'c.NotebookApp.open_browser = False
Remember we created the jnotebooks folder. Enter its full path for the notebook_dir parameter.
Here’s a short vim cheatsheet of the commands you will need.
Step 3. Set a Jupyter password. This is optional but highly recommended since we just opened HTTP access. From the command line enter the following and then enter your password. You will have to enter this every time you access this server from VSC so I recommend not going overboard here.
> jupyter notebook password
Step 4. Final step here. Let’s startup the Jupyter Notebook server. Here’s the command.
> jupyter-notebook
If your screen doesn’t closely resemble this then kill the process by pressing Control-c and go back and check the four parameters in the Jupyter config file.
I am a big Visual Studio Code fan. Microsoft open-sourced it some time ago and it has since evolved quickly to become a very user friendly and highly functional editor. And the thing I find really cool is that it now fully supports Jupyter Notebooks. I personally am not in love with running notebooks in a browser when I am in work mode. I prefer a real code editor that I use for Python and Java (and hopefully Swift will be supported soon). In VSC I can build my models in notebooks and then easily export to a Python file and start moving towards production’izing my project, all in the same editor. Enough rambling. Let’s connect to our server and train a model.
Step 1. Install and setup Visual Studio Code. If you use an Anaconda distribution then you are all set. Anaconda and VSC work well together. Otherwise click here to get the latest version for your operating system. VSC supports Jupyter natively so there’s nothing extra that needs to be installed.
Step 2. Open VSC and create a new Jupyter notebook. The easiest way to do this is by using the command browser. On MacOS the command is shift+command+p. Then type in Python: Create New Blank Jupyter Notebook
Step 3. Connect to your server. Did you note your external IP address when you started up your server? You’ll need it here. Once again access the command browser via shift+command+p. Then enter the command Python: Specify local or remote Jupyter server for connections
You see this next.Select Existing.
Here’s where you need the external IP address of you VM instance.
After entering your http address above and pressing return there’s a chance VSC will require a restart. If that does happen then close VSC, start it up again, and then follow this Step 3 back to this point. Type in some test Python code and execute the cell. VSC will send the code to the server and will display the returned result. Then you will see your Jupyter Server address in the top right corner like the highlight below.
SUCCESS! Now it’s time for some real fun!
I believe every data science hacker, coder, and professional is familiar with the Titanic data set at Kaggle. It’s like the “Hello World” of data science. I’ve toyed around with this data set as well. I’ve applied just basic reasoning like “assume all men die”. That will get you a mediocre score with just a few lines of code. For fun, I’ve done a lot of feature engineering and then applied all the Scikit-learn models and Xgboost. Then for real fun I thought I would apply a DL model. My model is available on Github. I recommend following along to see the workflow.
Step 1. Get model from Github here.
Step 2. Switch VSC to a local Python instance. shift+command+p followed by Python: Specify local or remote Jupyter server for connections then select Default from the menu. This will force a reload of VSC (hopefully this reloading requirement will go away in the future).
Step 3. Open the file titanic_feature_creation.ipynb in VSC and execute each cell. The final cell will write out the files titanic_train_wrangled.csv, and titanic_test_wrangled.csv into a subfolder called data. Those two files have to be copied to your VM instance.
Step 4. Copy data files to VM instance. From the titanic folder you retrieved from Github execute the following command. Substitute your instance name for inst_name.
> gcloud compute scp ./data/*.csv inst_name:~/jnotebooks/data/
Step 5. Connect VSC to your Server. Shift+command+p followed by Python: Specify local or remote Jupyter server for connections then select your server from the menu. This will force a reload of VSC.
Step 6. Open the file titanic_model_evaluation_cnn.ipynb. Note, in the first cell the data files that are read. You created these locally and then transferred them to your VM instance.
Step 7. Execute all the cells to generate a submission.csv file.
Disclaimer: The cell titled, Convert features to square PNG, is just a simple way to translate each row (or record) of both data files into an image for this exercise. There are many better ways to translate data for DL models. I wanted to keep it bare-bones simple for this demonstration. All created images are stored in a sub folder on the server called images_titanic.
Step1. Now let’s retrieve the file submissions.csv that was created in the last cell shown below.
Here’s the command. Once again, substitute your instance name for inst_name.
> gcloud compute scp inst_name:~/jnotebooks/data/submission.csv .
Step 2. Visit Kaggle’s Titanic site here and submit your work.
78% percentile is okay. Not bad. However, if you’re a real DS practitioner you would laugh at this. It’s an extreme case of over engineering. But this was not an exercise in optimal deep-learning-model engineering. This was all about how to set up a sandbox so you then go focus on building optimal models.
One last thing.