Build and Setup Your Own Deep Learning Server From Scratch | by Kitty Shum | Towards Data Science

Earlier this year, I completed the “Practical Deep Learning — Part 1” course by Jeremy Howard’s. It was a pragmatic course that teaches you how to practice various deep learning techniques using AWS. AWS was a way to get up and running really quickly but the cost adds up quickly. I had a p2 instance for about $0.9/hr and let’s not forget all the extra hidden cost that comes with AWS. I was racking up $50–$100/month. It was affecting my productivity because I was trying to limit my use of AWS to reduce cost. I wanted to pursue my more in deep learning and decided to take the plunge and build my own deep learning server. This stuff changes very quickly, so I will not be surprise if the content is outdated by the time you read this. However, I hope this will give you some insights on how to build your own server.
1. Select Components2. Hardware Assembly3. Install Operating System4. Install Graphics Card and Driver5. Setup Deep Learning Environment6. Setup Remote Access
It’s useful to do a bunch of research (i.e. reading blogs) to get an idea on what parts you need to buy.
Use pcpartpicker.com before you make your purchases. The site helps you do a “compatibility check” to make sure all your components are compatible with each other. My part list is here.
In general you need the following:
CPU — Intel — Core i5–6600K 3.5GHz Quad-Core Processor $289.50RAM — G.Skill — Ripjaws V Series 32GB (2 x 16GB) DDR4–2133 Memory $330GPU — EVGA — GeForce GTX 1070 8GB SC Gaming ACX 3.0 Video Card $589SSD — Samsung — 850 EVO-Series 500GB 2.5" Solid State Drive $200Motherboard — MSI — Z270-A PRO ATX LGA1151 Motherboard $140CPU Cooler — Cooler Master — Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler $30Power Supply — EVGA — SuperNOVA NEX 650W 80+ Gold Certified Fully-Modular ATX Power Supply $120Case — Corsair — 200R ATX Mid Tower Case $75Total: under $1800 CAD
Key Takeaways:
GPU is the most important component! If you are on a tight budget, make sure you spend your money on a good graphics card. NVIDIA has many years of experience building graphics card. Do some additional research and you’ll see what I mean. At the time of writing, GTX1080Ti was the latest best option available, however a GTX1070 will do just fine.
Think about components that enables you to upgrade in the future. Consider paying a little extra to get a motherboard that can support multiple 16e PCIe slots. This enables you to install an additional GPU if you want to get a performance boost later. Similarly, make sure there’s enough RAM slots for upgrades as well.
When in doubt go for parts that has the most customer reviews. I find that if there’s a lot of reviews, it means the part has been tested and you’re more likely to find help or how-to-guides online.
Buy from trusted sellers! pcpartpicker.com will give you pricing from various sellers. I noticed there was a reseller that offered components at a slightly lower price. It’s always nice to save a couple bucks whenever possible. I was about to purchase a few items from this *less well-known* reseller until I looked at their reviews. Thank goodness I did because their reviews were horrible! I opted on the safer side and bought my components from my stores I trust such as Amazon and newegg.
Free Shipping & Handling: Consider buying your parts from places that offer free shipping & handling such as Amazon. You can save yourself some money.
Other helpful resources:
Building your own deep learning box
Great Deep Learning Box Assembly Setup and Benchmarks
Deep Learning Hardware Guide
Setting up a deep learning machine in a lazy yet quick way
Build Personal Deep Learning Rig
I haven’t touched computer hardware *stuff* since I was in high-school so I was a little nervous at the beginning. If you’re really not comfortable putting the hardware together, you can head to Canada Computers, or the equivalent, and they can assembly it together for you for about $80. For me, I didn’t want to pass up on this great learning opportunity.
Before doing anything, I recommend looking through the instruction manuals and how-to videos to familiarize yourself with the various components. You’ve already spent so much $$$, you might as well take a little time to make sure you know what you’re doing!
Here are some helpful videos for the components that I used:
Key Takeaways:
Watch all the videos before starting. Words and manuals can only communicate so much. You’ll feel much more confident after watching someone else install it. There’s often a few gotchas that manuals don’t point out.
Make sure everything is plugged in properly. I was afraid of breaking my parts so I was really gentle with everything. The first time I plugged in the power cord and turned on the power, nothing happened — no lights, no sound. I thought I must have broken something and wasted $1000. I turned it off and checked all the plugs to make sure everything was plugged in properly. Load and behold everything worked afterwards.
Don’t install your GPU yet! Wait till you install your OS. I made this mistake and thought my motherboard was broken because I got a black screen on my monitor. Just leave your video card on a side for now. See details below — I dedicated a section just for it.
I chose to install ubuntu 16.04 with a USB. You can technically install Windows as well but I didn’t see the need for it at the moment.
3.1 Create a Bootable USB Drive
1. Get a 2GB or larger USB. 2. Download the ISO file from Ubuntu. I used 16.043. Follow these instructions to create a bootable USB stick.
3.2 Install Ubuntu
Reboot your system and the Ubuntu setup screen should pop up. Follow the instructions on the screen and it should be pretty straight-forward. Here’s a useful tutorial.
Some people may not automatically get the Ubuntu setup screen. If that’s the case for you, you just need to reboot into the BIOS (for me it’s pressing F11 when the computer starts) and configure your boot priority to load the USB drive first before the hard drive (SSD).
3.3 Getting Up-to-date
Open your terminal and run the following commands:
sudo apt-get updatesudo apt-get upgradesudo apt-get install build-essential cmake g++ gfortran git pkg-config python-dev software-properties-common wgetsudo apt-get autoremovesudo rm -rf /var/lib/apt/lists/*
NVIDIA graphics cards are tricky to install. It’s a bit of a chicken and egg because you can’t use the graphics card until you install the driver, but you can’t really install the driver unless the motherboard detects that your graphics card is installed. My motherboard also automatically boots the graphics card first if it detects that there’s a card in the PCI slot BUT because I don’t have the drivers installed yet, I just end up getting a blank screen. So what do you do? After doing a bunch of research, I found the following steps work. Hope it will save you some trouble.
4.1 Change the BIOS Settings
Motherboards should have an integrated graphics device so you can hook up your monitor to the motherboard directly. By now, you should have your monitor hooked up to the motherboard display device. Since we’re going to install a graphics card, this is where you need to tell the BIOS which graphics device to initiate first. There are two settings in the BIOS:
PEG — PCI Express Graphics: motherboard will boot the graphics card if there’s one detected on the PCI slot. If there isn’t one, then it will boot the built-in one from the motherboard.
IGD — Integrated Graphics Device: will always use the motherboard’s built-in card
By default the BIOS will have the setting set to PEG. In normal circumstances once you have the NVIDIA driver installed, this makes sense because you want to leverage the card. However, it causes problem when you have to install the NVIDIA driver.
Change the BIOS config by:
restart the computer and boot into the BIOS (F11).Goto: Settings -> Integrated Graphics Configuration -> Initiate Graphic AdapterChange settings from PEG to IGDSave and turn off your computer
restart the computer and boot into the BIOS (F11).
Goto: Settings -> Integrated Graphics Configuration -> Initiate Graphic Adapter
Change settings from PEG to IGD
Save and turn off your computer
4.2 Physically Insert The Graphics Card
Now that the system is turned off, insert the graphics card into the PCI slot. Sometimes you have to push harder than you think. Make sure it’s fully in.
4.3 Install The Driver
Now that the graphics card is connected to the motherboard, turn on your machine to install the NVIDIA driver.
Find your graphics card model
Find your graphics card model
lspci | grep -i nvidia
2. Determine the latest version of NVIDIA driver available for your graphics card
Visit the graphics drivers PPA page and see the versions of drivers available. As of August 2017, 384 is the latest version.
Visit the NVIDIA driver downloads site to determine the version that is compatible with your card and OS.
3. Add and update the graphics-driver PPA
sudo add-apt-repository ppa:graphics-driverssudo apt-get update
4. Install NVIDIA driver. Enter the version number that is compatible to your card. For example, I installed version 384
sudo apt-get install nvidia-384
5. Reboot your system and verify your installation
lsmod | grep nvidia
If you have issues and need to start again fresh, you can purge everything by using the following command. I hear that people who are upgrading graphics card often run into drivers issue.
sudo apt-get purge nvidia*
6. Change your BIOS graphics device priority to PEG again. (See step 2)
Before you start installing anything, or blindly follow instructions from a blog, take some time to understand what library version you need. These things change so fast that most blogs will be outdated by the time you read them. Also, don’t blindly download and install the latest version either. For example, Tensorflow 1.3 supports up-to python 3.6 (as of August 1st 2017) but Theano only supports up-to python 3.5. Anaconda’s latest version runs on python 3.6 but you can create a conda environment with python 3.5. The code I used in part 1 of my deep learning course was written with pythonn2.7 but part 2 of the course uses python 3.5 — for that reason I created 2 conda environments.
I ended up using the following list of libraries and its corresponding version.
CUDA (v8) —a parallel computing platform by leveraging the power of GPU
cuDNN (v6)— CUDA Deep Neural Network library that sits on-top of CUDA.
Anaconda (v3.6)— A good package and environment manager that comes with a lot of data scientific computing tools such as Numpy, Pands and Matplotlib. The other benefit of Anaconda is it makes it easy to create custom python environments.
Tensorflow (v1.3)— Google’s machine learning framework
Theano (v0.9) — An alternative machine learning framework.
Keras — higher level neural network library that runs on top of Tensorflow, Theano or other machine learning framework.
5.1 Install CUDA
Goto the NVIDIA website and find download the CUDA Toolkit.
Run the following code to install CUDA. Change the version that correspond to the package that’s compatible to your system.
wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.44-1_amd64.debsudo dpkg -i cuda-repo-ubuntu1604_8.0.44-1_amd64.debsudo apt-get updatesudo apt-get install cuda
Add the path to your environment
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrcecho 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrcsource ~/.bashrc
check that you have installed the right version
nvcc -V
check out details about your video card
nvidia-smi
5.2 Install cuDNN
Goto the NVIDIA website to download the library. You have to sign-up in order to download. The site says it may take up to a few days to approve but the turn-around time for me was instant.
Extract and copy the files over to where CUDA is installed (usually /usr/local/cuda)
cd ~/<folder you downloaded the file>  tar xvf cudnn*.tgz  cd cuda  sudo cp */*.h /usr/local/cuda/include/  sudo cp */libcudnn* /usr/local/cuda/lib64/  sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
5.3 Install Anaconda
Goto the website to download the installer and follow the instructions. The latest Tensorflow version (at the time of writing it is v1.3) needs Python 3.5. So download the python 3.6 version and you can create a conda environment with a specific python version.
Create a virtual environment with the following command. More details on how to manage the environment can be found here. You may want to create one environment for Theano and one for Tensorflow since they often support different versions of libraries.
conda create -n tensorflowenv anaconda python=3.5
Activate the conda environment you just created
source activate tensorflowenv
5.4 Install TensorFlow
Install the TensorFlow library. There’s a different package depending on your OS, Python version and CPU vs GPU support. Take a look here to determine the TensorFlow binary URL you should use.
pip install --ignore-installed --upgrade <tensorFlowBinaryURL>
For example, you would use the following command if you are running Python 3.5 on Ubuntu with GPU support.
pip install --ignore-installed --upgrade \https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.3.0-cp35-cp35m-linux_x86_64.whl
For more details about TensorFlow installation, see here.
5.5 Install Theano
You can find installation details here. However, it’s pretty much just one command.
sudo pip install Theano
5.6 Install Keras
Install Keras with the following command
sudo pip install keras
Depending on which backend you use (TensorFlow or Theano) you have to configure it accordingly.
vi ~/.keras/keras.json
The configuration file looks something like this. You just have to change “backend” to “tensorflow” or “theano” depending on what backend you’re using.
{    “image_data_format”: “channels_last”,    “epsilon”: 1e-07,    “floatx”: “float32”,    “backend”: “tensorflow” }
More details about Keras installation: 1, 2
5.7 Other useful resources:
NVIDIA GTX 1080 on Ubuntu 16.04 for Deep Learning
Setting up a Deep Learning Machine from Scratch
This step is optional, but if you plan to work remotely from your laptop, here are some options.
6.1 Teamviewer
Teamviewer is basically a screen share software. It’s a little clunky at times, but it’s a good option for remote control.
6.2 SSH
Most developers will want to SSH into the box via a terminal and don’t really need a graphical user interface. You can use OpenSSH.
You can use the following commands to install and check its status. By default, the OpenSSH server will start running on first boot.
sudo apt-get install openssh-serversudo service ssh status
6.3 Remote Jupyter Notebook
I use Jupyter notebook quite a bit. If you want to run it off of your laptop’s browser, this is a pretty cool trick. Check out a thorough tutorial here. In summary, just run the following code on your laptop and server.
$laptop: ssh -l <username>@<yourNewServerIP>$server: jupyter notebook --no-browser --port=8888$laptop: ssh -NL 8888:localhost:8888 <username>@<yourNewServerIP>
Then you can goto http://localhost:8888 on your laptop’s browser and remotely view/edit your Jupyter notebooks.
That’s it! I’d love to hear your setup experience!