Creating Your First Machine Learning Model
By: Samuel James
Tags: Machinelerning, Setup, Anaconda

Creating Your First Machine Learning Model

Show me How to Set up My Local Environment like I’m Five

Photo by Wes Hicks on Unsplash

This is my first post in a series I titled “creating your first machine learning model.” I assume you already know the basics of machine learning and want to leap towards creating your first machine learning model. My goal is to break the steps down for you as possible.

If you find any uncleared terms or something you don’t understand, it means I did not live up to my promise. Be aware that you’re free to comment and request for clarification.

In this post, you’ll learn:

How to set up your hardware for machine learning projects.

A repeatable process for managing your machine learning dependencies using conda.

How to create an identical machine learning project environment on a new computer

The limitations of your local hardware and when to you use cloud services for machine learning jobs.

Free cloud services that allow you research, build and train models for free.

If you’re going on a road trip, there are certain things you need to do before embarking on your journey. Otherwise, you might bring your road trip to a halt and may have to pay for some unexpected repairs and , perhaps, a waste of time. Likewise, before you start your machine learning journey, your environment must be fully setup.

Configuring your hardware

Machine learning job requires a lot of compute power; the first thing you want to do is to make sure your hardware is powerful enough to build stuff locally. For the hardware requirements, you should be good if you have the following:

Central Processing Unit (CPU)

Intel Core i5 or higher. You’re also good to go with an AMD equivalent

You have at least 8GB of RAM

A GPU (NVIDIA GeForce GTX 960) is recommended but optional

If you’re a developer, some of these steps would not be strange to you. The first thing you should do is to ensure your operating system is up to date.

As a PC owner, you’re probably familiar with the concept distribution — a collection of software components built, assembled, and configured for ease of use. For scientific tasks, you need a group of software built for scientific computations. One of these distributions that we’ll use in this tutorial is Anaconda.

Anaconda is just one of the many distributions that exist. For a beginner who is just getting into the world of “condas”, you may find the term “anaconda”, “miniconda” and conda confusing. You’re not alone. I felt the same at the beginning. If you take a closer look at the words, you’ld see that the word "conda" is common in the three names.

What is “conda” actually?

Conda

If you use pip and virtual env in python, then conda would be clearer. Conda is both a package manager and environment manager that is platform-independent. A package manager allows you to manage dependencies (code written by you or some else). That is, installing, updating, and removing packages. The environment manager, on the other hand, allows you to isolate software environment.

Environment isolation is an essential aspect of machine learning projects because different tools may have conflicting requirements and packages; without environment isolation, it would be difficult to manage multiple projects.

Miniconda

For many reasons, you might not want to install Anaconda. For instance, if you only need one package for your task, why installing 199 packages that you will not use? You can see miniconda as a bootstrap version of Anaconda that includes the package manager “conda”, python, plus a few basic packages. Miniconda requires about 400MB disk space, which could be a better choice when you’re running low on disk space.

Anaconda

Anaconda is a python and R distribution whose goal is to provide data science packages you might need out of the box for you. At the time of writing this post, anaconda has up to 200 data science packages pre-packaged. Since it has most packages, anaconda takes larger disk size on your machine. For a beginner, you might want to start your machine learning journey with Anaconda– that is why we use it for the rest of this tutorial.

So how do you install Anaconda?

Installing Anaconda

To install Anaconda, go to the anaconda’s download page and download a suitable binary for your choice of operating system .

Double click the binary distribution to install, click “continue” always and accept the license agreement. The first screen of Anaconda on my Mac OS looks like this:

Anaconda install screen

Please take a moment to relax; it could take some minutes to install. You can verify that anaconda was successfully installed by running the command below:

conda list

If everything works fine, you should see a list of packages and the pre-installed versions, as shown in the image below.

Conda list ouput

If you got an error, don’t panic, you might need to kill your terminal and restart it again.

Managing environments with conda

Science is based on reproducibility and manufactured objectivity. As strong as that makes its ability to generate claims about matter and energy, it also makes scientific knowledge inapplicable to the existential, visceral nature of human life, which is unique and subjective and unpredictable — Paul Kalanithi

Have you ever re-run code you or someone wrote ten years ago? It can be frustrating if you’re lucky enough to get it to run. The reason is not far from the fact that software and libraries do change rapidly. The version of libraries used, the operating systems, compiler, and interpreter may have changed or replaced by newer versions. In short, the environment and conditions under which the software ran ten years ago may have changed.

Good code must be re-runnable and replicable. One way to ascertain your software can be re-run is to ensure that condition and requirements under which is reproducible and replicable are constant. We do this by pinning the requirements and isolating the software environments.

On installing anaconda, conda creates a default environment named "base" for you. If you put all your code in this base environment, then the purpose of isolating code is defeated. What you want to do is create a new environment for each project to keep your projects isolated.

Creating an environment with conda

To create a new environment with a specific software version, for example, a particular version of python and scipy, execute the following.

conda create --name example_env python=3.7 scipy=1.4.1

Sometimes, it’s handy to define all your environment dependencies in the YAML file. This very handy if you want to share your code with others.

You can define an environment with a YAML file environment.yml as:

# filename: environment.yml

name: another_example_env

dependencies:

- python=3.7

- scipy=1.4.1

After you execute the code below in your terminal to create the environment

conda env create --file /path/to/environment.yml

Updating an environment with conda

Updating your environment is one of the everyday tasks you would do in your machine learning journey.

Libraries changes frequently, an older version are replaced, or sometimes you might even need to swap one library for another or remove redundant dependencies.

Conda provides a command to update your environment after making changes to your environment file.

For example, let’s add a new dependency to the environment.

# filename: environment.yml

name: another_example_env

dependencies:

- python=3.7

- scipy=1.4.1

- pytorch=1.5.0

With changes made to the environment file, you can update your environment by running the command:

conda env update --file /path/to/environment.yml --prune

Listing environments

You might want to see a list of your environments first when you have multiple machine learning projects before choosing the one to work on.

With conda list command, you see a list of your environments :

conda info --envs

Activating your environment

Environment activation is a prerequisite to making your application run. When you activate your environment, you’re setting arbitrary environment variables that may be necessary for your app to work.

To activate an environment using conda, run the command:

conda activate myenv

Launching jupyter notebook

Jupyter notebook is a tool that provides an interactive web-based interface that allows you to develop and present data science project interactively. It integrates your code and its output into a single document.

Once you’ve set up and activated your environment, you can launch a Jupyter notebook by executing the command jupyter notebook to start development.

jupyter notebook

You’ll be presented with a screen similar to the one below:

Jupyter notebook

Sharing and reproducing your environments

Software is written to be run multiple times. Oftentimes, we share code with colleagues, friends, co-workers, even random strangers. And when we do, we should make sure they are able to run the code we share with them — reproducibility is impossible without identical environments.

You can share your environment by exporting your environment so that your colleagues can build the same environment as yours to run your code. We export environments using the command:

conda env export --file exported_environment.yml

Building identical conda environment

There are times where you might want to build an identical environment you have on one machine on another on an identical operating system.

For example, if you buy new hardware or transfer your code from your work computer to your home computer, which has an identical operating system. Conda allows you to build an identical environment either on the same machine or on another machine running on the same operating system platform. We do this by exporting a specification list as a file.

conda list --explicit > current/working/directory/spec-file.txt

You can use the exported spec to create exactly the same environment on another machine using the command:

conda create --name identical_env --file spec-file.txt

The limitation of your local hardware

You’ve seen how to set up your local development environment for machine learning projects. Your local setup is suitable for small machine learning projects that require small amount of compute power.

In the real world, machine learning projects often require huge amount of data and huge compute power far beyond what your local machine can provide.

If you’re training a model to recognize patterns, it could require parallel compute resources, which could take days on traditional processors.

All these limitations and more make us turn to cloud services like Google Cloud Platform, Amazon Sagemaker which have hardware that’s optimized for machine learning jobs. They enable us to train large datasets in parallel with infinite compute power. Cloud services offered by Google, AWS, Azure etc. have hardware that’s optimized for machine learning jobs

Free cloud services that allow you to build models for free

You might be thinking, “Oh no, this is an introductory post, for my project, I don’t even need massive compute power, I just wanna try things out.”

I get it, I totally understand. If you’re not working on a massive project, you probably need little compute power to try things out.

Here is the catch, the cloud has become relatively cheap, some services like Google Colab even allow you to train models for free. I mean need no need for environment setup, most machine learning packages have been pre-installed. You can research fast at no cost.

The following are cloud services; you can use:

Conclusion

In this post, you’ve seen how to set up your local machine for machine learning projects. We also saw a list of free cloud services that allow you to build and train models for free.

In the meantime, while you wait for the second post, you can subscribe to my newsletter. Once a month, I send out a newsletter containing lots of exciting stuff on data science, software engineering, and machine learning. Expect quick tips, links to interesting tutorials, opinions, papers, and libraries.