Setting up Python platform for Machine Learning projects | by Aqeel Anwar | Towards Data Science

There are a lot of in-depth tutorials on how to get started with machine learning using python. These tutorials mainly focus on the use of Deep Learning frameworks (say TensorFlow, PyTorch, Keras, etc.) such as how to set up basic supervised learning problem, or how to create a simple neural network and train it, etc. But even before one can start experimenting with such tutorials, a working python platform must be available at the host machine on which these hands-on can be carried out.
A few years ago when I started with ML, I couldn’t find one good tutorial on how to set up a working platform on my machine. There were a lot of options/configurations available and I couldn’t decide on any single one. I had to go through a lot of webpages before I could decide on a configuration and took me about 2 weeks to finally have a working platform. A couple of months ago one of my colleagues was facing the same problem, so I assisted her with setting up the platform which saved her quite a bit of time. Hence I decided to write a detailed article about it.
I have divided this article into two parts and will be focusing on them separately
Part 1: Selecting and installing a python distribution and IDE
Part 2: Steps required when creating a new project
In this article, I will use the following Python distribution and IDE.
Python Distribution: Anaconda
Python IDE: PyCharm
What is Anaconda? Anaconda is a distribution of Python and R language with simplified package management and deployment. Using anaconda, it is easier to have multiple python environments with different configurations and to switch between them. The anaconda package manager makes it easier to resolve conflicts between multiple versions of a package required by different packages. A detailed explanation of the pros and cons of using Anaconda can be found here
Download & Install: Anaconda distribution can be downloaded from here. The installation instructions are pretty straightforward.
What is PyCharm? PyCharm is one of many IDEs available for python. I prefer PyCharm because it is much more user-friendly, powerful, and configurable as compared to other IDEs. It provides integration with git, has its own terminal and python console, provides support for various handy plugins, and a lot of useful keyboard shortcuts.
Download & Install: To download PyCharm, head over to this link and download the latest Community (free) version and follow the installation instructions
A different project that you will be working on, will require different resources and packages with different version requirements. So, it is always recommended that you use a separate virtual python environment for each project. This also makes sure that you don’t accidentally overwrite any of the existing working versions of certain packages with other versions hence rendering it useless for your current projects.
Following are the steps that you should take whenever creating a new project.
Open Anaconda Prompt Command and type
conda create -n myenv python==3.5
This will create a new virtual environment named myenv that will come installed and loaded with python version 3.5
Once you have created the environment, you can verify it by activating the environment by using
conda activate myenv     #For Windowssource activate myenv    #For MAC OS
You can find the location of the created env myenv by typing
which python# The output will look something like this# /Users/aqeelanwar/anaconda/envs/myenv/bin/python
We will be using this to locate our environment in step 2
After activating the environment, you can install any required packages using
#General Format:conda install package_name#Example:conda install numpy    #To install numpy
Following are the useful conda commands that will come in handy in managing conda environments
# Activate an environmentconda activate env_name# Deactivate an environmentdeactivate           #Windowssource deactivate    #Linux and macOS# Listing all the created environmentsconda env list# Listing packages install in an environmentconda list# Installing a packageconda install package_name# Cloning a conda environmentconda create --clone name_env_to_be_cloned --name name_cloned_env
Open PyCharm and select Create New Project.
Select the location and a name for your project (medium_tutorial in this case).
Expand the Project Interpreter option and select Existing interpreter
Locate your environment by clicking on the three dots on the extreme right under Existing interpreter
At this point, we will locate our environment myenv (if it is not already in the Project Interpreter list: more on this later) using the location that was displayed from the which python command in step 1. Also, we want this environment to be available for other projects that we create in the future, so we will select the ‘Make available to all projects’ checkbox
Hit OK and then Create.
The project will now be using the environment myenv. You can use the built-in terminal of PyCharm to install packages to this environment.
Create a new .py file (say main.py) and run it using
run >> run main.py
If in the future, you ever want to switch between different conda environments for the same project, you can do so by following the steps below
PyCharm can only select environments that have been included in its Project Interpreter list
To add a newly created (say named PyCaffe) environment to the Project Interpreter list
settings >> Project:project_name >> Project Interpreter
Hit the gear icon at the right top and select add
Select
Conda Environment > Existing environment > <three dots>
and locate the newly created environment and hit OK
Now the environment has been added to the Project Interpreter list and can be seen in the drop-down menu
This list shows the existing environments and whatever environment you select will be used for the project
Note: PyCharm terminal doesn’t automatically activate the current selected environment. If you have selected, say PyCaffe, env from the project interpreter list, and now want to install a new package in it, you will have to first activate the environment in the terminal and then you can use conda install package_name. Otherwise, the package will be installed in the previously activated conda environment
Now you are all set up with the platform. At this point, you can install the desired ML framework (TensorFlow, Keras, PyTorch) and start experimenting with ML tutorials.
In this tutorial, we went through the pre-requisites for working on ML projects (or any python project for that matter). We saw how using Anaconda and PyCharm we can have multiple python environments and switch between them.
If you have any issues in following this tutorial, just comment on the issue below and I will respond with the solution.
If this article was helpful to you, feel free to clap, share and respond to it. If want to learn more about Machine Learning and Data Science, follow me @Aqeel Anwar or connect with me on LinkedIn.