The complete guide to start your Data Science and AI journey. | by Defend Intelligence | Towards Data Science

AI & Data Science are already skills that everyone should develop. With this guide we will go through a fast tutorial to present you how to start this technical journey (platforms to learn and apply skills, installation on your computer, useful software). At the end of this article you will be able to run your first Machine Learning program ! This guide is made for the non expert, so no worries ! :-)
You are all set now to start your Data Science & Artificial Intelligence journey. Welcome aboard !
Faculty to learn by yourself
Data Science / AI / Machine Learning / Deep Learning, all those fields are currently being defined. So the first thing to learn is the faculty to be always active in learning. Not sure about something ? Looking for the definition of a supervised algorithm ? A bug in your program ? Google it (or use any other search engine as Qwant or Ecosia) !Be careful about your sources, since AI is a new field you may see a lot of different and sometimes contradictory information. No one has the absolute truth here about definitions or knowledge. Search, interact and construct your own ideas ! It is why I recommend to learn from collaborative platform. On hot topic such as AI, collaborative platforms are a good source of information as you can go across the discussion thread, identify the person, and be able to challenge any answer or ask for more details.If you have an issue or an interrogation, you can be sure that someone before you already asked for it on some website. You may also already be able to help people so do not hesitate to contribute :
Stack Overflow your new best friend for coding
Quora sharing knowledge platform
Wikipedia the universal encyclopedia
Medium by the way
When you will be a little more advanced in machine learning your key sources references will be documentations from libraries as for example this documentation from scikit-learn here or Tensorflow and Pytorch.
Mathematical and statistical knowledge
Machine Learning, Data Science and Artificial Intelligence requires strong mathematical and statistical knowledge. Here are the fields of knowledge you have to go through to start to understand Data Science(please do not hesitate to comment with some other field to complete the list) :
Common Algebraic Theory (Linear model and matrix Algebra)
Probability and Bayesian Theory
Decision Theory
Graph Theory
Functional Analysis
Game Theory
Algorithm Logic
Resources :
Data Science Math Skills by Duke University
Introduction to Mathematical Thinking by Stanford University
Introduction to Linear Models and Matrix Algebra by Harvard University
Mathematical lessons by Khan Academy
Python for Data Science
Python is a language used for many things (websites, games, applications, back-ends, database administration...). To use Python for Data Science you will need to:
understand the common logic of the language
learn how to use libraries
What are the most common librairies in Data Science ?Pandas for data manipulation, seaborn/matplotlib for data visualization, numpy for mathematical manipulation, sklearn for algorithms use, keras, pytorch and tensorflow for more advanced use (like neural network for example).
Resources :
Introduction to Python by Microsoft
Python Data Science Handbook by Jake Vanderplas
Applied Data Science with Python by University of Michigan
Data Science courses
Once you have all those mathematical and code knowledge, it’s time to go into the Data Science field. You will need to learn new skills. A Data Scientist should be able to have a wide overview of all those skills listed below. But depending on what you want to specialize in only a few should be mastered.
Database Management: To access different kind of data from different kind of sources (SQL, No SQL, structured, not structured, cluster, file...).
Data Structure: To easily manipulate and prepare your data including features engineering.
Data Visualizations: To understand your data by visualising it rather than reading it.
Algorithms: There is nothing worse than trying all algorithms available and take the one that seems “better” regarding specific metrics, this is a waste of time. Each well trained Data Scientist should know how commons algorithms of Machine Learning work. From this knowledge the Data Scientist will be able to choose an algorithm from the data analysis.
Evaluation: Combining data visualization and metrics knowledge a Data Scientist have to know if a model works well or not.
Interpretability: We have to solve the myth of the “Black Box”. Machine Learning interpretability (with some libraries such as Shap or Lime ) help us understand how a Data Science solution works.
Knowledge about web and infrastructure security : To not save, run or deploy a solution that could be dangerous for an organisation or your private laptop.
Resources :
Machine Learning by Stanford University
Data Science: Machine Learning by Harvard University
Learn by applying you Skills !
Perhaps my favorite method of learning, is to directly learn by applying immediate skills (and by applying skill I mean coding !). You will meet many people talking of Machine Learning, Data Science, Analytics or Artificial Intelligence without having ever touched a single line of code, and I think thzt it is not relevant. Like Google says about learning Data Science: Get your hands dirty !
Two popular platforms are here to help you to (truly) develop your Data Science coding skills:
Kaggle : the most popular Data Science platform with an awesome community that learns from each other. At the end of this guide you will be able to run a Machine learning project on Kaggle ! :-)
Hackerrank : This platform is really useful to learn, not from competition, but from courses specialisation with different challenges for different levels, go take a look !
Algorithm ? Model ? This common distinction has to be done before going any further. Machine Learning is made of algorithms like Random Forest or the famous XGBoost. Currently some geniuses around the world are developing more and more complex algorithms. Thanks to open-source libraries, you will not have to code them, just to know how to use them ! ;)A model is something more general, a model is made of several steps (including algorithms) to help you solve a specific problem.
A Machine Learning project (I include here Deep Learning) can be of different types:
Classification (i.e. discrete value): The goal of this kind of problem is to predict the “class” of an event, object, person... For example if you want to predict which brand a phone is using its performance, materials, country etc.Regression (i.e. continuous value): IThe goal of this kind of problem is to predict a specific value. For example if you want to predict the price of a phone.
Classification (i.e. discrete value): The goal of this kind of problem is to predict the “class” of an event, object, person... For example if you want to predict which brand a phone is using its performance, materials, country etc.
Regression (i.e. continuous value): IThe goal of this kind of problem is to predict a specific value. For example if you want to predict the price of a phone.
Inside those two common problems in Machine Learning, you have three kind of approaches:
Supervised: In supervised learning you have two different kind of columns. Features columns are columns that you will learn from to predict label columns (most of the time you will have to predict one label column). For example predicting the price of a phone for the next June regarding its past prices on previous months of June (and more other details as for example competitors, materials of the current version of the phone...)
Unsupervised: An unsupervised learning is more tricky, you have many features and you have to find a way to classify some events or values. For example predicting the price of a new phone in June, but since it is a new phone you don’t have any past data. You will have to find a price using other data you have, for instance data from other phones that will match the best our new phone.
Reinforcement: Most of the time during a reinforcement learning you don’t have any data, you have just some metrics that you want to maximize/minimize. Using different methods such as genetic algorithms, you will be able to perform an automatic task. For example to find the perfect combination of materials to maximize the margin of your new phone.
Once that you are data science ready, let’s use a machine that will be your ally during this AI journey.
Starting data science everyone’s first question is: Do I have enough power in my laptop to build and create a Terminator ?
Well... Good question ! The first step is to know the resource of your computer. After that it depends on which kind of task you want to do. If you want to do Deep learning, by training your own neural networks from thousands of images, you will need to have a high level computer (with at least a Nvidia GPU 1080 and 32Go of RAM). But wait... Stay here! Most current laptops can run basic machine learning problems with data coming from a database (.csv to begin with) with around 100.000 rows. If you are going to deal with more data and more complexes data(such as image or sound), you will need to upgrade your laptop. From my point of view, a starting machine learning laptop should have at least 16Go of RAM and 8 processor cores, you can have less but it will take more time...
If you need more resources for limited time , you can use cloud solutions, which may be more difficult for beginners to handle, as they require some advanced configuration. Do not hesitate to visit popular cloud solution providers for Machine Learning like Google Cloud Platform, Amazon Web Service or Microsoft Azur.
As this guide is for beginners we will install all the needed stuff directly on your computer.
Perhaps the first weirdest thing a beginner may see. You will find several word to define this black window : shell / bash / cmd / prompt, let’s call it just a terminal.The terminal allows you to access to access each folders and config of your computer and, believe it or not, it is faster than a mouse navigator when you master it ! Before the democratization of the mouse controller in each computer, everything was accessed using several terminal “command”.
Unix ? Windows ? MacOS ? Linux ? Is it all the same regarding the terminal ?
Unix is the core system of OSX and the inspiration of Linux, this mean that all commands are the same. Macbook and Linux users, you are already all set , jump to the next section !For Windows users stay here a little bit longer :-). Since Windows 10, Windows allow users to install unix command on the terminal, a huge success for many developers !You can install the unix terminal on windows with several methods, we will see below (Git section) the simplest I use, but here is a tutorial to allow you to install Unix command line with windows process : https://goo.gl/kYm1XC
Once you have unix command installed on your machine, open a search bar.
Type pwd in you terminal. pwd allows you to see in which folder you are.
Type ls to list visible folders and files. And type ls --all to list each file included the dissimulated ones (Why are they dissimulated ? To simplify the navigation you do not see all files and folders by default. Most of the time hidden files are some config files you do not need to touch, but with your terminal, everything is accessible now, awesome no ?)
choose a folder and type cd THE_NAME_OF_THE_FOLDER_HERE
type again pwd and you should see that you just jumped inside this folder, congrats you made your first terminal navigation !
Take a look at the complete set of commands of Unix (type man COMMAND to get the manual entry for the command)
Do not hesitate to play with it to feel more comfortable and Enjoy !
Now that you master the terminal, let’s install all the stuff we need to start to play with Artificial Intelligence !
Yes all it’s about Snakes, but no worries you will be fine, let’s become a snake charmer ;).
Anaconda is a framework that install in your system all the basics stuff (libraries, modules, packages, language) needed on your computer in a simple way.
Libraries ? Module ? Package ? Language ?
Here we will use as “code language” , Python. But you have several others languages available for coding. To choose of the language depends on what you want to do (javascript for web development, SQL for database access, Java for app development, Swift for iOS app...). Python is one of the most famous language by its faculty to be human readable with an easy and understandable logic.
As you may expect several programming functionalities have already been coded in Python. To not develop again functionalities others may have already developed, a common way to share code is to use libraries (i.e. modules, packages...) and code in open-source. The most famous platform to get all this open-source code is GitHub, and we will see how to install and use it later, no worries. Anaconda already installs all basic libraries listed below required to begin Data science.
Let’s install Anaconda. Jump to their website here to download the installer and follow the installation process (available for all platforms — Windows / macOS / Linux) :
I recommend to use the Python 3.7 version, as all libraries stop supporting Python 2.7. All libraries are not again already all set in this latest version, but do not hesitate to Google any difficulties you may encounter.
For information, once Python is installed on your computer, you will really often use the command pip . pip is a libraries system management used from the terminal.
After installing Anaconda, you will be able to use conda instead of thepip command line.I highly recommend you to use environments to manage all your projects, take a look here . With an environment you can choose the version of python you want and which librairies and versions of each you want to use. But you can come back later on this point when you will have more hours of practice in Data Science and coding.
Let’s start to go a little bit deeper ! Git is an awesome system to manage and share code. As you may except when sometimes hundreds of people are working on the same project you cannot merge all your code manually by USB or worse with e-mail transfers. Git brings the system of code management, or versioning.
To install Git on your machine download the installer here, depending of your system (MacOs, Linux, Windows). And here are the tutorial to install git on your machine.
Remembering how to manage Unix command on windows ?Windows users please follow the few lines under to get Unix commands !
Open the installer
Information > Click Next
Select Components > Click Next
Choosing the default editor used by Git > Use the Nano editor by default
Adjusting your PATH environment : STOP ! Choose “Use Git and optional Unix tools from the Command Prompt”
Choosing HTTPS transport backend > Use the OpenSSL library
Configuring the line ending conversion > Checkout Windows-Style, commit Unix-style line endings
Configuring the terminal emulator to use with Git Bash: STOP ! Choose “Use windows’ default console windows”
Configuring extra options > Enable file system caching & Enable Git Credential Manager
Install and welcome in this amazing new world of billions of line of code !
A fast guide here to allow you to install Git and understand how it works with your new terminal command line skills ;) :
Git PlatformAround Git several platforms have been built to allow in a simple view to manage “repositories”. (wut ?) A repository is a project in the Git world :-). Some of those platforms : Bitbucket , GitLab, CodeBase, Assembla ...The most famous platform which allows you to find all you need is Github.First step: go on Github and create an account, for regular use all everything is free by the way.You are now able to access billions of lines of code, some of them are from top tech companies, like Google and it will simplify your first step in DataScience. Do not hesitate to go across the platform and discover awesome projects as the Dense Pose project from Facebook.
Never forget that we all use open-source projects. Please do not hesitate to share your code with the community and contribute to open-source projects, thousand of projects are waiting for you ! Code is an awesome tool for sharing knowledge, sharing before thinking to make money is so much more profitable for everyone, let’s follow this mindset.
As .docx is for Word Office, .xlsx for Excel, .png for images, .py is the extension for Python Script.
To access and manage a Python Script, you have several software with different levels of complexity and different features, most of them are called IDEs for Integrated Development Environments. Let’s present four of them but again do not hesitate to try and search on the internet for some others, find the one that fits best with you :
Pycharm : The most complete, stable but the most complex, allowing you to view, stop, run, follow the process of your code, including Git and automtic error control.
Spyder : Less features than Pycharm but easier to install and has complete features for beginners.
Visual Studio Code : An IDE really simple of use, with less features than others used by professionals but also perfect for beginners.
Sublime text : The most simple of use, but Sublime text allow you just to edit your code you will need to go back on your terminal to run your script.
Once Python is installed on your computer, to run a .py script you will just need to type on your terminal inside the folder which contains your script: python my_awesome_script.py
.ipynb is the extension for notebooks. The most famous notebook runner is Jupyter.
Notebook ? Script?
They are both useful for Data Science projects. Here is an image from Kaggle, the most famous Data Science platform that explain you the difference. When you start a new project (named a kernel) you have to choose between script or notebook.
Globally a notebook is used for Data exploration / Analytics and run models and lines several times step by step without going back to the beginning of your code execution. But as you have to start a local server, it is slower and less scalable than a script.
To install notebook in the terminal, we will install jupyter lab:
with conda:conda install -c conda-forge jupyterlab
with pip: pip install jupyterlab
Here is the full website documentation but with your new git skills you can also access to Github repo ! :-)
Once installed you can run Jupyter Lab with the command line : jupyter lab
Let’s apply all this new stuff !
Time to practice all this new knowledge. This final part contains a tutorial to run your first machine learning project through all steps required :
Init a project
Mission understanding : We are gonna to work with the famous titanic dataset “to complete the analysis of what sorts of people were likely to survive. In particular, we ask you to apply the tools of machine learning to predict which passengers survived the tragedy”. (thanks to the Kaggle Challenge here )
Data Exploration / Analytics
Data Visualization
Features Engineering
Modeling
Prediction
I made a project that will help you to start this journey. We are going to use this github repo.I highly recommend you to fork the project to be able to modify it and play with it as you want. What is a fork ? Jump on this support Github website here.
Open a terminal
Navigate to the location desired and clone the repository, you can use the direct url of the repo (https://github.com/anisayari/your_first_machine_learning_project.git), but I recommend you to switch with your forked url which should look like : https://github.com/YOURUSERNAME/your_first_machine_learning_project.git
C:\Users\AAyari> cd DesktopC:\Users\AAyari> git clone https://github.com/anisayari/your_first_machine_learning_project.gitCloning into 'your_first_machine_learning_project'...remote: Enumerating objects: 22, done.remote: Counting objects: 100% (22/22), done.remote: Compressing objects: 100% (19/19), done.remote: Total 22 (delta 4), reused 11 (delta 0), pack-reused 0Unpacking objects: 100% (22/22), done.
Navigate to the file and start jupyter lab to see the notebook :
C:\Users\AAyari> cd your_first_machine_learning_projectC:\Users\AAyari\Desktop\your_first_machine_learning_project> cd my_first_machine_learning_projectC:\Users\AAyari\Desktop\your_first_machine_learning_project\my_first_machine_learning_project> jupyter lab[I 11:11:57.121 LabApp] The Jupyter Notebook is running at:
Great ! Our Jupyter Lab is running and you are now able to access your notebook. The Jupyter Lab should have opened in a new browser tab, click onthe file my_first_machine_learning_program_notebook.ipynb.The browser window should look like the image below :
Enjoy this notebook ! It is well commented, do not hesitate to go through those lines and try to reproduce them and improve them.
You should have noticed my_first_machine_learning_program.pyit is the script version of the kernel. Try to run it and see the differences ! You will better understand the difference between a notebook and a script. To run this script :
C:\Users\AAyari\Desktop\your_first_machine_learning_project\my_first_machine_learning_project> python my_first_machine_learning_program.py
Want to use it on Kaggle ? The notebook is available on kaggle here, do not hesitate to “Fork” it and to run it to submit your first Machine Learning program on Kaggle !
Now you should be able to :
know which skills and knowledge you have to develop, and where to learn them
understand basic words used in Data Science (such as notebook, supervised, model...)
got all basic software required to run and manage python code on your laptop
And last but not least, you should be able to say to your colleagues that you ran your first machine learning program ! Again ,congratulation and thank you for reading this guide.
The Data Science and Artificial Intelligence fields are exciting areas of knowledge, be curious and go around the internet to discover always more.
Welcome to your Data Science/AI journey.