Data Science and Machine learning- A Foreword
By: Bharadwaj Venkat
Tags: Data Science, Machine Learning, Algorithms

Google’s self-driving cars and robots get a lot of press, but the company’s real future is in machine learning, the technology that enables computers to get smarter and more personal.

– Eric Schmidt (Google Chairman)

Data Science is the field of generating insights from data by using mathematics, statistics and computer science.

As many of the other fields of the science, data science is performed by hypothesis, proving, disproving assumptions, slicing, combining data and generate observations for understanding of the given problem.

Machine learning is the subset of AI. It is performed by using various algorithms that perform certain tasks without being explicitly programmed.

Deep Learning is a subset of the Machine learning which is younger, advanced , more complex and computationally expensive.

Machine learning has 2 main types based on problem to be solved and ways to solve.

Supervised Vs Unsupervised

Supervised Learning : This requires labelled data to perform a task, the given task is performed under the supervision of the labelled data. Unsupervised Learning : This does not required labelled data , the given task is figuring out the data by itself and generating the trends.

Supervised learning has 2 sub-types based on the data types and task.

a) Regression : A technique to find the relationship between independent and dependent variable. It requires the dependent variable to be continuous.

b) Classification: A technique to classify the objects in to different classes. It requires the dependent variable to be discrete.

Unsupervised Learning to have 2 main sub-types

a) Clustering : This is a technique of grouping similar objects together

b) Dimension Reduction : This is a technique used to reduce the dimensions in data to reduce complexity without losing the information.

Algorithms are the building blocks used in machine and deep learning.

There are quite a few algorithms. Let’s discuss few here.

Linear regression Logistic regression Support vector machine K-Nearest Neighbor K-Means clustering Decision Tree Random Forest Bagging and Boosting Principal Component Analysis Neural Networks

Let’s talk about algorithms

Linear Regression : Arguably, the oldest and famous statistical technique. This is used to find the linear relationship between one dependent and one or more independent variables.

The goal of linear regression is to find the ‘best fit line’ that roughly passes through the actual points while reducing the distance between actual and predicted points.

The equation of the line is Y = B0 + B1X

Here Y is dependent variable, X is the independent variable and B0 , B1 are the values we need to find.

Gradient descent is the a method of updating the B0, B1 values to find the ‘best fit line’ while reducing the cost function or loss function or generally called as error.

In Linear regression, the error is called Mean Square Error. The MSE is calculated as mean of the difference between all the predicted points and actual points.

There are multiple ways to evaluate linear regression model

a) Mean Square Error

b) Root Mean Square Error

c) Absolute Mean square Error

d) R2 ( R Squared)

e) Adjusted R2

Logistic Regression : Most versatile and commonly used classification technique. Logistic regression is a supervised classification algorithm.

The goal of the algorithm is to separate the dependent variable into different classes. This concept is called Maximum likelihood estimate.

Since, the dependent variable is discrete in nature, applying the regression will not make sense. Hence, it uses a transformation function called logit function to convert the predicted output into discrete class.

The equation of the line that separates the classes forms an ‘S’ shaped line.

Here, the cost function is called Logarithmic loss function to calculate the cost of mis-classification.The goal is to reduce the cost function and gradient descent is used to update the values of the ‘S’ shaped line.

Logistic regression has few evaluation metrics

a) Confusion Matrix

b) F1 Score

c) AUC-ROC

Support vector machine : This is an supervised classification algorithm that separates the classes of the variables.

This goal of the algorithm is to find the hyperplane that separates the classes while increasing the margin of the distance between the hyperplane and the decision boundary formed with help of it’s closest vectors in each class, those vectors are called support vectors.

The support vectors determine the position and orientation of the hyperplane.

Here we want to maximize the margin of distance between the hyperplane and points for future new points to comfortably fall into a particular class.

The cost function here is called hinge loss.The cost function is calculated when the actual class and predicted class are different. Lower the mis-classification, the lower the cost function.Using gradients we update the weights in the equation of the plane.

For non-linearly separable data finding the hyperplane becomes complex and non-intuitive. SVM has an additional parameter called kernel that converts high dimension to lower dimensions.

There are few kernel functions like polynomial,radial, RBF etc…

Decision Tree :Decision tree is a supervised classification and regression algorithm, also called as CART algorithm, means it can be used for classification and regression problems. It forms the base for all the advanced tree based models.Hence, understanding decision tree is vital for any DS/ML enthusiast.So, lets try to understand.

As the name implies it follows a tree shaped approach which makes decisions for the given problem. The goal of the model is to obtain the pure nodes and decisions formed along the way.

There are 2 types of Decision trees based on target variable data type

Categorical variable decision tree Continuous variable decision tree

Before we dive in , let’s understand few terms here

a) Root Node : Where the features of the data enters the tree

b) Decision Node : Where the decision to split the node takes place

c) Leaf Node : Where the splitting into other sub-nodes stops.

d) Pruning : when we remove the sub-nodes of the tree.

Now, let’s see how the decision tree works.

The accuracy of the model depends on the variable selection and splitting decision. Decision Tree has to decide the variable at first and variables at internal nodes and its splits for splitting that gives us the best pure nodes at the end . The selection is done by using one of the below algorithms in the decision tree model.

They are

ID3- A Top-Down approach while building the tree.

2. C4.5- Advanced version of ID3

3. CART- Classification and regression tree

4. CHAID-Chi-square automatic interaction detection Performs multi-level splits when computing classification trees

5. MARS-multivariate adaptive regression splines

There are also few criteria(metrics) that are part of the variable selection and splitting process.

a) Entropy : It is a measure of randomness of data in the node. Higher the entropy , lower the model performance.Hence, we need variable with lower entropy to proceed further while building tree

b) Information Gain : Information gain is related to entropy while making the decision. This gives how much information the node has about the variable.Hence, we need higher information gain value from the node.

ID3 algorithm used both entropy and information gain to build model.

c) Gini Index

d) Gini Ratio

e) Reduction in variance

f) Chi-Square

To be continued…