Understanding the Basics of Data Science (when you are not a data scientist but work with or manage data scientists) — Part I
By: Sabahat Iqbal
Tags: Data Science, Datasciencemethodology, Newbie, Business Intelligence

Unlocking the data mining process behind data science principles

First, break up a data analytics problem into tasks for which a tool is available.

And what are these tasks?

Classification — predict the mutually exclusive class of values an individual belongs to. Let’s say I work for a mobile phone company and I want to identify which customers will respond to a marketing offer and which will not. In this case, the individual is the customer and the mutually exclusive classes are: Will Respond To Offer, Will Not Respond To Offer. The tool we use will create a model. That model can then be used on any customer to determine whether they will respond to offer or not and so whether the company should pay for ads directed at them. Will be covered in further detail. Class Probability Estimation — closely related to Classification. The model created from this task will tell us how likely the customer is to respond to an offer or not. Will be covered in further detail. Regression — rather than predicting which category a customer falls into, here the model will predict a numerical value for a customer. To continue the telecom example, maybe we want to know how much a customer will use the service being offered in the earlier marketing campaign. So, we want to estimate, service usage (maybe that is in terms of minutes or texts). Will be covered in further detail.

Rough summary of three of the data analytic tasks used in data science.

4. Similarity Matching — finds individuals that are similar to each other based on data known about them. For example, a telecom company may want to find customers similar to their most profitable customers. They can find matches based on similarities in characteristics of their customers. Will be covered in further detail.

5. Clustering — a type of similarity matching but without any underlying question. In the previous example, the telecom company wanted to know which customers were similar to their most profitable customer(s). In Clustering, a company would just ask which customers are similar to each other, i.e. sort customers into groups based on how similar they are in their characteristics. This task is usually one of the first steps in trying to understand patterns in the data and can lead to suggestions for other data mining tasks. Will be covered in further detail.

6. Co-occurrence Grouping — Rather than looking for similarities based on static characteristics (age, gender, salary of customer), this task looks for links based on transactions (by the customer, to continue with the telecom example). So if a customer upgraded their data plan while downgrading their messaging bundle, and this pattern occurs with some regularity, the telecom company could decide to roll out a new offer that includes those changes together and be able to offer customers more of what they want.

7. Profiling — characterizing typical behavior of a group or population and not an individual usually. For example, the telecom company might want to know typical cell phone usage of groups of people based on geography.

8. Link Prediction — predicts connections (and the strength of those connections) between two entities. If most of the customers in a geographic area are buying Data Package A over Data Package B, the telecom company could push all customers in that are to use Data Package A (and thereby have more efficient operations and happier customers). This task is the basis for movie recommendations a la Netflix.

9. Data Reduction — here, a trade-off is made between keeping data that reveals useful patterns and loss of information from removing data. Large datasets often need to be reduced so they are easier to process.

10. Causal Modeling — moves beyond finding correlations to finding causation. There are two methods: Randomized Control Trials (also sometimes known as A/B testing or experimental method) which require substantial investments in time and data or methods that draw causal conclusions based on observational data. Either are attempting to understand what would have happened if the “treatment” (or telecom marketing offer) had not happened. It is very important to know what assumptions were made as these assumptions may not hold in all scenarios when looking to scale.