A Business approach to Analysis of Data
By: Rahulraj Singh
Tags: Analytics, Business, Strategy, Data Science, Storytelling

A Business approach to Analysis of Data

What do you need to master Business Analytics and Decision Science?

There is a new buzz word in town these days. It is called Business Analytics. But what is it, and why is it important? Although every company and organization has a different meaning to what a Business Analyst does, but by university standards a business analytics course equips students with knowledge of data analysis and decision making for the business. I went through a lot of MOOCs and HarvardX courses to understand how a business analysis process is different from that of data analysis — and spoiler — it is not all that different. The only key difference is that a data analyst who can derive to decisions for his business from the analysis that is made is a business analyst. What I will do for you is, I will lay down all the major and minor details that a business analysis procedure follow and we will together build a better understanding of how data analysis done with the right goal in mind leads to successful outcomes. Let’s start with from the beginning of everything

Feasibility — Before diving deep into solving a problem, you need to have an extensive plan of the steps that are going to be followed to reach the decision. Most common blockers in this process will be the distribution of data sources and other requirements that are needed for the analysis. If you are lucky enough, then you might find all data sources in one place, say a data warehouse, in which case you need to extract all of the data from this source and perform your analysis. But in many cases, this might be distributed over web analytics data, BI reports, database servers and un-organized local data. It then becomes important to be able to find out whether this analysis is feasible or not. That is, the cost put in for this, will have a significant outcome or not. At the end of the day, both the timeline and the cost should justify the need for decision.

Information-Action Value Chain

Lets start with what a value chain is; it is a sequence of steps in any process where the end of every individual step adds a value to the overall system. Let us walkthrough how this chain is formed step-by-step:

First we start with data capturing and storing. In the real world when analyzing terabytes of data, the source of this data might not be a singular, and therefore we need to bring all data sources, data points and tools together. Generally, all this information is stored in a data warehouse. This is important as it saves a lot of time that will be spent in juggling through various files, folders and databases in the latter stages if all the information does not exist in one standardized location. Bringing the data together not only saves time, it is also useful in performing the initial steps of analysis: cleaning, establishing relationships, etc. Next in the value chain is extraction of data. Although, we have placed all of our data in a singular place, the location where data is stored will not have analysis capabilities every time. This step is important because it creates a path from the data source to the tool which is going to perform analysis operations, Relational Databases, like SQL Server Reporting Services (In DB analytics services), Python Data Libraries, etc. Next in the series comes the information that we would be concluding, the analysis itself. There are three major categories of data analytics that businesses tend to work with and I will walk through all of them briefly in the below sections.

Descriptive Analytics

How the environment works, or has historically been working. This information helps in understanding the system and therefore can help in developing insights on how to improve it. Descriptive Analytics can take the form of simple aggregations or cross tabulations of data, simple statistical measures like means, medians, and standard deviations. At times it can also be used with more sophisticated statistics like distributions, confidence intervals and test of hypotheses or advanced association or clustering algorithms. Take an example of a cellphone operator like Vodafone, a descriptive analytics scenario could be determining why customers are leaving a certain mobile data plan and switching to others.

Predictive Analytics

Predictions help in finding something that has not occurred yet but has a high probability of occurrence. It often uses advanced algorithms like linear or logistic regression, neural networks, simulation techniques and tree based algorithms like random forests to bring a conclusion. Considering the above example of why people cancelled a data plan, predictive analytics could be used on the historical data to find who could cancel in the future, or what factors lead to a plan cancellation. This information can further be used to take measures to stop these factors from coming up and hence lower the rate of drop in customers.

Prescriptive Analytics

This comes as the ending of the process and explicitly links the analysis to decision making by providing recommendations on what can be done from the analysis to reach a certain outcome. All the three are linked together as they form a lifecycle of analysis events. From the above set of examples imagine we already know from the predictions made, what customers are going to discontinue the cellular plans from Vodafone. Prescriptive analytics can be used to identify those customers from the predictions that can be reached out to, with an agenda of making them stop before they end the plan. This needs to be a targeted approach in order to maximize the return on investment.

University of Maryland describes Business Analytics as a collection of concepts like statistics for data operations and visualization of the data for communication

Analytical Tools and Technologies

Now that we are versed with the forms that analytics can take it now comes down to the tools that we could use to make the above happen.

Data Storage- Storage in the analysis phase can be put into 2 categories, one being the source where all of the data exists. The source mostly has a lot of data in tremendous details and we might not need all of it for analysis hence comes the second division where we move only the data that is required for analysis and leave the rest out. We have come across terms like OLTP (Online Transactional Processing) and OLAP (Online Analytical Processing). These are broad categories of what capacities data storage systems contain. Both these systems are optimized for their individual tasks. It is generally a bad idea to perform operations right in the place where raw data is stored.

Big Data and cloud are increasingly being called together, but both are not really the same. Cloud is more to do with the platform where the analysis occurs, and big data tools on the other hand are used to analyze large amounts of stored data, streaming data, etc. but yet both these technologies occur without each other.

This gives an overview of how the cloud infrastructure holds together all parts of the data analysis process. Each of the tools that exist on the infrastructure have a separate set of languages to work with.

Virtualization- When we mention storing data and making it available for analytics, we generally refer to what is called ETL (Extract, Transform, Load) which as a process extracts data from one location, transforms the data in some way, and then loads the data into a new location. The general notion of analyzing data is that it is stored in a location (generally a disc storage) and is accessed from there for analysis, often leaving the portion that is not required behind in the disc. This has to do with the cost of computing resources, since ETL is a very processing intensive operation it is recommended that it be done in the background, in memory. Although if both the data access and processing could happen in memory things would be extremely fast, but there is a reason why your computer storage is in Terabytes and memory in Gigabytes and that is because memory is extremely costly. This is a concept that is manipulated by data virtualization where the sources of data are distant but it is made to look like all the data is in one place for access to the users, from one common interface. This gives an advantage that data need not be stored in two separate locations, one for storage and one for analysis. But again, if there is a requirement of high ETL processing, virtualization will not do the trick and it is advised to use the general old-school approach.

Data Federation- With data federation, not only do we make it look like data is in one place, but we actually fit that data into a common integrated data model. We perform all the same transformations and establish all the same relationships among data entities that we would do in a physical database, but we do it all virtually. That is, without ever actually moving the data. Of course, this comes at the cost of even more complex processing that can result in slower performance when data is accessed or extracted. Both data virtualization and data federation are usually accomplished using specialized software applications that connect to a variety of different source systems. While they eliminate the need to move data using ETL processes, they still require development and maintenance to establish those connections and present a unified view of data to users. It is somewhat similar to what we do with code in GIT. Both virtualization and federation are great when the amount of data and processing required is not very big. As the complexity of operations increases, both gradually become inefficient.

In-Memory and In-Database are options that will speed up the analytical operations and give results faster than the above mentioned options, but also they come with a higher cost of operation and management. Finally, a Relational Database is the most popular storage and analysis service used in the industry. An RDBMS application is used to manage these databases and they operate in a row-and-column approach with a strictly defined schema. Maintaining a schema has its advantages of keeping data intact, performing normalization that eliminates redundancy by ensuring that we store unique data only once versus multiple times.

Tools of a Data Analyst

I will bring this discussion to an end with the most exciting conversation — the analyst’s toolbox. Data Analysis is an evolving process and so are the tools we use day to day, they evolve every day and will continue to expand as the quantity of data increases. I have grouped these in 6 broad categories that will be a go-to for any analysis process. Let us see each one of these with brief details.

Database Systems: Create, maintain and extract data. Databases are primarily used for keeping an updated score of where and how the data is stored and to keep all of it located together. This embarks the start of any analysis journey. Examples could be SQL Server, PostgreSQL, MySQL or some NoSQL systems like MongoDB. These are the skills you would need to start your analytical journey. Standard Reporting: These were the first ever reporting tools created. Stable repetitive use of data is presented using standardized reports and is meant for business stake holders. Automating generation of these reports periodically (monthly, weekly, daily, hourly, etc.) is something that a data analyst needs to know because many reports do not need specific tasks but just need updating, as and when new data arrives. These could include Power BI, Excel and others, although all of these tools are now much more advanced and can perform various complex tasks other than standard reporting. Dashboarding: It is a superset of standard reporting where from a large set of reports, the most important and relevant ones are pulled out, put into a visualization and then collectively presented on a web page or to a stake holder. This emphasizes on a concept called summary statistics where not everything is shown, but the most important parts are shown in a way that all the information is conveyed. Tools for Dashboarding are Tableau, Adobe Analytics (Report Builder), etc. Data Visualization: While we may or may not spend time developing reports or dashboards, the idea of data visualization is squarely within the domain of the data analyst. Visualization is the process of arranging data in such a way that we can more easily see what’s going on and draw conclusions based on what we see. It hides the complex algorithms running behind the scenes and shows data visually for building generalized conclusions. Some important skills to gather here are libraries in Python and R like Pandas, Matplotlib, Seaborn, ggplot and some standalone tools like Tableau, Power BI, Excel. Data Exploration: Data exploration is an intelligent extension of the idea of data visualization. Data exploration tools seek to proactively guide the data analyst by automatically scanning data and providing cues or suggestions on what thing the data analyst might look at next. Statistical Modelling & Advanced Programming: The last class of tools we talk about are modelling and programming tools which are the backbone of a data scientist’s work. These tools are used to execute highly sophisticated analytical procedures on data, often using statistical techniques.

This image from University of Colorado, Boulder explains perfectly the landscape of tools that are used for the various portions of a data analysis lifecycle.

Understanding the above image will make the entire process extremely simple. On the horizontal are the subjects of analysis mentioned above and from this horizontal up are the tools used for each of the processes. As we go vertically up the ladder, the cost of the tools and its complexity increases. Also, some tools have a horizontal span of more than one aspect, this states that those given tools can perform all the horizontals that they cover. For example, Excel is capable of doing reporting tasks, dashboarding, visualization, exploration and modelling. Similarly Power BI and Tableau can be used for dashboarding, visualization and exploration.

Approaches and Options to Data Analysis

Based on what environment and methodology you use, there are various approaches an analyst can take to derive to decisions. I will be discussing a few important derivates here:

Intermediate File Approach- This method relies on a complete file being transferred from the data source to the analytical tools, where the analysis will occur. This is best suited for data that is static and will not change over time and where a one time analysis is required on a very large dataset, usually an offline data source.

This is a consideration where a file is exported from a data storage and moved to a tool that will perform the analysis and reporting

This approach will not be ideal for live streaming data or constantly evolving data since transferring files to analytical tools might overlay the operation execution time and hence delay the entire process.

Direct Connection Approach- Here the tool is directly integrated with the database management system using an API or an ODBC connection string. These connection strings basically create a route for the data to be fetched by the analytical service engine and hence as the data in the source refreshes, it would automatically reflect in the API. Therefore, it can be used for evolving data and reduces the step of an external file creation.

All integration services occur in the database system

Although, it would require a connection setup and extraction of data without the connection (offline analysis) will be difficult to execute.

Downstream Integration Approach- It is quite often the case in data analytics that the information we need is located in a bunch of different locations and formats. While it would be nice to have everything on one warehouse, it doesn’t always make sense to spend the time and energy to put everything there prior to analysis. Especially if we’re not entirely sure those sources will turn out to be important.

Integrations happen directly on the analysis tools — this is helpful when more than one data sources exist and the upstream data is in varied formats

In cases like this we form a broader set of API and ODBC connections to the tool which is going to perform the analysis and operate all of them concurrently. We then use additional functionality in the tool to integrate the data from sources and construct the required dataset for analysis. This of course requires a higher quality toolset and lot of code to get along. Take an example of a Python script which connects data from SQL, Excel and CSV files and then forms a new data frame to perform analysis.

The Final Cut

Data Analysis and Reporting is a never ending process just like any other development task. Requirements are created, then the product is delivered. But with time the requirements keep evolving and the product continues to improve. What I have mentioned in this entire article is an overview for someone who is recently moving into the data analysis space and wants to know how the business aspect of projects function. I hope this gives you a fair picture of how things look from a business perspective. I will continue to author more stories and articles in this series with an aim to equip all data aspirants to stay hand-in-hand with business.