Business Analytics and Fundamentals of Data Storage and Data Interaction
By: Matthew G. Brown
Tags: Analytics, Data Science, Business Intelligence, Database, Data

A beginner’s guide to navigating terminology in an emerging field

The study of business analytics is a field which has many components, both technical and conceptual. The goal of the field is simple: find new ways to extract, process, and load data, and to drive insight from this data. While the goal of the field can be defined with a wide scope, understanding the different facets of the field is both helpful and necessary for most employees in today’s business environment. Additionally, such an understanding can enable employees to be better suited for the ever-changing enterprise business landscape. This blog will focus on some of the most important topics in business analytics and should serve as a starting point for any aspiring data analyst or data scientist.

What is Business Intelligence?

Before going into some of the tools and processes used by data scientists, it is important to understand some of the fundamental topics of data and analytics. While most larger companies have already begun to implement “business intelligence” teams in their organizations, many employees lack an understanding of what these types of departments do. Simply put, business intelligence is the applied usage of any tools or processes for gathering, storing, processing, transforming, and analyzing any business data. Business intelligence involves benchmarking data, data mining, managing dashboards, and the management of databases and data warehouses.

Data can come from any number of sources, and it is often the job of business intelligence teams to manage such data sources in addition to determining what data is necessary to the organization. Especially during a time in which data is being created and disseminated at unprecedented rates, it has never been more necessary for such business intelligence teams to “look past the noise” — that is, being able to understand what data can help drive insights and what data is worthless. However, as was mentioned, gathering and understanding data is not the sole role of employees within these business intelligence teams; another key component of business intelligence is that of the management of data.

Business intelligence teams are often the designers and managers of various databases and data warehouses within an organization, and it is extremely necessary for such teams to understand the design and storage tactics for their organization’s data. Why is this so important? Analysts are constantly using both current and historic data in order to drive insight or assist with decision making. Without an understanding of the data storage scheme, such analysts lack an ability to adequately employ all possible data. Additionally, in order to ensure that the right decisions are being made, business analysts must often create processes in which new data is loaded into some created database or data warehouse. Though these two concepts might sound the same to most, understanding the difference between a database and a data warehouse is another fundamental concept which is necessary for any aspiring business analyst to understand.

Databases vs. Data warehouses: What’s the Difference?

The term “database” is one that most in the general public understand. However, the usage of the term of often incorrectly referring to a data warehouse. A database is simply an organized collection of logically related data, which is designed such that it meets a certain defined need of an organization. Databases are designed and populated such that they serve a specific purpose for some application of data. These systems are the standard, traditional databases that most would think of when they hear the term. Databases are also referred to as online transaction processing systems, generally abbreviated “OTLP.” Such OTLP systems are generally hyper-focused in their usage and design, and as such, it becomes very difficult for OTLP systems to interact with one another. However, newer approaches to business intelligence have called for greater interaction and on-demand analytics. As such, these requirements often call for what is known as a “data warehouse.”

Unlike databases, data warehouses place a greater focus on analytical processing, and these systems are often referred to as online analytical processing systems, abbreviated “OLAP.” While databases largely involve the usage of CRUD operations — creating, reading, updating, and deleting data — data warehouses generally provide read-only operations. However, OLAP systems allow for both current and historical data reporting, and these systems are often used to aggregate and manipulate data in order to gain insight. Additionally, while databases are designed using what is known as an entity-relationship model, a conceptual design approach, data warehouses are designed with a specific schema consisting of facts and dimensions. Understanding the difference between these two concepts is an important foundation for the field of analytics. With such an understanding, employees can begin to understand both how and when to leverage these two components of a business environment in order to drive insights or create greater efficiencies within some process.

Extraction, Processing, and Profiling: The Key Steps

Now that an understanding of data storage systems and has been established, it is necessary to understand the steps an analyst needs to take in order to create actionable insight from data. In almost all business environments, data will be sourced from many different locations. Different departments might handle their data in different ways, and it is a crucial role of a data analyst to understand the first component of this process: information extraction.

Information extraction can manifest in a number of ways, but this step concerns itself with the retrieval and conglomeration of any data needed for a process. In this step, determining whether individual components of the retrieved information are important is not necessary. Rather, the information extraction phase relies on an automated retrieval of any data related to the question which is sought to be answered. This process can be iterative, and the determination of the necessary data comes in the second phase: data processing.

During the data processing phase of making data “business-ready,” data is adjusted and organized such that it follows some logical structure. It is in this phase that an analyst would make the data easier to read and understand, as well as making it ready for additional downstream processing. There are tools that have been created to perform this phase automatically, especially within newer visualization tools, but the goal of this phase is to enable seamless reporting and insight generation during the final phase: data profiling.

In this final phase, data profiling, all previously extracted and processed data is observed in addition to all available data sources. This phase is closely related with the previous processing phase, and the two are often performed in tandem, especially within systems which have created automation for these necessary steps. During this phase, insights about the data can begin to be drawn, and summaries about the data can be created. Additionally, this phase requires analysts to assess various risks. Such risks include the risk of system functionality with increased scale, the risk involved in integrating the data across multiple departments, and the risk of future relevance.

While actionable insight is not the goal of this phase, the necessity for data profiling is still very evident. Without properly extracted, processed, and profiled information, the job of a business intelligence team becomes exponentially harder. Rather, if processes are created such that data is manipulated and made easier to understand prior to future analytical processes, true insight can be drawn more quickly and with less energy involved.

Querying and SQL: Excel on Steroids

Understanding the necessity for data extraction, processing and profiling is certainly important for anyone looking to join a business intelligence team or pursue a career in data science. However, with the growing rate of data generation, many large-scale businesses generate terabytes of data each day. Given this fact, previously accepted business reporting tools such as Microsoft Excel or Apple Numbers are no longer adequate for working with data. These programs are limited in their capacity, with both placing a cap of one million on their allowed number of rows. As such, the need for databases and data warehouses are even more evident than ever. Though costs are variable depending on the amount of storage needed, these systems allow for virtually unlimited storage of data. However, with such immense amounts of data and a lack of ability to use commonly known tools like Excel, it has never been more necessary for all employees to understand the concept of querying and the applied usage of SQL programming language.

SQL is a programming language which stands for “Structured Query Language,” and it is among the most commonly used languages for database communication. While the first relational database concept was created in 1970, it was not until 1979 that SQL was brought to the public domain; it was then that Oracle released and marketed their first available relational database systems with integrated SQL language support. Over the years that followed, SQL was adopted by most large organizations with database systems, and SQL was ultimately recognized by the American National Standards Institute in 1986. Today, SQL has become the most well-known database interaction programming language, and it is generally the default for most database systems.

The SQL language involves three classes of commands: DDL, DCL, and DML. DDL, or “data definition language,” are commands that are used to define a database. In DDL, SQL is used to create, alter, and drop tables. DDL primarily concerns the CRUD operations that were previously touched on. While reporting analysts would not likely use the DDL class of SQL commands, it is important to understand that such analysts could invoke these commands accidentally or with malicious intent. In order to ensure that situations such as these do not arise, another class of SQL command is used: DCL.

Example of table creation using the DDL class of SQL commands

DCL, or “data control language” is the set of commands that allow access rights to certain users in a database system. These commands are employed to administer or revoke certain privileges in database environments, and such commands are extremely important. In a business environment, data control language commands can ensure that compliance is met regarding the viewing of certain confidential information or classified records. Additionally, DCL commands are used to commit data into the database, so it is extremely important that database engineers are aware of this power.

The third of the SQL command classes is that of DML, or “data manipulation language.” DML is the most commonly used component of an analyst, as DML-class commands are designed to display, or “query” data based on user-defined parameters. DML can be used to insert new records into a table, update tables, and delete records within a table, but DDL is the only class of commands which can delete or “drop” a table entirely. Within DML, the command which is likely the most used is that of the “SELECT” command, and this command is what enables true analyzation of data within some database.

The SELECT command can be used for queries on a single table or across multiple tables at once. In addition, when paired with other conditions or “clauses,” the SELECT command allows for the filtering and slicing of data in many different ways. Though many different DML commands exist, it is important to understand some of the most-used clauses within a SELECT statement, and how they allow for greater reporting abilities.

Example of a query using SELECT, FROM, and WHERE statements, components of the DML class of SQL commands. The resulting output is shown below the written script.

The FROM command is a clause which is almost always used in a SELECT statement, and this clause is used to indicate the table or view from which the data is to be pulled from. The WHERE clause is used to indicate some conditions under which a row would be included in the resulting query. The GROUP BY clause is used to categorize results based on some dimensional parameter. The HAVING clause is used to further filter the data through an indication of the conditions to which the group must satisfy in order to be in the final result. Lastly, the ORDER BY clause is used to sort the resulting query according to some user-defined criteria.

Though these clauses do not represent a full list of all clauses that can be used with a SELECT statement for querying, having an understanding of these clauses and their associated abilities to manipulate the data is a critical component for anyone interacting with a database. Furthermore, during a time in which companies are finally beginning to understand the necessity for large-scale databases, having the ability to effectively interact with these databases could give any prospective employee an edge in the search for employment.