Data Science for Everyone: Getting To Know Your Data — Part 1
By: Sumudu Tennakoon
Tags: Data Science, Machine Learning, Data Scientist, Artificial Intelligence, Education

Data: Formulating the Concepts

Definitions

The word data is the plural form of the word datum, which has the meaning of a “single piece of information, as a fact, statistic, or code” [5]. Another definition is “something given or admitted especially as a basis for reasoning or inference” [6]

In simple terms, data can be defined as numbers, characters, words, sounds, or symbols that can use to describe, quantify, recognize physical or virtual entities.

For example, if you can sufficiently describe a person with some data points (datums) such as name, date of birth, gender, appearance (colors and built), height, weight, etc. The same information can also be used to differentiate one person from another for recognition purposes.

Figure 3: Data field with a value assigned to it. (Image by author)

Let’s look at this conversation between two people “That [tall] [boy] with [brown hair] working as a [barista] at [ABC Cofee Shop] helped me when my car broke down in front of his shop. I think his name is [James]”. The words within [] are the data points you may use to recognize the specific person in a normal conversation as well in a systematic data application.

Data points are sometimes mentioned as features, data fields, characteristics, facts, and attributes. Which should be taken as the same concept at the high level.

A collection of data fields we can use to describe a person can be called a data model of a person. That becomes a record when the values are assigned to those fields. Similarly, we can represent other physical objects like vehicles, buildings, books using their data points which can describe their characteristics.

Figure 4: Data fields used to represent a person. (Image by author)

Several related records can be arranged into a structure such as a table or a list. Imagine a table containing data about 100 different people one row representing each person and each column used to store one data point.

Figure 5: Table

Many related data structures are combined into one larger structure that becomes a database. Depending on the application, there are multiple types of databases and database management systems to choose from.

Data and Information

We looked at the basic concept of the data everyone should know. Let’s quickly look at another related concept always mention with the data. That is Information. Let’s try to understand the difference between information and Data.

As we discussed above, data comes with two main components, structure and context. Without them, data has no meaning or value.

When data is taken with structure, context, and meaning, we call it information.

Here is an example, you got some data: a list of values with different color names. It is certainly data but can it alone give you any context? Is that data meaningful or useful? The answer could be no for both the questions.

Figure 6: Information (Photo by William Iven on Unsplash)

What if the same list is given with another data point for each color value: a car model and brand? The data now have some context and meaning. How about adding another data point: price? We now added some context to the data and can be used to derive useful information from it.

In the person data example illustrated in Figure 4, all the attributes “name”, “date of birth”, “height”, “weight”, etc. have no use unless they are connected to the person entity (arrows in the figure).

Data Organization

Based on how they are arranged, data collections can be categorized as structured, unstructured, and semi-structured.

Figure 7: Illustrating structured, unstructured, and semi-structured data. (Image by author)

The most traditional form of data collection is structured where the data is organized into tables which is easy to handle by both the human and machines. Structured data are easier to search and manage.

Unstructured data like images, videos, sound, and large text contents like books, letters, paragraphs are used by humans for many centuries of years even before the computer era. Managing and searching through unstructured data is quite cumbersome. According to a Forbes Technology Council post referring to Gartner, an “estimate that upward of 80% of enterprise data today is unstructured [7]”. Therefore, the research and development efforts in data science are heavy in this area.

With the advent of the internet and evolution of the computer technology, semi-structured data forms like mark-up languages, hierarchical data structures have become popular in storing and transmitting data. Semi-structured data considered self-describing data and it helps to store complex data that cannot easily organize into tables.

Figure 8: Semi-structured data formats (XML and JSON). (Image by author)

In the process of analyzing the data, unstructured data is converted into a structured or semi-structured form utilizing suitable data science methodologies.

Data types

Let’s now take a different perspective on understanding data.

Data can be represented in many different ways such as numbers, characters, symbols, pictograms, colors, signs, object arrangements, etc.

Figure 9: Different data representations and conversion into digital form. (Image by author)

In digitally computerized data representation every other representation will melt down into numbers and ends up in the binary form when storing, transmitting, and computing.

We can find a system of data types used in computing such as boolean, integer, float, and characters, as primary types.

The derived types of these are strings, arrays, lists, sets, vectors, matrices, tensors, complex numbers, structure, enumerators, dictionaries, tables, and objects made possible the complex computing applications we all benefit from in this era.

A special data type is known as null, none, or void depending on the programming language is used to represent “nothing”.

An in-depth discussion on the different data types and their uses is planned for a future article.

Figure 10: Data Types used in computing. (Image by author)

Data File Types

Traditionally you access data from printed materials, videos, display boards, etc. In a computer system, your data comes as files or streams. The common file types are text, binary, images/photos, audio, video, archive, compressed, database. A detailed discussion on these file formats and their uses is planned for a future article.

Data Encoding

The data you accessed is not always stayed in the same format as it presented to you. We learned the ultimate form of data will be binary (1/0) in the digital systems. However, there are intermediate representations used when storing and transmitting data.

When data is moving from one location to another their representations can also change. We call that encoding and decoding with reference to one representation. One of the common encoding schemes is the American Standard Code for Information Interchange (ASCII) [8] which use to represent characters (letters, digits, and special signs). Figure 10 illustrates the encoding of the text “DATA@8” into ASCII. The universally accepted encoding scheme for the same purpose is known and UNICODE [9].

Figure 11: Character Encoding Example (ASCII) (Image by author)

Encoding should not be confused with encryption, which hides data content from unintended parties. A detailed discussion on the encoding-decoding, encryption-decryption and their uses are planned for a future article.

Analog vs. Digital Data

In nature, the data exist in analog form and we need to convert them into machine recognizable binary form to be used with digital computing machines. The term “digitization” is used to name this conversion process [10]. In electronics, this sometimes called analog to digital encoding.

Some examples of digitization are scanning a paper document to create its digital copy, recording a sound from your mobile phone mic, recording your walking track using GPS data.

When the digitally stored data need to serve as an analog output, digital to analog conversion is used. You are using this analog to digital conversion and vise versa in your personal devices such as mobile phones, video displays, sound recorders, cameras, music players, etc.

Figure 12: Analogue and Digital conversion: digitization of a sound signal. (Image by author)

Qualitative and Quantitative nature of data

Each data measurement can also be classified as qualitative and quantitative and their subclasses by the nature of values they can take.

Figure 13: Classifying data measurements by their Qualitative and Quantitative nature. (Image by author)

Data is a broad concept that can be examined from a variety of perspectives. The more you combine those different perspectives, you will get a better grip of the data you are dealing with. Therefore, the concepts we discussed above are crucial at every stage in the data science workflow. They are also important at any time you engage with the data or data-driven applications.

Measuring Data

Data is quantifiable. Data is quantifiable. The smallest unit of digital data is called a bit, which is also used as a scale to measure data. A single bit can store a value of either 0 or 1.

In the data encoding example, we showed that ASCII uses 7 bits which makes 2⁷ = 128 possible combinations. In other words, 128 different characters can be represented using an ASCII value.

A group of 8 bits (octet) is considered as one byte which is the fundamental unit of measuring data. The symbol defined by the International System of Units (SI) is B. To measure large quantities of data the SI prefixes such as Kilo (K), Mega (M), Giga (G), etc. are used [11]. The use of these prefixes must not be confused with the binary interpretation of prefixes used in many applications like Ki, Mi, Gi, etc.[12,13]

Figure 14: System of units for measuring digital Information (Image by author, information source: [12])

Data visualization

You have seen charts, plots, and various infographics condensing data and information into graphical representations. Visualization is a very efficient way of communicating data. It is also important in the early stages of data science workflow to understand the data, and various quality control measures before moving into the later stages. Some argue data visualization is both an art and science. An in-depth discussion on data visualization methods and their uses are planned for a future article.

It is also important in the early stages of data science workflow to understand the data, and various quality control measures before moving into the later stages. Some argue data visualization is both an art and science. An in-depth discussion on data visualization methods and their uses are planned for a future article.