Programming with Databases in Python using SQLite
By: Parul Pandey
Tags: Database, Python, Sqlite, Data Science, Programming

Photo Credit: Pixabay

If you are aspiring to be a data scientist you are going to be working with a lot of Data. Much of the data resides in Databases and hence you should be comfortable accessing data from databases through queries and then working on them to find key insights.

Data forms an integral part of the lives of Data Scientists. From the number of passengers in an airport to the count of stationary in a bookshop, everything is recorded today in form of digital files called databases. Databases are nothing more than electronic lists of information. Some databases are simple, and designed for smaller tasks while others are powerful, and designed for big data. All of them, however, have the same commonalities and perform a similar function. Different database tools store that information in unique ways. Flat files use a table, SQL databases use a relational model and NoSQL databases use a key-value model.

In this article, we will focus only on the Relational Databases and accessing them in Python. We will begin by having a quick overview of the Relational databases and their important constituents.

Relational Database: A Quick Overview

A Relational database consists of one or more tables of information. The rows in the table are called records and the columns in the table are called fields or attributes. A database that contains two or more related tables is called a relational database i.e interrelated data.

The main idea behind a relational database is that your data gets broken down into common themes, with one table dedicated to describing the records of each theme.

i) Database tables

Each table in a relational database has one or more columns, and each column is assigned a specific data type, such as an integer number, a sequence of characters (for text), or a date. Each row in the table has a value for each column.

A typical fragment of a table containing employee information may look as follows:

The tables of a relational database have some important characteristics:

There is no significance to the order of the columns or rows.

Each row contains one and only one value for each column.

Each value for a given column has the same type.

Each table in the database should hold information about a specific thing only, such as employees, products, or customers.

By designing a database this way, it helps to eliminate redundancy and inconsistencies. For example, both the sales and accounts payable departments may look up information about customers. In a relational database, the information about customers is entered only once, in a table that both departments can access.

A relational database is a set of related tables. You use primary and foreign keys to describe relationships between the information in different tables.

ii) Primary and Foreign Keys

Primary and foreign keys define the relational structure of a database. These keys enable each row in the database tables to be identified and define the relationships between the tables.

Primary Key

The primary key of a relational table uniquely identifies each record in the table. It is a column, or set of columns, that allows each row in the table to be uniquely identified. No two rows in a table with a primary key can have the same primary key value.

Imagine you have a CUSTOMERS table that contains a record for each customer visiting a shop. The customer’s unique number is a good choice for a primary key. The customer’s first and last name are not good choices because there is always the chance that more than one customer might have the same name.

Foreign Key

A foreign key is a field in a relational table that matches the primary key column of another table.

The example above gives a good idea of the primary and foreign keys.

Database Management Systems

The Database management system (DBMS) is the software that interacts with end users, applications, and the database itself to capture and analyze data. The DBMS used for Relational databases is called Relational Database Management Systems(RDBMS). Most commercial RDBMSes use Structured Query Language (SQL), a declarative language for manipulating data, to access the database. The major RDBMS are Oracle, MySQL , Microsoft SQL Server, PostgreSQL , Microsoft Access, and SQLite .

We have barely scratched the surface regarding databases here. The details are beyond the scope of this article.However, you are encouraged to explore the database ecosystem since they form an essential part of a data scientist’s toolkit.

This article will focus on using python to access relational Databases. We will be working with a very easy to use database engine called SQLite.

SQLite

SQLite is a relational database management system based on the SQL language but optimized for use in small environments such as mobile phones or small applications. It is self-contained, serverless, zero-configuration and transactional. It is very fast and lightweight, and the entire database is stored in a single disk file. SQLite is built for simplicity and speed compared to a hosted client-server relational database such as MySQL. It sacrifices sophistication for utility and complexity for size. Queries in SQLite are almost identical to other SQL calls.

Python sqlite3 module

SQLite can be integrated with Python using a Python module called sqlite3. You do not need to install this module separately because it comes bundled with Python version 2.5.x onwards. This article will show you, step by step, how to work with an SQLite database using Python.

Before starting I would highly recommend you all to install DB Browser for SQLite. The browser can be downloaded from their official page easily. DB Browser for SQLite is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite. It will help us to see the databases being created and edited in real time.

DB Browsers’ view

Since everything is in place, let us get to work.

Contents: