Level Up Your SQL Skills
Tags: Programming, MySQL, Database, Sql, Software Development

Level Up Your SQL Skills

Image from Caspar Camille Rubin on Unsplash.

According to JetBrains’ 2020 Developer Ecosystem Survey, 56% of developers utilize or interact with a SQL database in their development environment. Even so, most developers view SQL concepts as a secondary concern, as they are often interacting via their chosen language and do not play much of a part in the design. Generally speaking, being able to connect and run basic queries will get you through the day.

At many companies, there is even a separate database team that will be handling the database design and optimization, so you will never get first-hand exposure. I think this is often the deterrent to learning more about SQL. It’s viewed as a utility that is often handled by others. Software developers have enough on their plate, right? Fair enough.

However, in our industry, having full-stack skills is always a good idea. You never know what will be thrown at you. We interact with SQL daily, so more knowledge will always come in handy. Additionally, if you end up at a smaller company that does not have the resources for a full database team, more of those responsibilities will fall on you. Being able to state that you are confident in your database skills during an interview will open up many interesting career paths.

I personally enjoy SQL development more than most. I interned early in my career with a team of outstanding database administrators and quickly fell in love. I had taken my database courses in school, so I had some basic knowledge, but what I learned during that internship has been invaluable to me. Not only did it teach me that I have a secret passion for data, but the technical understanding I gained has been a boon in every job I’ve gotten going forward.

http://ajax.actiup.com/out/Video-Columbus-Crew-Seattle-Sounders-v-en-gb-1umt-2.php

http://ajax.actiup.com/out/video-Columbus-Crew-Seattle-Sounders-v-en-gb-1nrc-14.php

Note: I have experience with Microsoft SQL Server, so that is what I will reference throughout. However, these concepts absolutely apply to other SQL DBMS options.

Normalization

If you’ve taken a course on relational databases in school, you have surely encountered the term normalization. It’s the core concept of database design. However, as a developer, here’s why you should care: It will make your life easier.

The big idea behind database normalization is to reduce data redundancy and make sure that data is stored in a clean, logical manner. The level to which a database is normalized is based on which “normal form” it achieves. To date, there are debatably up to six normal forms, but in reality, all you need to understand are the first three.

Here are the big rules in simple terms:

First Normal Form (1NF) states that a single field should store a single data point and that all rows should be unique. Don’t store a list of things in a field. Those should be individual records.

Second Normal Form (2NF) states that each table should have a single primary key column. Create unique IDs. Instead of using name and address as a compound key, just give the user a unique ID number or GUID.

Third Normal Form (3NF) states that no functional transitive dependencies are allowed. This one might be the most complicated, but essentially it means that if you have a field that is dependent on another field, it should be broken out to another table. This allows the foreign key to change — not the actual value.

The important point here is that your data should be atomic, so large tables with repeated data should be broken out into smaller tables with easily searchable keys. This makes filtering more efficient and querying more straightforward. It literally makes your job is easier!

Database Objects

If you are in a more full-stack role or with a company that doesn’t have a database team, you will likely be responsible for database development. In this case, you should have a good idea of which kinds of database objects to use in different scenarios.

In short, database objects are anything you can generate with a create script that stores or accesses data. The most obvious example is a table, which is the building block of the database. These are clearly going to be the go-to for data storage.

However, there are other kinds of objects that access and transform that data in different ways. Using the best object for the job is important. Here are the basic rules that I use when deciding which to use for a use case:

Note: Indexes are technically database objects since they affect how data is stored and accessed.

Stored procedure: This is the go-to for processing or transforming data. Stored procedures can be created and then run on a schedule if you need to make updates at certain times of the day or on a certain cadence.

View: A view is essentially a stored query and it allows you to select from a dynamic set of data. I have often replaced cached tables with views if the data needs to be fresh and possibly transformed in some way. No changes are actually being stored, but you can include calculated fields, which make views a great option when you need dynamic, up-to-date data.

With an understanding of tables, views, and stored procedures, you will likely be able to handle most database development tasks thrown your way.

Image from Benjamin Lehman on Unsplash

Indexing

The first major indexing concept you need to understand is the difference between a clustered and non-clustered index.

Clustered index: Stores all of the data in the table based on your index. There can only be one clustered index, and by default, your primary key is clustered.

Non-clustered index: Stores the indexed field separately from the table. This indexed field points to the data stored in the table, just like the index of a book. You can have as many non-clustered indexes as you want!

Note: The trade-off for more efficient queries is increased storage. Non-clustered indexes require their own storage since they are kept separate from the table data. Only index the fields that you need to filter or join on in order to find the best balance between storage and performance.

Indexing is the #1 way to optimize database performance. Indexing affects the way that data is stored in pages. By default, records are not stored in any guaranteed order, so essentially indexing a field causes the data to be stored in a predictable manner. Therefore, when you filter on that field, the query will run faster because SQL Server knows exactly where to start looking.

There are many indexes that you can set on a field for specific purposes, but if you are attempting to improve performance, you will generally be creating a generic non-clustered index.

Pro tip: If you are creating indexes, you have to maintain them. Inserts, updates, and deletes cause fragmentation of your indexes. Therefore, unless your data is completely stagnant, you will need to rebuild your indexes based on how often the data is changing. Ideally, you should do this with a scheduled stored procedure so that performance is maintained consistently.

The SQL statement to create an index is pretty straightforward. The following statement will create a non-clustered index with the name idx_orderdate on the given field of the Orders table:

CREATE INDEX idx_orderdate ON Orders (OrderDate);

Reading a Query Plan

If you are interested in understanding database performance on a deeper level, being able to read a query execution plan is a useful skill. When you run a query, SQL Server is processing it based on an execution plan. You can generate the plan upon running your query or view a cached plan if your query has been running. This allows you to see how much time is being spent on each step in the query.

There is plenty of nuance that an experienced SQL wizard can pull from a query plan. However, for the standard developer, what you need to know is the difference between a seek and a scan.

Scan: This can be an index scan or table scan, but either way, SQL Server is having to scan the entire table or index for the data.

Seek: In the case of an index seek, SQL Server is able to identify the data you are querying for based on the given index.

As a rule, seeks are more efficient. Typically, if you are looking to optimize a query, you should identify where scans are happening in your query and fix them. The easiest way to fix them is to make sure you’re joining on indexed fields.

Pro tip: SQL Server will let you know if you are missing an index that it could be using in your query. The suggestion will appear at the top of the query plan. If you right-click on the query plan where an index is suggested and select “Missing Index Details,” it will even generate the index creation script for you!

Congratulations!

You have now leveled up your SQL skills! Using these concepts, you can now optimize your database and query more efficiently. Whether you are querying from an application or building out your own stored procedures and views, these tips will make your job easier!

Good luck out there!