How I Learned to Stop Worrying and Love NULL in SQL
Tags: postgres, mysql, sql, null
Published: 2021-04-14 01:45:12

The NULL value is a data type that represents an unknown value. It is not equivalent to empty string or zero. Suppose you have an employee table containing columns such as `EmployeeId`, `Name`, `ContactNumber` and an alternate contact number. This table has a few mandatory value columns like `EmployeeId`, `Name`, and `ContactNumber`. However, an alternate contact number is _not required_ and therefore has an unknown value. Therefore a NULL value in this table represents missing or inadequate information. Here are other meanings NULL can have:
* Value Unknown
* Value not available
* Attribute not applicable
In this post we will consider how NULL is used in creating tables, querying, string operations, and functions. Screenshots in this post come from the [Arctype SQL Client](https://www.arctype.com).

## Allowing NULL in CREATE TABLE
Once we define a table structure, we need to define whether the respective column allows NULL or not. For example, look at the following customer's table. The columns such as `CustomerID`, `FirstName`, `LastName` do not allow NULL values, whereas the `Suffix`, `CompanyName`, and `SalesPerson` columns can store NULL values. 

``` sql 
CREATE  TABLE Customers(
  CustomerID SERIAL  PRIMARY  KEY,
  FirstName varchar(50) NOT  NULL,
  MiddleName varchar(50) NULL,
  LastName varchar(50) NOT  NULL,
  Suffix varchar(10) NULL,
  CompanyName varchar(128) NULL,
  SalesPerson varchar(256) NULL,
  EmailAddress varchar(50) NULL
)
```

Let’s insert a few records into this table using the following script. 
``` sql 
INSERT INTO Customers (FirstName,MiddleName,LastName,Suffix,CompanyName,
SalesPerson,EmailAddress)
 values('John',NULL,'Peter',NULL,NULL,NULL,NULL),
        ('Raj','M','Mohan','Mr','ABC','KRS','raj.mohan@abc.com'),
        ('Krishna',NULL,'Kumar','MS','XYZ',NULL,'Krishna.kumar@xyz.com')
``` 
 ![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/42ig64kjknuapp1kzj6t.png)

## Using NULL in the WHERE Clause
Now, suppose you want to fetch records for those customers who do not have an email address. The following query works fine, but it will not give us a row:
``` sql
Select * FROM Customers WHERE Emailaddress=NULL
``` 

![Values that are NULL cannot be queried using =](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m7te3bb49eqxc9t2ynj1.png)
 
In the above select statement expression defines “Where the email address is an UNKNOWN value”. In a SQL standard, we cannot compare a value to NULL. Instead, you refer to the value as  IS NULL for this purpose. Note: There is a space between IS and NULL. If you remove space, it becomes a function ISNULL().
 ![By using IS NULL instead of equals you can query for NULL values.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v8wxi3fgkbpouxnz96wl.png)

## Integer, Decimal, and String Operations with NULL
Similarly, suppose you declared a variable but did not initialize its value. If you try to perform an arithmetic operation, it also returns NULL because SQL cannot determine the correct value for the variable, and it considers an UNKNOWN value. 
``` sql
SELECT 10 * NULL
 ```
![Multiplying an integer by NULL returns NULL](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v7nsxddbxt485qqyj14w.png)
 
``` sql
SELECT 10.0 * NULL
``` 
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ka5mh82sf5qf7oas5jrh.png)
 
 
NULL also plays an important role in string concatenation.  Suppose you required the customer's full name in a single column, and you concatenate them using the pipe sign(||) .
``` sql
SELECT Suffix,  FirstName, MiddleName, LastName, Suffix, 
(Suffix || ' ' || FirstName || ' ' || MiddleName || LastName ) AS CustomerFullName  
  FROM Customers
``` 
![Setting a string to NULL and then concatenating it returns NULL](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tkzj4k1w0d6pgmtxuypy.png)
 
Look at the result set - the query returns NULL in the concatenated string if any part of the string has NULL. For example,  the person in Row 1 does not have a middle name. Its concatenated string is NULL as well, because SQL cannot validate the string value contains NULL.
 
There are many SQL functions available to overcome these NULL value issues in string concatenations. We’ll look at them later in this article.

## The NULL value in SQL Aggregates
Suppose you use aggregate functions such as SUM, AVG, or MIN, MAX for NULL values. What do you think about the expected outcome- NULL?
``` sql
SELECT Sum(values) AS sum
    ,avg(values) as Avg
    ,Min(Values) as MinValue
    ,Max(Values) as MaxValue
  FROM (VALUES (1), (2), (3),(4), (NULL)) AS a (values);
``` 
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d5cve8nzdvyjncy3sgbh.png)
 
Look at the below figure, and it calculated values for all aggregated functions. SQL ignores the NULLs in aggregate functions except for COUNT() and GROUP BY(). 
 
You get an error message if we try to use the aggregate function on all NULL values.
``` sql
SELECT 
    Sum(values) AS sum
    ,avg(values) as Avg
    ,Min(Values) as MinValue
    ,Max(Values) as MaxValue
           FROM (VALUES (NULL), (NULL), (NULL),(NULL), (NULL)) AS a (values);
```
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ldrjh9qjpgyhbxlgbpy3.png)
 
 
## ORDER BY and GROUP BY with NULL 
SQL considers the NULL values as the UNKNOWN values. Therefore, if we use ORDER By and GROUP by clause with NULL value columns, it treats them equally and sorts, group them. 
For example, in our customer table, we have NULLs in the [MilddleName] column. If we sort data using this column, it lists the NULL values at the end, as shown below.
``` sql
SELECT Suffix,  FirstName, MiddleName, LastName, Suffix, 
(Suffix || ' ' || FirstName || ' ' || MiddleName || LastName )
 AS CustomerFullName
 FROM Customers
 Order BY MiddleName
``` 
 ![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ajiiiut10ajka46cozcp.png)
 
Before we use GROUP BY, let's insert one more record in the table. It has NULL values in most of the columns, as shown below.
``` sql
INSERT INTO Customers (FirstName,MiddleName,LastName,Suffix,CompanyName,
SalesPerson,EmailAddress)
 values('Sant',NULL,'Joseph',NULL,NULL,NULL,NULL);
``` 
Now, use the GROUP BY clause to group records based on their suffix. 
``` sql
SELECT count(*) as Customercount , suffix
    FROM Customers
    Group BY Suffix
``` 
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mnz8kl49phn4gkdwjpqw.png)
 
As shown below, SQL treats these NULL values equally and groups them. You get two customer counts for records that do not have any suffix specified in the customers table. 

 
## Useful Functions for Working with NULL	
We explored how SQL treats NULL values in the different operations. In this section, we will explore a few valuable functions to avoid getting undesirable values due to NULL.

###Using NULLIF in Postgres and MySQL
The NULLIF() function compares two input values.
* If both values are equal, it returns NULL.
* In case of mismatch, it returns the first value as an output.
For example, look at the output of the following NULLIF() functions. 
``` sql
SELECT   NULLIF (1, 1); 
``` 
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c740n9kv59uyvrvd65dn.png)
 
 ``` sql
SELECT   NULLIF (100,0); 
``` 
 ![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bvzva7kvuxnvn21txwkr.png)
 
``` sql
SELECT   NULLIF ('A', 'Z'); 
```
 ![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zaqzzig1lgnylcjg4p2u.png)
 
### COALESCE function
The COALESCE() function accepts multiple input values and returns the first non-NULL value. We can specify the various data types in a single COALESCE() function and return the high precedence data type.
``` sql
SELECT COALESCE (NULL,2,5) AS NULLRESPONSE;
 ``` 
![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r5mv1q7j95lrlohcvjgf.png)
 
``` sql
SELECT coalesce(null,null, 8, 2, 3, null, 4);
``` 
 ![alt text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iw8of0k79f3ydgt6up99.png)
 
We get a blank value in the last query because SQL treats blank space as a first non-null value. 

## Summary
The NULL value type is required in a relational database to represent an unknown or missing value. You need to use the appropriate SQL function to avoid getting undesired output for operations such as data concatenation, comparison, ORDER BY, or GROUP BY. You should not try to prevent NULL values - instead, write your query in a way to overcome its limitations. 


For any additional questions or support regarding this or any other SQL-related topic, please feel free to [drop us a line on the Arctype Discord server!](https://discord.gg/qWy8C35P)