Understanding KeyConditionExpression and FilterExpression in DynamoDB
Tags: aws, dyanmodb, learning, database
Published: 2024-05-16 19:32:49

> **tl;dr;** Use `KeyConditionExpression` to define the primary criteria for your queries, and apply `FilterExpression` to refine the results as needed. This approach helps manage read capacity costs while ensuring you get the precise data you need.

## Introduction

[DynamoDB](https://aws.amazon.com/dynamodb/) is a powerful NoSQL database provided by AWS, designed to handle large amounts of data efficiently. However, for newcomers, understanding the nuances of querying DynamoDB tables can be challenging, particularly when it comes to the differences between `KeyConditionExpression` and `FilterExpression`. This blog post aims to clarify these concepts and provide practical examples of their usage.

## KeyConditionExpression vs. FilterExpression: The Basics

### KeyConditionExpression

`KeyConditionExpression` is used in DynamoDB queries to filter items based on their partition key and optionally their sort key. It is a fundamental part of querying DynamoDB tables and is required in any query operation.

### FilterExpression

`FilterExpression`, on the other hand, is used to filter the results of a query or scan operation after the data has been retrieved. It applies additional criteria to the data that is returned by the `KeyConditionExpression` or scan operation, but it does not affect the items read from the table.

## Detailed Comparison

### 1. Available Attributes

**KeyConditionExpression**:
- Can only be used with partition key and sort key attributes.
  
**FilterExpression**:
- Can be used with any non-key attributes, providing more flexibility in filtering the data.

### 2. Available Operators

**KeyConditionExpression**:
- Limited to basic comparison operators such as equals, not equals, greater than, less than, and between.
- Supports starts with operation for strings.
- Only one operator can be specified per attribute.

**FilterExpression**:
- Supports all operators available in `KeyConditionExpression`.
- Additionally, includes operators like `IN` (checks if an attribute is within a list of values) and `NOT` (checks if an attribute does not match a specified value).
- Allows for more complex and varied filtering conditions.

### 3. Read Capacity Consumption

**KeyConditionExpression**:
- Consumes read capacity units based on the size of the data stored in DynamoDB, regardless of how many items match the query.
  
**FilterExpression**:
- Applied after the data is retrieved, so it does not consume additional read capacity units. However, the initial query still consumes read capacity based on the size of the data being processed.

## Practical Examples

### Using KeyConditionExpression

The `KeyConditionExpression` is used to specify the partition key and optionally the sort key in a query. Here’s an example:

```sh
aws dynamodb query \
    --table-name YourTableName \
    --key-condition-expression "PartitionKeyName = :partitionValue" \
    --expression-attribute-values '{":partitionValue":{"S":"YourPartitionValue"}}' \
    --profile YourProfileName \
    --region YourRegionName
```

This command queries the table for items with the specified partition key.

### Using FilterExpression

The `FilterExpression` adds additional filtering criteria to the query results. Here’s an example:

```sh
aws dynamodb query \
    --table-name YourTableName \
    --key-condition-expression "PartitionKeyName = :partitionValue" \
    --expression-attribute-values '{":partitionValue":{"S":"YourPartitionValue"}}' \
    --filter-expression "NonKeyAttribute = :filterValue" \
    --expression-attribute-values '{":filterValue":{"S":"YourFilterValue"}}' \
    --profile YourProfileName \
    --region YourRegionName
```

This command filters the query results to include only items where the `NonKeyAttribute` matches the specified value.

### Points of Concern

- **Use KeyConditionExpression whenever possible**: This ensures efficient querying and minimizes read capacity consumption.
- **Use FilterExpression rarely**: Apply it only when necessary to avoid additional complexity and potential performance impacts.