Discover the power of FILTER() in DAX | by Salvatore Cagliari | Towards Data Science

Most of you know something about the FILTER() function in DAX.
But, there are chances that you misuse it or don’t use this function’s full power.
For example, some time ago, I saw a query similar to this:
EVALUATE    SUMMARIZECOLUMNS(        ‘Product’[BrandName]        ,”Sales”, CALCULATE([Sum Online Sales]            ,FILTER(‘Product’            ,’Product’[ProductCategoryName] = “Computers”            )        )    )
While this query is syntactic correct, it is not optimal.
In the following picture, you can see the Timing information from DAX Studio:
A much better version is this one:
EVALUATE    SUMMARIZECOLUMNS(        ‘Product’[BrandName]        ,”Sales”, CALCULATE([Sum Online Sales]            ,’Product’[ProductCategoryName] = “Computers”            )        )
And here is the Server timing without FILTER from DAX Studio:
When you look at the SE CPU time, you can see that the second query needs almost half the processing time without FILTER.
And, while the first query needs three storage engine operations to complete, the second query can be processed with only one SE operation.
Let’s look at why this happens and what we can do with the FILTER() function.
The FILTER function is an Iterator like SUMX and the other X-functions.
Consequently, you can use the Row-Context and Context transition to unleash the full power of FILTER().
If you are not familiar with context transition in DAX, look at my article about this topic:
https://towardsdatascience.com/whats-fancy-about-context-transition-in-dax-efb5d5bc4c01
The fact that FILTER() is an iterator explains why it’s slower than adding a “normal” filter when using it with CALCULATE() or CALCULATETABLE() as shown above.
But, when you need to work with row-based expressions or filter data based on Measures, you need to use FILTER().
Another fact is that FILTER() returns a table. For this reason, you can use FILTER() to generate filtered tables and use them in your data model or your measures.
Because FILTER() returns a table, you can use it to create a calculated table in Power BI or to query your model. Or you can use the function to create a calculated table in your model.
The following query shows an example with the use of FILTER to query a table and filter the result:
EVALUATE FILTER(Store        ,Store[StoreType] <> “Store”        )
And, as an Iterator, you can use the row context to filter the result:
EVALUATE FILTER(‘Online Sales’   ,’Online Sales’[UnitPrice] * ‘Online Sales’[SalesQuantity] > 1000   )
Of course, this is a very costly query, as the multiplication of the two columns has to be executed for every row in the Online Sales table. But it can be run entirely by the Storage engine and, consequently, use all available CPU cores, which makes this approach very efficient.
While the Storage Engine (SE) can process data on multiple CPU cores, the Formula Engine (FE) can use only one CPU core per query. Thus, whatever can be processed by the SE, is more efficient than the processing with the FE.
Next, we can use context transition to find the Stores, which had more than 1'000'000 of Sales:
EVALUATE FILTER(Store        ,[Sum Retail Sales] > 1000000        )
The possibility to use a Measure to filter a table is handy to calculate results, as we will see in the next section.
But, these queries always return all columns from the filtered table. What if I want to retrieve only a subset of columns?
In this case, I can use a table function to get only the columns I’m interested in.
For example:
EVALUATE    FILTER(SUMMARIZECOLUMNS(Store[StoreType]                            ,Store[StoreName]                            ,Store[StoreManager])             ,[Sum Retail Sales] > 1000000             )
Here is the result of the query above:
FILTER() accepts all table functions as the first parameter.
CALCULATE() and CALCULATETABLE() accepts tables as filter parameter.
Due to this fact, you can generate a table with FILTER() and modify the result of the measure with this mechanism.
Look at the following query with the measure [Large Stores]:
DEFINE MEASURE ‘All Measures’[Large Stores] =          CALCULATE([Sum Retail Sales]                     ,FILTER(Store                             ,[Sum Retail Sales] > 100000000                             )                     )  EVALUATE CALCULATETABLE(                SUMMARIZECOLUMNS(Store[StoreName]                      ,”Sales”, [Sum Retail Sales]                      ,”Is large Store”, IF([Large Stores]>0                                              ,TRUE(), FALSE())                      )                      ,’Date’[Year] = 2020                ) ORDER BY [Is large Store] DESC, [Sum Retail Sales] DESC
This query aims to find all Stores with a Sales Amount over 100'000'000 ($, €, or whatever) in 2020.
See the following picture for the result of the query:
Let’s analyse what’s happening here:
The query uses SUMMARIZECOLUMNS() to generate a list of all StoresCALCULATETABLE() is used to add a filter for the year 2020I call the [Sum Retail Sales] measure to get the sales for each Store for 2020I call the measure [Large Stores] and check the result if it’s larger than 0a. If yes, I return TRUE()b. If no, I return FALSE()
The query uses SUMMARIZECOLUMNS() to generate a list of all Stores
CALCULATETABLE() is used to add a filter for the year 2020
I call the [Sum Retail Sales] measure to get the sales for each Store for 2020
I call the measure [Large Stores] and check the result if it’s larger than 0a. If yes, I return TRUE()b. If no, I return FALSE()
The measure uses the existing filter context to calculate the result. The filter context contains the filter for the year 2020.
The FILTER() function considers this filter when evaluating the list of stores with more than 100'000'000 sales. FILTER() returns a list of Stores, which are above the threshold.
CALCULATE() perform the calculation of [Sum Retail Sales] only for those Stores, as the result of FILTER() is used as a filter-modifier in the measure. Therefore we can use a check in the query, like IF([Large Stores]>0,TRUE(), FALSE()), to generate the desired output.
Context transition is an important topic when you’re working with FILTER().
Let’s assume that we want to filter our Sales transactions to get only the rows, which have a value of 1000 or more.
You might be tempted to use FILTER() to construct a table and use this table as a source for SUMX():
Large Sales Amount =SUMX(    FILTER(           ‘Retail Sales’  ,’Retail Sales’[SalesQuantity] * ‘Retail Sales’[UnitPrice] >= 1000  )  ,[Sum Retail Sales] )
Here, I use FILTER() to generate a list of transactions above the threshold mentioned above.
While the result might be correct, I’m triggering a context transition by calling the [Sum Retail Sales] measure inside SUMX.
A better and faster approach is the following measure:
Large Sales Amount =CALCULATE ( [Sum Retail Sales],   FILTER (           ALL ( ‘Retail Sales’[SalesQuantity]                 ,‘Retail Sales’[UnitPrice] ),   ‘Retail Sales’[SalesQuantity] * ‘Retail Sales’[UnitPrice] >= 1000   ) )
Here I use the measure in CALCULATE(), and I use FILTER() to apply a filter on the ‘OnlineSales’ table to calculate the desired result.
The difference in performance is dramatic, as you can see in the following picture:
The first measurement shows the timing and query plan of the first measure with SUMX() and FILTER(). Below, you can see the timing of the second measure with CALCULATE() and FILTER().
The reason is visible in the Query plan, where you can see that the query processes 1'451'337 rows twice. In addition, two CallbackDataID steps are executed, which pushes data between the formula and the storage engine. These operations are very costly.
The FILTER() function is essential for your DAX toolbelt.
You need to understand his capabilities and the potential issues when using this function.
But, it gives you a lot of opportunities for enhancing your DAX expressions.
As a summary:
FILTER() is an iterator
FILTER() returns a table
FILTER() can be used in CALCULATE() to set the filter context
You can use context transition in FILTER()
If not used properly, you can slow down your DAX expressions
As soon as you understand this function correctly, you can start using it at the right places and unleash the full power of FILTER().
The FILTER() function is described on this page: FILTER — DAX Guide
This page has an embedded video with further information and examples.
Information about what to avoid when using FILTER() in CALCULATE(), can be found here: Avoid using FILTER as a filter argument in DAX — DAX | Microsoft Docs
I use the Contoso sample dataset, like in my previous articles. You can download the ContosoRetailDW Dataset for free from Microsoft here.
The Contoso Data can be freely used under the MIT License, as described here.
I enlarged the dataset to make the DAX engine work harder. The Online Sales table contains 63 million rows (instead of 12.6 million rows), and the Retail Sales table contains 15.5 million rows (instead of 3.4 million rows).