Time Intelligence calculations in R | by Hamza Rafiq | Towards Data Science

Time intelligence is simply doing calculations over periods of date or time. For example, a common request you could get is to get total sales aggregated by month for a year or to calculate total sales for products YTD and sales, same period last year to compare and calculate growth. Learning from my own practical experience, I will be sharing some of the ways you can make these calculations in R using the tidyverse and lubridate packages in R
For this exercise, I created a dummy dataset in Excel. This dataset contains sales from January 2nd, 2017 till 30th September 2019
library(tidyverse) #Data manipulationlibrary(lubridate) #For working with date and timesale_data <- read_xlsx("C:\\Users\\ACER\\Desktop\\regression_test.xlsx") %>%     mutate(Date = ymd(Date))
Used the read_xlsx function to read in our dataset
Within the mutate function, we passed the ymd function to convert our Date column to date type
This is what our dataset looks like
Let's start with some basic time intelligence calculations. A common request you would get is to show total sales overtime, aggregated on a monthly level
sale_data %>%     group_by(Year=year(Date),Month=month(Date,label = T)) %>%     summarise(total_sales=sum(Offered_Calls,na.rm = T))
After the group_by, I created the Year and Month columns and after that used the summarise function to get total_sales. There is another way to this is as well
sale_data %>%     group_by(Monthly=floor_date(Date,unit = "1 month")) %>%     summarise(total_sales=sum(Offered_Calls,na.rm = T))
Basically, I rounded the Date column to the nearest month using the floor_date function from lubdridate and then calculated the total sales
What if someone asked for total sales from January to June for 2019 and compare it with the same period for last year to see the change?
sale_data %>%    summarise(sales_2019 = sum(Offered_Calls[Date %within% interval("2019-01-01","2019-06-30")],na.rm = T),              sales_2018 = sum(Offered_Calls[Date %within% interval("2018-01-01","2018-06-30")],na.rm = T))
So basically I used the %within% and Interval functions from the lubridate package to pass conditional parameters for the date column for which I want to calculate the sum
But what if you are working with a live dataset? You wouldn't want to update your code every single time you have to run your regular report to account for the latest date parameters. I have a pretty neat solution for that as well
sale_data %>%    summarise(sales_current_mtd = sum(Offered_Calls[Date %within% interval(cut(Sys.Date(),"month"),Sys.Date())],na.rm = T),              same_period_last_year = sum(Offered_Calls[Date %within% interval(cut(Sys.Date()-years(1),"month"),Sys.Date()-years(1))],na.rm = T))
This is the same formula as before. The only change is that I have replaced the date parameters in the interval function with functions that automatically calculate the desired date interval whenever you run your report or refresh your data. Let's break it down:
cut(Sys.Date(),"month") floors the sale date to the nearest month so you will always get the first day of every month
Sys.Date() basically just gets the current date. So inputting these two parameters in the interval function, you will get an interval from the start of every month till the end of every month, resulting in totals for MTD
To get totals for the same period last year, you just add an additional argument. Adding the Years(-1) function for both intervals will get you the same intervals but for last year, resulting in totals for last year MTD
I will show another example where you have to do calculations on a yearly granularity like YTD which will show how flexible this calculation is
sale_data %>%    summarise(sales_ytd = sum(Offered_Calls[Date %within% interval(cut(Sys.Date(),"year"),Sys.Date())],na.rm = T),              same_period_last_year = sum(Offered_Calls[Date %within% interval(cut(Sys.Date()-years(1),"year"),Sys.Date()-years(1))],na.rm = T))
The only difference here is that I changed the parameter from “month” to “year” in the cut function. This floors the date to the start of the year. This allows us to do a YTD calculation for both the current and last year
These are some of the solutions that I use at work to do “time intelligence” calculations in R. I showed both the manual process for one-time ad-hoc requests and automated solutions for regular reporting. While my examples only consisted of yearly, monthly calculations, you can do calculations on any level like weekly, daily, and quarterly by following the same principles that I showed. I found this solution really useful for calculating KPIs that have to refreshed/reported regularly in our weekly/monthly/quarterly reports. It allows me to just refresh the report without having to manually calculate them every single time.