add filtering to reduce step so that once filtered by time, no data is allowed through.