Assign the frame start time when we get a slow path frame