Extra small improvement to context switch smoothing, to make it truly robust