Change staleThreshold to a time.Duration internally.

[#81878766]