Refactor and remove double cast on tv_nsec