Only refresh indexes that are valid in the attributedString