Perhaps `label_indices` is a better name for this.