Are you sure? Because moving to this style of approach in `np.pad` was a massive improvement in performance. It could be preallocated; the final dimension would be of length `len(unique_labels)` but I'm not sure this would actually be a notable gain.