Finally, I have found a simple way to perform the decoding with only numpy operation and without bounding the maximal number of classes.