ok, see the current Categorical, it is almost exactly this (but operates vectorized).