ok, we don't have a notion of a numeric Categorical vs a non-numeric Categorical at them moment. Should they have an underlying dtype? (e.g. say int64/float64/object)?