I could use `atleast2d_or_csc`. This way I can do the categorical / non-categorical splitting without densification. Also, if there are only few categorical variables, this means that the densification done by _fit_transform and _transform will be affordable.