I returned the "with nogil" statement, by keeping X as an "object" and using a "issparse" variable to determine whether to assign local variables for sparse or dense format.
Doing so, the local variables are used with nogil [here](https://github.com/eltermann/scikit-learn/commit/00c4dba6618f61345abe7ee824bc102aac97786e#diff-a2cead4f3702cc4b9f76562bb2777edbR2346).