BUG: fix windows pointer size problem

Fix windows test failure.
It's to fix
https://ci.appveyor.com/project/ogrisel/scikit-learn/build/1.0.9/job/rp7i3uw81vb92bai