I think it should be possible to fit the bias term directly (even with a kernel) but I didn't do the derivation and can't find it on internet. Right now, I set fit_intercept="auto" and automatically switch to False if the kernel is non-linear.