pydata/patsy — pull request #44 by jiffyclub (2014-05-27 21:09:12+00:00)
Title: Use boolean dtype detection to save time in categorical checks.

I've no idea if these changes are generally useful, but this PR has a couple small changes I'm planning to use internally to save processing time in our usage of patsy.

I work on [synthicity/urbansim](https://github.com/synthicity/urbansim) and our models frequently make use of boolean variables specified in patsy with expressions like `I(year_built < 1940)`. Our problem is that [including categorical variables makes patsy drastically slower](http://nbviewer.ipython.org/gist/fscottfoti/1cec32e70a60e440ebce). We see times approaching a minute to assemble design matrices, and we'd like to avoid that.

Looking at profiling data it's clear that most of the time is spent going through categorical arrays and checking each item individually to see if it is a nan value. In our application the vast majority of our categorical model components will be boolean pandas Series. In that situation it seems like noting the Series dtype should be all that's required to ascertain the categorical levels and lack of nan values. So the changes I've made do exactly that.

As I said, I've no idea if these changes may be generally useful, but I thought our situation might be interesting to the developers. Thanks for a very useful tool!