Don't install numpy until we actually use it

Seems that numpy is the way to do math for PIL (rounded corners,
gradients). But I ended up going so simple (aka "flat" ;) ) that I'm not
actually using numpy right now. It's still in the repo so the bloat is
there but we can at least avoid building it constantly.