Demo2

Well it turns out computing n-grams horizontal, vertical, and diagonal directions results in a respectable linear model for mnist.  Adding a single hidden unit improves it with almost no impact on wall clock.

Wouldn't it be great to integrate OpenCV or something equivalent as a reduction and be able to generate cool features for images on the fly?  Compose that with a reduction which creates dynamic deformations as invariant hints and you'd get something pretty cool for vision problems.