npinto/asgd — pull request #19 by jaberg (2012-01-30 22:49:41+00:00)
Title: TheanoBinaryASGD attempt 2

Second draft of the theano classifier. This is much improved relative to the first one both in speed and legibility.

Several files have been changed, so if you don't like all the changes, consider just cut & paste the theano_asgd and test files into the trunk. The revision history here is not important.

The new code seems to run about 3-4 times faster than the naive implementation on honeybadger, using the same blas setup in numpy and theano.

.theanorc
=======

[gcc]
cxxflags=-march=native -O3 -ffast-math -ftree-parallelize-loops=4 -ftree-loop-distribution -funroll-loops -foptimize-register-move -funswitch-loops

Speed test
=========

jbergstra@honeybadger ~/cvs/asgd/asgd (theano2)$ nosetests -sd tests/test_theano_asgd.py:run_theano_binary_asgd_speed
generating fake data...
done
noise:0.005 dtype:float32 N_FEAT:000100 Naive:0.180 Theano:0.041
noise:0.005 dtype:float32 N_FEAT:001000 Naive:0.221 Theano:0.065
noise:0.005 dtype:float32 N_FEAT:010000 Naive:0.908 Theano:0.311
noise:0.005 dtype:float32 N_FEAT:100000 Naive:7.403 Theano:2.486
noise:0.005 dtype:float64 N_FEAT:000100 Naive:0.147 Theano:0.037
noise:0.005 dtype:float64 N_FEAT:001000 Naive:0.203 Theano:0.061
noise:0.005 dtype:float64 N_FEAT:010000 Naive:0.953 Theano:0.313
noise:0.005 dtype:float64 N_FEAT:100000 Naive:8.316 Theano:2.669
generating fake data...
done
noise:0.050 dtype:float32 N_FEAT:000100 Naive:0.162 Theano:0.040
noise:0.050 dtype:float32 N_FEAT:001000 Naive:0.204 Theano:0.064
noise:0.050 dtype:float32 N_FEAT:010000 Naive:0.866 Theano:0.293
noise:0.050 dtype:float32 N_FEAT:100000 Naive:7.402 Theano:2.485
noise:0.050 dtype:float64 N_FEAT:000100 Naive:0.143 Theano:0.037
noise:0.050 dtype:float64 N_FEAT:001000 Naive:0.198 Theano:0.062
noise:0.050 dtype:float64 N_FEAT:010000 Naive:0.946 Theano:0.314
noise:0.050 dtype:float64 N_FEAT:100000 Naive:8.362 Theano:2.654
generating fake data...
done
noise:0.500 dtype:float32 N_FEAT:000100 Naive:0.170 Theano:0.039
noise:0.500 dtype:float32 N_FEAT:001000 Naive:0.220 Theano:0.062
noise:0.500 dtype:float32 N_FEAT:010000 Naive:0.874 Theano:0.291
noise:0.500 dtype:float32 N_FEAT:100000 Naive:7.420 Theano:2.485
noise:0.500 dtype:float64 N_FEAT:000100 Naive:0.149 Theano:0.037
noise:0.500 dtype:float64 N_FEAT:001000 Naive:0.213 Theano:0.061
noise:0.500 dtype:float64 N_FEAT:010000 Naive:0.957 Theano:0.313
noise:0.500 dtype:float64 N_FEAT:100000 Naive:8.327 Theano:2.657