finding that I need to be really explicit with how the indexing of the matrices is laid out for theano.functions.