Module Expr Str ImportFrom alias Import alias Import alias Import alias Import alias Import alias Import alias identifier ImportFrom alias ImportFrom alias ImportFrom alias alias ImportFrom alias ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults TupleLoad Num Num body Expr Str Assert CompareEq SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice Num Assign NameStore BinOpFloorDiv BinOpMult SubscriptLoad NameLoad Index Num Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice Num Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpDiv Num BinOpAdd NameLoad NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr keyword UnaryOpUSub NameLoad keyword NameLoad keyword NameLoad keyword AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad SubscriptLoad NameLoad Index Num keyword AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str Num Str Str Assign AttributeStore NameLoad attr ListLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad decorator_list decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Num Num Str ListLoad Num Num Num body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Num Assign NameStore Call NameLoad NameLoad Assign TupleStore NameStore NameStore SubscriptLoad NameLoad Index Num Assign TupleStore NameStore NameStore SubscriptLoad NameLoad Index Num Assign TupleStore NameStore NameStore SubscriptLoad NameLoad Index Num Assign NameStore SubscriptLoad AttributeLoad Call AttributeLoad NameLoad attr keyword NameLoad attr Index Num Assign NameStore SubscriptLoad AttributeLoad Call AttributeLoad NameLoad attr keyword NameLoad attr Index Num Assign NameStore SubscriptLoad AttributeLoad Call AttributeLoad NameLoad attr keyword NameLoad attr Index Num AugAssignFloorDiv NameStore NameLoad AugAssignFloorDiv NameStore NameLoad AugAssignFloorDiv NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Expr Call NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Num Num Num Assign NameStore Call NameLoad NameLoad keyword NameLoad keyword TupleLoad NameLoad Num Num Num keyword TupleLoad SubscriptLoad NameLoad Index Num Num Num Num keyword TupleLoad Num Num Assign NameStore Call NameLoad NameLoad keyword AttributeLoad NameLoad attr keyword TupleLoad NameLoad SubscriptLoad NameLoad Index Num Num Num keyword TupleLoad SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num Num Num keyword TupleLoad Num Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Num Assign NameStore Call NameLoad NameLoad keyword NameLoad keyword BinOpMult BinOpMult SubscriptLoad NameLoad Index Num Num Num keyword Num keyword AttributeLoad NameLoad attr Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr keyword Num keyword Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListLoad NameLoad Call AttributeLoad NameLoad attr NameLoad keyword Dict NameLoad NameLoad SubscriptLoad NameLoad Slice BinOpMult NameLoad NameLoad BinOpMult BinOpAdd NameLoad Num NameLoad SubscriptLoad NameLoad Slice BinOpMult NameLoad NameLoad BinOpMult BinOpAdd NameLoad Num NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListLoad NameLoad Call AttributeLoad NameLoad attr NameLoad keyword Dict NameLoad NameLoad SubscriptLoad NameLoad Slice BinOpMult NameLoad NameLoad BinOpMult BinOpAdd NameLoad Num NameLoad SubscriptLoad NameLoad Slice BinOpMult NameLoad NameLoad BinOpMult BinOpAdd NameLoad Num NameLoad Assign NameStore BinOpAdd BinOpAdd BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore ListComp TupleLoad NameLoad BinOpSub NameLoad BinOpMult NameLoad NameLoad comprehension TupleStore NameStore NameStore Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListLoad NameLoad NameLoad keyword NameLoad keyword Dict NameLoad NameLoad SubscriptLoad NameLoad Slice BinOpMult NameLoad NameLoad BinOpMult BinOpAdd NameLoad Num NameLoad SubscriptLoad NameLoad Slice BinOpMult NameLoad NameLoad BinOpMult BinOpAdd NameLoad Num NameLoad Expr Call NameLoad Str Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Call NameLoad NameLoad BinOpFloorDiv NameLoad Num Assign NameStore AttributeLoad NameLoad attr Assign NameStore Num Assign NameStore Num Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Num Assign NameStore NameLoad While BoolOpAnd CompareLt NameLoad NameLoad UnaryOpNot NameLoad body Assign NameStore BinOpAdd NameLoad Num For NameStore Call NameLoad NameLoad body Assign NameStore BinOpAdd BinOpMult BinOpSub NameLoad Num NameLoad NameLoad If CompareEq BinOpMod NameLoad Num Num body Expr Call NameLoad Str NameLoad Assign NameStore Call NameLoad NameLoad If CompareEq BinOpMod BinOpAdd NameLoad Num NameLoad Num body Assign NameStore ListComp Call NameLoad NameLoad comprehension NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call NameLoad BinOpMod Str TupleLoad NameLoad BinOpAdd NameLoad Num NameLoad BinOpMult NameLoad Num If CompareLt NameLoad NameLoad body If CompareLt NameLoad BinOpMult NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad BinOpMult NameLoad NameLoad Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore ListComp Call NameLoad NameLoad comprehension NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call NameLoad BinOpMod Str TupleLoad NameLoad BinOpAdd NameLoad Num NameLoad BinOpMult NameLoad Num If CompareLtE NameLoad NameLoad body Assign NameStore NameLoad Break Assign NameStore Call AttributeLoad NameLoad attr Expr Call NameLoad Str Expr Call NameLoad BinOpMod Str TupleLoad BinOpMult NameLoad Num BinOpAdd NameLoad Num BinOpMult NameLoad Num Expr Call NameLoad BinOpAdd BinOpAdd Str SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Index Num BinOpMod Str BinOpDiv BinOpSub NameLoad NameLoad Num keyword AttributeLoad NameLoad attr decorator_list If CompareEq NameLoad Str body Expr Call NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Call NameLoad AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr decorator_list
N/A N/A This tutorial introduces the LeNet5 neural network architecture
using Theano.  LeNet5 is a convolutional neural network, good for
classifying images. This tutorial shows how to build the architecture,
and comes with all the hyper-parameters you need to reproduce the
paper's MNIST results.


This implementation simplifies the model in the following ways:

 - LeNetConvPool doesn't implement location-specific gain and bias parameters
 - LeNetConvPool doesn't implement pooling by average, it implements pooling
   by max.
 - Digit classification is implemented with a logistic regression rather than
   an RBF network
 - LeNet5 was not fully-connected convolutions at second layer

References:
 - Y. LeCun, L. Bottou, Y. Bengio and P. Haffner:
   Gradient-Based Learning Applied to Document
   Recognition, Proceedings of the IEEE, 86(11):2278-2324, November 1998.
   http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf

 __future__ print_function N/A os N/A sys N/A timeit N/A numpy N/A theano N/A theano.tensor T theano.tensor.signal downsample theano.tensor.nnet conv2d logistic_sgd LogisticRegression load_data mlp HiddenLayer LeNetConvPoolLayer N/A object N/A N/A Pool Layer of a convolutional network  __init__ N/A N/A self rng input filter_shape image_shape poolsize N/A N/A 2 2 N/A N/A 
        Allocate a LeNetConvPoolLayer with shared variable internal parameters.

        :type rng: numpy.random.RandomState
        :param rng: a random number generator used to initialize weights

        :type input: theano.tensor.dtensor4
        :param input: symbolic image tensor, of shape image_shape

        :type filter_shape: tuple or list of length 4
        :param filter_shape: (number of filters, num input feature maps,
                              filter height, filter width)

        :type image_shape: tuple or list of length 4
        :param image_shape: (batch size, num input feature maps,
                             image height, image width)

        :type poolsize: tuple or list of length 2
        :param poolsize: the downsampling (pooling) factor (#rows, #cols)
         N/A N/A N/A image_shape N/A 1 N/A filter_shape N/A 1 N/A N/A self input input N/A fan_in N/A N/A numpy prod N/A filter_shape N/A 1 N/A fan_out N/A N/A N/A filter_shape N/A 0 N/A N/A numpy prod N/A filter_shape N/A 2 N/A N/A numpy prod poolsize N/A W_bound N/A N/A numpy sqrt N/A 6.0 N/A fan_in fan_out N/A N/A self W N/A N/A theano shared N/A N/A numpy asarray N/A N/A rng uniform low N/A W_bound high W_bound size filter_shape dtype N/A N/A theano config floatX borrow True N/A b_values N/A N/A numpy zeros N/A N/A filter_shape N/A 0 dtype N/A N/A theano config floatX N/A N/A self b N/A N/A theano shared value b_values borrow True N/A conv_out N/A conv2d input input filters N/A self W filter_shape filter_shape input_shape image_shape N/A pooled_out N/A N/A downsample max_pool_2d input conv_out ds poolsize ignore_border True N/A N/A self output N/A N/A T tanh N/A pooled_out N/A N/A N/A self b dimshuffle x 0 x x N/A N/A self params N/A N/A self W N/A self b N/A N/A self input input N/A N/A evaluate_lenet5 N/A N/A learning_rate n_epochs dataset nkerns batch_size N/A 0.1 200 mnist.pkl.gz N/A 20 50 500 N/A N/A  Demonstrates lenet on MNIST dataset

    :type learning_rate: float
    :param learning_rate: learning rate used (factor for the stochastic
                          gradient)

    :type n_epochs: int
    :param n_epochs: maximal number of epochs to run the optimizer

    :type dataset: string
    :param dataset: path to the dataset used for training /testing (MNIST here)

    :type nkerns: list of ints
    :param nkerns: number of kernels on each layer
     N/A rng N/A N/A N/A numpy random RandomState 23455 N/A datasets N/A load_data dataset N/A N/A train_set_x train_set_y N/A datasets N/A 0 N/A N/A valid_set_x valid_set_y N/A datasets N/A 1 N/A N/A test_set_x test_set_y N/A datasets N/A 2 N/A n_train_batches N/A N/A N/A N/A train_set_x get_value borrow True shape N/A 0 N/A n_valid_batches N/A N/A N/A N/A valid_set_x get_value borrow True shape N/A 0 N/A n_test_batches N/A N/A N/A N/A test_set_x get_value borrow True shape N/A 0 N/A n_train_batches batch_size N/A n_valid_batches batch_size N/A n_test_batches batch_size N/A index N/A N/A T lscalar N/A x N/A N/A T matrix x N/A y N/A N/A T ivector y N/A N/A print ... building the model N/A layer0_input N/A N/A x reshape N/A batch_size 1 28 28 N/A layer0 N/A LeNetConvPoolLayer rng input layer0_input image_shape N/A batch_size 1 28 28 filter_shape N/A N/A nkerns N/A 0 1 5 5 poolsize N/A 2 2 N/A layer1 N/A LeNetConvPoolLayer rng input N/A layer0 output image_shape N/A batch_size N/A nkerns N/A 0 12 12 filter_shape N/A N/A nkerns N/A 1 N/A nkerns N/A 0 5 5 poolsize N/A 2 2 N/A layer2_input N/A N/A N/A layer1 output flatten 2 N/A layer2 N/A HiddenLayer rng input layer2_input n_in N/A N/A N/A nkerns N/A 1 4 4 n_out 500 activation N/A T tanh N/A layer3 N/A LogisticRegression input N/A layer2 output n_in 500 n_out 10 N/A cost N/A N/A layer3 negative_log_likelihood y N/A test_model N/A N/A theano function N/A index N/A N/A layer3 errors y givens N/A x y N/A test_set_x N/A N/A index batch_size N/A N/A index 1 batch_size N/A test_set_y N/A N/A index batch_size N/A N/A index 1 batch_size N/A validate_model N/A N/A theano function N/A index N/A N/A layer3 errors y givens N/A x y N/A valid_set_x N/A N/A index batch_size N/A N/A index 1 batch_size N/A valid_set_y N/A N/A index batch_size N/A N/A index 1 batch_size N/A params N/A N/A N/A N/A layer3 params N/A layer2 params N/A layer1 params N/A layer0 params N/A grads N/A N/A T grad cost params N/A updates N/A N/A param_i N/A param_i N/A learning_rate grad_i N/A N/A param_i grad_i N/A zip params grads N/A train_model N/A N/A theano function N/A index cost updates updates givens N/A x y N/A train_set_x N/A N/A index batch_size N/A N/A index 1 batch_size N/A train_set_y N/A N/A index batch_size N/A N/A index 1 batch_size N/A N/A print ... training N/A patience 10000 N/A patience_increase 2 N/A improvement_threshold 0.995 N/A validation_frequency N/A min n_train_batches N/A patience 2 N/A best_validation_loss N/A numpy inf N/A best_iter 0 N/A test_score 0.0 N/A start_time N/A N/A timeit default_timer N/A epoch 0 N/A done_looping False N/A N/A N/A epoch n_epochs N/A done_looping N/A N/A epoch N/A epoch 1 N/A minibatch_index N/A range n_train_batches N/A N/A iter N/A N/A N/A epoch 1 n_train_batches minibatch_index N/A N/A N/A iter 100 0 N/A N/A N/A print training @ iter =  iter N/A cost_ij N/A train_model minibatch_index N/A N/A N/A N/A iter 1 validation_frequency 0 N/A N/A validation_losses N/A N/A validate_model i N/A i N/A range n_valid_batches N/A this_validation_loss N/A N/A numpy mean validation_losses N/A N/A print N/A epoch %i, minibatch %i/%i, validation error %f %% N/A epoch N/A minibatch_index 1 n_train_batches N/A this_validation_loss 100.0 N/A N/A this_validation_loss best_validation_loss N/A N/A N/A this_validation_loss N/A best_validation_loss improvement_threshold N/A N/A patience N/A max patience N/A iter patience_increase N/A best_validation_loss this_validation_loss N/A best_iter iter N/A test_losses N/A N/A test_model i N/A i N/A range n_test_batches N/A test_score N/A N/A numpy mean test_losses N/A N/A print N/A      epoch %i, minibatch %i/%i, test error of best model %f %% N/A epoch N/A minibatch_index 1 n_train_batches N/A test_score 100.0 N/A N/A patience iter N/A N/A done_looping True N/A N/A end_time N/A N/A timeit default_timer N/A N/A print Optimization complete. N/A N/A print N/A Best validation score of %f %% obtained at iteration %i, with test performance %f %% N/A N/A best_validation_loss 100.0 N/A best_iter 1 N/A test_score 100.0 N/A N/A print N/A N/A The code for file  N/A N/A N/A N/A os path split __file__ N/A 1 N/A  ran for %.2fm N/A N/A end_time start_time 60.0 file N/A sys stderr N/A N/A N/A __name__ __main__ N/A N/A N/A evaluate_lenet5 experiment N/A N/A state channel N/A N/A N/A N/A evaluate_lenet5 N/A state learning_rate dataset N/A state dataset N/A