Module Import alias identifier Import alias Import alias Import alias Import alias Import alias ImportFrom alias ImportFrom alias FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Str NameLoad body Assert CompareIn NameLoad TupleLoad Str Str Assign NameStore Num Assign NameStore Num Assign NameStore Call NameLoad Assign NameStore Num Expr Call AttributeLoad NameLoad attr For TupleStore NameStore NameStore NameStore Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str Call AttributeLoad NameLoad attr Str BinOpMult ListLoad NameLoad SubscriptLoad NameLoad Index Str body If CompareEq NameLoad Str body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad orelse If CompareEq NameLoad Str body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad AugAssignAdd NameStore Call AttributeLoad NameLoad attr NameLoad AugAssignAdd NameStore SubscriptLoad NameLoad Index Str If AttributeLoad NameLoad attr body Assign NameStore BinOpSub Call NameLoad NameLoad If CompareGt NameLoad Num body Assign NameStore BinOpMod Str TupleLoad BinOpMult BinOpDiv Call NameLoad NameLoad SubscriptLoad NameLoad Index Str Num BinOpDiv NameLoad NameLoad Assign NameStore Call NameLoad NameLoad Call NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpAdd NameLoad Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr If AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpAdd BinOpMult Str NameLoad Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad NameLoad attr BinOpDiv NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults body Assign NameStore Num Assign NameStore AttributeLoad NameLoad attr Assign NameStore NameLoad For NameStore Call NameLoad Num BinOpAdd NameLoad Num body Assign NameStore Call NameLoad Print BinOpMod Str NameLoad Print BinOpMod Str NameLoad Assign NameStore Call NameLoad NameLoad NameLoad keyword Str keyword NameLoad Print BinOpMod Str NameLoad Assign NameStore Call NameLoad NameLoad NameLoad keyword Str Print BinOpMod Str NameLoad Print BinOpMod Str BinOpSub Call NameLoad NameLoad If CompareGt BinOpMult Call AttributeLoad NameLoad attr NameLoad NameLoad Call AttributeLoad NameLoad attr NameLoad body If UnaryOpNot NameLoad body Assign NameStore NameLoad Print Str If CompareGt NameLoad NameLoad body Print Str Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd Str NameLoad orelse If CompareLt NameLoad NameLoad body Print Str Expr Call AttributeLoad NameLoad attr BinOpAdd Str NameLoad keyword NameLoad Break orelse Print Str Assign NameStore NameLoad Assign NameStore BoolOpOr CompareLt BinOpDiv NameLoad Num NameLoad CompareEq NameLoad NameLoad Expr Call AttributeLoad NameLoad attr BinOpAdd Str NameLoad NameLoad If NameLoad body AugAssignDiv NameStore Num If CompareLt NameLoad NameLoad body Break Print BinOpMult Str Num Print Str Print BinOpMod Str NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assert CompareEq SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str Assert CompareEq SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str Assert CompareEq SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str Print Str Print BinOpMod Str Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str Print BinOpMod Str SubscriptLoad NameLoad Index Str Print BinOpMod Str SubscriptLoad NameLoad Index Str Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword NameLoad Expr Call NameLoad NameLoad NameLoad NameLoad NameLoad SubscriptLoad AttributeLoad NameLoad attr Index Str SubscriptLoad AttributeLoad NameLoad attr Index Str SubscriptLoad AttributeLoad NameLoad attr Index Str If BoolOpOr UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Print Str Return Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assert CompareEqEq SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str AttributeLoad NameLoad attr Assert CompareEqEq SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str AttributeLoad NameLoad attr Assert CompareEqEq SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str AttributeLoad NameLoad attr Print Str Print BinOpMod Str SubscriptLoad NameLoad Index Str Print BinOpMod Str SubscriptLoad NameLoad Index Str Print BinOpMod Str IfExp SubscriptLoad AttributeLoad NameLoad attr Index Str Str Str Assign NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword NameLoad keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword SubscriptLoad AttributeLoad NameLoad attr Index Str keyword SubscriptLoad AttributeLoad NameLoad attr Index Str Expr Call NameLoad NameLoad NameLoad NameLoad NameLoad SubscriptLoad AttributeLoad NameLoad attr Index Str SubscriptLoad AttributeLoad NameLoad attr Index Str SubscriptLoad AttributeLoad NameLoad attr Index Str decorator_list
N/A N/A numpy np N/A sys N/A os N/A utils N/A models N/A conf time time itertools izip _process_corpus N/A N/A net dataset mode learning_rate N/A train None N/A N/A N/A mode N/A train test N/A counter 0 N/A total_neg_log_prob 0 N/A t0 N/A time N/A max_output_length 0 N/A N/A N/A net reset_state N/A N/A inputs outputs pauses N/A izip N/A dataset N/A inputs N/A dataset N/A outputs N/A N/A dataset get pauses N/A N/A None N/A dataset N/A total_size N/A N/A N/A mode train N/A N/A neg_log_probs N/A N/A net train inputs outputs pauses learning_rate N/A N/A N/A mode test N/A N/A neg_log_probs N/A N/A net neg_log_prob inputs outputs pauses N/A total_neg_log_prob N/A N/A np sum neg_log_probs N/A counter N/A dataset N/A batch_size N/A N/A conf SHOW_WPS N/A N/A elapsed_time N/A N/A time t0 N/A N/A elapsed_time 0 N/A N/A output N/A %d%% @ %.1f wps   N/A N/A N/A N/A float counter N/A dataset N/A total_size 100 N/A counter elapsed_time N/A max_output_length N/A max max_output_length N/A len output N/A N/A N/A N/A sys stdout write N/A output N/A N/A N/A N/A sys stdout flush N/A N/A conf SHOW_WPS N/A N/A N/A N/A N/A sys stdout write N/A N/A max_output_length N/A N/A N/A N/A sys stdout flush N/A ppl N/A N/A np exp N/A total_neg_log_prob counter N/A ppl N/A _train N/A N/A net training_data validation_data model_name learning_rate max_epochs min_improvement N/A N/A N/A min_learning_rate 1e-06 N/A best_validation_ppl N/A np inf N/A divide False N/A epoch N/A range 1 N/A max_epochs 1 N/A N/A epoch_start N/A time N/A N/A 
======= EPOCH %s ======= epoch N/A N/A 	Learning rate is %s learning_rate N/A train_ppl N/A _process_corpus net training_data mode train learning_rate learning_rate N/A N/A 	Train PPL is %.3f train_ppl N/A validation_ppl N/A _process_corpus net validation_data mode test N/A N/A 	Validation PPL is %.3f validation_ppl N/A N/A 	Time taken: %ds N/A N/A time epoch_start N/A N/A N/A N/A N/A np log validation_ppl min_improvement N/A N/A np log best_validation_ppl N/A N/A N/A divide N/A N/A divide True N/A 	Starting to reduce the learning rate... N/A N/A validation_ppl best_validation_ppl N/A N/A 	Loading best model. N/A net N/A N/A utils load_model N/A ../out/ model_name N/A N/A N/A validation_ppl best_validation_ppl N/A N/A 	Saving model. N/A N/A N/A net save N/A ../out/ model_name final True N/A N/A N/A 	New best model! Saving... N/A best_validation_ppl validation_ppl N/A final N/A N/A N/A learning_rate 2.0 min_learning_rate N/A epoch max_epochs N/A N/A N/A net save N/A ../out/ model_name final N/A divide N/A N/A learning_rate 2.0 N/A N/A learning_rate min_learning_rate N/A N/A N/A N/A - 30 N/A Finished training. N/A N/A Best validation PPL is %.3f

 best_validation_ppl N/A train N/A N/A model_name p1_train_data p1_dev_data p2_train_data p2_dev_data N/A N/A N/A training_data N/A N/A np load p1_train_data N/A validation_data N/A N/A np load p1_dev_data N/A N/A N/A training_data N/A batch_size N/A validation_data N/A batch_size N/A N/A N/A training_data N/A vocabulary N/A validation_data N/A vocabulary N/A N/A N/A training_data N/A punctuations N/A validation_data N/A punctuations N/A 1st phase data loaded... N/A N/A Vocabulary size is %d N/A N/A utils get_vocabulary_size N/A validation_data N/A vocabulary N/A N/A Training set size is %d N/A training_data N/A total_size N/A N/A Validation set size is %d N/A validation_data N/A total_size N/A net N/A N/A models T_LSTM N/A N/A N/A net initialize hidden_size N/A N/A conf PHASE1 N/A HIDDEN_SIZE projection_size N/A N/A conf PHASE1 N/A PROJECTION_SIZE in_vocabulary N/A training_data N/A vocabulary out_vocabulary N/A training_data N/A punctuations batch_size N/A training_data N/A batch_size hidden_activation N/A N/A conf PHASE1 N/A HIDDEN_ACTIVATION bptt_steps N/A N/A conf PHASE1 N/A BPTT_STEPS use_pauses False N/A N/A _train net training_data validation_data model_name N/A N/A conf PHASE1 N/A LEARNING_RATE N/A N/A conf PHASE1 N/A MAX_EPOCHS N/A N/A conf PHASE1 N/A MIN_IMPROVEMENT N/A N/A N/A N/A N/A N/A os path isfile p2_train_data N/A N/A N/A N/A os path isfile p2_train_data N/A N/A No second phase data. N/A N/A training_data N/A N/A np load p2_train_data N/A validation_data N/A N/A np load p2_dev_data N/A N/A N/A training_data N/A batch_size N/A validation_data N/A batch_size N/A net batch_size N/A N/A N/A training_data N/A vocabulary N/A validation_data N/A vocabulary N/A net in_vocabulary N/A N/A N/A training_data N/A punctuations N/A validation_data N/A punctuations N/A net out_vocabulary N/A 2nd phase data loaded... N/A N/A Training set size is %d N/A training_data N/A total_size N/A N/A Validation set size is %d N/A validation_data N/A total_size N/A N/A Trainging %s pause durations. N/A N/A N/A conf PHASE2 N/A USE_PAUSES with without N/A t_lstm net N/A net N/A N/A models TA_LSTM N/A N/A N/A net initialize hidden_size N/A N/A conf PHASE2 N/A HIDDEN_SIZE t_lstm t_lstm out_vocabulary N/A training_data N/A punctuations batch_size N/A training_data N/A batch_size hidden_activation N/A N/A conf PHASE2 N/A HIDDEN_ACTIVATION bptt_steps N/A N/A conf PHASE2 N/A BPTT_STEPS use_pauses N/A N/A conf PHASE2 N/A USE_PAUSES N/A N/A _train net training_data validation_data model_name N/A N/A conf PHASE2 N/A LEARNING_RATE N/A N/A conf PHASE2 N/A MAX_EPOCHS N/A N/A conf PHASE2 N/A MIN_IMPROVEMENT N/A