Recently , neural network based sentence modeling methods have achieved great progress . Among these methods , the recursive neural networks ( RecNNs ) can effectively model the combination of the words in sentence . However , RecNNs need a given external topological structure , like syntactic tree . In this paper , we propose a gated recursive neural network ( GRNN ) to model sentences , which employs a full binary tree ( FBT ) structure to control the combinations in recursive structure . By introducing two kinds of gates , our model can better model the complicated combinations of features . Experiments on three text classification datasets show the effectiveness of our model . Recently , neural network based sentence modeling approaches have been increasingly focused on for their ability to minimize the efforts in feature engineering , such as Neural Bag-of-Words ( NBoW ) , Recurrent Neural Network ( RNN ) ( Mikolov et al . , 2010 ) , Recursive Neural Network ( RecNN ) ( Pollack , 1990 ; Socher et al . , 2013b ; Socher et al . , 2012 ) and Convolutional Neural Network ( CNN ) ( Kalchbrenner et al . , 2014 ; Hu et al . , 2014 ) . Among these methods , recursive neural networks ( RecNNs ) have shown their excellent abilities to model the word combinations in sentence . However , RecNNs require a pre-defined topological structure , like parse tree , to encode sentence , which limits the scope of its application . Cho et al . ( 2014 ) proposed the gated recursive convolutional neural network ( grConv ) by utilizing the directed acyclic graph ( DAG ) structure instead of parse tree * Corresponding author . ( GRNNs ) . Left is a GRNN using a directed acyclic graph ( DAG ) structure . Right is a GRNN using a full binary tree ( FBT ) structure . ( The green nodes , gray nodes and white nodes illustrate the positive , negative and neutral sentiments respectively . ) to model sentences . However , DAG structure is relatively complicated . The number of the hidden neurons quadraticly increases with the length of sentences so that grConv can not effectively deal with long sentences . Inspired by grConv , we propose a gated recursive neural network ( GRNN ) for sentence modeling . Different with grConv , we use the full binary tree ( FBT ) as the topological structure to recursively model the word combinations , as shown in Figure 1 . The number of the hidden neurons linearly increases with the length of sentences . Another difference is that we introduce two kinds of gates , reset and update gates ( Chung et al . , 2014 ) , to control the combinations in recursive structure . With these two gating mechanisms , our model can better model the complicated combinations of features and capture the long dependency interactions . In our previous works , we have investigated several different topological structures ( tree and directed acyclic graph ) to recursively model the semantic composition from the bottom layer to the top layer , and applied them on Chinese word segmentation ( Chen et al . , 2015a ) and dependency parsing ( Chen et al . , 2015b ) tasks . However , these structures are not suitable for modeling sentences . … Softmax(W s × u i + b s ) u i P(•|x i ; θ ) w 2 w 1 ( i ) ( i ) … … … … … … … … w 3 w 4 w 5 ( i ) ( i ) ( i ) 0 0 0 Figure 2 : Architecture of Gated Recursive Neural Network ( GRNN ) . In this paper , we adopt the full binary tree as the topological structure to reduce the model complexity . Experiments on the Stanford Sentiment Treebank dataset ( Socher et al . , 2013b ) and the TREC questions dataset ( Li and Roth , 2002 ) show the effectiveness of our approach . 2 Gated Recursive Neural Network In this paper , we propose a gated recursive neural network ( GRNN ) to recursively summarize the meaning of sentence . GRNN uses full binary tree as the recursive topological structure instead of an external syntactic tree . In addition , we introduce two kinds of gates to model the complicated combinations of features . In future work , we would like to investigate the other gating mechanisms for better modeling the feature combinations .

Challenge: Despite the high performance, recursive neural networks requiring an external topological structure limit their applications and also quadraticly increase the computational costs with text lengths.
Approach: They propose a gated recursive neural network that uses a full binary tree to model recursive structures and also gating mechanisms for more complex modelling.
Outcome: They show that the proposed system model has more complex structures on classification tasks from the Stanford Sentiment Treebank dataset.