## 1 Introduction  
When compared with the traditional Deep Neural Network (DNN), the recurrent neural network is more suitable for time sequence modeling tasks [1]. To solve tasks with long-term sequence correlation, it is important to choose a neural network model with stronger long-term modeling capabilities. In recent years, Recurrent Neural Network (RNN) has greatly improved its performance on various time sequence tasks [2]. As a result, RNN has gradually become the mainstream solution for modeling sequential data.  
Unfortunately, the gradient of the simple RNN depends on the maximum eigenvalue of the state update matrix, and it may increase or decrease exponentially over time [3]. Therefore, basic RNNs are difficult to train. In fact, they can only model short-range effects and cannot remember long-term information [4]. There are many ways to try to solve this problem, such as pre-training with fine-tuning [5], gradient clipping, weight regularization, using ReLU activation function, batch normalization, and residual structure [6], etc. However, these methods only alleviate the gradient problem from the depth dimension. One of the most efficient and widely used methods is to introduce Long Short-Term Memory (LSTM) cells, which can alleviate vanishing gradients on the time axis. There is another popular RNN model variant, called Gate Recurrent Unit (GRU) [7], which is not only simpler than LSTM, but also can be used to model long short-term sequences. Although GRU has shown similar results to LSTM in a variety of machine learning tasks [8], it has not been widely used in speech recognition tasks. In addition, the multidimensional grid LSTM [9] provides a unified framework for calculating both depth and time dimensions by arranging the LSTM blocks into a multidimensional grid, which not only solves the problem that traditional RNN networks cannot memorize long-term information, but also alleviates the vanishing gradient in the depth dimension.  
It is well known that context-related tasks with time sequence characteristics depend on the content before and after the current moment. The Bidirectional Long-Short Term Memory (BiLSTM) recurrent neural network is an improvement of the RNN model, and its structure is particularly suitable for solving time sequence problems. Unlike LSTM, BiLSTM can use forward and backward information. [PERSON] and [PERSON] first proposed BiLSTM [10] in 2005 and applied it to the phoneme classification. Since then, BiLSTM has demonstrated state-of-the-art performance in speech recognition [11], natural language processing [12; 13], and others. The Bidirectional Grid Long Short-Term Memory (BiGridLSTM) recurrent neural network that is proposed in this paper, by combining the GridLSTM and bidirectional structure, modifies GridLSTM into a bidirectional structure, which not only has the advantages of GridLSTM mitigating the gradient phenomenon from two dimensions, but also can obtain context information at the same time. Validated on the LibriSpeech [14] corpus train-clean-100 dataset, the proposed BiGridLSTM achieves greater gain than other methods. There is no doubt that the improved speech recognition rate in the proposed approach could help a lot in executing text-based or structured queries over speech audio archives, as well as using NLP to transform these archives into machine interpretable resources. In addition, our proposed model can be widely used in time sequence tasks such as machine translation, weather forecasting, stock forecasting, and behavior recognition, etc.  
The paper is organized as follows. In Section 2, we will briefly discuss related work. In Section 3, we describe our model. The fourth part summarizes the experimental setup and the fifth part is the results and analysis. Finally, we conclude our work in Section 6.