# Approximate LSTM Computing for Energy-Efficient Speech Recognition
## 5 Simulation and Implementation Results
### Prototype Implementation  
To validate the proposed idea, we realized the prototype LSTM accelerator, whose internal architecture is depicted in Figure 5. For hardware analysis, we perform the post-layout simulation of the prototype LSTM accelerator, which is designed in a 65 nm CMOS technology. For synthesizing the prototype, more precisely, we adopted Synopsys Design Compiler [23], whereas the place and route steps were based on Synopsys IC compiler [24]. Figure 10a depicts the layout result of the proposed approximate LSTM accelerator, occupying 3.37 mm\({}^{2}\), where the area breakdown is summarized in Table 1. From the post-layout simulation, the prototype decoder can operate at the speed of 100 MHz, consuming only 33 mW by adopting the various hardware-level optimization methods. Figure 10b shows how the proposed schemes gradually improve the energy efficiency of the LSTM-based end-to-end speech recognition, where the energy efficiency is computed by dividing the maximum throughput of the proposed LSTM processing scheme by the corresponding power consumption, which is widely used to present the allowed number of operations for the given power budget [25].  
For the same processing architecture, note that the original bi-directional LSTM processing provides less than 100 GOPS/W. By introducing the algorithm-level innovations, combining the similarity-based cell skipping and approximation, we can enhance the energy-efficiency up to 160 GOPS/W. As the approximate LSTM operation reduces the required computing precision, as depicted in Figure 6, our accelerator architecture presents the flexible MAC architecture supporting different resolutions, further improving the energy-efficiency by 15.4% compared to the naive implementation of the proposed algorithm-level optimization. Finally, the sparsity-aware memory accessing achieves the most energy-efficient processing by eliminating the unnecessary on-chip buffer requests, increasing the energy-efficiency by 119% and 31% when compared to the original LSTM operations and the naive version, respectively. As a result, the proposed work provides the energy-efficient LSTM processing scenario by exploiting the co-optimization approaches between the algorithm and hardware levels, leading to the on-device end-to-end speech recognition.  
However, depending on the applications, there might be some limitations for directly applying the proposed approximate LSTM computing, which is dedicated to reducing the computational cost of bi-directional LSTM operations used for the recent DeepSpeech-based speech recognition systems.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Total Area** & **Similarity Check** & **Delta Gen** & **Non-Linear Operations** & **LSTM Engine** & **SRAM** & **Others** \\
**(\(\mu\)m)** & **(\(\mu\)m)** & **(\(\mu\)m)** & **(\(\mu\)m)** & **(\(\mu\)m)** & **(\(\mu\)m)** & **(\(\mu\)m)** \\ \hline
3,367,650 & 13,830 & 37,703 & 522,732 & 140,629 & 2,639,526 & 13,230 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Area breakdown of the prototype LSTM accelerator  
Figure 10: Implementation of the prototype LSTM accelerator. (**a**) Layout result, (**b**) Improvement of energy efficiency).  
When we consider the streaming applications for the real-time system [26], for example, we have to exploit a one-directional LSTM structure associated with the attention-based computations, and only a few LSTM cells in the attention region are involved in identifying the output characters. In this case, the proposed similarity-based LSTM cell approximation may severely degrade the recognition accuracy. Extending the proposed approximate LSTM operation to the attention-based processing could be solved by introducing more advanced similarity-checking criteria, which would be one of the future research directions for further improving the energy efficiency of the state-of-the-art DNN-based speech recognition systems.