# Approximate LSTM Computing for Energy-Efficient Speech Recognition
## 4 Accelerator Design for Approximate LSTM Processing  
Figure 5 shows the conceptual architecture of the proposed energy-efficient LSTM accelerator, including an external memory interface for receiving input vectors and weight matrices, a similarity check module for checking \(S_{t}\) of incoming input vectors, non-linear operators for computing activation functions, an LSTM engine including 32 parallel processing elements (PEs), and on-chip memories for storing input/output feature maps. When a 4096-sized 16-bit quantized input vector is transferred from the external memory, it is first stored in on-chip SRAM buffers. In order to provide a sufficient bandwidth required by the LSTM operations, the on-chip memory is configured by using eight 64-entry dual-port SRAM banks. Note that total \(n\) vectors are serially accessed from the external memory and issued to the LSTM engine for handling a sentence, which depends on the length of the recorded speech signal. To support the similarity-based LSTM cell operations, it is required to keep the prior input vector for calculating the similarity score as well as the sparse vector, as illustrated in Figure 4. The addresses of on-chip buffers are managed in a ping-pong manner, efficiently supporting the proposed LSTM processing without consuming the additional energy caused by vector movements. The similarity check module then easily accesses two most recent vectors to compute \(S_{t}\) and selects the current processing mode among three candidates, i.e., the normal LSTM mode, the approximate LSTM mode, and the skipping mode. Two thresholds \(\theta_{\text{s}}\) and \(\theta_{\text{p}}\) can be initialized at the run time based on numerous simulations as described in [14]. The LSTM engine performs the dedicated operations according to the selected processing mode. Note that the internal MAC processing arrays are totally disabled when we observe the highly similar inputs satisfying \(S_{t}>\theta_{\text{s}}\) whereas the large amount changes of input vectors (\(S_{t}<\theta_{\text{p}}\)) allow the fully-activated LSTM engine. As shown in Figure 5, the LSTM engine mainly consists of 32 PEs, each of which includes a MAC operator. Hence, 32 MAC operations are processed in parallel for activating the normal LSTM cell operations. The memory controller efficiently manages data flows between the LSTM engine and numerous dual-port on-chip buffers, maintaining the hardware utilization to be high enough. For the case of approximate mode, i.e., \(\theta_{\mathrm{p}}<S_{t}<\theta_{\mathrm{s}}\), the LSTM engine supports the approximate operations with the reduced computing resolution. More specifically, the original MAC operations performed at each PE are based on \(16\times 16\) multiplications, whereas the approximate mode requires \(8\times 8\) multiplications. In order not to degrade the energy efficiency, in this work, the MAC operator is designed to support two multiplications in parallel when the approximate mode is selected. Figure 6 is an overall structure of the LSTM engine of the proposed accelerator, which can support various computing resolutions required at the proposed approximate LSTM computing method. As shown in the figure, the proposed LSTM engine internally includes a total of 32 processing elements (PEs). By utilizing dual-port on-chip memories, we can perform the read and write accesses simultaneously, improving the energy efficiency of the overall LSTM accelerator by reducing the number of inactive PEs. In fact, each processing element is nothing but a MAC operator, which flexibly supports two different computing resolutions according to the LSTM processing mode delivered from the similarity check module. By supporting the optimal computing precision depending on the current input vector, we can additionally enhance energy efficiency. Figure 7 details the internal architecture of each PE, where the connections of multiplication steps are conditionally modified on demand.