# InMemQK: A Product Quantization Based MatMul Module for Compute-in-Memory Attention Macro
## 4 Simulation Results
### InMemQK Performance of MatMul Optimization  
To evaluate the performance of the proposed method, we built the proposed InMemQK based on the same parameter settings from the reliable time-domain accelerator Timely (T-MAC) and compared it with the 2024 digital multiply-accumulate macro (D-MAC) [34]. The D-MAC introduces a multi-precision adaptive accumulator (MPAA) and an SRAM buffer for serial-parallel conversion (SPBUF), effectively minimizing peripheral circuit and intermediate buffer area overhead while supporting multi-precision operations. As shown in Figure 5a, whether in terms of energy efficiency or area efficiency, InMemQK has significant advantages over the D-MAC, thanks to the inherent advantages of the in-memory computing architecture. Its energy efficiency is 184.9 times better than and its area efficiency is 15.8 times that of the D-MAC.  
Moreover, it should be noted that InMemQK, compared to the D-MAC, can avoid certain analog-to-digital conversion demands, thus further lowering power consumption. Figure 5b illustrates that in traditional in-memory accelerators (JSSC'2024 [35] and JSCC'2022 [36]), analog signals \(Q\) and \(K\) produced by the in-memory array must undergo quantization via ADC and TDC before being processed by the digital acceleration module, incurring extra power consumption and further diminishing energy efficiency. InMemQK, however, these can be directly integrated adjacent to the in-memory array, avoiding these additional conversions. Compared to JSSC'2024, power consumption is reduced by 13.9 times; compared to JSSC'2022, power consumption is reduced by 13.2 times.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Model** & & **STSB** & **COLA** & **SST2** & **MRPC** & **RTE** & **MNLI** & **QNLI** & **QQP** \\ \hline \multirow{4}{*}{BERT} & Baseline & 0.88 & 0.57 & 0.933 & 0.84 & 0.693 & 0.835 & 0.897 & 0.912 \\  & L2 & 0.665 & 0.07 & 0.79 & 0.31 & 0.472 & 0.497 & 0.542 & 0.688 \\  & HFR & 0.819 & 0.389 & 0.836 & 0.715 & 0.631 & 0.615 & 0.754 & 0.759 \\  & HFR * & 0.817 & 0.417 & 0.84 & 0.693 & 0.519 & 0.615 & 0.744 & 0.755 \\  & RL-FT & 0.868 & 0.433 & 0.87 & 0.819 & 0.642 & 0.757 & 0.832 & 0.884 \\ \hline \multirow{4}{*}{ALBERT} & Baseline & 0.904 & 0.576 & 0.865 & 0.845 & 0.667 & 0.842 & 0.89 & 0.901 \\  & L2 & 0.131 & 0.012 & 0.844 & 0.665 & 0.472 & 0.326 & 0.51 & 0.719 \\  & HFR & 0.814 & 0.384 & 0.877 & 0.754 & 0.592 & 0.501 & 0.64 & 0.828 \\  & HFR * & 0.807 & 0.356 & 0.858 & 0.752 & 0.61 & 0.502 & 0.66 & 0.825 \\  & RL-FT & 0.848 & 0.446 & 0.0.832 & 0.812 & 0.654 & 0.732 & 0.855 & 0.871 \\ \hline \multirow{4}{*}{RoBERTa} & Baseline & 0.903 & 0.603 & 0.928 & 0.887 & 0.667 & 0.866 & 0.9 & 0.909 \\  & L2 & 0.211 & 0.166 & 0.676 & 0.696 & 0.498 & 0.452 & 0.564 & 0.631 \\ \cline{1-1}  & HFR & 0.566 & 0.328 & 0.877 & 0.757 & 0.595 & 0.626 & 0.721 & 0.766 \\ \cline{1-1}  & HFR * & 0.567 & 0.311 & 0.876 & 0.718 & 0.566 & 0.624 & 0.744 & 0.755 \\ \cline{1-1}  & RL-FT & 0.855 & 0.459 & 0.893 & 0.801 & 0.631 & 0.795 & 0.848 & 0.858 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Optimized product quantization attention performance and comparison  
Figure 5: (**a**) Performance comparison with D-MAC. (**b**) InMemQK power saving over JSSC’2024, JSSC’2022 (only mapping relevance computing).