# Language Model Using Neural Turing Machine Based on Localized Content-Based Addressing
## 6 Conclusions and Future Work  
We presented the LM using the LCA-NTM architecture. The LCA methods selected a memory address that represented the maximum cosine similarity. This differed from vanilla content-based addressing, because vanilla content-based addressing does not search for the maximum cosine similarity. The specific memory area, including the selected memory address, was normalized using a softmax function. For the PTB LM task, when the selected memory address was 257 when applying the LCA, the performance of the LCA-NTM-based LM showed a BPC of 1.5648. For the enwik8 LM task, when the selected memory address was 97, the BPC performance of the LCA-NTM-based LM showed a BPC of 1.3887. These results indicate that the proposed approach outperformed the MDM-NTM-based LM.  
In a future work, we intend to modify the LCA-NTM architecture to select multiple addresses according to the cosine similarity. In addition, we will implement methods to improve the inference time of the LCA-NTM architecture. We will also evaluate the LCA-NTM architecture on web-scale LM tasks, such as WikiText-103 and One Billion Word.  
Conceptualization, D.L. and M.-W.K.; methodology, D.L.; software, D.L.; validation, D.L.; formal analysis, D.L.; investigation, D.L. and M.-W.K.; writing--original draft preparation, D.L.; writing--review and editing, D.L., J.-S.P., M.-W.K. and J.-H.K.; supervision, J.-S.P., M.-W.K. and J.-H.K. All authors have read and agreed to the published version of the manuscript.  
This research was funded by by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1F1A1076562). The authors declare no conflict of interest.  
The following abbreviations are used in this manuscript:  
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**nWP**} & \multirow{2}{*}{**nRV**} & \multirow{2}{*}{**nVEM**} & \multirow{2}{*}{**nSVEM**} & \multirow{2}{*}{**WD**} & \multirow{2}{*}{**IT**} & \multicolumn{3}{c}{**BPC**} \\ \cline{5-10}  & & & & & & & & **Validation** & & **Test** \\ \cline{5-10}  & & & & & & & & \(\mu\) & \(\sigma\) & \(\mu\) & \(\sigma\) \\ \hline LSTM & 18.1 M & - & - & - & - & - & - & - & - & 1.4610 & - \\ Transformer & 44.1 M & - & - & - & - & - & - & - & 1.1120 & - \\ MDM-NTM & 42.5 M & 4 & 128 & - & \(1\times 10^{-7}\) & 3644 & 1.3970 & 0.0008 & 1.3922 & 0.0009 \\ LCA-NTM & 42.5 M & 4 & 128 & 33 & \(1\times 10^{-7}\) & 320 & 1.4120 & 0.0018 & 1.4113 & 0.0021 \\  & 42.5 M & 4 & 128 & 65 & \(1\times 10^{-7}\) & 321 & 1.4007 & 0.0020 & 1.3995 & 0.0019 \\  & 42.5 M & 4 & 128 & 97 & \(1\times 10^{-7}\) & 318 & 1.3895 & 0.0017 & 1.3887 & 0.0020 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Evaluation results of LCA-NTM-based LM according to number of selected memory addresses on the enwik8 LM task (nWP, number of weight parameters; nRV, number of read vectors; nVEM, number of vectors in the external memory; nSVEM, number of selected vectors in the external memory; WD, weight decay; IT, inference time (ms/batch); \(\mu\), mean of BPC results; \(\sigma\), standard deviation of BPC results).