VP8-08 Binary neural networks with giant conductance changes in Co2MnSi/MgO/Co2MnSi magnetic tunnel junctions
Creators: Takao Marukame
Rei Kusunose and Tetsuya Uemura
Publisher: name: IEEE
Year: 2025

With the growth of generative AI such as large-scale language models (LLMs), there is a global concern about their environmental impact due to high energy consumption [1, 2]. Recently, LLM models have been investigated for efficient computation using vector matrix multiplications (VMMs) of neural networks (NNs) represented as binary -1/+1 weights [1, 2]. However, binary hardware with realistic memory cells has yet to be implemented in LSI. In this study, we focus on Co2MnSi/MgO/Co2MnSi magnetic tunnel junctions (MTJs) [3], whose conductance corresponds to binary weights, and analyze binary NN computation. Conventional VMMs have used resistance change devices, such as ferroelectric tunnel junctions, due to their multilevel resistances [4]. In contrast, MTJs are generally stabilized at two values. A pair of the resistance change devices can be used to represent binary values in analog circuits [5]. To use binary (-1/+1) signals in the circuits, the rate of the conductance change should be high. Figure 1 shows conductance-voltage characteristics of our Co2MnSi/MgO/Co2MnSi MTJ, indicating huge conductance changes for binary states. The tunnel magnetoresistance ratio was 362% at room temperature, which is due to a halfmetallic property of Co2MnSi and a coherent MgO tunnel barrier, along with linear regions. In NN learning, connection weights between neurons are iteratively updated during learning and then used for inference. We assume the MTJs (Fig. 1) as binary connections with weights of -1/+1 when used in NN. The NN simulation with MNIST handwritten digits results show the inference accuracy of binary NN during and after learning (Fig. 2). We further propose the learning protocol using binary values in the early epochs and continuous values after that. Thus, we confirmed that the accuracy is improved due to the unique weight distribution, suggesting the feasibility of the MTJs with giant conductance changes for VMM applications.References: [1] H. Wang, et al., arXiv:2310.11453 (2023) [2] S. Ma, et al., arXiv:2402.17764 (2024) [3] H-x, Liu, et al., Appl. Phys. Lett. 101(13), 132418 (2012) [4] R. Berdan, T. Marukame, et al., Nature Electronics 3, 259 (2020) [5] T. Marukame, et al., Jpn. J. Appl. Phys. 59, 040606 (2020)
With the growth of generative AI such as large-scale language models (LLMs), there is a global concern about their environmental impact due to high energy consumption [1, 2]. Recently, LLM models have been investigated for efficient computation using vector matrix multiplications (VMMs) of neural networks (NNs) represented as binary -1/+1 weights [1, 2]. However, binary hardware with realistic memory cells has yet to be implemented in LSI. In this study, we focus on Co2MnSi/MgO/Co2MnSi magnetic tunnel junctions (MTJs) [3], whose conductance corresponds to binary weights, and analyze binary NN computation. Conventional VMMs have used resistance change devices, such as ferroelectric tunnel junctions, due to their multilevel resistances [4]. In contrast, MTJs are generally stabilized at two values. A pair of the resistance change devices can be used to represent binary values in analog circuits [5]. To use binary (-1/+1) signals in the circuits, the rate of the conductance change should be high. Figure 1 shows conductance-voltage characteristics of our Co2MnSi/MgO/Co2MnSi MTJ, indicating huge conductance changes for binary states. The tunnel magnetoresistance ratio was 362% at room temperature, which is due to a halfmetallic property of Co2MnSi and a coherent MgO tunnel barrier, along with linear regions. In NN learning, connection weights between neurons are iteratively updated during learning and then used for inference. We assume the MTJs (Fig. 1) as binary connections with weights of -1/+1 when used in NN. The NN simulation with MNIST handwritten digits results show the inference accuracy of binary NN during and after learning (Fig. 2). We further propose the learning protocol using binary values in the early epochs and continuous values after that. Thus, we confirmed that the accuracy is improved due to the unique weight distribution, suggesting the feasibility of the MTJs with giant conductance changes for VMM applications.References: [1] H. Wang, et al., arXiv:2310.11453 (2023) [2] S. Ma, et al., arXiv:2402.17764 (2024) [3] H-x, Liu, et al., Appl. Phys. Lett. 101(13), 132418 (2012) [4] R. Berdan, T. Marukame, et al., Nature Electronics 3, 259 (2020) [5] T. Marukame, et al., Jpn. J. Appl. Phys. 59, 040606 (2020)