The 70th Japan Society of Applied Physics Spring Meeting 2023
Highlighted Presentations Press Release
April 18, 2023

Performance evaluation of reservoir computing with two-dimensionally connected Josephson junctions forimage recognition

Next-generation machine learning technology reservoir
computing demonstrated in an image recognition task thatis close to real-world applications
Graduate School of Engineering, Tohoku University1, Research Institute of Electrical Communication, Tohoku University2
Graduate School of Informatics and Engineering, The University of Electro-Communications3
Kohki Watanabe 1,2, Yoshinao Mizugaki3, Satoshi Moriya2, Hideaki Yamamoto1,2, Shigeo Sato 1,2

Summary• An image recognition task similar to that in real-world applications was performed in thesimulation of reservoir computing circuits using Josephson junctions, which aresuperconducting devices.
• Numerical images were successfully classified with 92% accuracy; power consumption of
12.8 µW, which is equal to or lower than that of other leading physical reservoir
candidates.
• Applications range from superconducting machine learning hardware to quantum
computers.

Currently, research on "reservoir computing," which can perform calculations suitable for high-speed machine learning, isaccelerating. This is owing to its simple processing structure, easier device implementation, and its power-saving andhigh-speed capabilities. A research group including Kohki Watanabe at the Graduate School of Engineering, Tohoku
University, simulated a reservoir computing circuit using Josephson junctions, which are superconducting devices. Theirsimulation achieved a classification of numeric images with 92% accuracy and a power consumption of 12.8 µW, which
is similar to or lower than other leading candidate materials for physical reservoirs. Based on the cryogenic operating
characteristics of this study, potential applications include superconducting machine-learning hardware and evenquantum computers.

DetailHigh speed, low power consumption, easy for device implementation
Large-scale data analysis using machine learning has become indispensable in modern society. As
the amount of electricity used for machine learning grows each year in response to social demand,
the research and development of low-power machine learning devices is being sought. In thiscontext, reservoir computing has attracted considerable attention. The advantages of thistechnology, such as high-speed learning, low power consumption, and a simple processing
structure that easily facilitates device creation, are currently accelerating research anddevelopment in various fields.
Reservoir computing has a processing structure consisting of input, reservoir (intermediate layer),and output layers (Figure 1). The reservoir layer uses a "recurrent neural network" (*1), which isindispensable for machine learning. In a typical recurrent neural network, learning adjusts thecoupling between neurons, updating the weights of the entire network. Therefore, physicalimplementation involves high implementation hurdles in hardware, such as the requirement to useelements that store changing weights, and inevitably consumes more power. However, reservoir
computing has a processing structure that does not change the weights within the reservoir layer.
In other words, the weights are adjusted only between the reservoir and output layers. Owing tothis processing structure, reservoir computing is characterized by easy physical implementationand low power consumption.
A research group including Kohki Watanabe at the Graduate School of Engineering, Tohoku
University has been studying a superconducting device, the Josephson transmission line (JTL), asa "physical reservoir" that replaces the reservoir layer with a physical device. A physical reservoir
must be nonlinear, which can be realized in memristors, spin devices, neurons, and opticalresonators. Previous studies on reservoir computing using a JTL, which is a superconducting
device, have been investigated; moreover, their feasibility has been reported. This study reports
the success of an image recognition task, which is close to real-world applications, by reservoir
computing using JTL through simulations. This study goes beyond previous research and realizes
realistic computing.
Reservoir computing is close to real-world applications
The JTL is being used in superconducting digital computers (SFQ circuits *2). It is a circuitcomposed of a chain of Josephson junctions (JJs). When a current pulse is applied as an input into
this circuit, a quantized magnetic flux or flux quantum is trapped within the circuit. In addition, inresponse to changes in the input signal, JJ switches from a superconducting state to a resistive
state, and the magnetic flux quantum propagates in the circuit. In this case, JJ acts as a thresholdelement, and thus, JTL exhibits a nonlinear response to the input signal.
“A JTL is a one-dimensional superconducting transmission line. A unique feature of this study is theconstruction of a JTL reservoir by arranging these lines in two dimensions. We constructed a more
complex network by making the network two-dimensional. The accompanying optimization of thecircuit parameters enabled us to perform an image-recognition task (Figure 2). As a result, wewere able to correctly classify numeric images with 92% accuracy. There has not yet been any
prior study on image recognition in reservoir computing using JTL. Previous studies only consistedof simulations of basic tasks. The novelty of our study is that we simulated a task similar to that inreal-world applications using reservoir computing. In the future, we hope to implement this inhardware,” said Watanabe.

The power consumption of the JTL reservoir was 12.8 µW when the input voltage to the circuit andthe bias voltage of each junction were calculated. Although the power consumption varies basedon the task to be performed and cannot be compared in general, it is 10 mW for photonics and1-100 µW for spin materials. These are the leading candidates for physical reservoirs, indicatingthat the JTL in this study has a low power consumption equivalent to or lower than that of theleading candidates.
Application to quantum computers“Reservoir computing uses a recurrent neural network in the reservoir layer, which indicates thatwhen a signal is input, it is stored for a certain period. Therefore, it is suitable for analyzing timeseries data. The JTL reservoir used in this study could also be applied to time series dataclassification,” says Watanabe about future prospects.
Professor Shigeo Sato of Graduate School of Engineering, Tohoku University, and ResearchInstitute of Electrical Communication, Tohoku University, who is involved in this study, alsoconsiders that "the seemingly disadvantageous features of JTL reservoirs, which operate at lowtemperatures, could open applications in quantum computers."
The JTL reservoirs are superconducting devices that operate at extremely low temperatures.
Therefore, it is unsuitable for edge computing (*3). However, most quantum computers operate at
low temperatures. Research on the use of machine learning to adjust the state measurement ofqubits is already underway, and there is a possibility that reservoir computing with JTL reservoirs
can be used in this machine learning process.
“Their advantage is that they can only operate at low temperatures when working with a quantum
computer. Currently, the integration of digital computing and machine learning and that of quantum
computing and machine learning are being explored. The recurrent neural networks used inreservoir computing can be applied to other machine learning; therefore, they are a good match.
Additionally, because quantum computers are currently being developed at the national level, thedevelopment of reservoir computing could also be an important theme,” said Shigeo Sato.
The group plans to conduct further research and development for the physical implementation of
JTL reservoirs and identify specific issues.

Annotation*1. RNN / recurrent neural network: A neural network that behaves dynamically over time by regressing the output of anode back on its input. It can be applied to tasks, such as sentence generation and speech recognition, and is used inmany modern machine learning applications.
*2. SFQ / single flux quantum circuit: A logic circuit in which the presence or absence of a magnetic flux quantum in asuperconducting circuit corresponds to a binary signal "1" or "0.” It was proposed by Nakajima et al. at Tohoku Universityand systematized by Likharev et al. at Moscow State University. They are used in various applications ranging fromdigital arithmetic circuits to readout circuits for superconducting qubits.
*3. Edge computing: Computational processing is performed at the edge (periphery) of a computer network. In anarrower sense, this refers to the computational processing within devices, such as those used in IoT.

Figures

Figure 1: Network structure in reservoir computing.

Figure 2 Image recognition in this study is the task of classifying images of handwritten numbers. The 5 × 4 image in Figure (a) is
divided into five rows and fed as input voltage for four time frames to a random node of the reservoir circuit in Figure (b). The readout
weights are computed by ridge regression, as shown in Figure (c). As a result, the JTL reservoir was able to classify the numeric images
with 92% accuracy.