# An Algorithm of Quantum Restricted Boltzmann Machine Network Based on Quantum Gates and Its Application

Authors: Peilin Zhang; Sheng Li; Yu Zhou
Journal: Shock and Vibration
 (2015)

---

## Abstract

We present an algorithm of quantum restricted Boltzmann machine network based on quantum gates. The algorithm is used to initialize the procedure that adjusts the qubit and weights. After adjusting, the network forms an unsupervised generative model that gives better classification performance than other discriminative models. In addition, we show how the algorithm can be constructed with quantum circuit for quantum computer.


---

## Body

## 1. Introduction

The restricted Boltzmann machine (RBM) is a probabilistic network model that has one visible layer and one hidden layer. In the RBM structure, the connections between visible variables and hidden variables are excluded, so a visible variable is conditionally independent influenced by the units of hidden layer, and a hidden variable is conditionally independent influenced by the units of visible layer. The RBM has been applied in many fields, such as text [1, 2] and images [3]. In [4], using complementary priors, a fast, greedy algorithm can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. In deep belief nets (DBN) for phone recognition [5], the mean covariance RBM is firstly introduced to learn features of speech data that serve as input into a DBN. The result of the proposed method is superior to all published results on speaker-independent TIMIT to date. For using RBM better, the research in RBM is introduced precisely for now in [6]. It provides a series of methods for the parameters of RBM to solve the existing problem. In [7], the spike and slab RBM are characterized by having both a real-valued vector, the slab, and a binary variable, the spike, associated with each unit in the hidden layer. The proposed method can achieve competitive performance on the CIFAR-10 object recognition task. As the classification RBM is established, different strategies for training and different approaches to estimating gradient are studied. The proposed method is applied in, namely, semisupervised and multitask learning [8]. The present paper describes a novel algorithm for automate brain MRI scan tissue segmentation which employs a continuous RBM. The results of a pilot performance test are presented. The proposed method can approach and possibly surpass the performance of existing segmentation algorithms [9]. For now, there are two ways. One is that a RBM is trained as an unsupervised neuron model for Deep Belief Network. The other is that a RBM is paired with some other evolution algorithms or computation methods to improve the performance of learning methods.Quantum computation was firstly proposed by Kak [10]. Nielsen and Chuang extended the field and applied the algorithm in factoring [11]. In quantum computation, qubit is the basis of computation, which can contain many system conditions. The parallelism makes the algorithm execute fast. Therefore, quantum computation has better performance in computational complexity, convergence, and arithmetic speed. Nowadays, quantum computation combines other evolution algorithms, such as quantum Fourier transform [12, 13], quantum neural network [14, 15], and quantum genetic algorithm [16, 17]. Quantum algorithm is applied in bilevel multiobjective programming, parameter optimization, and pattern recognition [18–20]. The best way to achieve quantum computation is to construct a quantum circuit for quantum computer. Maeda et al. [21] used a quantum circuit to construct a qubit neuron model to solve XOR problem. P. C. Li and S. Y. Li [22] established a learning algorithm of quantum neural network based on universal quantum gate, which got a superior performance in pattern recognition. Aiming at fast computation and efficient convergence, the quantum circuit has been applied in eigenvalues search [23], optimal method [24], probability calculation [25], and so on [26].In this paper, we argue that quantum computation can provide a novel, fast, and stable framework for RBM to solve unsupervised nonlinear problems. An algorithm of quantum restricted Boltzmann machine (QRBM) network is proposed. In the proposed method, quantum representation is used as input, and quantum gates are used for data training and parameters updating. By modeling the quantum circuit of visible layer and hidden layer, QRBM will be used for locating the best global solution in the network outputs. We compare different number of hidden layers, which is the key of convergence speed and complexity. We also describe how the QRBM can be used for pattern recognition. Using experiments on fault diagnosis of gear box, we show that the classification performance of QRBM is more competitive than other classifiers such as standard RBM (SRBM) network and standard neural network (SNN).The structure of the paper is organized as follows. In Section2, the principle of RBM network is introduced. Section 3 shows the quantum computation and quantum gates for QRBM. In Section 4, the execution steps of QRBM are given, and it shows how the quantum circuit is composed of quantum gates, which provides a simulation progress. Section 5 shows the pattern recognition performance of QRBM for gear box with different hidden layers.


## 2. Restricted Boltzmann Machine Network

In RBM with one hidden layer, consider the state ofn visible layer and the state of m hidden layer h; the energy function is given by(1)Ev,h=-∑i,jvihjwij-∑ibivi-∑jcjhj,where wij is the connection weight and bi and cj are the biases of two layers, i=1,2,…,n, j=1,2,…,m.According to energy function, the joint probability of a visible and a hidden variablev,h is assigned: (2)pv,h=1Ze-Ev,h,where Z=∑v,he-Ev,h.The probability for visible layerv is given by summing the whole possible variables in hidden layer:(3)pv=∑he-Ev,hZ.The probability that the RBM is assigned to a training set can be raised by finding the minimum energy in energy function. Define free energy as(4)FreeEnergyv=-ln∑he-Ev,h.Then the probability can be rewritten as(5)pv=e-FreeEnergyvZ.Because of probabilistic network, the performance of RBM, whose standard is likelihood function, is evaluated on probability theory. The method of maximum likelihood is used for estimation for model parameters, which makes the maximum probability for variables of visible layer and finds the minimum free energy of the RBM network:(6)∂lnpv∂wij=phj=1∣vvi-∑vpvphj=1∣vvi,∂lnpv∂bi=vi-∑vpvvi,∂lnpv∂cj=phj=1∣v-∑vpvphj=1∣v,where phj=1v=sigm∑i=1nwijvi+cj.For sampling problem in RBM, it starts at any random state in visible layer and performs Gibbs sampling [1]. In order to simplify the procedure of sampling, a gradient of objective function called contrastive divergence is shown, which needs fewer steps.


## 3. Quantum Computation

In quantum computation, because of the entanglement, superposition, and no-cloning of qubit, a random state can represent linear combination of two basic states:(7)ϕ=α0+β1,where α and β are the probability amplitude. The relationship between α and β satisfies the following: (8)α2+β2=1.In this paper, the independent variables can be described as follows:(9)xi=αiβiT,where i=1,2,…,n.In quantum computation, the quantum gates are used for random and complicated computation. In Hilbert space, a random unitary transformation can be decomposed into the combination of single qubit gates and two-qubit gates. The common quantum gates have Hadamard gate (H), rotation gate (R), and controlled-NOT gate (Ucnot), which are shown in Figures 1, 2, and 3.Figure 1
Hadamard gate.Figure 2
Rotation gate.Figure 3
Two qubit controlled-NOT gates.In order to carry out the proposed QRBM algorithm, the three quantum gates are the basis for constructing a quantum circuit to execute.


## 4. The Algorithm of Quantum Restricted Boltzmann Machine

### 4.1. The Quantum Representation

There is a set of dependent variablesH=h1,h2,…,hq and a set of independent variables X=(x1,x2,…,xp).Definition 1.
A formula which is to transform the real values into the quantum representation of variables is said to be(10)xi=αi0+βi1=cos2πxi-min1≤j≤pxjmax1≤j≤pxj-min1≤j≤pxj0+sin2πxi-min1≤j≤pxjmax1≤j≤pxj-min1≤j≤pxj1,(11)hk=θk0+γk1=cos2πhk-min1≤l≤qhlmax1≤l≤qhl-min1≤l≤qhl0+sin2πhk-min1≤l≤qhlmax1≤l≤qhl-min1≤l≤qhl1,where i=1,2,…,p, k=1,2,…,q.In Formula (9) and Formula (10), one can see that α,β,θ,γ∈-1,1. After transformation, the qubit sets are got:(12)X=x1,x2,…,xp=α1β1α2β2⋯⋯αpβp,H=h1,h2,…,hq=θ1γ1θ2γ2⋯⋯θqγq.


### 4.2. Quantum Circuit for QRBM

Based on quantum gates, a quantum circuit can be constructed. Set a quantum register0,0,…,0, which can store the state of assistant qubits. The specific quantum circuit of QRBM network is shown in Figure 4.Figure 4
The quantum circuit of QRBM.In Figure4, we show the variables of visible layer (x1,x2,…,xp) and the variables of hidden layer (y1,y2,…,yq) and a quantum register contained p qubits. Hadamard gate is the preprocessing step, which is the basis of unitary transformation. The matrix of quantum rotation gates is used for phase transformation to calculate the coefficients of variables in visible layer. With a controlled-NOT gate, a quantum state of each variable is switched, and the whole variables in visible layer are summed into one qubit. After phase transformation with Hadamard gate again, the states of one qubit in hidden layer are given, which represents the output matrix.As we all see, the quantum circuit in Figure4 is a simple circuit with one hidden layer. In order to enhance the performance of QRBM, we can increase the number of hidden layers to compose different QRBM network, such as QRBM1, QRBM2, and QRBMn. A suitable number of hidden layers can shorten execution time and maintain high classification accuracy. The number of hidden layers in learning algorithm is depended on the actual need for pattern recognition and fault diagnosis.


### 4.3. The Strategy of Parameter Update

In QRBM algorithm, there is a matrix of quantum rotation gate updated, which hasp×q parameters: (13)Rφ=φ11φ12⋯φ1qφ21⋱φ2q⋮⋱⋮φp1φp2⋯φpq.Set QRBM1 algorithm as an example; the output after quantum operations can be denoted as(14)hk=cosψk0+sinψk1=cosϕk+π40+sinϕk+π41,where ψk=ϕk+π/4, ϕk=arcsin∏i=1psinεi+φik+π/4.We put the coefficient of qubit1 as the actual output, so(15)hk=sinψk=sinϕk+π4.Quantum computation is also a probability computation. So the probability formula of hidden layer for QRBM is obtained:(16)phk=sin2ψk.Definition 2.
Fork=1,2,…,q, the free energy for hidden layer in QRBM can be said to be (17)FreeEnergyφ=FEφ=14∑k=1aph~k-phk2,where ph~k is the expected probability and phk is the actual probability.The bigger theFE is, the worse the robustness of neural network model is. It will be sensitive for the change of sample.For the updating of parameterRφ, the gradient descent method that can approach the minimum error is applied. Put (18)-∂FE∂φik=∑k=1aph~k-phkphkcos2ψkSkTk,where Sk=1/1-sin2ϕk, Tk=sinϕk∑i=1pcotεi+φik+π/4.So the formula of parameter updating is denoted as(19)φikt+1=φikt-∂FE∂φik,where t is the iteration step.


## 4.1. The Quantum Representation

There is a set of dependent variablesH=h1,h2,…,hq and a set of independent variables X=(x1,x2,…,xp).Definition 1.
A formula which is to transform the real values into the quantum representation of variables is said to be(10)xi=αi0+βi1=cos2πxi-min1≤j≤pxjmax1≤j≤pxj-min1≤j≤pxj0+sin2πxi-min1≤j≤pxjmax1≤j≤pxj-min1≤j≤pxj1,(11)hk=θk0+γk1=cos2πhk-min1≤l≤qhlmax1≤l≤qhl-min1≤l≤qhl0+sin2πhk-min1≤l≤qhlmax1≤l≤qhl-min1≤l≤qhl1,where i=1,2,…,p, k=1,2,…,q.In Formula (9) and Formula (10), one can see that α,β,θ,γ∈-1,1. After transformation, the qubit sets are got:(12)X=x1,x2,…,xp=α1β1α2β2⋯⋯αpβp,H=h1,h2,…,hq=θ1γ1θ2γ2⋯⋯θqγq.


## 4.2. Quantum Circuit for QRBM

Based on quantum gates, a quantum circuit can be constructed. Set a quantum register0,0,…,0, which can store the state of assistant qubits. The specific quantum circuit of QRBM network is shown in Figure 4.Figure 4
The quantum circuit of QRBM.In Figure4, we show the variables of visible layer (x1,x2,…,xp) and the variables of hidden layer (y1,y2,…,yq) and a quantum register contained p qubits. Hadamard gate is the preprocessing step, which is the basis of unitary transformation. The matrix of quantum rotation gates is used for phase transformation to calculate the coefficients of variables in visible layer. With a controlled-NOT gate, a quantum state of each variable is switched, and the whole variables in visible layer are summed into one qubit. After phase transformation with Hadamard gate again, the states of one qubit in hidden layer are given, which represents the output matrix.As we all see, the quantum circuit in Figure4 is a simple circuit with one hidden layer. In order to enhance the performance of QRBM, we can increase the number of hidden layers to compose different QRBM network, such as QRBM1, QRBM2, and QRBMn. A suitable number of hidden layers can shorten execution time and maintain high classification accuracy. The number of hidden layers in learning algorithm is depended on the actual need for pattern recognition and fault diagnosis.


## 4.3. The Strategy of Parameter Update

In QRBM algorithm, there is a matrix of quantum rotation gate updated, which hasp×q parameters: (13)Rφ=φ11φ12⋯φ1qφ21⋱φ2q⋮⋱⋮φp1φp2⋯φpq.Set QRBM1 algorithm as an example; the output after quantum operations can be denoted as(14)hk=cosψk0+sinψk1=cosϕk+π40+sinϕk+π41,where ψk=ϕk+π/4, ϕk=arcsin∏i=1psinεi+φik+π/4.We put the coefficient of qubit1 as the actual output, so(15)hk=sinψk=sinϕk+π4.Quantum computation is also a probability computation. So the probability formula of hidden layer for QRBM is obtained:(16)phk=sin2ψk.Definition 2.
Fork=1,2,…,q, the free energy for hidden layer in QRBM can be said to be (17)FreeEnergyφ=FEφ=14∑k=1aph~k-phk2,where ph~k is the expected probability and phk is the actual probability.The bigger theFE is, the worse the robustness of neural network model is. It will be sensitive for the change of sample.For the updating of parameterRφ, the gradient descent method that can approach the minimum error is applied. Put (18)-∂FE∂φik=∑k=1aph~k-phkphkcos2ψkSkTk,where Sk=1/1-sin2ϕk, Tk=sinϕk∑i=1pcotεi+φik+π/4.So the formula of parameter updating is denoted as(19)φikt+1=φikt-∂FE∂φik,where t is the iteration step.


## 5. Experiment

In this work, we applied the proposed algorithm, QRBM, into the pattern recognition of gear box. The computation operation has been carried out on a computer that had an Intel Core 2 Duo CPU with 2.00 GB of main memory. We compared quantum restricted Boltzmann machine (QRBM) with the other two traditional methods, standard neural network (SNN) and standard restricted Boltzmann machine (SRBM).In gear box experiment for collecting vibration signals, we set four working conditions, that is, normal condition, gear tooth wear fault, gear root crack fault, and gear tooth broken fault. The speed is 600 r/min. The sampling number is 2048. The sampling frequency is 12800 Hz. The mode of sensor is CA-YD-185. For each working condition, 50 samples are collected, in which there are 20 samples for training and 30 samples for testing. Figures5(a) and 5(b) give the demonstration of vibration signal in four working conditions.Figure 5
The quantum circuit of QRBM.
(a)
Normal condition
(b)
Gear tooth wear fault
(c)
Gear root crack fault
(d)
Gear tooth broken faultConsidering amplitude and its distribution, we can see some obvious differences in Figures5(a)–5(d). Meanwhile, the waves in Figure 5 are a little confused. In gear tooth wear fault, there is much noise. In gear root crack fault, there is a strong impact at the crack. In gear tooth broken fault, there is lots of noise and a stronger impact at the broken point.In order to find the best number of hidden layers, we compared the performance of four QRBM algorithms in execution time and classification accuracy. The number of hidden layers is set at 1, 2, 3, and 4, which are called QRBM1, QRBM2, QRBM3, and QRBM4. The parameters in four algorithms are listed as follows. The initial value of φ is the random value in 0,2π. The number of iteration t is 1000. The structure of three algorithms is separately 2048-1800-4, 2048-1800-1024-4, 2048-1800-1024-800-4, and 2048-1800-1024-800-400-4. So, the comparison results are shown in Figure 6.Figure 6
The comparison result with four algorithms.As we can see from Figure6, QRBM4 had the lowest accuracy 85.6%, which took 11.27 s. QRBM3 had the accuracy 90.3%, which took 8.41 s. The classification accuracy of QRBM2 was the highest accuracy 97.1%, which took 6.13 s. QRBM1 had the accuracy 95.8%, which took 4.28 s. As the number of hidden layers increases, the classification accuracy is lower and the run time is longer. The reason that QRBM4 had lowest accuracy and longest execution time was that there are too many quantum gates calculated and the value is stuck in local optimum. For QRBM2, because of suitable quantum gates, it had the highest classification accuracy, but the execution time was longer than QRBM1. Therefore, in this paper, we choose QRBM1 as the main learning algorithm.To evaluate the performance of QRBM1, we compared the classification performance to SNN and SRBM, which are the traditional methods for pattern recognition. In SNN, the parameters were set as follows. The structure is 2048-1024-4. The learning rate is 0.9. The number of iteration steps is 1000. The tolerated error is 0.05. In SRBM, the parameters were set as follows. The structure is 2048-1024-4. The number of iteration steps is 1000. The tolerated error is 0.05. The three algorithms were carried out 50 times, and these results are presented in Table 1.Table 1
Comparison results of pattern recognition for gear box.
Condition
Algorithms
SNN
SRBM
QRBM1
Normal condition
Correct number
50
50
50
Accuracy rate (%)
100
100
100
Execution time (s)
12.9
8.4
5.8
Gear tooth wear fault
Correct number
48
50
50
Accuracy rate (%)
96
100
100
Execution time (s)
13.7
8.9
6.3
Gear root crack fault
Correct number
43
47
49
Accuracy rate (%)
86
94
98
Execution time (s)
18.3
9.2
6.7
Gear tooth broken fault
Correct number
45
47
48
Accuracy rate (%)
90
94
96
Execution time (s)
15.8
8.1
6.8In Table1, SNN, SRBM, and QRBM1 can reach the accuracy rate 100%, but QRBM1 had the shortest execution time. Meanwhile, the performance of SRBM was better than SNN. With observing all the results of four working conditions, the best performance algorithm is QRBM1, which enhanced 2–4% of classification accuracy rate and saved 63% of execution time at most. So, QRBM algorithm has better feasibility and practicability.


## 6. Conclusions

The intention of this paper is to propose an algorithm of quantum restricted Boltzmann machine based on quantum gate. We considered the universality of quantum computation for RBM for pattern recognition and evaluated it. In particular, we highlighted the quantum circuit for QRBM and explored the suitable number of hidden layers. We also extended the application field where the QRBM can be employed and multiclass problem can be solved.As discussed and analysed previously, the algorithm of QRBM is an effective and practical learning method, which can achieve superior classification performance. As a prognosis for quantum computation applied in RBM, with these improvements to QRBM, this review could help us construct a more precise quantum circuit for quantum computer in future.



---

*Source: 756969-2015-09-15.xml*