# Modified Conditional Restricted Boltzmann Machines for Query Recommendation in Digital Archives
## 4 Modified CRBMs (M-CRBMs)  
We propose M-CRBMs in this paper, which incorporate three conditional layers that can take extra information into account rather than only the observed visible variables, as shown in Figure 3. The conditional layers can easily be added or removed according to the number of types of extra information. In our experiment on a dataset of DAs, three types of extra information were used. Therefore, we implemented a three-conditional-layer M-CRBM in this paper. Our proposed M-CRBMs are much more scalable on large-size training data and suitable for DAs compared with the conventional CRBMs described in Section 3.3. First, in the query recommendation task, the training data can be extremely large in size, because in query recommendation, the ordinary embedding method is to represent an item (or a URL) with the bag-of-words (BoW) encoding method, where the ones represent the formulated queries that have been used to access that item. Therefore, to represent all the items, the number of dimensions of the representative vector of the item would be the number of all the queries in the access log. Next, the queries were difficult to embed for many of the natural language processing (NLP) models, which often reported efficiently reducing the dimension of the representative vector. The reason for this is that the queries in DAs are special, often being from a unique field, and can easily fail to be embedded well  
Figure 2: A CRBM with one conditional layer that connects to both the hidden layer and visible layer. by those NLP models trained on a general text corpus. Moreover, our proposed M-CRBM has the ability to incorporate more information, such as the metadata, which are abundant in DAs. Lastly, the extra information is considered to alleviate the difficulty of dealing with the sparse query representative vector. This is also one of the key advantages of M-CRBMs when compared with RBMs. Algorithm 3 states our proposed method. The idea of M-CRBMs is to capture the configurations (or dependencies) of the co-occurring queries formulated by high-level experts and low-level expertise users. Each M-CRBM represents an item (an ukiyo-e print), each visible layer unit represents a query, each hidden unit represents a latent feature of the queries, and each conditional unit represents a word from the corresponding metadata field. In this model, we regarded the queries that retrieved the same item as relevant, and these relevant queries were semantically similar. By learning the configurations of high-level and low-level expertise queries from all of the query log, the relevant queries could be detected. ```
Input: training data \(v^{(t)}\), conditional data \(r\), learning rate \(\eta\), initialized parameters \(\theta\) Result: update all the parameters \(\theta\) begin while\(step<N\)do if\(step=0\)then  sample \(\mathbf{v}^{(step+1)}\sim p\Big{(}\mathbf{v}\mid\mathbf{h}^{(0)}\Big{)}\); else  sample \(\mathbf{v}^{(step+1)}\sim p\Big{(}\mathbf{v}\mid\mathbf{h}^{(current)}\Big{)}\);  end if  sample \(\mathbf{h}^{(step+1)}\sim p\Big{(}\mathbf{h}\mid\mathbf{v}^{(step+1)},\mathbf{ r}\Big{)}\);  set \(\mathbf{h}^{(current)}\leftarrow\mathbf{h}^{(step+1)}\)  end if  update all the parameters \(\theta\) by: set \(\theta\leftarrow\theta+\eta\cdot\Big{(}\partial F^{c}\Big{(}\mathbf{v}^{(t)} \Big{)}/\partial\theta-\partial F(\mathbf{v}^{(step+1)})/\partial\theta\Big{)}\)  end for
```  
**Algorithm 3**CD in M-CRBMs with free energy  
Algorithm 3 states our proposed method. The idea of M-CRBMs is to capture the configurations (or dependencies) of the co-occurring queries formulated by high-level experts and low-level expertise users. Each M-CRBM represents an item (an ukiyo-e print), each visible layer unit represents a query, each hidden unit represents a latent feature of the queries, and each conditional unit represents a word from the corresponding metadata field. In this model, we regarded the queries that retrieved the same item as relevant, and these relevant queries were semantically similar.