# Semi-Supervised Convolutional Neural Network for Law Advice Online
## 4 The Implementation of Our Scheme
### _tv-Embedding Model_  
The semi-supervised learning framework includes the following two steps: tv-embedding (tv represents two-view) learning and supervised learning. Semi-supervised model directly learns the embedding of text regions from unlabeled data with the tv-embedding learning model and then integrates the learned embedding into supervised CNN. The learned embedding regions (the output of the tv-embedding learning model) are used as the additional input to the supervised model. The first step of the semi-supervised learning model is to learn the region embedding from no-label text data, as illustrated in Figure 5. The definition of tv-embedding is that if there exists a function \(g(e.g.,P\left(X_{2}|X_{1}\right)=g_{1}\left(f_{1}\left(X_{1}\right),X_{2} \right))\), the function \(f_{1}\) is the tv-embedding of \(\chi_{1}\quad w.r.t.\quad\chi_{2}\) for any \(\left(X_{1},X_{2}\right)\in\chi_{1}\times\chi_{2}\). A tv-embedding of a view (X1) keeps the required structure in order to generate another view (X2), which learns useful feature vectors from no-label text data. For the tv-embedding learning model, a neural network is trained to predict the adjacent regions from each small text region. That is, convolution layers produce useful feature vectors from each small text region. Then, the classifier in the top layer uses the feature vectors. Since each small text region is associated with its own output, the tv-embedding model has a distinguishing difference from CNN. We conclude the goals that the tv-embedding model mainly implements as the following aspects:  
* It implements the goal to predict the adjacent text region from each region of size \(p\). What is more, it can capture the internal information between data. * It can assign a sub-label (e.g., positive/negative) to each small text region of size \(p\). For example, in Figure 5 above, with \(p=4\) and \(stride=2\), three regions \"[PERSON] was injured\", \"was injured by [PERSON]\" and \"by X Xiao gang\" are assigned sub-labels 0, 0, 1, respectively. Note that 1 means positive, 0 means negative. It is worth noting that assigning a sub-label to each small text region of size \(p\) is sensible because these small text regions are helpful for making classification. * It also assigns a total label (e.g., positive/negative) for the entire sentence. For example, the entire sentence \"[PERSON] was injured by Xiao gang.\" is assigned a total label 0, as illustrated in Figure 5. Note that 1 means positive, 0 means negative. This is sensible because CNN makes classification by analyzing the entire sentence as well. * A convolution layer learns the embedding of small regions from no-label text data. The high-dimensional feature vectors are converted to the low-dimensional feature vectors; in addition, the predictive structure is preserved. Given a document \(x\), for a computation unit associated with the \(i\)-th region, the convolution layer of the tv-embedding learning model calculates:  
\[\
u_{i}\left(x\right)=\sigma^{\left(v\right)}\left(W^{\left(v\right)}\cdot r_{i} ^{\left(v\right)}\left(x\right)+b^{\left(v\right)}\right) \tag{1}\]  
where \(\sigma\) is a predefined non-linear activation function ReLU (namely, \(f\left(x\right)=\max\left(0,x\right)\)), which is applied to each vector component. \(r_{i}^{\left(v\right)}\left(x\right)\) is the input region vector for the \(i\)-th text region, which can be either the seq-one-hot-vector, the bow-one-hot vector, or the bag-of-n-gram. In this paper, we use the bow-one-hot vector, which solves the problem of high-dimensional data. \(W^{\left(v\right)}\) and \(b^{\left(v\right)}\), which are learned through training, are shared by the computation units in the same layer. The top layer uses \(v_{i}\left(x\right)\) as the input vector. The goal here is to learn a vector representation of the small text region and assign a sub-label (e.g., positive/negative) to each small text region and a total label (e.g., positive/negative) for the entire sentence.