# Semi-Supervised Convolutional Neural Network for Law Advice Online
## 4 The Implementation of Our Scheme
### One-Hot Embedding for Small Text Regions  
The CNN-based model is a kind of DNN framework, which can use the internal structure of the data. In this paper, we study CNN on text processing to use the 1D structure (namely, word order) of text data for the effective semantic analysis task. We directly learn vector representations of the high-dimensional text regions to obtain the predictive ability of co-present and absent words in the classification task. In contrast, a low-dimensional word vector is trained to only represent individual words in isolation. In this sense, a text region can be more expressive than the simple average of word vectors in isolation. In practical application, the feature values are not always continuous values. Sometimes, they are classified values (namely, discrete values) such as gender characteristics, which can be divided into boys and girls. In machine learning, such feature information is usually digitized. In this paper, we introduce the one-hot vector model to implement effective mathematical representation for the discrete values. For text classification, one-hot CNN has been successfully applied, as illustrated in Figure 4, where it applies CNN to high-dimensional one-hot vectors directly and learns directly the embedding of small text regions [53]. Now, we consider the application of one-hot embedding to text data. Suppose that the text description of a law case \(L\) with vocabulary \(V\), which is represented by \(L=(\tau_{1},\tau_{2},\cdots,\tau_{n})\), is given. For text, a straightforward representation would be to treat each word as a pixel and the entire sentence as an image, which is \(|L|\times 1\) pixels with \(|V|\) channels. The goal here is to represent each word as a \(|V|\)-dimensional one-hot vector. For example, suppose that vocabulary \(V=\{\text{a},\text{developers},\text{default},\text{migrant},\text{pay},\text{time}, \text{on},\text{workers}^{\prime},\text{wages}\}\), which is in alphabetical order, and that the text description of a law case L = (Developers default on migrant workers' wages). Then, we have a one-hot-vector: \(t=\left| \begin{array}{l}010000000\\ 00100000\\ 00000100\\ 000000010\\ 00000001\end{array} \right|\). Next, we introduce the seq-one-hot-vector (\"seq\" for keeping sequences of words) and bow-one-hot-vector. For text representation, each text region is represented by a concatenation of the pixels (namely, each word). Each region, which each computation unit responds to, makes \(r|V|\)-dimensional region vectors where \(r\) is the region size fixed in advance. For example, on the example text vector \(t\) above, with \(r=3\) and \(stride=2\), we would have three regions \"developers default on\", \"on migrant workers\", and \"workers' wages\", represented by the following  
Figure 4: One-hot CNN example. Region size three, stride one. seq-one-hot-vector: \(r_{0}\left(t\right)=\left| \begin{array}{c}010000000\\ 001000000\\ 000000100\end{array} \right|,r_{1}\left(t\right)=\left| \begin{array}{c}00000010 \\ 000100000\\ 000000010\end{array} \right|,r_{2}\left(t\right)=\left| \begin{array}{c}00000001 0\\ 000100000\\ 0000000010\end{array} \right|,r_{2}\left(t\right)=\left| \begin{array}{c}0000000 10\\ 0000000010\end{array} \right|\). However, a potential problem of the seq-one-hot-vector method is that the dimensionality of each region vector \(r_{i}\left(t\right)\) could be very high if the number of the vocabulary \(V\) is very large. Therefore, it raises the urgency of considering other alternatives to the word vector representation. Here, an alternative is provided to perform the bow-one-hot-vector, which is converted from the \(r|V|\)-dimensional region vector to the \(|V|\)-dimensional vector. For example, the seq-one-hot-vector above would be converted to: \(r_{0}\left(t\right)=\left|011000100\right|,r_{1}\left(t\right)=\left|0001001 10\right|,r_{2}\left(t\right)=\left|00000011\right|\). With bow-one-hot-vector representation, fewer parameters are learned.