## 4 Neural Network Architecture  
In this paper, a two-layer convolutional neural network is presented for cross-domain product review sentiment classification (**LM-CNN-LB**). This classification model is deployed with its convolution layers interleaved with pooling layers. The architecture of LM-CNN-LB is illustrated in Figure 1.  
A CNN works only with fixed length inputs; thus, every input length is standardized to \(l\) by trimming the longer sentences and padding the shorter sentences with zeros. Given an input instance \(X_{i}\in R^{l}\), \(x_{i}\) is the \(i\)-th word in this instance. **layer-0**: The first layer is the embedding layer. Words in sentences are converted to low-dimensional word vectors \(v_{i}\in R^{k}\) through pre-training by Word2 Vec [11], where \(k\) is the dimension of the word vector. Thus, the input instance \(X\in\mathbb{R}^{l\times k}\) is concatenated with the word vector as follows, where \(\oplus\) is the concatenation operator:  
\[X_{1:l}=v_{1}\oplus v_{2}\oplus...\oplus v_{1} \tag{1}\]  
The word vectors are applied to initialize the weight of the embedding layer, and this layer is fine-tuned during training in the source domain. In subsequent steps, the network is regularized by dropout. The output can then be used to augment the neural network layer. **layer-1**: This layer consists of a one-dimensional convolutional operation and a max pooling operation. The convolutional operation can learn an internal feature representation. The essence of the convolutional layer is to convert text regions of a fixed size (e.g., \"do not waste your time\" which has a size of 5) to feature vectors. The vector \(\mathbf{m}\in R^{n}\) is the filter of the convolution, namely, the weight. A filter width of \(n\) enables the convolution layer to make use of text word order to capture the contextual feature of a word. For a word vector \(v_{i}\), the feature vectors around \(v_{i}\) are concatenated within \(n\). Then, the vector is input to the convolution operation to take the dot product of the weight vector \(\mathbf{m}\in R^{n}\) and an input vector \(v_{i}\in R^{k}\) with the activation function. The new feature representation is as follows:  
\[f_{j}=\mathbf{m}^{T}v_{j+n-1} \tag{2}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Domain** & \(\mathbf{D_{\textit{I}}}\) & \(\mathbf{D_{\textit{b}}}\) & \(\mathbf{D_{\textit{I}}}\)\(\rightarrow\)\(\mathbf{D_{\textit{b}}}\) \\ \hline D\(\rightarrow\)B & 0.2643 & 0.2052 & 0.3017 \\ E\(\rightarrow\)B & 0.6069 & 0.4864 & 0.5113 \\ K\(\rightarrow\)B & 0.6048 & 0.5064 & 0.5126 \\ B\(\rightarrow\)D & 0.2294 & 0.1925 & 0.2439 \\ E\(\rightarrow\)D & 0.5225 & 0.4587 & 0.4734 \\ K\(\rightarrow\)D & 0.5677 & 0.5043 & 0.4938 \\ B\(\rightarrow\)E & 0.6153 & 0.3171 & 0.4014 \\ D\(\rightarrow\)E & 0.5363 & 0.3025 & 0.3610 \\ K\(\rightarrow\)E & 0.3010 & 0.2379 & 0.2189 \\ B\(\rightarrow\)K & 0.6280 & 0.3029 & 0.3746 \\ D\(\rightarrow\)K & 0.6211 & 0.2642 & 0.3425 \\ E\(\rightarrow\)K & 0.3033 & 0.1936 & 0.1800 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: KL divergence of different domains. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Domain** & \(\mathbf{D_{\textit{b}}}+\mathbf{D_{\textit{I}}}\) & **Positive** & **Negative** \\ \hline \(B_{D_{\textit{M}}}\) & \(B_{D_{\textit{D}}}+B_{D_{\textit{I}}}\) & 50,000 + 1000 & 50,000 + 1000 \\ \(D_{D_{\textit{M}}}\) & \(D_{D_{\textit{D}}}+D_{D_{\textit{I}}}\) & 50,000 + 1000 & 50,000 + 1000 \\ \(E_{D_{\textit{M}}}\) & \(E_{D_{\textit{D}}}+E_{D_{\textit{I}}}\) & 50,000 + 1000 & 50,0 The output of the layer is as follows:  
\[\mathbf{O}=\sigma(f_{j}+b) \tag{3}\]  
where \(\sigma\) is activation function _relu_. \[f(x)=max(0,x) \tag{4}\]  
The weight vector \(\mathbf{m}\in R^{n}\) and the bias vector \(b\in R^{n}\) are shared by all units in the same layer and are learned through training. This filter is applied to each possible window of words in the sentence \(\{x_{1:h},x_{2:h+1},...,x_{n-h+1:n}\}\) to produce a feature map.