# Sentiment Classification Using Convolutional Neural Networks
## 3 The Proposed Method  
CNN, which has been widely used on image datasets, extracts the significant features of the image, as the \"convolutional\" filter (i.e., kernel) moves through the image. If the input data are given as one-dimensional, the same function of CNN could be used in the text as well. In the text area, while the filter moves, local information of texts is stored, and important features are extracted. Therefore, using CNN for text classification is effective. Figure 1 shows a graphical representation of the proposed network. The network consisted of an embedding layer, two convolutional layers, a pooling layer, and a fully-connected layer. We padded the sentence vectors to make a fixed size. That is, too long sentences were cut to a certain length, and too short sentences were appended with the [PAD] token. We set the fixed length \(S\) to be the maximum length of the sentences. An embedding layer that maps each word of a sentence to an \(E\)-dimensional feature vector outputs an \(S\times E\) matrix, where \(E\) denotes the embedding size. For example, suppose that 10 is king, 11 is shoes, and 20 is queen in the embedding space. 10 and 20 are close in this space due to the semantic similarity of king and queen, but 10 and 11 are quite far because of the semantic dissimilarity of king and shoes. In this example, 10, 11, and 20 are not numeric values, they are just the simple position in this space. In other words, the embedding layer is a process of placing words received as input into a semantically well-designed space, where words with similar meanings are located close and words with opposite meanings are located far apart, digitizing them into a vector. The embedding is the process of projecting a two-dimensional matrix into a low-dimensional vector  
space (\(E\)-dimension) to obtain a word vector. The embedding vectors can be obtained from other resources (e.g., Word2 Vec) or from the training process. In this paper, our embedding layer was obtained through the training process, and all word tokens including the [UNK] token for unseen words would be converted to numeric values using the embedding layer. The \(S\times E\) matrix, the output of the embedding layer, is then laid down as the first convolutional layer. The first convolutional layer is the \(C_{1}\times E\) matrix, which stores the local information needed to classify the sentiment class in a \(S\times E\) matrix and convey information to the next convolutional layer. The \(C_{1}\times E\) matrix slides (i.e., convolves) all the values of the \(S\times E\) matrix with an arbitrary stride, calculates the dot product, and passes the dot product result to the next layer. The second convolutional layer uses the \(C_{2}\times 1\) matrix to extract features from the contextual information of the main word based on the local information stored in the first convolutional layer. \(C_{1}\) and \(C_{2}\) denote the filter size of each convolutional layer, and the two convolutional layers have \(K_{1}\) and \(K_{2}\) distinct filters, respectively, to capture unique contextual information. In other words, the first convolutional layer is utilized to look at simple contextual information while looking over the \(S\times E\) matrix, and the second convolutional layer is utilized to capture key features and then extract them (e.g., worst, great) that contain sentiments affecting classification. The matrix that passed through the consecutive convolutional layer is used as the input to the pooling layer. While average-pooling and L2-norm pooling have been used as the pooling layer position, in this paper, we used the max-pooling, which is a technique for selecting the largest value as a representative of the peripheral values. Since the sentiment is often determined by a combination of several words rather than expressing the sentiment in every word in the sentence, we adopted the max-pooling technique.