# Deep Learning for Information Triage on Twitter
## 3 Preliminary Study: Types of Information Found on Twitter
### Classification Methods  
In classification, we compared seven classifiers, beginning from simple kNN and Naive Bayes, through SVMs and Tree-based classifiers, up to Neural Networks. After all thorough experiments, we also propose a Convolutional Neural Network-based approach with a best-matching data-preprocessing method.  
The **Naive Bayes** classifier is a supervised learning algorithm applying [PERSON] theorem with the assumption of a strong (naive) independence between pairs of features, traditionally used as a baseline in text classification tasks.  
**kNN**, or the k-Nearest Neighbors classifier takes, as input, k-closest training samples with assigned classes and classifies input samples to a class by a majority vote. It is often applied as a baseline, next to Naive Bayes. Here, we used the k = 1 setting, in which the input sample is assigned to the class of the first nearest neighbor, up to k = 5.  
**JRip**, also known as Repeated Incremental Pruning to Produce Error Reduction (RIP-PER) [22], learns rules incrementally to further optimize them. It has been especially efficient in the classification of noisy text [23].  
**J48** is an implementation of the C4.5 decision tree algorithm [24], which firstly builds decision trees from a labeled dataset, and each tree node selects the optimal splitting criterion further chosen to make the decision.  
**Random Forest** in the training phase creates multiple decision trees to output the optimal class (mode of classes) in the classification phase [25]. An improvement of RF to standard decision trees is their ability to correct over-fitting to the training set common in decision trees [26].  
**SVM** or support-vector machines [27] are a set of classifiers well-established in AI and NLP. SVM represent data belonging to specified categories, as points in space, and find an optimal hyperplane to separate the examples from each category. We used four types of SVM functions, namely: linear, the original function which finds the maximum-margin hyperplane dividing the samples; polynomial kernel, in which training samples are represented in a feature space over polynomials of the original variables; radial basis function (RBF) kernel, which approximates multivariate functions with a single univariate function, further radialised to be used in higher dimensions; and sigmoid, that is, hyperbolic tangent function [28].  
**CNN** or Convolutional Neural Networks are an improved type of a feed-forward artificial neural network model (i.e., multilayer perceptron). Although originally CNN were designed for image recognition, their performance has been confirmed in many tasks, including NLP [29] and sentence classification [30].  
We applied a Convolutional Neural Network implementation with Rectified Linear Units (ReLU) as a neuron activation function [31], and max pooling [32], which applies a max filter to non-overlying sub-parts of the input to reduce dimensionality and in effect, corrects over-fitting by down-sampling input representation. We also applied dropoutregularization on the penultimate layer, which prevents co-adaptation of hidden units by randomly omitting (dropping out) some of the hidden units during training [33].  
We applied two versions of CNN. First, one hidden convolutional layer containing 100 units was applied as a proposed baseline. Second, the final proposed method consisted of two hidden convolutional layers, containing 20 and 100 feature maps, respectively, both layers with a \(5\times 5\) patch size and \(2\times 2\) max-pooling, and Stochastic Gradient Descent [34].