Question: What is applied to encounter class imbalance?
Evidence: In this paper, we explore the feasibility of automatically recognising signals of cyberbullying. A crucial difference with state-of-the-art approaches to cyberbullying detection is that we aim to model bullying attacks, as well as reactions from victims and bystanders (i.e., all under one binary label `signals of cyberbullying'), since these could likewise indicate that cyberbullying is going on. The experiments described in this paper focus on the detection of such posts, which are signals of a potential cyberbullying event to be further investigated by human moderators.
The English and Dutch corpus contain 113,698 and 78,387 posts, respectively. As shown in Table TABREF36 , the experimental corpus features a heavily imbalanced class distribution with the large majority of posts not being part of cyberbullying. In classification, this class imbalance can lead to decreased performance. We apply cost-sensitive SVM as a possible hyperparameter in optimisation to counter this. The cost-sensitive SVM reweighs the penalty parameter INLINEFORM0 of the error term by the inverse class-ratio. This means that misclassifications of the minority positive class are penalised more than classification errors on the majority negative class. Other pre-processing methods to handle data imbalance in classification include feature filtering metrics and data resampling BIBREF56 . These methods were omitted as they were found to be too computationally expensive given our high-dimensional dataset.
For the automatic detection of cyberbullying, we performed binary classification experiments using a linear kernel support vector machine (SVM) implemented in LIBLINEAR BIBREF57 by making use of Scikit-learn BIBREF58 , a machine learning library for Python. The motivation behind this is twofold: i) support vector machines (SVMs) have proven to work well for tasks similar to the ones under investigation BIBREF38 and ii) LIBLINEAR allows fast training of large-scale data which allow for a linear mapping (which was confirmed after a series of preliminary experiments using LIBSVM with linear, RBF and polynomial kernels).
The classifier was optimised for feature type (cf. Section SECREF38 ) and hyperparameter combinations (cf. Table TABREF37 ). Model selection was done using 10-fold cross validation in grid search over all possible feature types (i.e., groups of similar features, like different orders of INLINEFORM0 -gram bag-of-words features) and hyperparameter configurations. The best performing hyperparameters are selected by F INLINEFORM1 -score on the positive class. The winning model is then retrained on all held-in data and subsequently tested on a hold-out test set to assess whether the classifier is over- or under-fitting. The holdout represents a random sample ( INLINEFORM2 ) of all data. The folds were randomly stratified splits over the hold-in class distribution. Testing all feature type combinations is a rudimentary form of feature selection and provides insight into which types of features work best for this particular task.
Feature selection over all individual features was not performed because of the large feature space (NL: 795,072 and EN: 871,296 individual features). BIBREF59 , among other researchers, demonstrated the importance of joint optimisation, where feature selection and hyperparameter optimisation are performed simultaneously, since the techniques mutually influence each other.
The optimised models are evaluated against two baseline systems: i) an unoptimised linear-kernel SVM (configured with default parameter settings) based on word INLINEFORM0 -grams only and, ii) a keyword-based system that marks posts as positive for cyberbullying if they contain a word from existing vocabulary lists composed by aggressive language and profanity terms.
We apply cost-sensitive SVM as a possible hyperparameter in optimisation to counter this.
Answer: Cost-sensitive SVM is applied to enconter class imbalance.