## 2 Materials and Methods
### Metrics and Algorithms
#### 2.3.3 Sequential Neural Network  
In this approach, keras neural network models were created using a sequential architecture, and used similar processing functions as the prior algorithms to prepare the data. For word-level neural networks, the data were tokenized by jieba, stop words were removed from the text, and whitespace was inserted between the tokens. These inputs were incorporated into the neural network via a _keras_ TextVectorization layer, which used the TF-IDF method for feature selection. The TextVectorization layer was also used to examine character-based and word-based tokenization approaches. This was the first layer of the network and was fed into an embedding layer for dimension reduction. A rule of thumb suggested an embedding dimension approximately equal to the fourth root of the vocabulary [33]. This was calculated as 8 dimensions for the 5888-token vocabulary character-level models and 20 dimensions for the 167,506-token word-level models. For the word-level models, the maximum tokens were limited to 20,000 as it is rare that performance increases can be identified beyond that level [34] and to avoid memory crashes. To change the dimensions of the embedding layer in preparation for the Dense layers, a Flatten layer was used. A Global Average Pooling 1D layer was investigated as a substitute for the Flatten layer but did not alter performance.  
Stochastic gradient descent (SGD) and adaptive moment estimation (Adam) were investigated as optimizer options, and each Dense layer used the ReLU activation function for its neurons. Because each headline was selected into one of many categories, the sparse categorical cross-entropy loss function was used, which penalizes the model when it estimates the incorrect category [26]. To prevent overfitting, L2 regularization was used at a fixed value of 0.001, and a range of dropout levels was investigated. Overfitting is monitored through the accuracy of the training and validation datasets at each epoch of training\(-\)it is known that overfitting occurs when the accuracy of the validation set plateaus while the training set accuracy continues to increase.  
To aid in training, a callback was established to halve the learning rate every 15 epochs, and early stopping was used with a patience level of 10 epochs. The [PERSON] recommendation for the number of neurons is related to the number of data points P = 296,493 non-duplicate entries, the number of weights (neurons \(\times\) (inputs + 1)), and the desired error level according to Equation (1) below [35].  
\[P=\frac{neurons\;*\;(inputs+1)}{error} \tag{1}\]  
According to the [PERSON] recommendation, for the approximate 10% error level in this work, 30 embedded dimensions are used as inputs and 15 outputs, and the recommendation is a maximum of 63 neurons. In the hyperparameter tuning process, the neuron count was limited to within an order of magnitude of this recommendation.  
Within Python, the Adaptive Experimentation Platform (Ax) library was installed and used to tune the neural network model's hyperparameters. The training dataset was split into training and validation, and the hyperparameters were tuned using the validation dataset. After tuning, the final model performance was calculated using the test/holdout dataset. Bayesian optimization was used for numeric features, except for the categorical optimizer feature, where Bandit optimization was used. Further optimization information is given in Section 3.3.