# Feature Drift in Fake News Detection:
## 4 Feature Drift
### Interpretable Analysis
Skewness is defined as the third standardized moment of a random variable \(X\), which can be calculated as follows:  
\[SK(X)=\mathbb{E}\Bigg{[}\bigg{(}\frac{X-\mu}{\sigma}\bigg{)}^{3}\Bigg{]} \tag{5}\]where \(\mu\) is the mean value of \(X\), \(\sigma\) is the standard deviation, and \(\mathbb{E}\) is the expectation operator. In order to control the distribution of the two datasets, we consider each event in the Twitter dataset and sample a skewness-matching event in the Gossipcop dataset. The tolerance error is \(\Delta=\pm 0.2\times 5K(X_{Twitter})\). If no event of the same skewness value exists in the Gossipcop dataset, the event in the Twitter dataset is not included in the subsampled dataset. After matching, we obtain two subsampled datasets from the Twitter dataset and the Gossipcop dataset with the same skewness value distribution. Figure 4: The distribution of _creation time_ in two datasets. Figure 3: Partial dependency plot for the Gossipcop dataset. (**a**) _Cut-off time_ is 2.4 h. (**b**) _Cut-off time_ is 12 h. The shaded area represents the confidence interval. To intuitively show the change in features in the sampled datasets and the original datasets, we calculate the mean (\(|\)SHAP value\(|\)) based on Random Forest on the sampled and the original datasets, where \(|*|\) denotes the absolute value, and the larger the mean (\(|\)SHAP value\(|\)) is, the more critical the corresponding features are. Figures 5 and 6 demonstrate the mean (\(|\)SHAP value\(|\)) changes with the subsampling of skewness. Comparing the _Characteristic time_ and _Network diameter_ (_Diameter_), the mean (\(|\)SHAP value\(|\)) on the subsampled Gossipcop dataset (Figure 5a,b) is larger than that on the original Gossipcop dataset (Figure 5c,d). This result shows that the importance of features does change after the matching method. Meanwhile, comparing most of the features (i.e., _Characteristic time_ (_Ctime_), _Average shortest path_ (_AvePath_), and _Tweet number in largest subgraph_ (_MaxS_)), the mean (\(|\)SHAP value\(|\)) does not change in the subsampled Gossipcop dataset (Figure 5a,b) while these features are different for the original Gossipcop dataset (Figure 5c,d), which indicates that the feature drift disappears in the subsampled Gossipcop dataset. Furthermore, as shown in Figure 6, the importance of the features does not change significantly between the subsampled and original datasets for the Twitter dataset, which implies that the difference in distribution may be the reason for the feature drift phenomenon. Overall, by controlling the distribution of tweets' _creation times_, we find that the phenomenon of feature drift disappears on Gossipcop, and the feature importance becomes closer between the two subsampled datasets, which implies that the difference in the distribution of the tweets' _creation time_ plays an important role in the feature drift phenomenon. Figure 5: Mean (\(|\)SHAP value\(|\)) on the subsampled and original Gossipcop datasets at different _cut-off times_. (**a,b**) On the subsampled Gossipcop dataset, the _cut-off time_ is 2.4 h and 12 h, respectively. (**c,d**) On the original Gossipcop dataset, the _cut-off time_ is 2.4 h and 12 h, respectively. Although SHAP and PDP could be used to interpret fake news detection methods and prove the existence of the feature drift phenomenon, they are global interpretable models that are costly when the data size is large. Fortunately, Anchors [55] provides a local interpretable perspective. Anchors is a local model-agnostic interpretability algorithm based on 'if-then' rules, which has high precision and clear coverage of the black-box model. The method of Anchors can be easily understood as follows: given an instance \(x\) to be explained, a rule (or an anchor) \(A\) that applies to \(x\) is to be found, while the prediction of \(x\)'s neighbors is the same as \(x\)'s under a high probability (which means 'precision' in Anchors). The definition of 'coverage' in Anchors represents the correct probability of \(x\)'s neighbors.