Paper: Fake News Detection via an Adaptive Feature Matching Optimization Framework (ICLR-2024)

Reviewer comment (Reproducibility):
Many details have been left out. For text features, do you take the embeddings corresponding to all positions (instead of the usual approach of taking just the first position) and, if so, why?

Author response:
Comment 1.1: For text features, the mean of embeddings from all segments of the text is used to represent the overall embedding of that text.

Reviewer comment (Reproducibility):
Many details have been left out. Can you describe the "array of fully connected layers" at the end of each feature extractor and that at the end of classifier?

Author response:
Comment 1.2: "Array of fully connected layers" refers to using multiple layers of fully connected layers to perform the final classification on the processed features.

Reviewer comment (Experiments):
Which multimodal Weibo dataset did you use? Jin et al., 2017 or Zhang et al., 2021a?

Author response:
Comment 2.2: The Weibo dataset used is from Jin et al., 2017.

Reviewer comment (Experiments):
Why do the statistics in Table 2 not match those in other papers (see Shu et al. 2018)? Is it showing the reduced data? Could this also explain the discrepancies between the results in Tables 4-5 and those in Hu et al., Deep learning for fake news detection: A comprehensive survey, AI Open, 2022?

Author response:
Comment 2.3: The statistical data in Table 2, except for Gossipcop, was obtained from open-source datasets by other authors. Gossipcop was crawled based on FakeNewsNet mentioned in the paper. FakeNewsNet is a web crawler on websites such as Twitter. The crawled content may be missing text or images due to reasons such as the news being banned by Twitter or the original post being deleted by the user or the image being expired. Therefore, after crawling, news data with empty text or images were removed, which may cause some differences in statistics from the dataset.

Reviewer comment (Experiments):
What is the performance gain of the feature selection step?

Author response:
Comment 3: The performance gain of feature selection is around 2%.

Reviewer comment (Novelty):
The advantages and innovativeness of the proposed method appear to be somewhat limited.

Author response:
Comment 1: The use of annealing algorithms can eliminate the mutual interference generated during multimodal feature fusion.

Reviewer comment (Theory):
The proposed model seeks to minimize potential interference between textual and visual information, but whether the feature selection through simulated annealing algorithm actually achieves the above-mentioned purpose seems to lack a detailed mechanism explanation.

Author response:
Comment 1: The use of annealing algorithms can eliminate the mutual interference generated during multimodal feature fusion.

Reviewer comment (Experiments):
The example in the 4.4.2 case study seems to be able to identify correctly using some methods based on capturing the similarity or ambiguity of images and texts (e.g. Cross-modal Ambiguity Learning for Multimodal Fake News Detection). I don’t know if there is a similar model in the baseline selected by the authors. This example does not seem to explain the advantages of the simulated annealing algorithm very well.

Author response:
Comment 4: The baseline does not include a model for capturing image and text similarity.

Reviewer comment (Reproducibility):
The paper does not provide enough details about the outlier removal step, such as how to choose the threshold for Mahalanobis distance and the number of neighbors for KNN.

Author response:
Comment 1: Provide the threshold value of Mahalanobis distance (0.08) and the number of domains for KNN (domain number k=3). Currently, the threshold value is set manually, such as 0.08 in the experiment. Different threshold values will result in different abnormal features being removed, which will affect the performance of the final retrained model. Bayesian parameter tuning can be used to find the optimal threshold ratio, but the time cost will increase exponentially.

Reviewer comment (Experiments):
The baseline compared in this paper is relatively weak. As far as I know, there are many more advanced multi-modal fake news detection works.

Author response:
Comment 2: Baseline selected the most advanced CARMN (2021) and TRIMOON (2023) in recent years. In the TRIMOON paper, it was mentioned that it was already the most advanced model, so no new baselines were added.

Reviewer comment (Experiments):
Though CARMN leveraging the attention to fuse the multi-modal features is used as a baseline, the experimental results of another fake news detection method based on the co-attention MCAN are also expected.

Author response:
Comment 4: MCAN's performance on Weibo is 0.899. In addition, it was only tested on mediaeval2016, which has a total of 14,480 data items, but only 512 of them have images. We believe that this dataset does not well reflect the multimodal task scenario.

Reviewer comment (Experiments):
Why does Table 5 miss the results of MKN? MKN serves as one of the baselines on the Weibo dataset in Table 4, but misses on the Gossipcop Dataset in Table 5. The performance of MKN on the Gossipcop Dataset is also expected.

Author response:
Comment 5: MKN does not disclose its source code, so it does not provide results on the gossipcop dataset. Only the data from MKN's original paper on the weibo dataset is used.