# DIFshilling: A Diffusion Model for Shilling Attacks
## 5 Experiments
### Attack Performance (RQ1)  
Figure 3 presents heatmaps showing the overall attack performance of various shilling methods across victim RS models and datasets. Concrete values of HR@10 and NDCG@10 are provided in Tables 3 and 4 for the ML100K and Fashion datasets, respectively, while results for other datasets follow similar patterns. In Figure 3, each cell represents the HR@10 value for a specific attack method targeting a victim RS on a dataset, with lighter colors indicating higher HR@10 values. From the heatmaps, we observe that DIFshilling consistently achieves superior attack performance across all victim RS models and datasets. Examining the rows of the heatmap, it is evident that DIFshilling performs particularly well against DMF and GCMC on the three smaller datasets in the first row. Conversely, for the two larger datasets in the second row, DIFshilling notably increases the frequency of targeted items appearing in the recommendation lists for NeuMF and NGCF. The heatmaps also allow us to differentiate between heuristic-based and GAN-based shilling attack methods, with the dividing line being the row where DIFshilling is located. A comparative analysis reveals that GAN-based methods generally outperform heuristic-based methods against victim RS models. Furthermore, the color bands in the heatmaps highlight that shilling attacks are more effective on smaller and denser datasets, while their impact diminishes on larger and sparser datasets.  
From Tables 3 and 4, we note that shilling attack methods, including DIFshilling, have a more pronounced effect on smaller and denser datasets. On the ML100K dataset, DIFshilling consistently delivers the best attack performance on all victim RS models, as reflected by both HR@10 and NDCG@10. DIFshilling significantly increases the frequency of targeted items appearing in recommendation lists and improves their rankings. For example, when DGCF is the victim RS, DIFshilling increases the average appearance frequency of targeted items by a factor of 15 compared to other attack methods. Similarly, when GCMC is the victim RS, DIFshilling improves the rank of targeted items in the recommendation list by three positions compared to the best alternative shilling methods. Table 4 summarizes the experimental results on the larger and sparser Fashion dataset. DIFshilling maintains its superior performance on most victim RS models, outperforming other methods. However, it is worth noting that the impact of shilling attacks diminishes on large and sparse datasets. For instance, in some cases, such as when the victim RS is GCMC, and the attacker is AUSH, DIFshilling fails to improve the hit ratio of targeted items in recommendation lists.  
These results underscore the effectiveness of DIFshilling, particularly in small and dense datasets, while also highlighting the challenges of executing effective shilling attacks in large and sparse environments. More importantly, DIFshilling consistently outperforms existing shilling attack methods across diverse recommender system models and datasets, demonstrating its strong transferability. Unlike heuristic-based and GAN-based attack methods, which exhibit varying levels of effectiveness depending on the dataset and victim model, DIFshilling maintains high attack performance across different RS models and dataset characteristics. This suggests that DIFshilling is not only adaptable to different recommendation environments but also robust in achieving consistent attack success, further validating its transferability across various settings.  
Figure 3: Heatmaps showing attack results. The lighter a cell is, the larger the corresponding HR@10.