## 5 Experiments
### Results  
Table 2 presents the outcome of experimentally evaluating competing approaches. Since GRec and LibSeek follow the same evaluation methodology as we do, we pull evaluation results for these approaches from respective publications. We do not run the publicly available code of GRec and LibSeek to avoid biasing our comparison with lower-quality results arising from post-publication experimental probing by development teams. For instance, we failed to set up GRec's latest published code version to reach the same high evaluation scores as those reported by their paper and found architectural inconsistencies (including different types of layers and activations) between the paper and the code while investigating the issue. Thus, we decided to err on the side of caution and present the better reported values. Graph filters were implemented by building on the filter definition framework provided by the _pygrank_ Python package [19] and were run on its _numpy_ backend to \(10^{-12}\) mean absolute error numerical tolerance. (_pygrank_'s _numpy_ backend implements the graph shift operator by wrapping the C++ code for sparse matrix multiplication and runs faster than the respective operation provided by existing GPU computing frameworks. We ran experiments five times and reported measure averages across runs. Standard deviations are less than 0.007 for coverage and less than 0.002 for other recommendation quality measures and thus facilitate robust pairwise approach comparisons. An implementation of the symmetric absorbing random walk filter and the experiment methodology are publicly available online ([[https://github.com/maniospas/libFilter](https://github.com/maniospas/libFilter)]([https://github.com/maniospas/libFilter](https://github.com/maniospas/libFilter)) accessed on 2 March 2022). For all recommendation quality measures aside from MAP, there is a clear evaluation order where GRec is the best approach and is followed by LibFiter (our approach), where the latter lags behind by an at most 5-23% relative decrease that shrinks as more dependencies are omitted from the training graph. Although the two approaches do not always enjoy similar levels of recommendation quality, they can be considered roughly comparable when factoring in the much lower predictive quality of LibSeek and LibPPR. In fact, these last two approaches lag significantly behind, especially in terms of coverage, for which they exhibit near-half or less of GRec. Characteristically, LibFilter lies approximately mid-way between GRec and LibSeek in terms of evaluation measures. Furthermore, it outperforms the other two filter-based alternatives LibPPR and LibARW by a large and small margin, respectively, across all experiments. With regard to practical deployment, we ran graph filters (LibPPR, LibARW, LibFilter) in a machine with 2.6 GHz CPU base clock and 16 GB DDR3 RAM. This extracts library recommendation scores for each project approximately within a fifth of a second--and well within 0.1 second when LibFilter is deployed. By comparison, the out-of-the-box running of the publicly available implementation of GRec on the same machine's GPU with 1680 MHz base clock and 6 GB DDR6 graphics memory requires over 5.5 h for training alone (approximately 25 s per training epoch for 800 epochs); this would be the minimum recommendation delay in case of deployment as a query-able service. \begin{table}
\begin{tabular}{l l l l l} \hline
**Approach** & **Citation** & **Type** & **Training Time** & **Inference Time** \\ \hline LibSeek & [14] & Repr. learning & \(O\) (\(E\cdot dims\cdot layers\cdot epochs\)) & \(O\) (\(I\cdot dims\)) \\ GRec & [16] & Repr. learning & \(O\) (\(E\cdot dims\cdot epochs\)) & \(O\) (\(I\cdot dims\)) \\ LibPPR & [22], this work & Graph filter & — & \(O\) (\(-E\cdot\log\epsilon\)) \\ LibARW & [39], this work & Graph filter & — & \(O\) (\(-E\cdot\log\epsilon\)) \\ LibFilter & [this work] & Graph filter & — & \(O\) (\(-E\cdot\log\epsilon\)) \\ \hline \end{tabular}
\end{table}