Paper:
Random Feature Stein Discrepancies
Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational cost that grows quadratically in the sample size. While linear-time Stein discrepancies have been proposed for goodness-of-fit testing, they exhibit avoidable degradations in testing power-even when power is explicitly optimized. To address these shortcomings, we introduce feature Stein discrepancies ( SDs), a new family of quality measures that can be cheaply approximated using importance sampling. We show how to construct SDs that provably determine the convergence of a sample to its target and develop high-accuracy approximations-random SDs (R SDs)-which are computable in near-linear time. In our experiments with sampler selection for approximate posterior inference and goodness-of-fit testing, R SDs perform as well or better than quadratic-time KSDs while being orders of magnitude faster to compute.

Review:
The authors have addressed most of my concerns, and I maintain my overall evaluation.  ----------  Based on recent work in graph Stein discrepancy and kernel Stein discrepancy (KSD), this paper proposes feature Stein discrepancies (\PhiSDs) as a new family of discrepancy measures that can be efficiently approximated using importance sampling (termed R\PhiSDs) which can be computed in near-linear time. By finding suitable KSDs that are upper-bounded by \PhiSDs, the paper shows that \PhiSDs provably determine the convergence of a sample to its target. The application of R\PhiSD to goodness-of-fit testing is discussed, and its asymptotic distribution is derived. Experiments are conducted on sampler selection in approximate inference as well as goodness-of-fit testing.  Overall, the writing of the paper is fairly clear and the mathematical development quite rigorous, although at times it would help to provide some more intuition/motivation for the definitions/assumptions being introduced (such as the name “tilted base kernel” and the proposed hyperbolic secant kernel).  I believe that this work is a nice addition to the existing literature on Stein discrepancies and goodness-of-fit testing. In particular, it is interesting to see that \PhiSDs include KSD and the finite-set Stein discrepancy (FSSD) as special cases.  More specifically, I have the following specific questions/comments:  - Since it is not a commonly used kernel function, what is the intuition and motivation for considering the hyperbolic secant kernel? Is it merely for mathematical convenience for verifying the various assumptions or are there certain application scenarios in which it would arise as a natural choice?  - If one wishes to develop another instance of R\PhiSD, rather than checking all of Assumptions 3.1 through 3.6, is there a more concise but possibly slightly stronger set of conditions that could be more intuitive or easier to verify in practice and guide the choice of the various quantities in the definition of R\PhiSD?  - While the authors have given two examples of R\PhiSDs, is it possible to provide a general characterization of how the choice of r affects the properties of the resulting R\PhiSD?  Regarding the experiments:  - In Figure 3(b) and (c), the behavior of L1-IMQ is quite strange, do the authors have any intuition/conjecture behind its counterintuitive behavior as dimension increases? Is this behavior consistent as one increase the number of features used?  - For the computational complexity experiment, while the KSD test serves as a simple sanity check, I would be more interested in seeing how R\PhiSD compares with FSSD, since both should run in near-linear time, and it’s unclear which would be more efficient in practice.  - For the goodness-of-fit testing experiments, it would be interesting and (more informative) to perform experiments on more complicated distributions (in addition to the standard Gaussian), such as those with multiple modes (e.g., Gaussian mixture models). In particular, should one expect better performance of \RPhiSD over KSD and KSSD in the presence of multi-modality?