text: 340,0,1404497507774865409,1014263782493818880,Pierre Ablin,"New paper: Kernel Stein Discrepancy Descent ! A method to sample from unnormalized densities by optimization of the Kernel Stein Discrepancy (KSD)🎯🎯🎯 Paper: <LINK> With @Korba_Anna, P-C Aubin & @sjm_majewski  Accepted for a long talk @icmlconf 🍾 1/8🧵👇 <LINK> Kernel Stein Discrepancy is a measure of distances between densities. It is a Maximum Mean Discrepancy (MMD) for a special Kernel, the Stein Kernel. 2/8 <LINK> The Stein Kernel only involves the score s (derivative of the log) of the target distribution. We can therefore compute the KSD in the formula above using only the score of the target: unlike most MMD's, we do not need samples. 3/8 <LINK> As a consequence, it is easy to evaluate the KSD between the target and a discrete measure of particles. In order to sample from the target, we can simply optimize this cost function w.r.t. the positions of the particles 4/8 <LINK> We then have a simple objective function of the positions of the samples, which can then be minimized using (stochastic) gradient descent or the fast and robust algorithm L-BFGS. Here, particles reach an equilibrium on a simple Gaussian problem 5/8 <LINK> Yet, we also uncover bad behaviors when the problem is not log-concave : some particles tend to get stuck in spurious local minima, for instance here on a mixture of Gaussian with low variances 6/8 <LINK> We give a mathematical analysis for these phenomena, and also show surprising results like lack of exponential convergence. 7/8 If you want to try it yourself, we have made a python package: pip install ksddescent Site: <LINK> Source: <LINK> It also features pytorch/numpy code for the awesome SVGD algorithm (<LINK>) 8/8 @umutsimsekli @Korba_Anna @sjm_majewski @icmlconf Of course it's here !  For Bayesian ICA, KSD descent does not really work, because it is a hard non convex problem with many spurious local minima <LINK>",https://arxiv.org/abs/2105.09994,"Among dissimilarities between probability distributions, the Kernel Stein Discrepancy (KSD) has received much interest recently. We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution $\pi$ on $\mathbb{R}^d$, known up to a normalization constant. This leads to a straightforwardly implementable, deterministic score-based method to sample from $\pi$, named KSD Descent, which uses a set of particles to approximate $\pi$. Remarkably, owing to a tractable loss function, KSD Descent can leverage robust parameter-free optimization schemes such as L-BFGS; this contrasts with other popular particle-based schemes such as the Stein Variational Gradient Descent algorithm. We study the convergence properties of KSD Descent and demonstrate its practical relevance. However, we also highlight failure cases by showing that the algorithm can get stuck in spurious local minima. ",Kernel Stein Discrepancy Descent,9,"['New paper: Kernel Stein Discrepancy Descent !\n\nA method to sample from unnormalized densities by optimization of the Kernel Stein Discrepancy (KSD)🎯🎯🎯\n\nPaper: <LINK>\n\nWith @Korba_Anna, P-C Aubin &amp; @sjm_majewski \n\nAccepted for a long talk @icmlconf 🍾\n\n1/8🧵👇 <LINK>', 'Kernel Stein Discrepancy is a measure of distances between densities. It is a Maximum Mean Discrepancy (MMD) for a special Kernel, the Stein Kernel.\n\n2/8 https://t.co/B9AZGpafys', ""The Stein Kernel only involves the score s (derivative of the log) of the target distribution.\n\nWe can therefore compute the KSD in the formula above using only the score of the target: unlike most MMD's, we do not need samples.\n\n3/8 https://t.co/4tYJfGAk5v"", 'As a consequence, it is easy to evaluate the KSD between the target and a discrete measure of particles. In order to sample from the target, we can simply optimize this cost function w.r.t. the positions of the particles\n\n4/8 https://t.co/DkeDjpP5ok', 'We then have a simple objective function of the positions of the samples, which can then be minimized using (stochastic) gradient descent or the fast and robust algorithm L-BFGS.\n\nHere, particles reach an equilibrium on a simple Gaussian problem\n\n5/8 https://t.co/ZUEIF7ShxS', 'Yet, we also uncover bad behaviors when the problem is not log-concave : some particles tend to get stuck in spurious local minima, for instance here on a mixture of Gaussian with low variances\n\n6/8 https://t.co/7MMd8DsQvc', 'We give a mathematical analysis for these phenomena, and also show surprising results like lack of exponential convergence.\n\n7/8', 'If you want to try it yourself, we have made a python package:\n\npip install ksddescent\n\nSite: https://t.co/ckGZtfGYWN\nSource: https://t.co/rrrwBE67Ew\n\nIt also features pytorch/numpy code for the awesome SVGD algorithm (https://t.co/L9mJFciENx)\n\n8/8', ""@umutsimsekli @Korba_Anna @sjm_majewski @icmlconf Of course it's here ! \n\nFor Bayesian ICA, KSD descent does not really work, because it is a hard non convex problem with many spurious local minima https://t.co/BB5nfDdJ5l""]",21,05,1876