<table><caption>Table 5: Results for bi-modal Gaussian mixtures with **diagonal** covariance and **medium** conditioning, averaged over 16 sampling runs (same setting as Table 2). Bold font indicates best result, **orange** cells refer to settings with uninformative mode weight estimation (i.e., uniform mixture), **red** cells denote mode collapse. N/A denotes settings where results could not be obtained due to numerical issues. The MMD and KS results are displayed with 100-factor rescaling.</caption><thead><tr><th rowspan="2">Algorithm</th><th colspan="3">d = 16</th><th colspan="3">d = 32</th><th colspan="3">d = 64</th></tr><tr><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th></tr></thead><tbody><tr><td>SMC</td><td>11.4%±1.9%</td><td>0.38±0.00</td><td>13.91±10.68</td><td>10.80±7.84</td><td>15.8%±8.5%</td><td>0.70±0.01</td><td>18.68±9.67</td><td>14.63±7.59</td><td>15.2%±7.5%</td><td>1.16±0.01</td><td>17.87±8.72</td><td>14.14±6.71</td></tr><tr><td>RE</td><td>16.5%±1.3%</td><td>0.38±0.00</td><td>20.57±1.78</td><td>15.20±1.31</td><td>15.9%±1.4%</td><td>0.70±0.00</td><td>19.23±2.03</td><td>14.54±1.55</td><td>17.0%±1.4%</td><td>1.17±0.00</td><td>20.36±1.83</td><td>15.68±1.39</td></tr><tr><td>LV-PIS</td><td>6.0%±3.4%</td><td>0.43±0.00</td><td>7.63±3.07</td><td>6.86±2.49</td><td>33.2%±6.1%</td><td>1.01±0.03</td><td>34.90±0.58</td><td>28.43±1.52</td><td>33.0%±0.1%</td><td>1.62±0.02</td><td>34.03±0.71</td><td>29.33±0.75</td></tr><tr><td>LV-DDS</td><td>11.8%±9.3%</td><td>0.40±0.00</td><td>13.83±10.23</td><td>11.29±8.06</td><td>31.5%±3.9%</td><td>0.86±0.04</td><td>33.96±1.29</td><td>28.00±2.65</td><td>33.1%±0.1%</td><td>1.49±0.02</td><td>34.45±0.61</td><td>28.95±0.89</td></tr><tr><td>LV-DIS</td><td>16.4%±10.5%</td><td>0.42±0.00</td><td>21.36±10.75</td><td>16.16±8.54</td><td>16.5%±4.0%</td><td>0.77±0.00</td><td>21.13±0.96</td><td>16.18±8.76</td><td>16.8%±5.0%</td><td>1.24±0.01</td><td>20.71±0.81</td><td>15.93±0.56</td></tr><tr><td>LV-CMCD</td><td>32.2%±14.9%</td><td>0.60±0.00</td><td>46.89±17.35</td><td>33.46±11.01</td><td>50.1%±8.8%</td><td>1.96±0.52</td><td>76.17±1.71</td><td>48.79±8.03</td><td>16.3%±10.6%</td><td>3.47±0.31</td><td>53.15±1.85</td><td>41.79±2.64</td></tr><tr><td>iDEM</td><td>33.7%±6.0%</td><td>1.75±0.00</td><td>53.82±10.75</td><td>36.86±9.92</td><td>66.7%±6.0%</td><td>4.16±0.22</td><td>45.52±1.53</td><td>61.10±1.26</td><td>11.7%±10.4%</td><td>117.82±0.14</td><td>90.13±0.12</td><td>N/A</td></tr><tr><td>PDDS</td><td>0.8%±0.6%</td><td>0.40±0.00</td><td>1.66±0.68</td><td>2.59±0.29</td><td>66.7%±6.0%</td><td>11.22±0.08</td><td>105.11±0.24</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td></tr><tr><td>GMM-LRDS</td><td>1.7%±0.6%</td><td>0.38±0.00</td><td>2.58±0.96</td><td>2.69±0.66</td><td>2.7%±0.8%</td><td>0.71±0.00</td><td>3.64±1.07</td><td>3.38±0.76</td><td>4.1%±0.6%</td><td>1.19±0.00</td><td>5.25±0.98</td><td>4.50±0.70</td></tr></tbody></table>

<table><caption>Table 6: Results for bi-modal Gaussian mixtures with **diagonal** covariance and **isotropic** conditioning, averaged over 16 sampling runs. Bold font indicates best result, **orange** cells refer to settings with uninformative mode weight estimation (i.e., uniform mixture), **red** cells denote mode collapse. N/A denotes settings where results could not be obtained due to numerical issues. The MMD and KS results are displayed with 100-factor rescaling.</caption><thead><tr><th rowspan="2">Algorithm</th><th colspan="3">d = 16</th><th colspan="3">d = 32</th><th colspan="3">d = 64</th></tr><tr><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th></tr></thead><tbody><tr><td>SMC</td><td>16.0%±10.1%</td><td>0.65±0.01</td><td>19.11±12.55</td><td>14.06±8.74</td><td>12.3%±8.6%</td><td>1.18±0.01</td><td>14.57±11.16</td><td>11.03±8.03</td><td>11.3%±8.8%</td><td>1.94±0.00</td><td>12.98±10.51</td><td>9.80±7.38</td></tr><tr><td>RE</td><td>15.2%±1.2%</td><td>0.66±0.00</td><td>18.72±1.66</td><td>13.42±2.13</td><td>16.1%±1.3%</td><td>1.19±0.00</td><td>19.50±11.70</td><td>14.11±1.26</td><td>16.5%±1.3%</td><td>1.94±0.00</td><td>19.46±1.71</td><td>14.39±1.24</td></tr><tr><td>LV-PIS</td><td>1.9%±1.2%</td><td>0.66±0.00</td><td>2.66±1.45</td><td>2.63±0.89</td><td>1.3%±0.6%</td><td>1.20±0.00</td><td>2.15±0.68</td><td>2.23±0.42</td><td>2.8%±0.6%</td><td>1.98±0.00</td><td>2.94±0.90</td><td>2.81±0.61</td></tr><tr><td>LV-DDS</td><td>0.7%±10.5%</td><td>0.65±0.00</td><td>0.86±0.70</td><td>1.59±0.30</td><td>0.87±0.5%</td><td>1.19±0.00</td><td>1.09±0.60</td><td>1.56±0.24</td><td>1.6%±0.8%</td><td>1.95±0.00</td><td>2.13±0.97</td><td>2.23±0.61</td></tr><tr><td>LV-DIS</td><td>16.4%±1.3%</td><td>0.66±0.00</td><td>19.99±1.73</td><td>14.16±1.21</td><td>16.7%±6.7%</td><td>1.20±0.00</td><td>20.12±1.00</td><td>14.51±0.70</td><td>16.6%±0.6%</td><td>1.96±0.00</td><td>19.79±0.90</td><td>14.59±0.65</td></tr><tr><td>LV-CMCD</td><td>0.90%±0.60%</td><td>0.68±0.00</td><td>2.84±0.55</td><td>2.92±0.45</td><td>1.7%±0.7%</td><td>1.24±0.00</td><td>2.77±0.36</td><td>2.76±0.36</td><td>1.1%±0.8%</td><td>2.05±0.00</td><td>3.43±0.75</td><td>3.15±0.65</td></tr><tr><td>iDEM</td><td>66.7%±0.6%</td><td>2.23±0.17</td><td>83.92±0.50</td><td>59.68±2.16</td><td>33.3%±0.6%</td><td>2.48±0.15</td><td>49.83±0.35</td><td>35.02±0.62</td><td>0.3%±0.2%</td><td>179.86±0.12</td><td>97.17±1.05</td><td>N/A</td></tr><tr><td>PDDS</td><td>0.9%±0.6%</td><td>0.70±0.00</td><td>1.27±1.25</td><td>2.50±0.49</td><td>0.9%±0.6%</td><td>1.22±0.00</td><td>1.53±0.30</td><td>2.41±0.31</td><td>0.0%±0.7%</td><td>1.96±0.00</td><td>0.85±1.25</td><td>2.05±0.56</td></tr><tr><td>GMM-LRDS</td><td>1.6%±0.6%</td><td>0.66±0.00</td><td>2.07±0.99</td><td>2.23±0.54</td><td>2.8%±0.5%</td><td>1.19±0.00</td><td>3.72±0.89</td><td>3.23±0.58</td><td>4.6%±0.6%</td><td>1.96±0.00</td><td>5.65±0.88</td><td>4.56±0.61</td></tr></tbody></table>

<table><caption>Table 7: Results for bi-modal Gaussian mixtures with **diagonal** covariance and **hard** conditioning, averaged over 16 sampling runs. Bold font indicates best result, **orange** cells refer to settings with uninformative mode weight estimation (i.e., uniform mixture), **red** cells denote mode collapse. N/A denotes settings where results could not be obtained due to numerical issues. The MMD and KS results are displayed with 100-factor rescaling.</caption><thead><tr><th rowspan="2">Algorithm</th><th colspan="3">d = 16</th><th colspan="3">d = 32</th><th colspan="3">d = 64</th></tr><tr><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th></tr></thead><tbody><tr><td>SMC</td><td>12.4%±3%</td><td>0.24±0.00</td><td>15.40±10.16</td><td>12.00±7.57</td><td>16.5%±9.5%</td><td>0.47±0.00</td><td>20.22±11.66</td><td>15.71±8.58</td><td>11.4%±9.8%</td><td>0.80±0.00</td><td>13.88±11.92</td><td>11.16±8.79</td></tr><tr><td>RE</td><td>16.3%±1.4%</td><td>0.25±0.00</td><td>20.76±10.95</td><td>15.40±1.38</td><td>17.0%±1.7%</td><td>0.17±0.00</td><td>21.31±1.19</td><td>16.11±1.58</td><td>16.4%±1.4%</td><td>0.81±0.00</td><td>19.05±1.75</td><td>15.54±1.38</td></tr><tr><td>LV-PIS</td><td>8.4%±1.4%</td><td>0.45±0.00</td><td>13.52±2.50</td><td>12.48±2.01</td><td>32.2%±3.4%</td><td>1.29±0.02</td><td>33.89±0.60</td><td>29.92±5.57</td><td>32.5%±2.8%</td><td>2.24±0.04</td><td>33.91±0.76</td><td>30.49±0.61</td></tr><tr><td>LV-DDS</td><td>24.7%±8.8%</td><td>0.32±0.03</td><td>26.89±1.93</td><td>22.92±7.59</td><td>40.5%±13.9%</td><td>0.73±0.02</td><td>44.97±19.24</td><td>37.20±12.88</td><td>38.1%±15.4%</td><td>1.59±0.04</td><td>42.64±20.91</td><td>35.46±13.67</td></tr><tr><td>LV-DIS</td><td>16.6%±10.6%</td><td>0.38±0.02</td><td>23.00±0.87</td><td>19.69±0.69</td><td>16.7%±0.7%</td><td>0.65±0.03</td><td>23.01±1.12</td><td>19.33±0.99</td><td>16.8%±0.6%</td><td>1.06±0.02</td><td>22.94±0.84</td><td>18.97±0.65</td></tr><tr><td>LV-CMCD</td><td>21.5%±17.8%</td><td>4.37±1.19</td><td>62.31±14.76</td><td>N/A</td><td>31.5%±2.2%</td><td>1.54±0.35</td><td>46.02±12.29</td><td>37.50±13.10</td><td>44.1%±15.3%</td><td>0.87±0.03</td><td>51.17±10.36</td><td>41.30±14.11</td></tr><tr><td>iDEM</td><td>33.3%±9.0%</td><td>2.12±0.11</td><td>51.43±0.83</td><td>38.07±1.02</td><td>66.7%±0.6%</td><td>7.53±0.52</td><td>85.77±0.64</td><td>55.56±1.01</td><td>33.3%±0.6%</td><td>175.16±0.83</td><td>102.13±0.29</td><td>N/A</td></tr><tr><td>PDDS</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td></tr><tr><td>GMM-LRDS</td><td>2.1%±1.0%</td><td>0.25±0.00</td><td>3.41±1.26</td><td>3.46±0.90</td><td>1.7%±0.9%</td><td>0.47±0.00</td><td>3.00±1.12</td><td>2.99±0.80</td><td>4.5%±1.8%</td><td>0.82±0.00</td><td>6.12±2.21</td><td>5.25±1.62</td></tr></tbody></table>

• Table 5 corresponds to Gaussian mixtures with diagonal covariance and medium conditioning, completing the results of Table 2 presented in the main paper,

• Table 6 corresponds to Gaussian mixtures with diagonal covariance and isotropic conditioning,

• Table 7 corresponds to Gaussian mixtures with diagonal covariance and hard conditioning,

• Table 8 corresponds to Gaussian mixtures with full covariance and medium conditioning,

• Table 9 corresponds to Gaussian mixtures with full covariance and hard conditioning.

For all of these bi-modal settings, we observe that the mode weight estimation error is consistent with the values of probability metrics (except for the regularized Wasserstein distance, which is not discriminative between the methods). In particular, GMM-LRDS is on par or superior to all competitors in each setting, except the least challenging setting ('Isotropic' conditioning, see Table 6), where LV-DDS is more performant.
<table><caption>Table 8: Results for bi-modal Gaussian mixtures with **full** covariance and **medium** conditioning, averaged over 16 sampling runs. Bold font indicates best result, **orange** cells refer to settings with uninformative mode weight estimation (i.e., uniform mixture), **red** cells denote mode collapse. N/A denotes settings where results could not be obtained due to numerical issues. The MMD and KS results are displayed with 100-factor rescaling.</caption><thead><tr><th rowspan="2">Algorithm</th><th colspan="4">d = 8</th><th colspan="4">d = 16</th><th colspan="4">d = 32</th></tr><tr><th>Mode Err. ↓</th><th>W<sub>2,x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2,x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2,x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th></tr></thead><tbody><tr><td>SMC</td><td>17.3%±7.1%</td><td>0.17±0.00</td><td>21.55±8.84</td><td>16.07±6.33</td><td>13.5%±4.5%</td><td>0.38±0.00</td><td>16.35±5.47</td><td>12.52±4.05</td><td>10.3%±5.8%</td><td>0.70±0.00</td><td>12.18±6.67</td><td>9.72±1.15</td></tr><tr><td>RE</td><td>16.6%±1.0%</td><td>0.17±0.00</td><td>21.34±1.54</td><td>15.30±1.11</td><td>16.4%±1.4%</td><td>0.38±0.00</td><td>20.65±1.90</td><td>15.23±1.34</td><td>16.3%±1.2%</td><td>0.70±0.00</td><td>19.90±1.74</td><td>15.04±1.24</td></tr><tr><td>LV-PIS</td><td>6.8%±2.4%</td><td>0.20±0.00</td><td>9.89±2.78</td><td>8.67±1.98</td><td>25.4%±2.6%</td><td>0.52±0.06</td><td>28.21±1.13</td><td>23.37±10.61</td><td>33.1%±2%</td><td>1.07±0.39</td><td>35.02±0.70</td><td>27.85±3.09</td></tr><tr><td>LV-DDS</td><td>1.6%±0.8%</td><td>0.17±0.00</td><td><b>2.64±0.87</b></td><td><b>2.63±0.53</b></td><td>3.3%±2.0%</td><td>0.39±0.00</td><td>4.83±1.08</td><td>4.27±0.75</td><td>4.0%±1.1%</td><td>0.75±0.00</td><td>6.85±1.23</td><td>5.95±1.01</td></tr><tr><td>LV-DIS</td><td>16.5%±0.9%</td><td>0.22±0.00</td><td>22.50±0.88</td><td>17.33±0.66</td><td>16.3%±1.1%</td><td>0.46±0.02</td><td>21.65±1.55</td><td>16.93±0.13</td><td>16.7%±0.6%</td><td>0.82±0.01</td><td>21.95±0.94</td><td>17.16±0.72</td></tr><tr><td>LV-CMCD</td><td>16.6%±0.5%</td><td>0.18±0.00</td><td>21.76±0.01</td><td>16.06±0.81</td><td>20.8%±1.2%</td><td>1.23±0.96</td><td>36.92±5.06</td><td>26.70±4.59</td><td>16.6%±1.0%</td><td>0.75±0.00</td><td>20.89±0.95</td><td>15.90±1.07</td></tr><tr><td>iDEM</td><td>66.7%±0.9%</td><td>2.06±0.19</td><td>85.27±0.40</td><td>61.42±0.09</td><td>66.7%±0.9%</td><td>3.95±0.20</td><td>84.18±0.53</td><td>62.98±1.19</td><td>66.7%±0.9%</td><td>6.89±0.24</td><td>85.60±0.41</td><td>62.87±1.31</td></tr><tr><td>PDDS</td><td>1.3%±1.1%</td><td>0.19±0.00</td><td><b>1.52±1.58</b></td><td><b>2.65±0.77</b></td><td>1.9%±0.9%</td><td>0.40±0.00</td><td><b>2.63±1.53</b></td><td><b>3.09±0.77</b></td><td>0.7%±0.6%</td><td>0.73±0.00</td><td><b>1.56±0.80</b></td><td><b>2.61±0.37</b></td></tr><tr><td>GMM-LRDS</td><td>1.3%±0.6%</td><td>0.17±0.00</td><td><b>1.97±0.85</b></td><td><b>2.36±0.52</b></td><td>1.8%±0.5%</td><td>0.38±0.00</td><td><b>2.65±0.87</b></td><td><b>2.72±0.58</b></td><td>2.4%±0.5%</td><td>0.71±0.00</td><td>3.48±0.89</td><td>3.24±0.61</td></tr></tbody></table>

<table><caption>Table 9: Results for bi-modal Gaussian mixtures with **full** covariance and **hard** conditioning, averaged over 16 sampling runs. Bold font indicates best result, **orange** cells refer to settings with uninformative mode weight estimation (i.e., uniform mixture), **red** cells denote mode collapse. N/A denotes settings where results could not be obtained due to numerical issues. The MMD and KS results are displayed with 100-factor rescaling.</caption><thead><tr><th rowspan="2">Algorithm</th><th colspan="4">d = 8</th><th colspan="4">d = 16</th><th colspan="4">d = 32</th></tr><tr><th>Mode Err. ↓</th><th>W<sub>2,x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2,x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2,x</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th></tr></thead><tbody><tr><td>SMC</td><td>14.2%±15.0%</td><td>0.10±0.00</td><td>17.83±19.13</td><td>13.83±11.58</td><td>10.1%±6.9%</td><td>0.24±0.00</td><td>12.91±8.77</td><td>10.03±6.21</td><td>17.9%±10.2%</td><td>0.47±0.00</td><td>21.61±12.58</td><td>16.96±9.50</td></tr><tr><td>RE</td><td>16.1%±1.2%</td><td>0.11±0.00</td><td>20.99±1.86</td><td>15.12±1.33</td><td>16.8%±1.2%</td><td>0.25±0.00</td><td>21.54±1.76</td><td>15.89±1.22</td><td>16.3%±1.3%</td><td>0.47±0.00</td><td>20.28±1.79</td><td>15.39±1.32</td></tr><tr><td>LV-PIS</td><td>10.9%±1.1%</td><td>0.20±0.01</td><td>15.75±1.22</td><td>14.13±0.83</td><td>2.2%±1.5%</td><td>0.43±0.00</td><td>10.36±0.80</td><td>10.50±0.70</td><td>32.7%±0.3%</td><td>1.34±0.03</td><td>34.25±0.75</td><td>30.15±0.78</td></tr><tr><td>LV-DDS</td><td>3.2%±1.8%</td><td>0.12±0.01</td><td>5.53±1.63</td><td>5.36±1.05</td><td>7.5%±4.1%</td><td>0.29±0.00</td><td>11.39±4.88</td><td>9.91±3.53</td><td>10.5%±7.8%</td><td>0.62±0.00</td><td>16.02±0.81</td><td>14.10±6.05</td></tr><tr><td>LV-DIS</td><td>16.9%±0.5%</td><td>0.28±0.03</td><td>24.00±0.90</td><td>21.85±0.90</td><td>16.6%±0.6%</td><td>0.51±0.03</td><td>23.85±0.96</td><td>22.1±0.77</td><td>16.8%±0.6%</td><td>0.91±0.07</td><td>24.34±0.91</td><td>22.76±0.77</td></tr><tr><td>LV-CMCD</td><td>16.6%±0.5%</td><td>1.98±0.02</td><td>18.76±0.56</td><td>34.42±1.16</td><td>16.7%±0.6%</td><td>3.67±0.02</td><td>19.86±0.52</td><td>36.03±0.10</td><td>16.7%±0.5%</td><td>6.05±0.02</td><td>22.65±0.42</td><td>35.88±0.99</td></tr><tr><td>iDEM</td><td>66.7%±0.9%</td><td>0.23±0.00</td><td>84.95±0.61</td><td>60.96±1.39</td><td>66.7%±0.9%</td><td>5.49±0.12</td><td>86.18±0.47</td><td>59.71±1.59</td><td>66.7%±0.9%</td><td>10.33±0.09</td><td>81.57±0.65</td><td>53.73±1.36</td></tr><tr><td>PDDS</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td></tr><tr><td>GMM-LRDS</td><td>1.8%±0.7%</td><td>0.11±0.00</td><td><b>2.85±1.00</b></td><td><b>3.22±0.67</b></td><td>1.0%±0.7%</td><td>0.25±0.00</td><td><b>2.22±0.91</b></td><td><b>2.60±0.62</b></td><td>2.2%±1.3%</td><td>0.47±0.00</td><td><b>3.19±1.63</b></td><td><b>3.13±1.17</b></td></tr></tbody></table>

**Execution time.** To demonstrate the applicability of LRDS in practice, we provide in Figure 9 the execution clock-time of each sampling method for several Gaussian mixture sampling settings. The computations were all ran on the same V100 GPU. We notice that (i) the initial MCMC sampling step to build the reference dataset, then used to initialize each sampler, and the reference fitting time from GMM-LRDS are completely negligible compared to the training time, and that (ii) the training time of GMM-LRDS is on par with previous variational diffusion-based approaches.

<figure><img src="image_11.png" /><figcaption>Figure 9: Execution clock-time for bi-modal Gaussian mixtures with **diagonal** covariance (left) and **full** covariance (right), averaged over the conditioning settings.</figcaption></figure>

**Ablation study on GMM-LRDS: effect of the reference distribution.** To assess the robustness of GMM-LRDS with respect to the setting of the reference distribution, we conduct an ablation study that reveals the impact of modifying the location and the entropy of the reference modes. To do so, we set $d = 8$ and define the target distribution $\pi : x \in \mathbb{R}^d \mapsto (2/3)N(x; -1_d, 0.05) + (1/3)N(x; 1_d, 0.05)$, as detailed in Appendix H.1. Instead of defining the reference distribution in GMM-LRDS as a Gaussian mixture fitted on MCMC samples, we propose to consider a flexible reference distribution given by the Gaussian mixture $x \in \mathbb{R}^d \mapsto (2/3)N(x; \mathbf{m}_1, 0.05\alpha^2 I_d) + (1/3)N(x; \mathbf{m}_2, 0.05\alpha^2 I_d)$ where $(\mathbf{m}_1, \mathbf{m}_2) \in \mathbb{R}^d \times \mathbb{R}^d$ and $\alpha > 0$.
<figure><img src="image_12.png" /><figcaption>Figure 10: Results of GMM-LRDS for an 8-dimensional bi-modal Gaussian mixture when varying the location of the reference modes: the performance degrades as soon as the reference modes and the target modes are further from each other.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure 11: Results of GMM-LRDS for an 8-dimensional bi-modal Gaussian mixture when varying the variance of the reference modes: except for small reference mode variance, the performance is unchanged.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure 12: Results of GMM-LRDS for an 8-dimensional bi-modal Gaussian mixture when varying the distance between the modes of the target distribution: the performance does not significantly degrades, despite the increasing sampling complexity.</figcaption></figure>

To observe the impact of the location of the reference modes on GMM-LRDS, we take $\alpha = 1$ (same variance setting as the target) and $(\mathbf{m}_1, \mathbf{m}_2) \sim N(-\mathbf{1}_d, \sigma_m^2 \mathbf{I}_d) \otimes N(\mathbf{1}_d, \sigma_m^2 \mathbf{I}_d)$ where $\sigma_m \in \{0, 0.25, 0.5, 0.75, 1.0\}$ is a mean perturbation parameter. Hence, the higher $\sigma_m$, the further the reference modes might be with respect to the target modes. We display in Figure 10 the results of GMM-LRDS for this setting, averaged over 16 sampling runs. This ablation study notably reveals the major need of precision on the location of the target modes (in practice, brought by MCMC sampling) to build the reference distribution.

On the other hand, to understand the effect of the entropy of the reference modes, we take $(\mathbf{m}_1, \mathbf{m}_2) = (-\mathbf{1}_d, \mathbf{1}_d)$ (same mean setting as the target), and $\alpha \in \{0.5, 0.75, 1, 1.5, 2\}$, which can be seen as a variance perturbation parameter. We display in Figure 11 the results of GMM-LRDS for this setting, averaged over 16 sampling runs. Interestingly, our ablation study demonstrates that GMM-LRDS works well as soon as the support of the reference distribution includes the support of the target distribution. In practice, this is verified by MCMC sampling.
<table><caption>Table 10: Results of GMM-LRDS for bi-modal Gaussian mixtures with **diagonal** covariance and **various conditioning** settings ($d = 128$), averaged over 16 sampling runs. The MMD and KS results are displayed with 100-factor rescaling.</caption><thead><tr><th colspan="4">Isotropic conditioning</th><th colspan="4">Medium conditioning</th><th colspan="4">Hard conditioning</th></tr><tr><th>Mode Err. ↓</th><th>W<sub>2,ε</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2,ε</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th><th>Mode Err. ↓</th><th>W<sub>2,ε</sub> ↓</th><th>MMD ↓</th><th>KS ↓</th></tr></thead><tbody><tr><td>6.6%±0.6%</td><td>3.04±0.00</td><td>7.84±0.92</td><td>6.16±0.67</td><td>6.8%±0.9%</td><td>1.86±0.00</td><td>8.33±1.29</td><td>6.81±0.98</td><td>6.2%±3.6%</td><td>1.32±0.00</td><td>8.09±3.83</td><td>6.73±2.92</td></tr></tbody></table>

**Ablation study on GMM-LRDS: effect of the distance between the target modes.** To assess the robustness of GMM-LRDS with respect to the complexity of the target distribution, we conduct a second ablation study that illustrates the behaviour of GMM-LRDS for target with higher energy barrier. To do so, we set $d = 8$ and define the target distribution $\pi: x \in \mathbb{R}^d \mapsto (2/3)\mathbb{N}(x; -a\mathbf{1}_d, 0.05) + (1/3)\mathbb{N}(x; a\mathbf{1}_d, 0.05)$, where $a \in \{1, 2, 4, 8, 16\}$ indicates the complexity level of the target. For each target, we conduct 16 sampling runs of GMM-LRDS, and display the results in Figure 12. Our numerics show that the performance GMM-LRDS remains significantly consistent with increasing $a$, proving the interest of our method for complex multi-modal targets.

**Limitation of LRDS: high dimension.** Although LRDS outperforms its competitors when sampling from challenging bi-modal Gaussian mixtures, its performance tends to decrease when the dimension is large, like most of samplers. To highlight this limitation, we provide in Table 10 the results of GMM-LRDS for various mixture settings with $d = 128$, where we observe that it fails to recover a good estimation of the strongest mode weight (roughly, 10% of relative error).

### I.5.2 MULTI-MODAL GAUSSIAN MIXTURES

We display in Figure 13 the results of all considered samplers, including LRDS, when targeting the multi-modal Gaussian mixtures defined in Appendix H.1 with fixed dimension $d = 8$ and increasing number of modes $L \in \{4, 8, 16, 32, 64\}$. To assess the sampling quality at a global level, we dispense for each sampler the total variation distance between the true weight histogram and the weight histogram obtained via Monte Carlo approximation. All metrics are computed based on 8192 samples. We notably observe that GMM-LRDS has the best performance compared to all other sampling methods, independently of the number of modes in the target distribution. In particular, Figure 13 demonstrates that GMM-LRDS is able to recover both global features and local features for a complex multi-modal distribution.

**Execution time.** To demonstrate the applicability of LRDS in practice, we provide in Figure 14 the execution clock-time of each sampling method for all multi-modal Gaussian mixture sampling settings. The conclusions are the same as in the bi-modal setting: GMM-LRDS has equivalent training time to previous variational diffusion-based approaches, with negligible extra cost due to reference fitting.

**Limitation of LRDS: high number of modes.** As depicted in Figure 13 (top left), the recovery of the mode weights for a multi-modal target distribution gets worse as soon as the number of modes is large. Although GMM-LRDS performs better than its competitors, it suffers from this limitation.

### I.5.3 RINGS AND CHECKERBOARD DISTRIBUTIONS

In Figure 15 and Figure 16, we illustrate the impact of the number $J$ of Gaussian mixture components when running GMM-LRDS to sample from Rings and Checkerboard distributions, detailed in Appendix H.1, by taking $J \in \{1, 8, 16, 32, 64\}$. In particular, we consider mixture models with diagonal covariance (first two rows) and full covariance (last two rows), fitted on MCMC samples from the target via EM algorithm. For both settings, we observe that setting $J$ large enables to recover with more precision the support of the target distribution, leading consequently to better performance of GMM-LRDS. In the special Rings setting, where the target energy landscape is narrow and complex, we also notice that mixtures with full covariance provide better estimation of the support of the distribution. On the other hand, we remark that GMM-LRDS struggles to recover the geometry of the Checkerboard distribution, while EBM-LRDS is more performant, see Figure 6.
<figure><img src="image_15.png" /><figcaption>Figure 13: Results for multi-modal Gaussian mixtures, averaged over 16 sampling runs. (Top Left): Total variation distance between weight histograms. (Top Right): Results with regularized Wasserstein distance. (Bottom Left): Results with MMD distance. (Bottom Right): Results with KS distance. Incomplete curves refer to settings with missing results due to numerical issues.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Figure 14: Execution clock-time for multi-modal Gaussian mixtures with increasing number of mixture components (from left to right).</figcaption></figure>

### I.5.4 $\phi^4$ FIELD SYSTEM

Table 11 shows that all competing algorithms fail at recovering the weight ratio of the $\phi^4$ system as they are highly prone to mode collapse either on the 'negative' mode or on the 'positive' mode.

### I.5.5 BAYESIAN LOGISTIC REGRESSION MODELS

Finally, we display in Table 12 the results of all considered samplers, including LRDS, when targeting the Bayesian posterior distributions obtained from real-world data and defined in Appendix H.1. To assess the sampling quality, we compute the average predictive log-likelihood (i.e. the expected posterior distribution on the test dataset) via Monte Carlo approximation with 8192 samples. Since these distributions do not exhibit explicit multi-modal characteristics, we observe that standard MCMC-based techniques such as SMC and RE perform better than deep learning-based approaches. Interestingly, LRDS is on par or superior to previous diffusion-based approaches. For completeness, we also provide in Figure 17 the execution clock-time of all samplers in this setting.
<figure><img src="image_17.png" /><figcaption>Figure 15: Results of GMM-LRDS for Rings distribution with increasingly expressive GMM for reference distribution - (Top two rows) GMM density and GMM-LRDS samples for diagonal covariance parameterization (Bottom two rows) GMM density and GMM-LRDS samples for full covariance parameterization.</figcaption></figure>

<table><caption>Table 11: Estimation of the weight ratio for the $\phi^4$ system with different values of $h$, averaged over 16 runs. The Laplace approximations, that stand for ground truth, are displayed at the bottom of the table. Orange cells refer to settings with uninformative mode weight estimation (i.e., uniform mixture), red cells denote mode collapse. N/A denotes settings where results could not be obtained due to numerical issues.</caption><thead><tr><th>Algorithm</th><th>h = 0</th><th>h = 9 × 10<sup>-4</sup></th><th>h = 2 × 10<sup>-3</sup></th><th>h = 2.5 × 10<sup>-3</sup></th><th>h = 3.5 × 10<sup>-3</sup></th></tr></thead><tbody><tr><td>SMC</td><td>1.28±1.19</td><td>1.71±1.47</td><td>1.51±1.38</td><td>1.60±1.52</td><td>1.57±1.43</td></tr><tr><td>RE</td><td>1.00±0.12</td><td>1.01±0.12</td><td>1.04±0.13</td><td>1.11±0.13</td><td>1.07±0.14</td></tr><tr><td>LV-PIS</td><td>∞</td><td>∞</td><td>∞</td><td>∞</td><td>∞</td></tr><tr><td>LV-DDS</td><td>0.00±0.00</td><td>∞</td><td>0.00±0.00</td><td>∞</td><td>∞</td></tr><tr><td>LV-DIS</td><td>1.07±0.03</td><td>1.05±0.03</td><td>1.05±0.06</td><td>1.16±0.12</td><td>1.17±0.07</td></tr><tr><td>CMCD</td><td>1.01±0.05</td><td>0.98±0.07</td><td>1.01±0.07</td><td>0.96±0.07</td><td>0.96±0.02</td></tr><tr><td>iDEM</td><td>0.59±0.01</td><td>35.9±1.43</td><td>98.6±10.6</td><td>1.81±0.03</td><td>170±17.8</td></tr><tr><td>PDDS</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td></tr><tr><td>Laplace 0<sup>th</sup> order</td><td>1.00</td><td>1.35</td><td>1.95</td><td>2.30</td><td>3.22</td></tr><tr><td>Laplace 2<sup>nd</sup> order</td><td>1.00</td><td>1.34</td><td>1.90</td><td>2.23</td><td>3.08</td></tr></tbody></table>
<figure><img src="image_18.png" /><figcaption>Figure 16: Results of GMM-LRDS for Checkerboard distribution with increasingly expressive GMM for reference distribution - (Top two rows) GMM density and GMM-LRDS samples for diagonal covariance parameterization (Bottom two rows) GMM density and GMM-LRDS samples for full covariance parameterization.</figcaption></figure>

<table><caption>Table 12: Average predictive log-likelihood for Bayesian posterior distributions obtained from logistic regression tasks, averaged over 16 runs. Bold font indicates best performance.</caption><thead><tr><th>Algorithm</th><th>Ionosphere (d = 36) ↑</th><th>Sonar (d = 62) ↑</th><th>German Credit (d = 26) ↑</th><th>Breast Cancer (d = 32) ↑</th></tr></thead><tbody><tr><td>SMC</td><td>-139.4±0.3</td><td>-191.4±0.2</td><td>-122.2±0.1</td><td>-89.2±0.5</td></tr><tr><td>RE</td><td>-139.5±0.2</td><td>-191.3±0.1</td><td>-122.2±0.1</td><td>-90.1±0.3</td></tr><tr><td>LV-PIS</td><td>-156.0±1.0</td><td>-202.2±0.2</td><td>-153.7±2.2</td><td>-203.9±3.3</td></tr><tr><td>LV-DDS</td><td>-147.7±0.5</td><td>-195.2±0.7</td><td>-151.2±1.3</td><td>-128.4±0.9</td></tr><tr><td>LV-DIS</td><td>-224.3±22.6</td><td>-318.7±47.0</td><td>-206.5±1.0</td><td>-152.1±1.9</td></tr><tr><td>CMCD</td><td>-970.6±2.7</td><td>-1645.4±6.2</td><td>-171.9±0.0</td><td>-175.3±5.7</td></tr><tr><td>iDEM</td><td>-153.7±0.2</td><td>-4634.1±12.9</td><td>-132.4±0.1</td><td>-320.8±0.9</td></tr><tr><td>PDDS</td><td>-139.6±0.3</td><td>-191.3±0.1</td><td>-122.2±0.1</td><td>-102.2±0.1</td></tr><tr><td>GMM-LRDS</td><td>-138.0±0.5</td><td>-190.5±1.0</td><td>-129.0±2.7</td><td>-100.1±2.1</td></tr><tr><td>EBM-LRDS</td><td>-148.0±15.4</td><td>-191.8±0.9</td><td>-131.2±5.2</td><td>-92.7±4.0</td></tr></tbody></table>

<figure><img src="image_19.png" /><figcaption>Figure 17: Execution clock-time for Bayesian posterior distributions obtained from logistic regression tasks 'Ionosphere', 'Sonar', 'German Credit' and 'Breast Cancer' (from left to right).</figcaption></figure>