least as successful as CV, but clearly dominates it for the higher values of ρ. Finally, the iceberg lasso (IL) is marginally outperformed by CV and BL when ρ = 0.75, outperforms CV but is again marginally outperformed by BL when ρ = 0.9, but has a substantial advantage over all other methods for ρ = 0.99.<sup>18</sup> Table 3: Percentage of times x1 is selected & average number of variables selected

||||ρ= 0.75|||ρ= 0.90|||ρ= 0.99||
|---|---|---|---|---|---|---|---|---|---|---|
|n||κ=5|κ=10|κ=20|κ=5|κ=10|κ=20|κ=5|κ=10|κ=20|
|250|CV|100.0<br>(8.65)|99.7<br>(8.55)|99.3<br>(8.74)|96.6<br>(8.87)|91.8<br>(8.66)|85.5<br>(8.64)|55.2<br>(8.52)|37.7<br>(8.22)|23.4<br>(7.93)|
||AL|99.7<br>|99.4<br>|97.9<br>|93.9<br>|87.4<br>|80.4<br>|45.3<br>|29.4<br>|17.7<br>|
|||(7.22)|(7.21)|(7.05)|(7.34)|(7.21)|(7.05)|(6.99)|(6.72)|(6.26)|
||PI|91.6<br>|89.9<br>|88.1<br>|80.6<br>|72.1<br>|63.7<br>|41.1<br>|26.8<br>|16.9<br>|
|||(1.26)|(1.52)|(1.89)|(1.45)|(1.73)|(2.06)|(1.23)|(1.33)|(1.41)|
||BL|100.0<br>|100.0<br>|99.8<br>|96.6<br>|98.4<br>|96.7<br>|90.4<br>|79.2<br>|64.2<br>|
|||(11.11)|(12.81)|(15.27)|(11.31)|(13.25)|(15.66)|(11.27)|(12.77)|(14.03)|
||IL|95.7|95.9|95.2|95.9|95.8|93.0|95.3|93.4|80.1|
|||(4.80)|(9.14)|(15.97)|(4.81)|(9.43)|(17.00)|(4.78)|(9.32)|(15.65)|
|1000|CV|100.0<br>(9.43)|100.0<br>(9.59)|100.0<br>(10.05)|100.0<br>(9.76)|100.0<br>(10.10)|99.9<br>(10.69)|81.0<br>(9.92)|69.8<br>(10.11)|56.4<br>(10.51)|
||AL|100.0|100.0|100.0|100.0|99.7|99.7|68.3|54.8|40.8|
|||(3.93)|(4.19)|(4.49)|(4.71)|(5.22)|(5.85)|(5.37)|(5.97)|(6.22)|
||PI|99.8|99.8|99.7|99.2|98.4|97.5|71.4|55.9|41.4|
|||(1.31)|(1.54)|(1.88)|(1.63)|(2.02)|(2.57)|(1.75)|(2.02)|(2.34)|
||BL|100.0|100.0|100.0|100.0|100.0|100.0|98.0|93.70|87.1|
|||(8.88)|(10.89)|(13.91)|(9.26)|(11.67)|(15.23)|(9.36)|(11.85)|(14.81)|
||IL|100.0|100.0|100.0|100.0|100.0|100.0|100.0|100.0|98.8|
|||(5.01)|(10.00)|(19.22)|(5.00)|(10.01)|(19.69)|(5.01)|(10.01)|(19.72)|
|4000|CV|100.0<br>(10.46)|100.0<br>(10.85)|100.0<br>(11.24)|100.0<br>(10.78)|100.0<br>(11.28)|100.0<br>(11.88)|99.0<br>(11.18)|97.8<br>(12.06)|94.9<br>(12.63)|
||AL|100.0<br>(1.00)|100.0<br>(1.00)|100.0<br>(1.00)|100.0<br>(1.03)|100.0<br>(1.00)|100.0<br>(1.03)|91.9<br>(1.18)|86.0<br>(1.30)|79.1<br>(1.70)|
||PI|100.0<br>(1.23)|100.0<br>(1.43)|100.0<br>(1.68)|100.0<br>(1.53)|100.0<br>(1.96)|100.0<br>(2.42)|98.0<br>(2.00)|93.9<br>(2.60)|88.1<br>(3.18)|
||BL|100.0<br>(7.86)|100.0<br>(9.91)|100.0<br>(13.03)|100.0<br>(8.44)|100.0<br>(11.04)|100.0<br>(14.94)|100.0<br>(8.93)|99.9<br>(11.94)|99.8<br>(16.27)|
||IL|100.0|100.0|100.0|100.0|100.0|100.0|100.0|100.0|100.0|
|||(5.00)|(10.00)|(19.99)|(5.00)|(10.00)|(20.00)|(5.01)|(10.00)|(20.00)|

The performance of all methods improves for the larger sample sizes, but the IL maintains its advantage in the more challenging cases with ρ = 0.99, with BL having a very similar performance. Overall, the two extensions of the PI we consider, the BL and IL, lead to greatly improved ability of identifying the relevant regressor, and there is generally little to choose between them, except in the extreme cases with n = 250 and ρ = 0.99 where the IL has a very clear advantage.

The results for the average number of variables selected are also interesting. In all cases considered, CV tends to lead to a high average number of selected regressors. On the other extreme, PI is generally the most parsimonious except for when n is large, in which case the oracle property of AL starts to become salient. Turning now to the

> 18Part of the reason why in some cases IL does not perform well is that sometimes PI selects no regressors at all, and in those cases IL cannot improve on it but BL can.

17