[MRG] Make tests faster to run

Ongoing fix for #3402.