I had this before but it breaks in an ugly way when used with sklearn cross_val_score
I want to avoid bugs in users code.