This may  be related to different scoring standards in possibly different versions of sklearn. 

Not sure though.

In any case, I will have to deal with a strong class imbalance problem while doing OvR for each label. Chance level for 0-1 scoring is at 87.5%
I am now going for f1_score. Any reason why I shouldn't?
If I go for AUC, I will have to dig into the classifier and move an offset around, which may not be ideal for a simple example