So `ProbabilisticPCA` stays broken with a warning? Just so it doesn't change the output of (broken) scripts using it?
I thought deprecation should be bypassed when it comes to fixing bugs. In my opinion the warning should say something like:
"WARNING: the covariance estimation was previously incorrect, your output might be different than under the previous versions"
and I would have the ProbabilisticPCA class just be the same as the new PCA class (they're API compatible right?)

I'd love to see other opinions on this, though.