Newer techniques like the support vector machine and Bayes-point machine are not error driven and come closer to true probabilistic averaging. The effect of fulfilled predictions can be even more extreme than described here. The optimal classifer can change in an arbitrary way, with new decision boundaries appearing or disappearing, when a prediction is confirmed. The set of consistent classifiers is a convex polytope in the space of classifiers. The optimal classifier is the polytope's center of mass.
New information cuts away pieces of the polytope, thus moving the center of mass.