Consider the proposition of formalism - the idea that the stated public rules must, in fact, drive the expression of political power.

I'm not familiar with this usage of "formalism". I searched your site, but didn't find other mentions. Your gloss helps[1], but is there a reference you could point to?

It's pretty clear that we don't live in a world governed by formalism.

Are you saying that we don't live in such a world, or that we can safely assume that almost everyone realizes that we do not live in such a world? The first seems arguably true, the second strikes me as likely false.

we claim to favor individual fairness (the proposition that individuals should be judged on their own individual merits) yet in practice we push for equality of outcomes (the proposition that processes should make decisions that do not vary across certain reference classes)

Your use of "we" here seems forced. You[2] are the only clear party to this "we", but since you offer this example as a contradiction, presumably you do not actually put yourself into both categories. I suspect there is actually a division between these groups, with few claiming to belong to both.

political power in the world strives to have proportional representation of different racial groups in college

I'm not sure if I disagree with this, or if I missed the beginning of a counterfactual. I'd agree that some people claim to strive for proportional representation of races in college. Some of these think of this as a goal in and of itself, some assume it's a convenient metric for "equal opportunity" or a related goal, and others find it beneficial to claim to be for this without doing any striving at all.

But to say that "political power in the world strives" either seems at odds with the evidence --- which I think shows that there are still racial disparities in college attendance and outcomes both in the US and the world --- or implies that "political power" is a relatively weak force that is prevented from having its way by something stronger.

Or maybe you are defining power differently than I would. I tend to a simplified Foucauldian worldview[3], where "power" is strong and "truth" is weak. In my view, the best way to understand what "power" is striving for is to ignore everything it claims and look instead at how the world actually behaves.

While not always accurate, I think the actual behavior of the world is usually a better starting point in determining what "power" is striving for than listening to its consciously crafted claims. When the world does not behave in a way that most benefits those with power, it's likely we have misidentified the true holder of power, or have mistaken its stated goals for its true goals.

As such, algorithms are a huge threat to existing power structures, because they both implement formalism and they also reveal the lies behind our existing stated principles.

I liked the distinction that the authors of the linked article make between "syntactic transparency" and "algorithmic accountability". They claim that the real harm comes from "opaque algorithms", and that "interpretability" is something that should receive more emphasis. I think this is compatible with your theory, since an opaque algorithm is less likely to reveal the divorce between what is espoused and what is practiced, and an easily interpretable one is more threatening. What do you think?

[1] In your linked post, you said "Unlike me in this blog post, they didn't gloss over the statistical test - instead, the used the one that came built into R." I wasn't sure what you meant here. Do you maybe mean "they did gloss over" (made it superficially lustrous by omitting essential details) or maybe that "didn't gloss" (failed to define their terms as with a glossary)?

[2] I don't know if it's clear to the audience that 'yummyfajitas' and the 'chrisstucchio' of the linked blog are related. I think they are the same person, but maybe I'm missing a subtle distinction of persona?

[3] http://www.powercube.net/other-forms-of-power/foucault-power...