> They were just examples of systemic bias. The fact that so many are racial is just a reflection of the unfortunate state we find ourselves in.

Do you think racial cases of bias outweigh other demographic biases? And what's your opinion of the general level of intent (or lack of, to fix) relative to other issues?

The medical one made me think about the similar case of differing heart-attack symptoms in men and women. In this case actually because doctors just focused on men.

> That's not what systemic bias means.

> "Systemic bias, also called institutional bias, and related to structural bias, is the inherent tendency of a process to support particular outcomes."

That just shifts the definitional question to the scale of the institution, structure, or process.

At one end, if everyone follows the same rules it's a system, but what about if one clinic got bad data, do we write that up as a systematic failure? That's okay if it's your definition, but cases at the other end seem a lot more important and a lot more amenable to systematic fixes.

> If something affects a population, that is system-wide by definition, since the definition of "system" is pretty broad

Is the definition, "a system is broad" or (I think you mean) "broad as in flexible", such that a system can be anything? Any micro or macro population?

I guess where I was going with this is that the list seemed to be about some things that are very broadly impactful but not specifically racist, like facial-recognition not recognizing anyone but white men very well, but then a related issue of how this is encoded in the system. Is usage required, or was it random, etc.

Such that maybe top-to-bottom it'd be sorted by breadth. How many people does each impact, and how deeply-mandated or intertwined are the issues. And then different populations and problem to both give contrast, but also to indicate (when complete) the demographics of the problems.

So I was wondering if you'd focused on black issues to make the list.

> Unfortunately, this is seen over and over again, more so than would happen by randomness. That's the point of the list.

So, statistically, how much subpopulation misrepresentation would you expect in various ways? And are you saying there's more in general, or more racial, than expected?

I interpreted the one about kidneys as an error because of changing demographics and how people had caught it, made note that this sort of thing happens, and started checking other research for similar sampling bias. It read as a success story where the one about facial-recognition was more actively black mirror.

> You seem to be agreeing that this was in fact a badly biased algorithm

No. But not a good one either. More that I'm wondering if it was an attempt to model, like for predictive policing, or a tool sold to simplify the sorting of people? Because models are good, even when they're wrong, but crappy predictive tools are worse than useless and - where I was ultimately going with this - perhaps fraudulent to sell.

The 'good' case would be if someone built a criminality model and the city was trying to work with police and communities to intervene in a predicted pattern. It's not unreasonable that the societal harm from a non-criminal becoming criminal could be worse than an existing criminal remaining that way. So modeling and discussing this isn't bad, even if the data has a racial component and some of the questions are of bias.

But yeah, to recommend sentences, total crap.