> For instance, at least here in the U.S., it is illegal for police to profile people based on race

It must also be mentioned, sadly, that data shows that shows that police profile racially despite it being illegal, including, sadly, judges, who seem to punish minorities about whom there are negative stereotypes more harshly for similar crimes.

> in the aggregate, have some predictive value.

This data does have predictive value, for sure, studies have shown. But it's important to recognise that it's not the ethnicity that is a causal factor, which studies have shown over and over again, too. (at least in the countries I'm familiar with, not sure how it is in the US but I wouldn't be surprised if the results are similar).

i.e., it may well be true that a minority has a higher probability of engaging in crime, such that racial profiling has value for the police. But studies show that this is because this particular minority is: 1) more likely to be less educated, live in a poorer neighbourhood, have fewer well-connected friends, be of a younger age, more likely to be unemployed, more likely to have parents who are unemployed or illiterate etc etc. Once you adjust for all these things, i.e. compare a minority person of age x, employment y, neighbourhood z, education t etc, to a majority person with similar parameters, the probability of crime is not significantly different. Such that it's not the ethnicity or culture, but rather the socioeconomic standing of a person. And that standing is often a function of the system of government, the institutions, the (migration) history etc of a group of people. For example, Italians in their first years were socioeconomically worse off, explained a lot by their recent migration history (poor migrants in a new country), and they tended to crime more than the average person. Today that's no longer the case. None of that had much to do with their skin color, ethnicity or culture.

So while ethnicity has predictive values, it's not a factor itself. Rather it's a proxy for recognising a person's probable socioeconomic status, which is a key factor in crime. If you can establish that by looking at a person's ethnicity, that's useful, but it's also racist by definition.

But that does mean the following, if a bot is supplied ample data, it may not need to use ethnicity as a proxy for key factors (e.g. socioeconomic status, broadly) that can predict crime. We can skip racial profiling all together.

And then we'll be stuck with a new problem, not racism, but some form of classism that we've seen before but in a new way. I felt that China recently made its first steps towards such a future with the scoring of citizens on a whole range of parameters. It's hugely valuable, but it reduces people to being scored on a number of parameters, e.g. income, education of parents, favorite sport etc, just like skin color, for which you can run all kinds of regressions to figure out which factors are 'good' and 'bad' for any given scoring... and then a priori judge people, rather than allow for the possibility that someone who has parameters that are statistically likely to be bad, may be perfectly fine and do nothing wrong (like skin color, where statistically being 'black' is 'worse' on a whole range of topics, from which it does not follow that any black individual is worse.) That's the story of racism, making judgements before the fact on the basis of a parameter that reduces a person to a society's perception of that parameter, skin color.

It's bad, but I'm afraid it may become worse. Racism will continue to be 'not done', but it may be replaced by proxies that are just as bad, which are considered perfectly fine. One example is where in my country, it's not incredibly popular to demand X amount of income for a home. The idea is that, if you don't have X, you can't afford it. The truth is, that X is way higher than the amount you need to reasonable afford that home, and so it's essentially creating a neighbourhood of certain affluence that weeds out socioeconomic groups that could afford to live there, but are discriminated against on the basis of their income, which, surprise, happens to create segregated communities because ethnicity are income are still pretty closely linked. But because it's done under the guise of 'we're just protecting ourselves and renters by renting only to those who can afford it', it doesn't create a single word of discussion. It's a tricky situation because I tend to appreciate the concept of minimum income for homes, but the level they set it at made me suspicious, and I'm seeing neighbourhoods segregate in my city. Further, OF COURSE it must be allowed to judge people on certain parameters, like say a degree when getting a job as a proxy for how qualified someone is, even though a non-degree holder may be more qualified, it's a reasonable form of discrimination. But at some point, we're also reducing a person to a small set of data and its associated probability set, much like skin color. I find this a tricky issue, although fortunately it seems that non-racial discrimination is limited to only a few things, like gender and age. You hear stories of some companies rejecting a person on the basis of his post code or address (reject poor neighbourhoods), looks or favorite sport (ski vacations?), but it's not widespread.

Anyway so much for my rambling. In short, I think it's reasonably easy to not give a bot racial data, but I think the bot will recreate the probabilities on the basis of other data that proxies socieconomic status, and that this is concerning in a way that is similar to racial profiling, because it reduces a person to the parameters he has and how those parameters happen to correlate with e.g. crime in a large group of people that he may behave differently to. This is how the race-parameter works, you're black? well that correlates to crime in a group of people, so we'll treat you like you're likely to be a criminal. That's bad, but if you ignore that parameter, there are others that are similarly bad, that a bot will find. Like oh you're poor, or oh you love in that neighborhood, or oh your parents are illiterate? Well that correlates to crime, so we'll treat you like you're likely to be a criminal. Non-racial parameter profiling, bots will be champs in that.