  Artificial intelligence algorithms are increasingly adopted as decisional
aides by public organisations, with the promise of overcoming biases of human
decision-makers. At the same time, the use of algorithms may introduce new
biases in the human-algorithm interaction. A key concern emerging from
psychology studies regards human overreliance on algorithmic advice even in the
face of warning signals and contradictory information from other sources
(automation bias). A second concern regards decision-makers inclination to
selectively adopt algorithmic advice when it matches their pre-existing beliefs
and stereotypes (selective adherence). To date, we lack rigorous empirical
evidence about the prevalence of these biases in a public sector context. We
assess these via two pre-registered experimental studies (N=1,509), simulating
the use of algorithmic advice in decisions pertaining to the employment of
school teachers in the Netherlands. In study 1, we test automation bias by
exploring participants adherence to a prediction of teachers performance, which
contradicts additional evidence, while comparing between two types of
predictions: algorithmic v. human-expert. We do not find evidence for
automation bias. In study 2, we replicate these findings, and we also test
selective adherence by manipulating the teachers ethnic background. We find a
propensity for adherence when the advice predicts low performance for a teacher
of a negatively stereotyped ethnic minority, with no significant differences
between algorithmic and human advice. Overall, our findings of selective,
biased adherence belie the promise of neutrality that has propelled algorithm
use in the public sector.
