# Framework for Assessing Ethical Aspects of Algorithms and Their Encompassing Socio-Technical System
## 1 Introduction  
A considerable amount of innovation by private and public sector organizations is driven by the use of data. In 2018, 2.5 exabytes of data were generated per day [1] and it is expected that this amount will only grow. For example, fifty percent of the municipalities in The Netherlands use data to a certain extent and 56% of governmental institutions perform research on algorithms [2].  
The issues created by artificial intelligence, or rather algorithms in general, extend far beyond the technical sphere. For a proper implementation and use of algorithms, they are embedded in a socio-technical system consisting of rules and regulations, organizational structure, etc. In our experience, the rules and regulations and the organizational structure are especially important for properly addressing the ethical concerns algorithms may have, hence a well-functioning socio-technical system should be well-designed for this purpose. A major consultancy firm (KPMG) described the AI ethicist as an essential position for a good AI implementation [3].  
Illustrative examples can be found among fraud detection algorithms developed by municipalities and other governmental institutions. These algorithms typically scan through individuals' data and create a subset of cases with a high suspicion of fraud. This subset can then be investigated further by a human aiming to confirm and decide whether fraud was indeed committed and to collect evidence to build a case. This is done because municipalities have the duty to monitor the proper and fair use of their societal services, hence to find and punish fraudsters [4]. The damage caused by fraud can be huge;estimated at more than a yearly EUR 1 billion of benefits fraud in the Netherlands alone [5]. A further reason for the interest in applying data-driven algorithms for fraud detection is that the available resources for human inspection are limited. Municipalities often have only a small team of people responsible for catching fraudsters, so an algorithm allowing them to focus their attention on a small set of suspicious cases is very attractive.  
Although attractive and seen as an important future direction, concerns related to the use of data-driven algorithms may arise. For example, the Dutch government came into trouble because of what is now known as the \"Dutch childcare benefits scandal\" (Dutch: \"Toeslagenafaire\") where, between 2013 and 2019, authorities wrongly accused an estimated 26,000 parents of making fraudulent benefit claims for day-care, requiring them to pay back the allowances they had received in their entirety, which drove many families into severe financial hardship. A parliamentary interrogation committee concluded among other things that \"unprecedented injustice\" was done and that \"affected parents did not receive the protection they deserved\" by the system [6]. Another notable example is when the Dutch government introduced the Systeem Risico Indicatic (SyRI) system. This was a social security fraud detection algorithm of the Dutch government. It received an ironic privacy prize for the invasion of privacy of people [7]. There were five reasons for this according to Bits for Freedom: citizens were a suspect in advance, it felt like a violation of their privacy, data might have been used without purpose limitation, it might have been discriminating, and it would be the first step towards a control society. Parliamentary discussions were held about the system [8]. It became clear that in the design and realization of the system, too little attention was paid to ethical concerns and that valid points raised by the public were insufficiently addressed.