# Framework for Assessing Ethical Aspects of Algorithms and Their Encompassing Socio-Technical System
## 6 Conclusions  
In this paper, we propose a framework for identifying and assessing ethical concerns regarding the use of data-driven algorithms. The framework can be used during the design for identification of relevant concerns so that they can be addressed early in the design of the entire socio-technical system contributing to a more robust, complete, and effective introduction of such systems, improve the quality of their designs from the ethical perspective, and avoid injustice and associated uproar. The framework has been developed by merging ethical frameworks from the literature and enriching and fine-tuning it on the basis of expert interviews and brainstorm sessions. The resulting framework is based on five ethical principles that algorithms and the surrounding socio-technical system should respect: benefice, non-maleficence, autonomy, justice, and explicability. The framework has been validated by applying it to two fraud detection case studies: the Systeem Risico Indicatie (SyRI) of the Dutch government and the fraud detection system being developed by the municipality of Amersfoort. For SyRI, we analyzed media articles and the court case. We showed that all of the concerns mentioned there were also addressed by the framework, and more. Importantly, it showed that a substantial number of aspects were not or only partially addressed by the design of the SyRI system. About 24% of the questions asked by the framework could not be answered though, because of lack of inside knowledge as opposed to the Amersfoort case where only 3% of questions could not be answered. For the Amersfoort case, he framework showed that a much smaller but nevertheless still significant number of aspects were either not addressed or insufficiently addressed. However, also here, the validation showed that all concerns mentioned in the media and interviews we held, were addressed by the framework. Here too, the framework addressed more aspects that were not explicitly 'on the radar' of the designers of the Amersfoort system. Common to both cases is that they are of high societal importance. Fraud is an important phenomenon and large sums of money are involved. Hence the reason why the solutions were or are created are more than clear. Second, in both cases the data subjects, citizens or customers, do not have a choice to participate. They cannot say that they do not want their data to be used. This makes that their autonomy is limited, hence there is need for proper safeguards and appeal protocols such that they receive sufficient control. The framework proved to be an effective tool in both cases. Furthermore, it indeed seems useful to apply the framework early in the design phase having a diverse group of people involved and having sufficient knowledge of the design and realization of all components of the whole socio-technical system. The framework helped discover initially unidentified concerns in both cases, concerns that, were they identified in an early phase, could have been properly discussed and incorporated in the design. It can also hopefully avoid the introduction of new systems with ethical flaws and associated scandals such as happened with SyRI. The validation of the framework focused on two cases. One important part of future work should be focused on validation with more cases and gaining more experience with its use for the purpose of further improvement and fine-tuning. It is important to choose such cases in other important domains, such as healthcare and e-commerce, where algorithms are increasingly used and ethical concerns may have much impact as well. Furthermore, the framework is currently designed with a focus on algorithms with limited to no autonomy for the data subject. Future research could re-evaluate the desirability of context-dependent concerns like'stakeholders' have the right to decide for themselves' and how these should be integrated in the framework.