# Framework for Assessing Ethical Aspects of Algorithms and Their Encompassing Socio-Technical System
## 5 Validation
### SyRI
#### 5.1.2 Comparison of the Analysis to the Framework  
A summary of the completeness results on the comparison of the analysis of the court case and media on SyRI with the proposed framework can be found in Figure 3. More information on the topics can be found in Figure 4. The topic of pseudonymization/anonymization was omitted from the table, because it is in most of the cases too ambiguous to ascertain which of the two is meant. Note that the intended use of the framework is _during_ the development of an algorithm and not afterwards, but the amount of information available from the court case and media attention made it very suitable for validation. The difference to be noted is, that if the framework would have been applied by the organization that developed SyRI itself, more (inside) information would have been available to them.  
Figure 4: Comparison of specific ethical concerns between framework, the court case and the media on whether questions in the framework could be answered with a yes, maybe, no, or were not answerable.  
Figure 3: Results of a comparison of specific ethical concerns between the framework, the court case and the media on whether questions in the framework could be answered with a yes, maybe, no, or were not answerable. (**a**) Exact numbers. (**b**) Percentages.  
There are several discrepancies between the author's application of the framework with the analysis of the general media an court case.
* The media and the author applying the framework are of the opinion that it is doubtful that this large amount of data is necessary for the purpose of the algorithm. The court also mentioned this briefly, however, it did not judge on this.
* The issue of independent parties reviewing the system is not a black and white case. It can be established that it has been reviewed by multiple governmental people. The media seem to agree on this, however, they mentioned that many of these governmental people are not educated enough to make decisions on this. There is something to say for that, looking at the number of questions that were asked about SyRI, but nonetheless some critical questions were asked by governmental people. Competence of (certain people within) the government is often something of debate and it seems not to be different here.
* The media claimed that the results of SyRI are unclear and unverifiable. The court case does not agree with this. It was mentioned that the algorithm is nothing more than a simple decision tree, which should be sufficiently transparent for the people working with it. The author applying the framework evaluates this as a case of a truth that is probably somewhere in the middle.
* The media often emphasize that SyRI is discriminating. It is difficult to say something about this since so much of SyRI is kept a secret. The court left it somewhere in the middle and the author chose the same approach with the framework, with a mention that the decision to apply SyRI only to vulnerable neighborhoods does feel doubtful. The framework's conclusion might have been different if it was applied by a team member of SyRI with more information. For example, the question 'Are there any potential discriminating factors in the algorithm?' could very well uncover more issues besides this decision to apply it only to vulnerable neighborhoods.