Arguing with incomplete information
Authors:   , Odekerken Daphne 
Published: December 2024

Every year, the Dutch police receives tens of thousands of reports on online trade fraud. Two subtasks within the process of handling these reports are the intake with the citizen and the investigation of potentially mala fide web shops. A significant part of the tasks consists of routine work, but some reports and web shops require detailed manual investigation. Neither humans nor automatic procedures are able to quickly perform these tasks on their own, but maybe they can complement each other. Can we develop AI systems that support humans in decision-making?
Since AI systems in law enforcement should be transparent, I focus on AI techniques that are able to reason with rules (and exceptions) or with earlier (precedent) cases. These types of reasoning can be modelled using formalisms from the fields of computational argumentation and AI & Law. However, in most research in these areas it is assumed that all information that is required for making a decision is available, which is not the case for our use cases.

In my dissertation research, I therefore extend these formalisms so that they can express not only certain, but also uncertain information. In addition, I introduce the notions of stability and relevance in the context of argumentation- and precedent-based reasoning. The goal of the notion of stability is to identify situations in which a decision can be made, even though there is some uncertainty about the information. In an application, this means that the advice will not change, regardless of information updates. In situations where the topic is not stable, it is not clear which decision should be made as some decisive information is missing. It is therefore necessary to identify relevant information updates in that investigation into the presence of this information possibly leads to a stable topic.

For each of the studied formalisms, the extension for modelling incomplete information represents a space of possible situations that grows exponentially with the amount of uncertain information. This raises concerns related to the computation of stability and relevance. Our complexity analysis reveals that many of these problems are in high complexity classes, implying (under the common assumption that P ̸= NP) that it is impossible that any exact algorithm can solve these problems in polynomial time. Thus the question arises to which extent the stability status and relevant updates can still be computed within reasonable time.

We answer this question by developing and testing various algorithms for stability and relevance in both structured and abstract approaches to argumentation, as well as precedent-based reasoning. We introduce heuristic and exact algorithms. Some of these are based on answer set programming.

Finally, we report on applying these algorithms for the use cases of fraud intake and web shop classification. The system for fraud intake was launched in September 2019 and has been used by hundreds of thousands of citizens, while the system for web shop classification is used by analysts at the Dutch police. In addition, the algorithms have been implemented in open-source software packages.