Human-centered computing in legal NLP - An application to refugee status determination (2022)

Abstract: This paper proposes an approach to the design of an ethical human-AI reasoning support system for decision makers in refugee law. In the context of refugee status determination, practitioners mostly rely on text data. We therefore investigate human-AI cooperation in legal natural language processing. Specifically, we want to determine which design methods can be transposed to legal text analytics. Although little work has been done so far on human-centered design methods applicable to the legal domain, we assume that introducing iterative cooperation and user engagement in the design process is (1) a method to reduce technical limitations of an NLP system and (2) that it will help design more ethical and effective applications by taking users’ preferences and feedback into account. The proposed methodology is based on three main design steps: cognitive process formalization in models understandable by both humans and computers, speculative design of prototypes, and semi-directed interviews with a sample of potential users.

## scope of the research proposal at the core of the global refugee crisis is the legal procedure of refugee status determination (rsd), i.e. the decision of granting refugee status or not. refugee adjudication is a high-stakes, life-altering decision that impacts vulnerable people. our project aims at helping and supporting all parties involved in refugee status adjudications to make better decisions by using data-driven intelligence. it looks at building an ethical human-ai decision support system and focuses on augmenting human legal reasoning through the use of machine learning models. the aim is neither to output a decision nor to recommend one, as we think refugee status determination should ultimately be made by human experts. potential users of the system are stakeholders in the legal decision process such as a lawyer, counsel, judge, civil servant, or case worker. although not the direct users, asylum-seekers are essential interested parties as they should directly benefit from improvements in the procedure. text data in refugee law includes cases and decisions, country reports, international conventions and local refugee status regulations. our work is based on a data set containing the text of first instance decisions rendered in canada over the past 25 years (approx. 20,000 decisions). given the importance of text and language, its interpretations and levels of meaning in law, we want to explore the application of state-of-the-art natural language processing (nlp) methods to extract and organize information from past decisions. we hypothesize that human-centered computing (hcc), design and human-computer interaction (hci) methods can be exploited in legal nlp systems to enhance trust and overall performance by providing easier access to information and reducing risks associated to the use of ai in the legal field. trust in our system is not immediate for users and we will need to provide rational guarantees and good evidence of safety, understood as effective avoidance of risks and harms. precisely, we assume that trust can be warranted by modeling features of interpersonal trust, by ensuring usefulness of the system and its functionalities and demonstrating its benefits. as a starting point, we assess potential risks and describe them as well as potential unwanted events or consequences. while there is little specific literature on humancentered computing and human-computer interaction in law, we build on general hcc and hci literature for high-stakes decision making. given the above stated hypothesis, this document aims at exploring relevant methodologies and design processes that can support the conception of our system. ## legal nlp background legal ai focuses on building ai-powered tools for the legal domain. much of it specifically relies on nlp methods to help accomplish legal tasks @xcite @xcite . here, common functionalities include information retrieval @xcite , database management (refworld), similar case matching @xcite @xcite @xcite ), text summarization, legal advice, contract and document automation and review. work on legal design also looks at legal procedure and systems with the aim of developing user-centered methodologies and designs approaches @xcite . although it does not necessarily imply the use of ai systems, automation and text analysis is a major field of investigation and legaltech has recently received a lot of attention. as it is arguably difficult for a machine learningpowered system to capture qualitative data, any textual representation that can be processed with nlp and text analytics tools will be partial and subject to errors. text analytics is limited when it comes to capturing meaning, context and legal arguments, and is only able to try and generalize knowledge based on past decisions and historical data that were contained in the training data set @xcite . ## risks and obstacles this section identifies some risks that we anticipate to arise from this project. risks associated with the design of our system are both technical constraints and ethical considerations, especially in terms of impact on the users. we specifically assessed how the design of our system could negatively impact individuals whether legal practitioners or claimants. we will link this approach of ethics and impact assessment with human-centered computing and try to combine human and ai learning and reasoning. a literature review and preliminary research has highlighted the following risks and limitations. the first risk concerns asylum seekers needs throughout their application process: risk of unjustified decision as to the determination of their status, risk to refuse refugee status to someone who would be granted the status had our tool not been used, lack of support and information and risk that the application process becomes more painful for the asylum seeker. other potential risks include: narrow ai in law and need for manual engineering, combining human and machine legal reasoning, accuracy bias, fairness and interpretability, accountability, privacy concerns, impact of the use of ai on the legal process and the law. from this assessment, we chose to gather risks in four categories that represent clear requirements to work on the design of the system. since each one of these concerns user requirements, it is worth noting that different users may have different requirements for each one of these risks and that design should facilitate tradeoffs. we conclude that the main challenges to design our system will be to guarantee trust, usefulness, usability, and provide benefits for refugees. ## human-ai cooperation human-centered computing is commonly defined as the use of computing technologies centered on human experiences @xcite . human-ai cooperation is the proposed way to mitigate the risks listed above by combining benefits from ai systems such as computational power with human abilities including intuition and context-aware reasoning. based on our review of the literature, we find that human-ai interaction may provide an interesting way to try and mitigate the uncertainty of legal procedure while also addressing some limitations of ai algorithms, which will hopefully lead to higher acceptance from legal practitioners. users indeed need guarantees to use the system, which would require several qualities such as transparency and justification, but also improved user experience and design. it is assumed that involving the user through interaction and cooperation with the application naturally generates more trust. this approach is also called "mutualism" @xcite ), cognitive computing @xcite , interactive machine learning @xcite and has the advantage of reducing the need for comparison or even competition between humans and ai, lack of accountability, and to mitigate the problem of control over an ai application. this method typically involves trade-offs, leading us to think in terms of the balance of cooper-ation between an ai application and its user. the question is to find what methods can be used to translate this theoretical approach in terms of design of the model, functionalities and user interface. ## effective human-centered design this section aims at analyzing functionalities of our system in sight of ways of working, procedures and design approaches across the three domains involved in our research: nlp, refugee law, and human-centered design. ## proposed design methodology the general idea that underpins our approach is that we aim to develop algorithms that not only learn from data, but also through exposure to human practices and interactions with human experts. to achieve this, we will employ methods of participatory design, value-sensitive design and rapid prototyping. building on ## evaluation methods to evaluate our system, we plan to use both quantitative metrics and qualitative analysis and to expand the scope of our evaluation beyond accuracybased measures. metrics should be three-fold: as our system aims at benefiting refugees, we want to add an additional evaluation metric in the form of "design for dignity" @xcite that accounts for the beneficial use of ai and its positive inputs toward a specifically vulnerable population as refugees. ## conclusion and future work this document highlights some solutions and methods for designing an nlp-powered decision support system aiming at providing additional insight to the refugee status determination process. it should be treated as a starting point towards exploring how nlp tools could be beneficial to asylum seekers and help understand reasons and steps leading to a decision outcome. in the future, we plan to test empirically this methodology and implement the above listed functionalities.