Unsupervised Anomaly Detection in Parole Hearings using Language Models (2020)

Abstract: Each year, thousands of roughly 150-page parole hearing transcripts in California go unread because legal experts lack the time to review them. Yet, reviewing transcripts is the only means of public oversight in the parole process. To assist reviewers, we present a simple unsupervised technique for using language models (LMs) to identify procedural anomalies in long-form legal text. Our technique highlights unusual passages that suggest further review could be necessary. We utilize a contrastive perplexity score to identify passages, defined as the scaled difference between its perplexities from two LMs, one fine-tuned on the target (parole) domain, and another pre-trained on out-of-domain text to normalize for grammatical or syntactic anomalies. We present quantitative analysis of the results and note that our method has identified some important cases for review. We are also excited about potential applications in unsupervised anomaly detection, and present a brief analysis of results for detecting fake TripAdvisor reviews.

## introduction california houses america's largest "lifer" population, with 25% of its 115,000 prisoners serving life sentences. each year, the board of parole hearings (bph) conducts thousands of parole hearings to decide whether to grant prisoners early release. as california has enacted legislation to reduce its prison population, the number of hearings is scheduled to double this year and continue to rise for the foreseeable future. while each hearing is transcribed into about 150 pages of dialogue and sent to the bph and governor's office for review, capacity constraints mean that, in practice, only grants of parole are reviewed. legal scholars who painstakingly analyzed small subsets of transcripts have found that parole decisions are sometimes made presiding commissioner: let me ask you a question, mr. [redact]. are you angry? inmate [redact]: no. presiding commissioner: you seem kind of like you're a smart ass. i don't mean to say that rudely, but are you a smart ass? figure 1 : example of a semantic anomaly in an arbitrary and capricious manner @xcite , but they lack the resources for ongoing review. to help alleviate these capacity constraints and allow for greater review of parole denials, we propose an automatic anomaly detection system that allows reviewers to focus their attention on the most anomalous portions of text in each hearing. 1 the lack of gold anomaly labels precludes the use of many supervised anomaly detection techniques, so instead we propose using language models trained on the parole transcripts to perform unsupervised anomaly detection. defining an "anomaly" in this context is challenging. there are many ways in which a piece of text might be unusual without constituting grounds for additional review. we distinguish primarily between non-semantic, semantic, and procedural anomalies. we define a non-semantic anomaly as an irregularity in the linguistic structure of a piece of text (for instance, a sentence fragment). a semantic anomaly, by contrast, is one caused by the meaning of the text. in the context of a parole hearing, a conversation that deviates substantially from the typical topics of discussion would constitute a semantic anomaly. finally, a procedural anomaly is an irregularity that indicates the hearing differed substantively from the prescribed guidelines. often, a procedural anomaly will also be a semantic anomaly. figure 1 represents such a case, as it both includes language atypical for a parole hearing and, more generally, indicates a breakdown in communication between the commissioner and the parole candidate. we note that there are also, of course, legal anomalies that do not manifest as atypical language. a language model (lm) provides an organic way to identify unusual text through its perplexity score. we hypothesize that many procedural anomalies can be identified by examining statistical anomalies in the texts of transcripts, which would seemingly allow for their detection by an lm. however, most instances of unusual text found by a naive lm are non-semantic, consisting of typos, ungrammatical sentences, etc. to solve this problem, we instead use a pair of language models. we define our anomaly metric, the contrastive perplexity score, as the scaled difference between the perplexity of one lm, which has been fine-tuned on the target domain, and the perplexity of another lm, which has only been pre-trained on out-of-domain text. non-semantic anomalies will have high perplexity under both lms (and thus low contrastive perplexity), so the second lm acts as a "normalizer" for non-semantic content. we present our results on a human-annotated subset of the parole data. our method recalls 71% of human-labeled procedural anomalies while only asking experts to review 50% of the text of each transcript. we also show that our method can be extended to other domains where a large labeled corpus of anomalous text is unavailable, namely the task of opinion spam detection in tripadvisor reviews. text is a challenging regime for ad because of the importance of domain-dependence: what is shocking in one case might be mundane in another. few, if any, universal features for ad exist. general approaches for text ad include non-negative matrix factorization @xcite and the use of "selectional preferences" @xcite . one notable approach, studied in the dis-course coherence literature, is to focus on local abnormalities in topics. @xcite and @xcite present deep models for identifying incoherent passages of text, but discourse coherence studies much shorter text than parole hearings. to address longer text, our approach, like that of @xcite , splits each document into segments ranked by anomaly score. our strategy of using lms for ad has precedents, but primarily much simpler lms, and for ad contexts that require more supervision than is available in the parole hearing setting. @xcite and @xcite use n-gram lms to identify anomalous sections and documents in a corpus of american bills presented before congress. axelrod et al. (2011) and xu et al. (2019) also explore using a "baseline" lm for translation and discourse coherence, respectively. ## approach our model uses gpt-2, a transformer-based lm pre-trained on webtext, a corpus scraped from the internet @xcite . the following three observations motivate our approach to identifying anomalous text: (1) the perplexity of a fine-tuned lm on a target domain yields a score that measures both genre-specific semantic anomalies and general language anomalies (e.g. ungrammatical inputs, misspellings, incoherence). we define the contrastive perplexity lm anomaly score to be the scaled difference in perplexities observed from two models. one model, the fine-tuned lm, is fit to a target corpus of text, without any supervision on which passages are anomalies. the other model, the normalizer lm, is the out-of-the-box gpt-2 model @xcite . lm anom. = pplx fine-tuned -β•pplx normalizer . for a mundane piece of text, both pplx fine-tuned and pplx norm. are low. for a non-semantic anomaly, both are high. in both cases, contrastive perplexity is low. however, for a semantic anomaly, we expect pplx fine-tuned to be high, because of its sensitivity to the text's context domain, and pplx norm. to be low, because the text may not otherwise be unusual in general english, leading to high contrastive perplexity. because the fine-tuned lm achieves a lower perplexity, we use β to re-scale the perplexity output of the normalizer and ensure the models operate at the same scale. while β can be tuned as a hyperparameter, a reasonable and balanced choice is the ratio between the mean perplexities achieved by the fine-tuned model and the normalizer model on a validation dataset. ## discussion & conclusion we present a novel contrastive perplexity-based approach for unsupervised anomaly detection. we define semantic and non-semantic anomalies, and present evidence that our model can distinguish between them better than other unsupervised baselines. detecting procedural anomalies in legal cases is easier with structured data, but that data is often not readily available. our approach seeks to support legal decision makers in identifying anomalous cases for review when structured records are unavailable. our experiments on an unexplored dataset of 30,734 parole hearing transcripts have identified troubling cases for review. however, our quantitative evaluations also show the difficulty of defining a semantic anomaly consistently. our results on detecting fake hotel reviews indicate that our approach becomes more powerful when anomalyfree documents are available to perform an out-ofdistribution test. in future work, we seek to use conditional lms to bridge the gap between our unsupervised method and settings in which some structured data is available.