NLP for Consumer Protection: Battling Illegal Clauses in G erman Terms and Conditions in Online Shopping (2021)

Abstract: Online shopping is an ever more important part of the global consumer economy, not just in times of a pandemic. When we place an order online as consumers, we regularly agree to the so-called “Terms and Conditions” (T&C), a contract unilaterally drafted by the seller. Often, consumers do not read these contracts and unwittingly agree to unfavourable and often void terms. Government and non-government organisations (NGOs) for consumer protection battle such terms on behalf of consumers, who often hesitate to take on legal actions themselves. However, the growing number of online shops and a lack of funding makes it increasingly difficult for such organisations to monitor the market effectively. This paper describes how Natural Language Processing (NLP) can be applied to support consumer advocates in their efforts to protect consumers. Together with two NGOs from Germany, we developed an NLP-based application that legally assesses clauses in T&C from German online shops under the European Union’s (EU) jurisdiction. We report that we could achieve an accuracy of 0.9 in the detection of void clauses by fine-tuning a pre-trained German BERT model. The approach is currently used by two NGOs and has already helped to challenge void clauses in T&C.

## introduction nlp, and technology more broadly, has improved the access to knowledge in many domains. it is no longer necessary to pay thousands of dollars for a lexicon like the encyclopaedia britannica or to hire a translator to understand texts in other languages. the legal domain is arguably one of the biggest resistance to digitisation efforts. while, in some aspects, it still struggles to catch up with other industries, technology has started to change the landscape of legal service provision. so far, consumers rarely benefit from this development. on the contrary, mostly big companies and law firms benefit. most of the existing so-called "legaltech" tools, like lexis advance foot_0 , lexical labs foot_1 , and anvi foot_2 , to name just a few, are tailored to the needs of companies and law firms, rather than consumers and consumer protection agencies. thereby, legal-tech tools are not only missing the opportunity to democratise access to legal advice, by making it more affordable and available, they are actively increasing the current imbalance of power between companies and consumers. in this paper, we describe, how we apply nlp technology to automatically assess clauses in german t&c from consumer online shops, to find void clauses and help to protect consumers from them. unlike the, relatively little, existing work (see section 2), we focus on organisations that represent consumer interests as users. by focusing on such organisations, rather than individual consumers, we hope to be able to increase the impact of our work. while tools for individual consumers usually only benefit those who are using them, consumer protection agencies legally challenge void t&c they find, forcing their change and hence benefiting all consumers. we also believe that the task of ensuring that companies adhere to consumer contract and distance selling laws should not be left to consumers alone. ## related work as mentioned before, the existing research in the area of the legal analysis of t&c focuses on individual consumers as users. the project "terms of service; didn't read" (tos;dr) from @xcite uses crowd-sourcing to provide manually generated summarisations of the tos from many major online platforms, like facebook and twitter. however, the fact that tos;dr is crowd-sourced affects the scalability and topicality of the project. the satos project (software-aided analysis of terms of services) @xcite @xcite automatically summarise and assess t&c for consumers using dependency parsing and other rule-based approaches, however, only covering a few selected aspects of t&c. claudette is a project at the european university institute @xcite @xcite @xcite which focuses on the detection of unfair clauses in terms of the legislation of the eu. originally focused on terms of services from tech giants like netflix, google, microsoft, and snapchat, claudette now mainly focuses on the analysis of privacy policies. since the introduction of the general data protection regulation (gdpr) in the eu, the interest in the analysis of privacy policies has increased in general (see e.g. @xcite and torre et al. ( 2020 )). ## the role of ngos in consumer protection the folk wisdom that being right does not automatically lead to getting justice is specifically true for the area of consumer protection, where there is regularly a strong imbalance of power between the involved parties, a single consumer on one side and a potentially large corporation on the other side. in acknowledgement of this fact, many legislators have given ngos in the area of consumer protection special and extensive rights to assist and represent consumers and their interests. at the same time, consumer advocates and consumer protection agencies are chronically underfunded in many countries. with their limited financial means, consumer advocates all over europe struggle to keep up with the demand generated by the increasing importance of digital offerings. in 2018, the consumer protection agencies in germany received in total 184,579 complaints from consumers. 65,370 of these complaints (more than 35%) were related to digital offerings. in comparison, only 36,945 complaints (20%) were received about products and services from the financial industry (verbraucherzentrale bundesverband e.v., 2019). in addition to providing individual counselling to consumers, consumer advocates increasingly try to monitor (digital) markets proactively and react to negative developments before consumers are harmed. monitoring markets as big as ecommerce and proactively act against void clauses in standard form contracts is, at scale, simply not possible without automation of the underlying processes. for the work presented in this paper, we collaborated with two consumer protection ngos from two different german states, which are mainly funded by the government and enjoy special privileges when it comes to taking legal actions on behalf of consumers. we worked with five legal experts from these organisations over a period of three years, from 2017 to 2020. ## data corpus building a corpus for the automated legal assessment of t&c is far from trivial. on the one hand, we want to have a realistic distribution of clauses in our corpus, with regard to their legality and topics, on the other hand, we need a sufficient number of void clauses in order to be able to train statistical classification models. if we would only use complete t&c, we would need thousands of contracts to find a sufficient number of void clauses. ## approach the bert language model @xcite has been shown to be effective on a wide range of tasks in the legal domain, including named entity recognition @xcite , annotation of legal concepts @xcite , and evidence retrieval @xcite . additionally, there is a pre-trained german language model available "bert-base-german-cased" @xcite that was trained, among other sources, on a large corpus of legal texts. it is trained on cased german texts and, like the original bert model, has 12 hidden layers with a size of 768, 12 attention heads per attention layer, and 110 million parameters. the model was trained on the german wikipedia and a web corpus gathered by @xcite , which account for more than 90% of the data the model was trained on. however, the model was also trained on the open legal data set from @xcite , which consists of more than 100,000 german court decisions. ## evaluation we used the huggingface transformers library @xcite to fine-tune the pre-trained language model with our data set on the binary classification task of deciding whether a clause is void or not. we split our corpus into a training (80%) and a test set (20%) and first perform a stratified five-fold cross-validation on the training set to identify the best performing hyper-parameters for the fine-tuning. we started our search with the values suggested in the original bert paper: batch size 16 or 32, learning rate 5e-5, 3e-5 or 2e-5, and 2, 3 or 4 epochs @xcite . however, the authors also note that the optimal hyper-parameters are task-specific and that small data sets (which they define as less than 100,000 labels) are more sensitive to the choice of parameters than larger ones, therefore we also tried a smaller batch size (8) and higher numbers of epochs @xcite @xcite . in the end, we found that batch size 16, learning rate 3e-5, and three epochs performed best. with these hyper-parameters, we evaluated the approach on our test data set, which consists of 237 clauses, of which 192 are valid and 45 are void. bert performed very well in the classification of void clauses and achieved an accuracy of 0.9, as well as a precision and recall of 0.9. out of the 45 void clauses in the test data, only four clauses have falsely not been identified as void (false negatives). since our approach is meant to be a support tool for experts, all results will be doublechecked by a human expert, which makes a high recall desirable. a deeper analysis of the results showed that, while some types of clauses, as mentioned before, are virtually always void in the data set, others are virtually never. this might have (positively) influenced the classification performance. ## ethical and societal implications the goal of this work is to support consumer advocates in order to further consumer protection and address the imbalance of power between corporations and consumers. while these are, by most standards, worthy and ethical goals, just because something is well-intended does not mean it can not have critical or at least ambivalent consequences. in this section, we want to highlight some of the issues that can arise from the research presented in this thesis and the goals it pursues. the laws governing t&c are changing comparably fast. for small companies, without in-house legal counselling, it can therefore be expensive and challenging to keep up with the changing legislation and keep t&c always up to date. in such cases, honest mistakes might be made in drafting and maintaining t&c which do not intend to harm consumers. nevertheless, such mistakes can make companies vulnerable to cease-and-desist orders from competitors and organisations which specialise in sending out cease-and-desist orders, not in order to protect consumer interests but for personal financial benefit. therefore, we choose organisations to collaborate with that are dedicated to consumer protection and bound to that aim by their statute and their state given mission. however, it can not be prevented that our research can also be used by less wellintended actors. while this poses a potential threat, it can also allow companies on the other side to use our results in the same way on their own t&c and hence make sure they match the rule of law. a second, arguably more philosophical issue that arises, not just from our research, but from the perspective of consumer-focused legaltech in general, is whether our legal system is prepared for lowering the bar for accessing the system. the legal and moral standpoint on this issue is quite clear. the charter of fundamental rights of the eu guarantees in article 47 that "everyone whose rights and freedoms guaranteed by the law of the union are violated has the right to an effective remedy before a tribunal". while the legal situation is clear, it is also clear that there are, in fact, barriers in place which make access to justice harder, whether they are of financial or procedural nature. and while it could be denied that they purposefully do so, it is difficult to deny that these barriers help to keep up the in many countries already stretched legal systems. if we would be able to denounce our neighbours by the click of a button every time they disturb the nighttime, this could not just have implications for the viability of our legal systems but also for the kind of society we live in and how we interact with each other. concerning our work, we would argue that, if it has any influence on the legal system at all, it is designed to reduce its load. while the number of cease-and-desist orders sent out by consumer advocates might rise, we would hope that subsequently, this would lead to fewer cases brought on by consumers about void clauses in t&c. finally, if a system that automatically t&c for their lawfulness would be successful and widely adopted, one of the implications would very likely be that companies could start trying to "gamble" the system. this is a phenomenon that can be observed very well in the area of search engine optimisation @xcite and security @xcite . this could potentially lead to a situation where such a system would mostly fail to detect clauses that were purposefully drafted in a consumer-aversive way and would potentially be left detecting mostly clauses that are unintentionally void, e.g., by honest mistake, and were never intended to harm consumers. if we can learn anything from search engine optimisation and security, then that there is no easy or permanent fix to such problems. we, therefore, try to build our system in a way that it can be easily adapted, so that consumer advocates can change the system in a way that it will be able to detect such clauses, once they became aware of it, entering an "arms race" with malicious companies. and while "security through obscurity" is generally discouraged, search engine providers have shown that obfuscating the exact criteria helps to stay ahead of attempts to manipulate the ranking of websites. therefore, our decision to focus on consumer advocates as users, rather than consumers themselves, can also help to mitigate the problem since companies will not be able to directly test different versions of their clauses. ## conclusion in this paper, we have given an example of how nlp can be used to further the goal of consumer protection and address the existing imbalance of power between consumers and companies. we have argued that, in order to support consumers as broadly and effectively as possible, one should not (only) target individual consumers as potential users, but rather target organisations that represent consumers and their interests and have the power and means to pursue legal battles. together with experts from consumer protection agencies, we labelled a corpus of more than 1,100 german clauses from t&c from online shops with regard to their lawfulness. we showed that the labelling process already generated an impact on consumer protection, by enabling consumer advocates to send cease-and-desist orders against clauses that were identified as void and by providing new insights to consumer advocates, e.g. about the average share of void clauses in t&c.