# Evaluation of Diversification Techniques for Legal Information Retrieval +
Footnote †: _Algorithms_  
[PERSON]  
1 Knowledge and Database Systems Laboratory, Division of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, Greece; [EMAIL_ADDRESS]  
[PERSON]  
2 Department of Computer Science and Biomedical Informatics, School of Sciences, University of Thessaly, Papassiopoulou 2-4, 35131 Lamia, Greece; [EMAIL_ADDRESS]  
Telin: +30-210-7721602  
multiple search facilities, so as to assist users to find the information they need. For instance, the user can perform simple search operations or utilize predefined classificatory criteria, e.g., year, legal basis, subject matter, to find relevant to her/his information needs legal documents. At the same time, the legal domain generates a huge and ever-increasing amount of open data, thus leveraging the awareness level of legislation stakeholders. Judicial ruling, precedents and interpretations of laws create a big data space with relevant and useful legal resources, while many facts and insights that could help win a legal argument usually remain hidden. For example, it is extremely difficult to search for a relevant case law, by using Boolean queries or the references contained in the judgment. Consider, for example, a patent lawyer who want to find patents as a reference case and submits a user query to retrieve information. A diverse result, i.e., a result containing several claims, heterogeneous statutory requirements and conventions, varying in the numbers of inventors and other characteristics, is intuitively more informative than a set of homogeneous results that contain only patents with similar features. In this paper, we propose a novel way to efficiently and effectively handle similar challenges when seeking information in the legal domain. Diversification is a method of improving user satisfaction by increasing the variety of information shown to the user. As a consequence, the number of redundant items in a search result list should decrease, while the likelihood that a user will be satisfied with any of the displayed results should increase. There has been extensive work on query results' diversification (see Section 4), where the key idea is to select a small set of results that are sufficiently dissimilar, according to an appropriate similarity metric. Diversification techniques in legal information systems can be helpful not only for citizens, but also for law issuers and other legal stakeholders in companies and large organizations. Having a big picture of diversified results, issuers can choose or properly adapt the legal regime that better fits their firms and capital needs, thus helping them operate more efficiently. In addition, such techniques can also help lawmakers, since deep understanding of legal diversification promotes evolution to better and fairer legal regulations for society [1]. In this work, extending our previous work [2], we address result diversification in the legal information retrieval (IR). To this end, in parallel with the search result diversification methods utilized in our previous work (maximal marginal relevance (MMR) [3], Max-sum [4], Max-min [4] and Mono-objective [4]), we extended the methods utilized by incorporating methods introduced for text summarization (LexRank [5] and Biased LexRank [6]) and graph-based ranking (DivRank [7] and Grasshopper [8]). We present the diversification methods utilized with an algorithmic presentation, alongside with a textual explanation of each diversification objective. We evaluate the performance of the above methods on a legal corpus annotated with relevant judgments using metrics employed in TREC Diversity Tasks, fine-tuning separately for each method, trade-off values between finding relevant to the user query documents and diverse documents in the result set.