A Retrieval Model for Automatic Resolution of Crossword Puzzles in Italian Language
Authors: Gianni Barlacchi, Massimo Nicosia, Alessandro Moschitti (Università di Trento, Qatar Computing Research Institute, 2014)
Topic: Language Models

Abstract
In this paper we study methods for improving the quality of automatic ex-traction of answer candidates for an ex-tremely challenging task: the automatic resolution of crossword puzzles for Italian language. Many automatic crossword puz-zle solvers are based on database system accessing previously resolved crossword puzzles. Our approach consists in query-ing the database (DB) with a search engine and converting its output into a probability score, which combines in a single scoring model, i.e., a logistic regression model, both the search engine score and statisti-cal similarity features. This improved re-trieval model greatly impacts the resolu-tion accuracy of crossword puzzles.

Introduction
Crossword Puzzles (CPs) are probably one of the most popular language game. Automatic CP solvers have been mainly targeted by the artificial intelligence (AI) community, who has mostly fo-cused on AI techniques for filling the puzzle grid, given a set of answer candidates for each clue. The basic idea is to optimize the overall probability of correctly filling the entire grid by exploiting the likelihood of each candidate answer, fulfilling at the same time the grid constraints. After several failures in approaching the human expert perfor-mance, it has become clear that designing more accurate solvers would not have provided a win-ning system. In contrast, the Precision and Recall of the answer candidates are obviously a key fac-tor: very high values for these performance mea-sures would enable the solver to quickly find the correct solution. Similarly to the Jeopardy! challenge case (Fer-rucci et al., 2010), the solution relies on Ques-tion Answering (QA) research. However, although some CP clues are rather similar to standard ques-tions, there are some specific differences: (i) clues can be in interrogative form or not, e.g., Capi-tale d’Italia: Roma; (ii) they can contain riddles or be deliberately ambiguous and misleading (e.g., Se fugge sono guai: gas ); (iii) the exact length of the answer keyword is known in advance; and (vi) the confidence in the answers is an extremely important input for the CP solver. Questo e-book appartiene a AlessandroLenci There have been many attempts to build automatic CP solving systems. Their goal is to outperform human players in solving crosswords more accurately and in less time. Proverb (Littman et al., 2002) was the first system for the automatic resolution of CPs. It includes several modules for generating lists of candidate answers. These lists are merged and used to solve a ProbabilisticConstraint Satisfaction Problem. Proverb relies on a very large crossword database as well as several expert modules, each of them mainly based on domain-specific databases (e.g., movies, writers and geography). WebCrow (Ernandes et al., 2005) is based on Proverb. In addition to its predecessor, WebCrow carries out basic linguistic analysis such as Part-Of-Speech tagging and lemmatization. It takes advantage of semantic relations contained in WordNet, dictionaries and gazetteers. Its Web module is constituted by a search engine, which can retrieve text snippets or documents related to the clue. WebCrow uses a WA* algorithm (Pohl, 1970) for Probabilistic-Constraint Satisfaction Problems, adapted for CP resolution. To the best of our knowledge, the state-of-the-art system for automatic CP solving is Dr. Fill (Ginsberg, 2011). It targets the crossword filling task with a Weighted-Constraint Satisfaction Problem. Constraint violations are weighted and can be tolerated. It heavily relies on huge databases of clues. All of these systems queries the DB of previously solved CP clues using standard techniques, e.g., SQL Full-Text query. The DB is a very rich and important knowledge base. In order to improve the quality of the automatic extraction of answer candidate lists from DB, we provide for the Italian language a completely novel solution, by substituting the DB and the SQL function with a search engine for retrieving clues similar to the target one. In particular, we define a reranking function for the retrieved clues based on a logistic regression model (LRM), which combines the search engine score with other similarity features. To carry out our study, we created a clue similarity dataset for the Italian language. This dataset constitutes an interesting resource that we made available to the research community.

Conclusion
In this paper we improve the answer extraction from DB for automatic CP resolution. We combined the state-of-the-art BM25 retrieval model and an LRM by converting the BM25 score into a probability score for each answer candidate. For our study and to test our methods, we created a corpora for clue similarity containing clues in Italian. We improve on the lists generated by WebCrow by 8.5 absolute percent points in MRR. However, the end-to-end CP resolution test does not show a large improvement as the percentage of retrieved clues is not high enough.