Intrinsic Plagiarism Detection and Author Analysis by Utilizing Grammar

With the advent of the world wide web the number of freely available text documents has increased considerably in the last years. As one of the immediate results, it has become easier to find sources that serve as the basis for plagiarism. On the other side, it has become harder for detection tools to automatically expose plagiarism due to the huge amount of possible origins. Moreover, sources may even not be digitally available, resulting in an unsolvable problem for such tools, whereas experienced human readers might find suspicious passages based on an intuitive style analysis. In this thesis, intrinsic plagiarism detection algorithms are proposed which operate on the suspicious document only and circumvent the problem of incorporating external data. The main idea is thereby to analyze the style of authors in terms of the grammar that is used to formulate sentences, and to expose significantly outstanding text fragments according to the syntax, which is represented by grammar trees. By using a similar style analysis, the idea has also been applied to the problem of automatically assigning authors to unseen text documents. Moreover, it is shown that grammar also serves as a distinguishing feature to profile an author, namely to predict his/her gender and age. Reusing all previous analyses and results, the idea has finally been adapted in order to be used to automatically detect different authorships in a collaboratively written document. Zusammenfassung Die Anzahl an frei verfügbaren Textdokumenten ist in den letzten Jahren aufgrund des enormen Aufschwungs des Internets erheblich gestiegen. Eine der Konsequenzen ist, dass Quellen für mögliche Plagiate leicht gefunden werden können, während es auf der anderen Seite für automatische Erkennungstools aufgrund der großen Datenmengen immer schwieriger wird, Plagiate zu erkennen. Zudem sind Quellen oft nicht in digitaler Form vorhanden, was für Tools, die auf Vergleiche mit bekannten Dokumenten basieren, ein unlösbares Problem darstellt. Andererseits können geübte menschliche Leser verdächtige Passagen oft über eine intuitive Stilanalyse ausfindig machen. In dieser Arbeit werden verschiedene Algorithmen zur intrinsischen Plagiatserkennung entwickelt, welche ausschließlich das zu prüfende Dokument untersuchen und so das Problem umgehen, externe Daten heranziehen zu müssen. Dabei besteht die Grundidee darin, den Schreibstil von Autoren auf Basis der von ihnen verwendeten Grammatik zur Formulierung von Sätzen zu untersuchen, und diese Information zu nutzen, um syntaktisch auffällige Textfragmente zu identifizieren. Unter Verwendung einer ähnlichen Analyse wird diese Idee auch auf das Problem, Textdokumente automatisch Autoren zuzuordnen, angewendet. Darüber hinaus wird gezeigt, dass die verwendete Grammatik auch ein unterscheidbares Kriterium darstellt, um Informationen wie das Geschlecht und das Alter des Verfassers abzuschätzen. Schlussendlich werden die vorherigen Analysen und Resultate verwendet und so adaptiert, dass Anteile von verschiedene Autoren in einem gemeinschaftlich verfassten Text automatisch erkannt werden können. Acknowledgements ”First of all, I have to say”, that a comprehensive work like this thesis can obviously not be completed without the continuous help of many others. Because I am usually one of the kind that has some problems expressing gratitude especially in a private environment this is a perfect opportunity for me to thank those who guided, assisted, advised, motivated, mothered and also distracted me at the right moments in order to complete this thesis and as a fact of which I’m proud of: in time! Every thesis needs a relaxing but yet productive environment to be developed and written in: and this was provided by the DBIS group. At first I want to thank my advisor Günther Specht for giving me the opportunity to write the thesis, but also for always having an open ear for any emerging problems and providing the mentioned environment, leaving space for developing and following ideas. Moreover, I want to thank all DBIS members I work/ed with: Domi, Doris, Eva, Gabi, Martin, Michael, Niko, Peter, Robert, Seppi, Sylvia, Wolfi thank you for always having time for helping me in any matter, discussing scientific and also non-scientific topics. Undoubtedly the biggest thank goes to my wife Claudia for her loving care and never-ending support: not only for the time this thesis was developed and written, but basically for all decisions I made and for all life-changing directions I went in the last decade. You steadily supported me as much as possible while always keeping an eye on the maintenance of our relationship, which includes taking the necessary steps like suggesting a spontaneous mindfreeing short trip. I also want to thank my son Noah (you’re exactly one year at this time) especially for the ”distraction”-part: one cannot think about a thesis improvement while changing diapers or playing guitar and singing children’s songs. Consequently, you forced me to work even harder and more efficiently during office times thanks! I also want to thank my family: my parents, my brothers and my parents-inlaw for providing me a 24/7 open and solid home, helping me in many matters and for generally having an open ear for anything. To my grandparents: thank you for the many nice and endless discussions, and also for cooking regularly since the beginning of my study: you waited long and patiently, but you finally have a ”Doctor” within your family congratulations! A special thank goes to my uncle for continuously giving me interesting popular-scientific input and also for the time spent reading and commenting many of my publications.