Towards Practical and Knowledgeable LLM s for a Multilingual World: A Thesis Proposal (2025)

Abstract: The frontier of large language model (LLM) development has largely been substantiated by knowledge-intensive tasks specified in English. In this proposed thesis, I argue for the key role that multilinguality occupies in the development of practical and knowledgeable LLMs.First, I consider practical methods to improve LLM’s performance on standard natural language processing (NLP) tasks by leveraging their existing multilingual knowledge.Then, I investigate the underlying multilingual knowledge of LLMs with two benchmarks: on complex reasoning, and on territorial disputes. These benchmarks reveal LLMs’ inconsistent performance across languages. I then design efficient techniques, both at inference-time and training-time, to address these discrepancies. Finally, I extend the territorial disputes benchmark to retrieval-augmented generation (RAG) setting, comparing the effects of different retrieval settings on cross-lingual robustness. My proposal shows that informed use of multilinguality enhances LLMs’ capabilities, and our understanding thereof.

## introduction the vast diversity of languages is both a contemporary and historical reality, with more than 7000 languages spoken throughout the world today @xcite . each language is strikingly different at a surface level, with its own vocabulary, syntax, grammar. however, to quote @xcite , "all known languages are at a similar level of complexity and detail." all languages build meaning in recursive units, from words, to sentences, to discourses. and anything can be expressed as validly in one language as in another. thus, multilinguality serves as a dual lens into human intelligence. first, any human possesses the capacity to, with enough practice and exposure, acquire fluency in any one or more language. second, any language can be used to enable communication in a society. that is, multilinguality demonstrates how knowledgeable individual humans are and serves a practical purpose for societies. if multilinguality comes so naturally to humans, then in our quest to develop machines that possess artificial intelligence (ai) capabilities, then it is also natural that these machines should be able to think in different languages. developers of an advanced ai chatbot would like to adapt their system for different linguistic communities. and users within them would like to access information about current events in their language and preferences. indeed, many of the major advancements in nlp have been substantiated by multilingual concerns. of particular note is machine translation (mt), the task of translating text from one language to another language. mt is a well-defined task with clear use-cases and a lot of data. key to neural language models, has been the introduction of the attention mechanism @xcite , and the transformer model @xcite ; these were first developed with mt as an illustrative task, before researchers soon found that the strong language representations learned here lead to effective models for all nlp tasks. this has led to our current era of large language models (llms), which are large in both their size -over 1 billion parameters -and their datasets -over 1 trillion tokens. despite this, there has been a widespread public sentiment that the current brisk pace of nlp development is leaving behind most of the world's languages, and the people that speak them. from the new york times @xcite to the world economic forum @xcite , articles abound about the phenomenon of the 'linguistic gap.' how do we feel about the state of multilinguality in our field of nlp? certainly, multilinguality has been and remains a primary area of research. taking inventory of conferences run by the association for computational linguistics from 2020-2025, we see that of the 16 conferences that have theme tracks of special interest, 4 directly concern multilinguality 1 -not to mention workshops and other events. still, sentiment on the state of multilinguality among nlp researchers remains mixed. we can thus say that multilinguality has become a primary but parallel concern. the frontier of llm development has largely been substantiated by knowledge-intensive tasks specified in english. only in parallel are multilingual efforts. one approach to building frontier llms is to start by training an english model, then adding multilingual support later. such is the case with the open-weight llms llama-3 @xcite , mistral @xcite , and gemma @xcite , and their multilingual follow-ups llama 3.1, mistral 2, and gemma 2. a second approach is to pursue llm development where multilinguality is considered from the ground up, such as aya @xcite and @xcite . these work well but have been largely relegated to non-english or multilingual use cases. this is because of the popular view is that supporting more languages decreases llm ability in any one of them. in this proposal, i will show that this need not be the case. ## evaluations of llms' knowledge i consider three characteristics of knowledgeability: factuality concerns utilization of factual information, complex reasoning concerns using logic and analytical abilities, and consistency concerns giving similar responses to similar queries. i thus introduce two benchmarks which by design evaluate factuality and complex reasoning. these benchmarks highlight the issues llms have with consistent responses across languages, by eliciting responses for the same underlying queries, but specified in different languages. 2