On Tables with Numbers, with Numbers (2025)

Abstract: This paper is a critical reflection on the epistemic culture of contemporary computational linguistics, framed in the context of its growing obsession with tables with numbers. We argue against tables with numbers on the basis of their epistemic irrelevance, their environmental impact, their role in enabling and exacerbating social inequalities, and their deep ties to commercial applications and profit-driven research. We substantiate our arguments with empirical evidence drawn from a meta-analysis of computational linguistics research over the last decade.

## introduction throughout its evolution, computational linguistics has undergone multiple identity crises. in its present from, and despite its logical origins and linguistic ambitions, it is almost entirely aligned with positivist principles and ideals @xcite . the imprint of this alignment is an idealization of experimental quantification, most commonly manifesting in the form of tables with numbers. tables with numbers can certainly be useful. that said, their centrality in contemporary computational linguistics research is indicative of both scientific reductionism and technological obsession. beneath the numbers lie signs of a field in disarray: a waning reliance on theory (linguistic or otherwise), nowadays substituted by model scale; a disproportionate representation of big industry and big academia, in turn associated with a lack of transparency, accessibility and inclusion; an experimental paradigm dominated by stagnant "taskand-benchmark" practices, detached from technical rigor as well as scientific insight; and a progressive estrangement from societal, humanistic and environmental context. and while the community seems to be both alert to and uneasy with the current state of affairs @xcite , a holistic analysis of these issues has been long missing from the literature. in this paper, we brave a look under the number rock. we conduct a critical assessment of the epistemic culture of computational linguistics, focusing specifically on its relation to tables with numbers. we narrow down on four axes of interest: ## the multiple facets of number the field's dominant scientific approach embodies a wildly exaggerated version of positivism. this is evident both in the themes prevalent in the mainstream discourse, and in those notably absent from it. in this context, two critical perspectives arise. first, how faithfully does computational linguistics actually adhere to its positivist posture? and second, what are the implications of computational linguistics as a singularly positivist discipline? we begin by addressing the former, setting off with a simplified introduction to the positivist worldview and its tenets. ## resource exhaustion as the field is witnessing a constant influx of progressively larger models, each vying for supremacy over increasingly more challenging benchmarks, tables are growing in both size and count; see fig. 1 . meanwhile, the numbers within are getting more resource-intensive by the day @xcite . as a result, the environmental footprint of contemporary research is expanding at an alarming rate @xcite . ## institutional bias & privilege besides environmental concerns, keeping up with contemporary research trends comes at a (literal) heavy price. as the cost of the "state of the art" explodes at a super-exponential rate @xcite @xcite , inter alia), the severe budget inequalities in higher education become further pronounced (o'sullivan, 2016; goyes and skilbrei, 2023), and the minimum requirements for scientific relevance becoming prohibitively high for smaller and lesser-funded institutions to acquire and maintain @xcite ; see also fig. 2 . consequently, a few dominant institutions get to consolidate their competitive advantage by effectively gatekeeping the means necessary to conduct exactly the kind of research that is perceived as groundbreaking and impactful @xcite . this is problematic on multiple levels. ## ways ahead the paradigm shift advocated for might seem radical or untenable. in reality, it is neither. the epistemic rewiring it calls for can be set in motion with as little as individual adjustments in research consumption and production attitudes. as readers, we need to stop allowing ourselves to be dazzled by big numbers. we must ask what their utility and cost are, who benefits from them, and who bears their expense. we should not only grow resilient to hollow benchmarking hypes, but also openly refute and disarm them. as authors, colleagues and advisors, we have to be conscious of our (and each other's) research goals and practices. we ought to look beyond numbers and benchmarks and focus on what questions our research really answers. we must challenge the notion of science as a competition or enterprise, and scorn endeavors that depend solely on experimental superiority to be deemed successful. we must be mindful and explicit of the resources we use and their accessibility, but also of the artifacts we produce and their inclusivity. above all, it is our responsibility to be vocal and assertive about the issues in our field; despite -or rather in spite ofnormative resistance and calls for conformity and "moderation". as reviewers, we should each recognize our respective academic privileges, and be cautious in our technical demands; not everyone has access to the same number of gpus. conversely, we should not be intimidated by big tables and bold face fonts; we need to be critical of the research we are exposed to, and call out opaque methodologies, exclusionary practices and useless flourishes. finally, our exclusive access to the reviewing process means it is our own duty to monitor it; each one of us has a role in identifying and confronting poor practices. ## conclusion we discussed tables with numbers, and related them to several issues that affect contemporary computational linguistics research. we argued that the focus on experimental superiority has shifted research priorities towards technical optimization, at the expense of theoretical depth and societal context. this has led to an inflationary effect on the epistemic value of experimental results, rendering them (and, by extension, the field itself), increasingly meaningless. we explained how the pressure for experimental superiority, while advancing technology, has fostered environmental degradation, institutional biases, and the commodification of research. to address these issues, we urge the field to critically reassess its methodologies, and prioritize a more holistic and socially responsible approach to scientific inquiry, balancing technical achievements with ethical and environmental considerations. such a shift is essential for ensuring that advancements in computational linguistics positively contribute to scientific knowledge, societal well-being, cultural diversity, and environmental sustainability.