A Major Obstacle for NLP Research: Let’s Talk about Time Allocation! (2022)

Abstract: The field of natural language processing (NLP) has grown over the last few years: conferences have become larger, we have published an incredible amount of papers, and state-of-the-art research has been implemented in a large variety of customer-facing products. However, this paper argues that we have been less successful than we *should* have been and reflects on where and how the field fails to tap its full potential. Specifically, we demonstrate that, in recent years, **subpar time allocation has been a major obstacle for NLP research**. We outline multiple concrete problems together with their negative consequences and, importantly, suggest remedies to improve the status quo. We hope that this paper will be a starting point for discussions around which common practices are – or are *not* – beneficial for NLP research.

## introduction why did i get nothing done today? is a question many people ask themselves frequently throughout their professional careers. psychologists agree on good time management skills being of utmost importance for a healthy and productive lifestyle @xcite @xcite . however, many academics and industry researchers lack time management skills, working long days and getting not enough done -not even the interesting experiment they had wanted to start over a year ago. in this position paper, we argue that natural language processing (nlp) as a field has a similar problem: we do not allocate our time well. instead, we spend it on things that seem more urgent than they are, are easy but unimportant, or result in the largest short-term gains. this paper identifies the largest traps the authors believe the nlp community falls into. we then provide, for each of the four identified problems (p1-p4), suggested remedies. while we know that -just as for individuals -change takes time, we hope that this paper, in combination with the emnlp 2022 special theme open questions, major obstacles, and unresolved issues in nlp, will ignite critical discussions. related work over the last couple of years, multiple papers have provided critical reflections on the state of affairs in nlp research: bender and koller (2020) criticizes the hype around language models and argues, similarly to @xcite , that true understanding is impossible when language is detached from the physical world. in contrast, @xcite talks about the risks associated with underclaiming. turning to evaluation, @xcite provides a critical view on benchmarking, and @xcite proposes ways to improve leaderboards in order to truly track progress. other position papers discuss the importance of data curation @xcite and the need for focusing on the user for natural language generation @xcite . @xcite identifies general concerning trends in nlp research. @xcite discusses our use of the term multimodality and proposes to use task-specific definitions of multimodality in the machine learning era. @xcite discusses downward trends in reviewing quality and whether these can be mitigated. we add to those meta-level papers by discussing subpar use of time as a major problem. ## conclusion in this paper, we outlined how several problematic practices in nlp research lead to a waste of the most important resource we have -our timeand, thus, constitute major obstacles for nlp research. we suggested multiple possible solutions to existing problems. we hope to foster much-needed discussion around how we, as a community, envision moving forward in the face of these concerns.