Here is the future work part of a faculty job application, which paragraphs (state first words only) would most benefit from improvements? 
Consider improvements in style, clarity, concreteness, vision and in general writing and content for a proffessor considering this application:

My future work will keep the above spirit of open and community-driven development of LLMs and AI. In the future, I plan to pursue open, modular and efficient pretraining and post-training the evaluation and understanding to support that, and metascience to allow others to do the same.

While my interests in \textbf{NLP} came from language, cognition and learning, my focus shifted to the latter. Perhaps the most prominent place to study it today is pretraining. I plan to pursue questions around \textbf{collaborative and modular pretraining} such as what training regimes create generalization and shared representations, bootstrapping pretraining with existing models outputs or weights, could pretraining accommodate replaceable experts, adaptable memory, or other submodules developed separately by a community.  I will also research \textbf{metascience} methodological questions on how to make informed decisions in pretraining, for example through training small models, approaches to scaling laws, continual pretraining, and community efforts. To exemplify the difficulties of such research let's consider small model approaches. For relevant and trustworthy results, such approaches require ways to tell if large models would also benefit from the solutions and ways to train and evaluate less-performing models on the tasks that matter -- to the large models' development.

The second stage of LLM building, alignment through \textbf{Human feedback} is the engine of progress behind closed models and is all but nonexistent in open research. Following that I gathered a thinking group from 15 institutions with a diverse set of expertise to discuss what is needed for open human feedback to be openly gathered \citep{DonYehiya2024TheFO}. I  see this direction as a crucial part of the future of open model development and my research. Accordingly, I currently lead an initial effort to build it with 2 MIT students and (mainly) Hugging Face collaborators, co-lead a joint effort of 30 IBM researchers to build foundations for this (as additional duty) and an RPI-IBM 200k\$ collaboration grant proposal under review. A core idea for open feedback collection is a platform where specialized models are created and updated like Wikipedia articles, collaboratively by the relevant community. A dedicated model will discuss the topic of choice (e.g., Cancer, Pokemon, or Bengali) and learn from the communities' feedback given, creating a feedback loop. The overall data will be shared openly to allow further research and general model development. This opens countless research avenues including fast improvement from feedback, interactivity, automatic training, iterations (known to be used in closed models but without any published research \citep{dubey2024llama}), types of feedback, communal feedback governance/moderation and use of contradicting feedback, diversity and accessibility, licensing and legal issues. All of which I aim to pursue in an open and collaborative way. 

I also intend to study more about the \textbf{science of deep learning}, complementing the methods with the necessary insights and sparking ideas. This is a vast field striving to understand what models learn (\textbf{interpretability}) and to characterize learning. Specifically, I am interested in analyzing and allowing generalization \citep{zhang2024unforgettable,DBLP:conf/acl/AkyurekACWA24}, how same and different knowledge might be represented in models and in loss landscapes \citep[following e.g.,;][]{DBLP:conf/coling/DinKCG24,knots,ifergan2024beneath,DBLP:journals/corr/abs-2311-07682} and how knowledge is acquired throughout training leading to the final representations that are more often studied \citep{hild,DBLP:conf/icml/HacohenCW20,DBLP:conf/acl/ChoshenHWA22}. Some key challenges this implies are how to compare different models and levels of representations (e.g., weights, outputs, activations, projected dimensions), how different abilities are learned and interact (e.g., knowledge, reasoning, syntax, multilingualism) and when complex behaviors appear.

A related part of understanding and guiding progress is evaluation. I plan to keep addressing the needs in \textbf{evaluation} as they arise and change. In the short term, focusing on prompt sensitivity and efficient methods to evaluate pretraining. Along those lines, we currently collaborate to create a 1B outputs dataset of model evaluation scores across tasks, prompts, models and other aspects, this dataset would allow analysis research on those questions without excessive costs for the researchers. In addition, we plan to release a call for proposals and extend the dataset to contain missing information the community requests. To do so, we plan to hold a shared task and evaluation workshop on the topic. 

An undercurrent to my work that I will continue pursuing is supporting scientific communication and collegiality between researchers of different statuses, institutes, backgrounds, and scientific goals. While this is not a research agenda per se, it is leading some of my research choices and sometimes has the potential to aid science more than any single paper, in other times, it helps a human being. It led me to publish with almost 200 researchers to date (according to DBPedia), to give a hand to many who reached out from diverse backgrounds (from AlphaXiv undergrads developers to STLM Singapore/Thai independent researchers \citep{stlm}) and needs (career, social and scientific advice). I am actively endorsing others’ research,  through a critical scientific lens, sharing it on 4 platforms. In this, I aim to set scientific norms (e.g., new knowledge first, affiliation last). I plan to continue supporting open science and building such bridges, wherever I can. Concretely, beyond continuing the above, I advise open science and nanopublication efforts, as well as collaborate on writing an open license for evaluation data which takes LLMs training on it into account. I will continue supporting similar efforts in the future.