Enhancing LLM Capabilities Beyond Scaling Up (2024)

Abstract: General-purpose large language models (LLMs) are progressively expanding both in scale and access to unpublic training data. This has led to notable progress in a variety of AI problems. Nevertheless, two questions exist: i) Is scaling up the sole avenue of extending the capabilities of LLMs? ii) Instead of developing general-purpose LLMs, how to endow LLMs with specific knowledge? This tutorial targets researchers and practitioners who are interested in capability extension of LLMs that go beyond scaling up. To this end, we will discuss several lines of research that follow that direction, including (i) the adaptation of LLMs to assimilate new information in situations where conflicts arise, (ii) the adaptation of LLMs to address target problems with inherent constraints, (iii) the customization of LLMs to align with user-specific instructions and preference, (iv) the defense against potential attacks and threads by malicious users, and (v) the collaboration with external models directly or through APIs. At last, we will conclude the tutorial by outlining directions for further investigation.

## introduction the advancement of ai can be broadly attributed to two technical trajectories: one involving generalpurpose models, and the other centering around task-specific models. in the earlier phases of deep learning and even before its inception, the focal point of research predominantly revolved around the integration of domain-specific and task-specific expertise into model architectures. nonetheless, the landscape underwent a transformation with the advent of pretrained large language models (llms), e.g., bert @xcite and gpt series @xcite . recent years have witnessed substantial achievements of those general-purpose models in a variety of ai problems. however, the advancements facilitated by llms are primarily rooted in larger scales of model parameters and confidential training data. these factors make llms increasingly costly, uninterpretable, unreproducible, uncontrollable, and unmanageable for most users. consequently, while acknowledging the substantial benefits offered by llms, it becomes crucial to address several pertinent inquiries. firstly, does the path to enhancing llms' capabilities solely involve scaling up? the resource-intensive nature of training large-scale llms prompts the exploration of potential bottlenecks and the feasibility of further expansion. secondly, despite llms' versatility, challenges persist in their application to specific disciplines, tasks, and even users. thus, strategies to augment llms' capabilities for these distinctive challenges warrant consideration. this tutorial delves into some research lines that extend the capabilities of llms beyond mere scale amplification. specifically, it presents a comprehensive analysis of this objective, identifying challenges across five key dimensions: optimizing llm inputs, enhancing llm responses, updating llms' internal knowledge, maximizing supervision from the target task, and improving llm trustworthiness. in line with these dimensions, the tutorial will address recent advancements in: (i) prompt optimization ( §2.2), (ii) llm self-improvement and inter-llm collaboration ( §2.3), (iii) adapting preexisting knowledge to integrate new, potentially conflicting information ( §2.4), (iv) aligning llm performance with the constraints and structures of target problems ( §2.5), and (v) defending against adversarial threats and malicious attacks ( §2.6). we believe it is necessary to present a timely tutorial to comprehensively summarize the new frontiers in llm capability extension research and point out the emerging challenges that deserve further investigation. participants will learn about recent trends, emerging challenges, and representative tools in this topic, and how related technologies benefit end-user nlp applications. ## outline of tutorial content this half-day tutorial presents a systematic overview of recent advancements in extending llms' capabilities without scaling up. the detailed contents are outlined below. ## specification of the tutorial the proposed tutorial is considered a cutting-edge tutorial that introduces new frontiers in llm capability extension beyond scaling up its size and data. the presented topic has not been covered by any * cl tutorials in the past 4 years. ## tutorial instructors the following are biographies of the speakers. past tutorials given by us are listed in appx. §a.1. wenpeng yin is an assistant professor in the department of computer science and engineering at penn state university. prior to joining penn state, he was a tenure-track faculty member at temple university (1/2022-12/2022), senior research scientist at salesforce research (8/2019-12/2021), a postdoctoral researcher at upenn (10/2017-7/2019), and got his ph.d. degree from the ludwig maximilian university of munich, germany, in 2017. dr. yin's research focuses on natural language processing with three sub-areas: (i) nlp/llm for scientific research, (ii) human-centered ai, and (iii) multimodal learning. additional information is available at www. wenpengyin.org . muhao chen is an assistant professor in the department of computer science at uc davis, where he directs the language understanding and knowledge acquisition (luka) group. his research focuses on data-driven machine learning approaches for natural language understanding and knowledge acquisition. his work has been recognized with an nsf crii award, two amazon research awards, a cisco faculty research award, an emnlp outstanding paper award, and an acm sigbio best student paper award. muhao obtained his phd degree from ucla department of computer science in 2019, was a postdoctoral researcher at upenn, and worked as as assistant research professor of computer science at usc prior to joining uc davis. additional information is available at http://luka-group.github.io . rui zhang is an assistant professor in the computer science and engineering department of penn state university and a co-director of the psu nlp lab. his overarching research goal is to build natural language interfaces for efficient information access and knowledge sharing including summarization for unstructured documents, question answering for semi-structured web tables and pages, and semantic parsing for structured knowledge. he has led a tutorial on con-trastive data and learning for natural language processing at naacl 2022. he is the co-organizer of several workshops including suki at naacl 2022, mia at naacl 2022, and intex-sempar at emnlp 2020. additional information is available at https://ryanzhumich.github.io/ . ben zhou is an assistant professor in the school of computing and augmented intelligence at arizona state university. ben's research uses data and symbolic cognitive processes to improve model reasoning, controllability, and trustworthiness from learning/inference schemes and architectural perspectives. he has more than 10 recent papers on related topics. ben obtained his ph.d. degree from the university of pennsylvania. he is a recipient of the eniac fellowship from the university of pennsylvania and a finalist for the cra outstanding undergraduate researcher award. additional information is available at http://xuanyu.me/ . fei wang is a ph.d. student in the department of computer science at university of southern california. his research interests lie in natural language processing and machine learning. his recent work focuses on enhancing the trustworthiness of llms with dynamic knowledge integration and robust alignment. fei is a recipient of an amazon ml fellowship and an annenberg fellowship. additional information is available at https://feiwang96.github.io/ . dan roth is the eduardo d. glandt distinguished professor at the department of computer and information science, upenn, the chief ai scientist at oracle, and a fellow of the aaas, acm, aaai, and acl. in 2017, roth was awarded the john mccarthy award, the highest award the ai community gives to mid-career ai researchers. roth was recognized "for major conceptual and theoretical advances in the modeling of natural language understanding, machine learning, and reasoning." roth has published broadly in machine learning, nlp, krr, and learning theory, and has given keynote talks and tutorials in all acl and aaai major conferences. roth was the editor-in-chief of jair until 2017, and was the program chair of aaai'11, acl'03 and conll'02; he serves regularly as an area chair and senior program committee member in the major conferences in his research areas. additional information is available at www.cis.upenn.edu/~danroth .