MPC oder: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learning (2024)

Abstract: Large Language Models (LLMs) have demonstrated great potential for assisting developers in their daily development. However, most research focuses on generating correct code, how to use LLMs to generate personalized code has seldom been investigated. To bridge this gap, we proposed MPCoder (Multi-user Personalized Code Generator) to generate personalized code for multiple users. To better learn coding style features, we utilize explicit coding style residual learning to capture the syntax code style standards and implicit style learning to capture the semantic code style conventions. We train a multi-user style adapter to better differentiate the implicit feature representations of different users through contrastive learning, ultimately enabling personalized code generation for multiple users. We further propose a novel evaluation metric for estimating similarities between codes of different coding styles. The experimental results show the effectiveness of our approach for this novel task.

## introduction nowadays, llms have been successfully used to support developers' daily development, such as code generation, test generation, etc. however, existing code llms are usually general models trained with large programming corpus @xcite , therefore the generated code is difficult to adapt to personalized and/or customized requests. consider the following practical scenarios: alice is a software developer. to improve programmers' daily efficiency, her company provided the base llms that can be used for code generation. nonetheless, different developers/projects have their own coding standards and specifications. if alice needs to generate code satisfying specific conventions, the base llms may fail to capture these nuanced differences. as shown in fig. 1 , alice has to painstakingly revise and review the generated code. if the custom style of the generated code conforms to the standard style of different developers/projects and is correct, it can greatly increase developer productivity @xcite @xcite and reduce code maintenance costs @xcite . recent researchers have explored code generation task by using llms; however, most studies @xcite @xcite focus on generating "correct" code. there is limited research investigating how to generate "personalized" code, especially for multi-user personalization, with no research conducted yet. automatically generating code according to developers' preferences or projects' consistency is a challenging task: (i) considering different programmers have their own coding styles, it is too expensive to fine-tune an llm for each user @xcite . therefore, how to build an efficient model to generate personalized code for multiple users poses a significant challenge. (ii) coding styles are hard to learn and capture. coding styles include different aspects of the code, such as code naming, formatting, and structures. how to distinguish coding style differences between different users and obtain good style representations is another challenge. (iii) coding styles are hard to evaluate. unlike code correctness, which can be evaluated by executing test cases, there is no evaluation metric to estimate coding styles of different code fragments. how to evaluate coding styles quantitatively becomes another challenge for our study @xcite . to tackle the above challenges, we propose a novel approach named mpcoder, which is designed to generate personalized code for multiple users according to their individual coding styles. after training, our model can be easily queried with the id of the user and generate personalized code consistent with his/her desired coding styles. to better capture styles within the raw code, we encode the explicit style features and implicit style features to obtain an effective coding style representation. regarding the explicit style features, we apply the coding style checking tool (checkstyle foot_0 in our study) to detect different coding style attributes explicitly, then the model learn and encode these style attributes by residual learning, which guides the model in identifying the coding style attribute by contrasting two sets of coding style attributes (section 2.2). for the implicit style features, to capture the subtle and unnoticed style differences between different users, we design a multi-user style adapter to further distinguish coding style differences among different users by using contrastive learning (section 2.3). finally, by combining the explicit coding style features and userspecific implicit style features, we can generate the user's personalized code that both contain the syntax and semantic styles of the code (section 2.4). due to the limited prior work on exploring personalized code generation, there is currently no effective way to estimate whether two pieces of code have similar coding styles. in this paper, we propose a novel evaluation metric, coding style score (css), to quantitatively estimate the coding styles between two given codes. in summary, our paper makes the following contributions: firstly, current research mainly focuses on generating correct code, to the best of our knowledge, no prior work explored how to generate personalized code for multiple users. secondly, we develop a novel model, named mpcoder, to learn multi-users' coding style features and generate personalized code. thirdly, we propose a novel evaluation metric for estimating coding styles quantitatively, and additionally, we also release our dataset which contains source code written by multiple users. the experimental results show the effectiveness of our model over a set of baselines, showing its ability to generate personalized code while minimizing the degradation of code correctness. we hope our study can lay the foundations for this research topic and provide valuable insights into the potential for personalized generation capabilities of general llms. ## methodology the coding style can be categorized into syntax style and semantic style, according to the structure and meaning of the code. syntax style refers to the formatting rules of the code (e.g., indentation, spacing, capitalization of variables) which can be easily defined @xcite . on the other hand, semantic style refers to the use of language features and constructs to convey intent (e.g., design patterns, and meaningful names of code) which is hard to precisely define in language @xcite . both syntax and semantic style are important for making the code more readable and maintainable. details of the syntax and semantic coding style differences can be found in appendix b.3. to capture these two types of coding styles, as illustrated in fig. 2 , we propose a novel approach mpcoder, which utilizes explicit coding style learning (section 2.2) to capture the syntax style standards pre-defined by industry and implicit coding style learning (section 2.3) to capture the semantic style that is learned from the code itself. moreover, a multi-user style adapter (section 2.3) is trained to estimate the style probability distribution for multiple users. after two stages of training , mpcoder can generate personalized code for multiple users simultaneously(section 2.4). ## related work code pre-trained language models. with the latest developments in the transformer-based model @xcite , recent work has attempted to apply llm to code to advance software engineering and code intelligence. code-bert @xcite pretrains the nl-pl data. codet5 @xcite leverages the t5 (raffel et al., 2020) architecture to leverage code semantics through identifier tokens. llm @xcite has made many achievements in the field of natural language processing. the openai codex @xcite model with 12b parameters pioneered and demonstrated the potential of large code generation models pre-trained on billions of lines of public code. subsequently, models dedicated to code generation emerged, such as codellama @xcite and codegeex @xcite . personalized generation. most of the existing work in personalized generation focuses on attribute-based controlled text generation @xcite , such as emotions and topics @xcite @xcite . the text-description-based approach @xcite focuses on promoting character consistency through pre-trained language models. the embedding-based utilizes user id information @xcite or embedded user dialogue @xcite history as an implicit profile. within the domain of personalized code generation, existing approaches @xcite involves fine-tuning for specific software projects, providing java unit tests for a single coding style, while we focus on providing coding styles for multiple users. ## conclusions this research aims to generate personalized code for multiple users to satisfy the coding styles of different developers or projects. to perform this novel task, we propose an approach mpcoder which utilizes explicit coding style residual learning to capture the syntax style standards and implicit coding style learning to capture the semantic style of each user. we propose a multi-user style adapter to bridge the gap between the generic outputs and user-specific personalized outputs. mpcoder can ultimately generate personalized code for multiple users simultaneously. the experimental results show the effectiveness of our for this task. we hope our study can lay the foundations for this new research and provide valuable insights into the potential for personalized generation capabilities of general llms.