A Simple, Yet Effective Approach to Finding Biases in Code Generation (2023)

Abstract: Recently, high-performing code generation systems based on large language models have surfaced. They are trained on massive corpora containing much more natural text than actual executable computer code. This work shows that current code generation systems exhibit undesired biases inherited from their large language model backbones, which can reduce the quality of the generated code under specific circumstances. To investigate the effect, we propose the “block of influence” concept, which enables a modular decomposition and analysis of the coding challenges. We introduce an automated intervention mechanism reminiscent of adversarial testing that exposes undesired biases through the failure modes of the models under test. Finally, we demonstrate how our framework can be used as a data transformation technique during fine-tuning, acting as a mitigation strategy for these biases.

## introduction large language models (llm) have recently demonstrated their ability to generate code @xcite @xcite or solve challenging programming/math tasks on par with human coders @xcite @xcite ; these models are trained with the data-driven paradigm. on the other hand, an increasing body of work also questions whether the data-driven approach leads to acquiring reasoning skills @xcite @xcite , showing that if left alone, it might not be sufficient for achieving truly human-level performance on tasks such as logical or visual reasoning. in many studied cases, models still rely on various hints in their reasoning process. this work extends the results above, i.e., the lack of reasoning capabilities, to the code generation domain. more specifically, we devise a framework that automatically identifies subtle cues a code generation model might exploit. changes or removal of those cues stands as a reasoning test towards the generational capabilities of the model at hand. we presume that the reasoning process of code generation models should remain invariant under changes that still provide enough context or pose little, if any, additional challenge to a human coder. to this end, we propose an automatic and modelagnostic framework that modifies the following: (1) function names, (2) keywords in a problem specification, and (3) examples provided in the problem prompt. we refer to these parts as blocks-of-influence; see figure 1 . each block contributes partially to the context needed for correct completion. we show that minor modifications of these blocks are sufficient to "fool" llm-based code generation methods. our results reveal biases such as keyword preference and memorization effects, which can be identified across multiple models. during our experiments, we ensure that any modifications maintain the global semantics of the coding challenge. this is achieved through a context-aware filtering mechanism that guarantees any information altered or removed still exists and/or can be deducted from the remaining unaltered part. contributions. the main contributions of our work can be summarized in three points. first, we propose a novel automated framework that identifies possible biases in code generation models. our framework removes subtle hints, introducing minimal changes such as keyword replacement or partial code-block omission, ultimately acting as an adversarial test. since the framework operates on a data level, it is agnostic to the model's structure and internal workings. the framework can be easily adjusted to any input format or programming language. second, we introduce the "blocks of influence" concept. we suggest that every instance of a typical coding challenge can be analyzed into three parts (blocks). each part is correlated with a different method of hinting and is used as a target of our transformations. a model's reasoning process is informed by all three blocks, making them perfect analyzing tools for cases of failing code generation. third, we explore new ways of mitigating biases during code generation. in section 6, we study the effects of adversarial training against our proposed perturbations, and the benefits of including examples with longer descriptions during fine-tuning. our results show that combining these techniques leads to more accurate code completions. ## related work our approach is inspired by works of various research directions, which we briefly describe here. solving coding and math challenges. the emergent abilities of large language models to generate, summarize and translate textual information, have recently sparked interest in their aptitude for math, logic, and programming challenges. tasks such as code-completion @xcite @xcite , code summarization and code translation @xcite have been proposed, with models constantly progressing towards near-human performance. similarly, @xcite @xcite have proposed tests measuring a model's ability to perform math and logic, ranging from school problems to competition-grade challenges. impressive results in multiple programming languages have also been achieved by decoder-only works @xcite . @xcite created the first generative model to perform infilling using a novel masking objective. finally, massive-scale models such as @xcite demonstrated breakthrough capabilities in language, reasoning, and code tasks achieving state-of-the-art performance in multiple domains simultaneously. social biases in large language models. trained on ever-increasing amounts of publicly available data, large language models have been studied for adopting social biases commonly found among humans. @xcite show that generative models can be conditioned to produce toxic content, with the use of nonsense, adversarial prefixes. similarly, @xcite suggest that models might adopt biases and social stereotypes found among their training data and provide ways to ap-ply fairness during generation. countermeasures have been proposed by @xcite , claiming that sanitized zero-shot examples contribute to mitigating biases during generation. ## benchmarks in this section, we describe the datasets used in our experiments. we employed widely used coding challenges humaneval (he) and mbpp and a more complex dataset with lengthy descriptions of problems (dmcc). more information about the datasets can be found in the appendix 10.2. humaneval (he). this is a human-curated problem-solving dataset described in @xcite . it consists of 164 original programming challenges assessing language comprehension, alright: we demonstrate three possible transformations, one for each block: swap the function name with "func", remove keywords, and remove examples. transformations can be applied alone or in combinations of two as described in section 5.2 gorithms, and simple mathematics. each problem is presented as an incomplete function, accompanied by a docstring. the docstring contains the task and a few example cases. for each task, we are provided with a set of unit tests. a task is considered solved when all unit tests are passed. ## evaluation models. in our experimental setup, we test five models representing different approaches to code generation. codeparrot @xcite comes with an open-source dataset and can be easily used for fine-tuning experiments due to its size. its smaller variant (110m) achieves competitive results to other open-source llms at larger parameter budgets. by exploring its dataset, we tested our hypothesis that function names act as biases during code generation. models can be heavily inspired by similarly named snippets in their training set and resort to copying whole or parts of the solution instead of performing reasoning. (see appendix a.9) we also test the incoder @xcite model, which is trained under a novel bi-directional causal objective, being able to handle context more efficiently than its causal counterparts. against our initial hypothesis, our methods cause significant performance drops despite the model's enhanced context-understanding capabilities ( despite not being a code generation model, it performs equally well with code-specific models in the same parameter budget. theoretically, bias effects can be reduced when a model is exposed to diverse training examples. our experiments reveal that this is still not the case under our setup, and post-training solutions are explored. codegen @xcite is a high-performing model trained in natural language understanding and code. we test its mono variant, further fine-tuned on the python language. finally, we have the powerful codex model, which can tackle most of the proposed coding challenges in the humaneval and mbpp datasets. a list of the tested models, as well as keybert @xcite that is used in our framework, can be found in ## limitations some limitations and possible research directions exist in our work. our study focuses on the python3 programming language, with many coding challenges existing in different popular choices (e.g., c, c++, java, scala). although the blocks of influence identification mechanism could be easily adapted to each case, an off-the-shelf application of our framework in another language would lead to errors. similarly, the framework assumes that each coding challenge will be in a "competition-style" format, meaning that a proper problem description, in-docstring examples, and each input types are present for each example. in appendix section 10.1, we present how an adaptation to less formatted codebases would be possible, but for now, we leave it as a future investigation. finally, there is no guarantee that the improved performance against the suggested perturbations reflects an equivalent performance increase in realworld code assistant applications. real-time coding suggestions and completions that are more user aligned are out of the scope of this work. ## risks and ethical considerations our research aims to discover and remove biases in code-generation scenarios through adversarial intervention. however, we acknowledge that insecure or malicious code can still be generated after finetuning with our suggested augmentations. furthermore, our work is focused only on cognitive biases that affect the reasoning and logic behind the coding process of large language models. social biases and stereotypes can still appear when general-purpose llms such as codex or bloom are used in typical text generation scenarios. signs of robustness against our methods are not to be confused with indicators of other forms of biases not existent. regarding the finetuning experiments of