C ode J udge: Evaluating Code Generation with Large Language Models (2024)

Abstract: Large Language Models (LLMs) have shown promising performance in code generation. However, how to reliably evaluate code generated by LLMs remains an unresolved problem. This paper presents CodeJudge, a code evaluation framework that leverages LLMs to evaluate the semantic correctness of generated code without the need for test cases. We investigate different ways to guide the LLM in performing “slow thinking” to arrive at an in-depth and reliable evaluation. We experimented with four LLMs as evaluators on four code generation datasets and five programming languages. The results show that CodeJudge significantly outperformed existing methods in most settings. Furthermore, compared with a SOTA GPT-3.5-based code evaluation method, CodeJudge achieved better results even when using a much smaller model, Llama-3-8B-Instruct. Our code and datasets are available on GitHub https://github.com/VichyTong/CodeJudge.

## introduction there is an increasing interest in leveraging large language models (llms) to generate code @xcite . however, reliably evaluating llms on code generation tasks remains a challenging problem @xcite . test-based methods, such as measuring pass@k @xcite , rely on manually written test cases to evaluate code quality. this reliance presents a significant limitation, since many tasks do not come with test cases or only have insufficient test cases that miss corner cases @xcite . moreover, it is challenging to write test cases for many coding tasks, such as object serialization and web scraping, since they require extensive effort to construct and configure test stubs and mock objects. when there are no test cases, existing work often relies on token-based metrics, such as bleu @xcite , rouge-l @xcite and @xcite , to evaluate modelgenerated code. however, these metrics do not account for cases where model-generated code is semantically equivalent to the ground truth while having syntactic variations, e.g., using a while loop instead of a for loop, following different naming conventions, etc. in particular, @xcite shows a statistically significant disagreement on code assessment between human judges and these token-based metrics. recent studies show that llms are promising alternatives to human evaluators in different tasks @xcite @xcite . inspired by these findings, we propose an llm-based code evaluation framework called codejudge . codejudge supports two kinds of assessment: (1) determines whether the model-generated code is correct or not, and (2) estimates to what extent the generated code is aligned with user-intended code. while the former is the typical way of evaluating llms in code generation, we argue that the latter provides a more informative evaluation, since llms often generate partially correct code, which provides a good starting point or hints to developers @xcite . thus, it is useful to account for partial correctness and the severity of code errors when evaluating llms for code generation. we design two methods to guide the llm to perform "slow thinking" @xcite for reliable code evaluation. for the first assessment, codejudge instructs the llm to perform a stepby-step analysis of the code functionality and then asks it to summarize the analysis results into a binary decision. for the second assessment, code-judge provides the llm with a taxonomy of common coding errors and instructs the llm to identify what types of errors the generated code contains. then, it computes a code correctness score based on the severity of identified errors. notably, our framework does not require any test cases or any classification task @xcite . the evaluation method f simply determines whether the generated code is correct or not (i.e., f (c, t) ∈ {0, 1}). for instance, test-based evaluation treats the generated code as correct if it passes all test cases. otherwise, the code is considered wrong. recent studies indicate that code evaluation should not be simply treated as a yes-or-no question @xcite . in practice, llms often generate code that is not fully correct, e.g., not handling corner cases, missing one computation step, etc. despite these errors, many developers find the generated code a good starting point compared with writing code from scratch, since they can fix the errors by changing a few lines of code or at least get some inspiration. thus, it would be helpful if the evaluation method f could measure to what extent the generated code deviates from the user-intended code based on the task description (i.e., f (c, t) ∈ r). evaluation without reference code. many evaluation methods assume that the correct code is available as the ground truth so that they can directly compare the generated code with the ground-truth code. all token-based methods, such as codebleu @xcite and code-bertscore @xcite , fall into this category. however, this assumption does not always hold in practice, especially in online settings. in many cases, human programmers can make a good assessment only through code inspection and reasoning based on their programming knowledge, without the need for the ground truth. since llms have been demonstrated as promising alternatives to human judges @xcite @xcite , it is appealing to investigate whether llms can make accurate code assessments without reference code. in this work, we consider code evaluation without references as a special and more challenging evaluation task. challenges. the challenge of code evaluation is two-fold. first, the generated code may have many syntactic variations compared with the correct code while being semantically equivalent, e.g., using different variable names, using a different ordering of independent program statements, using a for loop instead of a while loop, etc. second, there can be multiple alternative solutions for a code generation task. for instance, for the task of sorting integers, there are many different sorting algorithms that have drastically different implementations. in task description: alphabetize letters in each word of a sentence, keeping the words and spaces in the same order. def anti_shuffle(s): return ' '.join([ ''.join(sorted(list(i))) for i in s.split(' ') ]) (a) reference code (e.g., ground truth) def anti_shuffle(s): return ' '.join([ sorted(list(i)) for i in s.split(' ') ]) (b) partially correct code def anti_shuffle(s): pass (c) completely useless code def anti_shuffle(s): return ' '.join([ ''.join(sorted(list(word))) for word in s.split(' ') ]) (d) correct code with syntactic variations def anti_shuffle(s): def sort(word): return ''.join(sorted(list(word))) @xmath0: if s[i] != " ": current_word += s[i] else: word_list.append(sort(current_word)) @xmath1.append(sort(current_word)) return ' '.join(word_list) (e) correct code with a different implementation the following section, we will illustrate these challenges using existing code evaluation methods on a running example in figure 1 . ## codejudge our key insight is to guide llms to perform "slow thinking" @xcite in code evaluation, instead of predicting an evaluation score in one step. we design two methods for the two kinds of code evaluation assessment defined in section 2.1. ## conclusion we propose codejudge , a framework that leverages llms to evaluate code generation without the need for test cases. we demonstrate that by guiding llms to perform slow thinking, code-judge outperforms all existing code evaluation methods. this demonstrates a promising future direction to replace human evaluators with llm evaluators. this is also beneficial for alignment methods that rely on human evaluation as feedback. finally, we release our code and dataset at https://github.com/vichytong/codejudge .