Predictive Modeling of Human Developers’ Evaluative Judgment of Generated Code as a Decision Process (2025)

Abstract: The paper presents early results in the development of an approach to predictive modeling of human developer perceiving of code generated in question-answering scenarios with Large Language Model (LLM) applications. The study is focused on building a model for the description and prediction of human implicit behavior during evaluative judgment of generated code through evaluation of its consistency, correctness, and usefulness as subjective perceiving characteristics. We used Markov Decision Process (MDP) as a basic framework to describe the human developer and his/her perceiving. We consider two approaches (regression-based and classification-based) to identify MDP parameters so it can be used as an “artificial” developer for human-centered code evaluation. An experimental evaluation of the proposed approach was performed with survey data previously collected for several code generation LLMs in a question-answering scenario. The results show overall good performance of the proposed model in acceptance rate prediction (accuracy 0.82) and give promising perspectives for further development and application.

## introduction today, large language models (llms) are widely applied in the practice of software development, with both general-purpose solutions like chatgpt by openai and solutions dedicated to code writing such as copilot by microsoft. one of the important capabilities of llms is support of code generation in various scenarios @xcite : bug fixing, code completion, question answering with code snippets, and many others. still, practical implementation of solutions for these tasks reveals several fundamental issues related to the complexity of the software development domain and the specificity of human developers as solution users. an important problem is the evaluation of the solutions. a straightforward approach for llm output evaluation is linguistic metrics such as bertscore, bleu, and others. complex semantics and non-linearity of code structure lead to the development of code-specific metrics such as code-bleu @xcite , ruby @xcite , and others. nevertheless, the real-world application of such metrics shows significant limitation in llm evaluation @xcite . another approach is application of test-based evaluation of generated code with such metrics as pass@k (passing tests with k generated answers). still, application of such an approach remains limited due to the lack of tests and limited applicability of automatically generated tests to the real-world problem. the problem significantly influences the performance of llms in real-world complex projects, which is clearly seen in modern project-based benchmarks like swe-bench @xcite , repobench @xcite , codereval @xcite , etc. the benchmarks show relatively weak performance even for state-of-the-art solutions. one of the best-known solutions for the evaluation problem is evaluation of llms with human-centered metrics like acceptance rate (ar). more complicated approaches may involve value, accuracy, and other human-centered metrics @xcite . still, involvement of human developers requires significant time and effort, with the involvement of multiple human users. another problem is proper understanding of real human developers roles, needs, intents, and expectations. the practical application and surveys of the developers applying llm-based solutions in daily tasks reveal several important issues @xcite @xcite . users often report a lack of personalization, efforts needed to understand generated code, differences between code generated by humans and by llms, etc. as a consequence, this leads to limited trust of developers @xcite , possible vulnerability in generated code @xcite , weak performance in real-world issues @xcite , etc. on the other hand, investigations on interaction with copilot show that proper time to show suggestions @xcite and configuration of interaction patterns @xcite show possible increases in the acceptance rate of suggestions generated by considered intelligent assistants. a key open problem here is understanding how human developers perceive, comprehend, and evaluate code in proper context @xcite . formal structuring of project context is currently approached by many solutions (see, e.g., projectspecific benchmarks mentioned above). but the context of the human mind in evaluative judgment of code generated by both humans and machines is weakly investigated. resolving the mentioned problems (human-level evaluation of code and human developer internal context representation) is limiting many directions in the application of llm to software development. the list of directions benefiting from resolving the problem includes training and fine-tuning llm for better code generation (e.g., with reinforcement learning with human feedback, rlhf); building a complete pipeline for software development @xcite ; improving human developer experience through better selection of available actions in ai agents. in the presented paper, we are focused on approaching the mentioned problems through modeling of human developer perceiving of the code obtained from generative llm. with the data collected in the previous developers' survey, we've modeled key perceiving characteristics that influence developers' actions in code acceptance evaluation. the approach is based on the idea of sequential decision on accepting or rejecting considered information (code). thus, the basic idea of human developer perceiving modeling is formulated as a markov decision process (mdp). the paper presents early, still promising results of the ongoing study in human developer perceiving modeling. the remaining paper is structured as follows. the next section briefly describes problem definition, background for this work, and data collection for the experimental study. following, section 3 presents key elements of the proposed approach to human developer modeling. section 4 presents the results of the experimental evaluation of the proposed approach. finally, the last sections conclude the paper and discuss possible further directions of research. ## problem definition and background works the problem of a human developer perceiving the code generated by an intelligent assistant (ia) such as copilot usually considered within some scenario (e.g. code completion, bug fixing, etc.). the developer posts a query to the ia and perceives the answer. in some cases, the query is proactively posted automatically, depending on the developer activity. ia answers with a block of information containing the answer, suggestions, explanation, etc. within our work, we are narrowing to the scenario where the user asks a question in natural language to ai powered by llm and expects a piece of code as an answer. according to the classification of the codexglue benchmark @xcite , the problem is text-to-code generation. the goal is to build a model for predicting a human developer's subjective evaluation and final ar for code generated with llm as an answer. the examples of practical problems being solved in such a way may be widely found in online forums where users ask questions to resolve some programming issues. one of the most popular forums for the software development domain is stackoverflow 1 (so) which is also widely used as an original source for training and evaluation of llms. a common pattern for such questions is "how to..." (use api, implement algorithm, fix bug, etc.) where an expected answer is a piece of code. as a result, so is widely adopted as a source for datasets and benchmarks building in text-to-code problem investigation: see, e.g., such datasets as conala @xcite , staqc @xcite , so-ds @xcite , etc. within our previous study @xcite , we used the data originating from so and containing queries to fine-tune and evaluate llms. we used two datasets for that purpose. first, we collected 42k pairs of questions (text) and answers (code) from so in "conceptual" and "api usage" classes (according to @xcite ) with answers shown as short code snippets in the python programming language. second, we used the publicly available conala dataset @xcite with 2379 entries of similar structure. we used the datasets for fine-tuning of several llms (codegen, gpt) for further evaluation. both queries and answers collected in the dataset were relatively short: the average lengths of queries and answers were 214 and 154 characters, respectively. next, we performed a survey with human developers evaluating the code generated by the finetuned models. we considered a set of seven different models applied to two datasets. also, for reference, we considered answers generated by copilot as a reference industrial solution. the survey was structured as a sequential evaluation of randomly selected pairs of questions (text) and answers (code). figure 1 shows elements of the user web interface developed for this survey. the evaluation was performed with three criteria inspired by the theory of planned behavior @xcite and includes evaluation of the following subjective perceiving characteristics: the selection of metrics reflects key categories of metrics for subjective evaluation of data quality according to @xcite : accuracy of data, relevancy of data, and representation of data, except for accessibility of data, which is beyond the considered scenario. the evaluation was performed with a 5-level likert scale (from -2 to +2). we collected the evaluations for 614 question-answer pairs from 42 developers of different levels, including msc students in computer science, ai, and mathematical modeling, as well as junior, middle, and senior software developers (mainly working in the area of machine learning, data science). more details on dataset collection, methodology, obtained scores, and dataset analysis can be found here @xcite . the analysis of the previously collected data showed that the human-centered metrics are weakly correlated with the linguistic metrics (including code-specific metrics) like bertscore, codebleu, ruby, etc. on the other hand, the collected metrics are well interconnected and may be considered as filters toward code acceptance. seeing this empirical evidence as a motivation example, we focused on the development of a dynamic model of internal perceiving, evaluative judgment, and acceptance of software code, which is described in the next section. ## modeling human developer perceiving process we propose the following conceptual approach for modeling human developer code perceiving (see figure 2 ). the basic interaction loop involves a human developer posting a query to a code generation model, which answers with a code snippet. we use query q and code context c as arguments that describe the external context of user decisions on answer accepting. in our experiment, q is a natural language request with a short description of a problem to be resolved with generated code, c is code generated by llm as an answer to the query and perceived by a user. next, we consider user-specific information, which may include user profile, code repositories or artifacts authored by the user, personal skills, etc. the idea is to identify robust groups of users with similar perceiving behavior. we can use such information for identification of user personality, groups of similar users (e.g., via clustering @xcite ). the size of such a group can vary from a single user (ultimate personalization) to large communities of groups working in the same area (e.g., "front-end developers"). here, we see human-related characteristics h as representation of internal context. h can be considered as a way of flexible model personalization, where we can select different levels of model generalization (personal model per each developer, group level developer, or general model for a wider group of developers). next, we use q × c × h as an input to the perceiving model identification. the model can be used to predict ar, which, in turn, can be applied to control the code generation model in different ways. we can consider at least three such ways: filtering of the code generation model c 1 can be used to block unwanted and weak suggestions, improving overall user satisfaction of ia use; filtering of queries c 2 can additionally reduce the computational resource load, as running llms multiple times may be costly; internal code generation control c 3 may be applied directly during sequential code generation by blocking or re-weighting candidate tokens. additionally, we can consider offline procedures involved in the approach. first, we consider human feedback as an important source for model identification and parametrization. second, the result model can be used for evaluation of generated code as an "artificial developer" assessing code and providing human feedback to the model (e.g., in the rlhf framework), which may enable significant scaling of the training/fine-tuning process with limited involvement of real human developers. ## conclusion and future works the paper presents early results in the investigation of human developer perceiving of code generated by llm as an answer to an explicit or implicit query. with the mdp-based model, we showed higher performance in predicting acceptance of generated code by human developers. we see the results as promising evidence in the prospective application of structured human perceiving models with implicit internal context involved in the modeling. moreover, the development of such models may be actively involved in practical application for llm control. additionally, we believe that the approach is generalizable and could be applied to different scenarios and various problem domains where the implicit internal context of an expert plays a role. we see several directions for further development of the model and approach in general. first, we consider further development of the proposed approach and detailed investigation of methodological basis we used. for example, we are planning to extend and reconfigure the set of metrics we are using for more detailed representation of diverse metrics considered in the area of subjective information evaluation @xcite . also, we are aimed at the development of more detailed and structured representation of cognitive state and transfers between those states to extend the proposed basic sequential model. within the experimental study, we are going to consider more realistic scenarios of human developer behavior with newly collected datasets or existing project-level datasets like @xcite . we are planning to perform more detailed analysis of internal context embedding space with possible dimensional reduction. for example, we can assess similarity between human developers and try to train a model for unobserved developers with a certain level of personalization. next, we want to implement the mentioned control scenarios in order to increase llm human-centered performance. in particular, the developed model can be considered as a "critic" model in the actor-critic machine learning approach in llm @xcite . finally, we want to evaluate more existing methods from the irl field in order to identify parameters of the proposed mdp.