# Automated Reporting of Code Quality Issues in Student Submissions

Oscar Karnalim, Unknown Simon, William Chivers, Billy Susanto Panca

► To cite this version:

Oscar Karnalim, Unknown Simon, William Chivers, Billy Susanto Panca. Automated Reporting of Code Quality Issues in Student Submissions. IFIP World Conference on Computers in Education (WCCE), Aug 2022, Hiroshima, Japan. pp.517-529, 10.1007/978-3-031-43393-1_47. hal-05533337

HAL Id: hal-05533337

https://inria.hal.science/hal-05533337v1

Submitted on 2 Mar 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Automated Reporting of Code Quality Issues in Student Submissions

Oscar Karnalim<fnref n="1" />,<fnref n="2" />✉️, Simon<fnref n="3" />✉️, William Chivers<fnref n="1" />✉️, and Billy Susanto Panca<fnref n="2" />✉️

**Abstract.** Despite its importance in industry, code quality is often overlooked in academia. A number of automated tools to report code quality have been developed but many of them are impractical to use. They either are developed as a standalone tool, require the use of a particular IDE, and/or need historical data. This paper presents code quality issues reporter (CQIS), a tool that can be embedded in an assessment submission system; it identifies code quality issues for each student submission via static analysis, and reports those in an HTML page whose link is sent via email. The tool covers 52 code quality issues specifically curated for academia, 32 for Java and 20 for Python. According to four quasi-experiments with a total of 274 students, students with CQIS are likely to have fewer code quality issues so long as the expected solutions are long and complex and code quality is considered as part of the marking. These students are also more aware of code quality, and readability in particular.

**Keywords:** code quality · static analysis · automation · programming · computing education

## 1 Introduction

The code written by computing undergraduates is expected not only to function well but also to be of high quality [1]; the code should be understandable by itself without the need to read additional documentation or to conduct functional tests. Writing high-quality code is a desirable skill in industry, but is often overlooked in academia.

A number of automated tools to check the quality of one's code have been developed. Many of them are designed for general use and can be applied in both

<footnote n="1">¹ University of Newcastle, University Drive, Callaghan, NSW 2308, Australia
oscar.karnalim@uon.edu.au, william.chivers@newcastle.edu.au</footnote>

<footnote n="2">² Maranatha Christian University, Surya Sumantri Street No. 65, Bandung, West Java 40164, Indonesia
billy.sp@it.maranatha.edu</footnote>

<footnote n="3">³ Callaghan, NSW 2259, Australia</footnote>
industry and academia. Two examples are checkstyle<fnref n="1" />, for Java, and Flake8²,
for Python.

General-purpose tools often cover code quality issues that are not relevant for academia, and give explanations that are somewhat too technical for novices. Consequently, a number of dedicated tools for academia have been developed [2]. Integrating such tools into an existing teaching environment may be challenging as they either require the use of a particular integrated development environment (IDE) [3,4], are not readily integrated in both IDE and assessment submission system [2,5], or need both historical data and manual labeling [6].

In response to this gap, this paper proposes an automated tool, code quality issues reporter (CQIS), which can be integrated in many teaching environments. The tool is embedded in an assessment submission system, and reports code quality issues for each student submission via static analysis. Instructors can thus freely choose the IDE for their students, and are not required to provide historical data and do manual labeling.

CQIS currently works for Java and Python submissions. It identifies 32 Java code quality issues: eight concerning comments and identifier names, and the remainder selected from checkstyle by considering their academic relevance and rewording the explanations. For Python, CQIS identifies 20 code quality issues: eight concerning comments and identifier names, and the remainder selected from Flake8 in the same manner as from checkstyle for Java. CQIS, the code, and the documentation are publicly available<fnref n="3" />.

Our study has two research questions, following the introduction of CQIS:

- RQ1: Do students with CQIS have fewer code quality issues in their submissions?

- RQ2: Are students with CQIS explicitly aware of various aspects of code quality?

## 2 Related Work

While educating students to improve the quality of their code, it is important to identify common code quality issues and focus on them first. Three studies have been conducted partly for that purpose. Keuning et al. [7] identify code quality issues relating to algorithms and structure in a large BlueJ data set of Java programs, and find that many of the issues are related to inefficient code and bad design. Aivaloglou & Hermans [8] identify code quality issues in programs collected from a Scratch repository, and find that more than a quarter of the programs contain both unused and reused code. De Ruvo et al. [9] identify 16 code quality issues that relating to program flow in introductory programming, including unnecessary if-else branches, unused variables, and assigning a variable

<footnote n="1">¹ https://checkstyle.sourceforge.io/.</footnote>

<footnote n="2">² https://flake8.pycqa.org/en/latest/.</footnote>

<footnote n="3">³ https://github.com/oscarkarnalim/CQS.</footnote>
to itself. Half of the programs that they examined have at least two distinct issues.

There are some tools that set out to improve student awareness of code quality by automatically reporting code quality issues. FrenchPress [3], an Eclipse plug-in for Java programs, reports four kinds of misuse covering booleans, fields, loop controls, and public modifiers. The study shows that at least a third of students provided with the tool are motivated to look further into their programs.

WebTA [4] is an IDE-like environment that constantly gives students feedback about their Java programs, covering stylistic issues in addition to common errors and failed tests. The stylistic issues themselves concern five anti-patterns: misplaced code, pseudo-implementation, localised instance variables, ‘knee-jerk’ (pointless use of a recently learnt feature), and repeated resource instantiation. The study does not report the effectiveness of the tool on student perceptions or performance.

Three automated tools are distinct from common IDEs. Refactor Tutor [2] is a web-based tool that specifically educates students about refactoring Python code for better quality. Covered aspects include branching, expressions, loops, and statements. Unique to the tool, the hints are given gradually so that they are not overwhelmed with too much technical detail at once. By analysing student log data [10], the authors show that students with programming background are able to identify and fix code quality issues.

Style++ [5] is a standalone tool to report C++ code quality issues covering comments, identifier names, and code size. Students can voluntarily use the tool before submitting their final programs. The evaluation found that the tool is perceived as beneficial for students. Further, the quality of the submitted programs is somewhat improved.

AutoStyle [6] is an automated tool that is integrated with an assessment submission system. Code quality issues are reported for each submission, and students can resubmit their programs as many times as they wish. To define the set of code quality issues, instructors are expected to provide submissions from a previous offering of the course and to manually label clusters of the submissions.

Some studies consider code quality as part of their analysis. Pettit et al. [11] analyse how students improve the quality of their C++ programs in sequential submissions. Breuker et al. [12] compare code quality of first-year programs with that of second-year programs. Tempero et al. [13] confirm that a count of objects can be useful to assess the quality of student design in object oriented projects.

## 3 The Tool

This paper proposes CQIS, an automated tool to report code quality issues in student submissions. Unlike existing tools, CQIS allows instructors to choose the IDE for students, and does not require yjrm to provide labeled data from a previous offering of the course.

CQIS is embedded in an assessment submission system, and reports code quality issues for each student submission via static analysis. In total, CQIS
covers 32 Java code quality issues and 20 Python code quality issues, and can
provide explanations in both English and Indonesian.

CQIS can be embedded in other systems since it is designed as a library with
console-based arguments. It accepts a student program and generates an HTML
page that reports code quality issues. An example report is shown in Fig. 1.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** CQIS generated HTML page; the text is not expected to be readable; the labels in boxes are added for clarity.</figcaption></figure>

A CQIS HTML report contains four panels. The general guideline panel summarises all code quality rules that have been applied. The student program panel shows the input student program displayed with Google Prettify<fnref n="4" />. The code quality suggestions panel lists code quality issues from the student program, including their hint text and position. If a row is clicked, an explanation of the issue and suggestions for fixing it will be displayed in the detailed explanation panel. Further, the student program panel will locate and highlight the hint text.

The selected Java code quality issues can be seen in Table 1. J01-J04 relate to comments. J01 occurs when a comment is empty or has less than three characters; J02 occurs when a comment contains only common words (e.g., conjunction words), which are typically meaningless on their own. The common words are detected with Apache Lucene<fnref n="5" />. J03 occurs when at least one word is misspelled, as checked by Apache Lucene with two English dictionaries (American<fnref n="6" /> and British<fnref n="7" />) and one Indonesian<fnref n="8" />. J04 occurs when a syntax block has no explaining comment.

<footnote n="4">⁴ https://github.com/google/code-prettify.</footnote>

<footnote n="5">⁵ https://lucene.apache.org/.</footnote>

<footnote n="6">⁶ https://github.com/dwyl/english-words.</footnote>

<footnote n="7">⁷ https://www.curlewcommunications.uk/wordlist.html.</footnote>

<footnote n="8">⁸ http://indodic.com/SpellCheckInstall.html.</footnote>
<table><caption>**Table 1.** Selected Java code quality issues.</caption><thead><tr><td>ID</td><td>Issue</td></tr></thead><tbody><tr><td>J01</td><td>Overly short comment</td></tr><tr><td>J02</td><td>Words in comment not meaningful</td></tr><tr><td>J03</td><td>Misspelled words in comment</td></tr><tr><td>J04</td><td>No comment before or after the first line of syntax block</td></tr><tr><td>J05</td><td>Overly short identifier</td></tr><tr><td>J06</td><td>Words in identifier not meaningful</td></tr><tr><td>J07</td><td>Misspelled words in identifier</td></tr><tr><td>J08</td><td>Inconsistent naming style for identifier</td></tr><tr><td>J09</td><td>Abstract class and prefix 'abstract' in class name inconsistently used</td></tr><tr><td>J10</td><td>Array declaration: brackets used after variable name</td></tr><tr><td>J11</td><td>Inline conditionals</td></tr><tr><td>J12</td><td>Too many boolean operators in expression</td></tr><tr><td>J13</td><td>Lowercased constant attribute name</td></tr><tr><td>J14</td><td>Unexpected declaration order expected order: static variables, attributes, constructors, other methods</td></tr><tr><td>J15</td><td>Empty syntax block</td></tr><tr><td>J16</td><td>Empty catch block</td></tr><tr><td>J17</td><td>Only one empty line needed as separator</td></tr><tr><td>J18</td><td>Empty statement</td></tr><tr><td>J19</td><td>Too many statements in method</td></tr><tr><td>J20</td><td>Overly long line</td></tr><tr><td>J21</td><td>Multiple variable declarations</td></tr><tr><td>J22</td><td>Missing braces in branching or looping statements</td></tr><tr><td>J23</td><td>Overly deep nested looping</td></tr><tr><td>J24</td><td>Overly deep nested branching</td></tr><tr><td>J25</td><td>Overly deep nested try-catch</td></tr><tr><td>J26</td><td>Too many statements per line</td></tr><tr><td>J27</td><td>Non-static attribute access without 'this' keyword</td></tr><tr><td>J28</td><td>Unnecessarily complex boolean expression (except return)</td></tr><tr><td>J29</td><td>Unnecessarily complex boolean return statements</td></tr><tr><td>J30</td><td>String comparison with '=='</td></tr><tr><td>J31</td><td>Too many parentheses</td></tr><tr><td>J32</td><td>Unnecessary semicolon</td></tr></tbody></table>

J05-J08 relate to identifiers. The first three are similar to J01-J03 but are now applied to identifier sub-words, tokenised with a module inspired by a software search engine [14]. The module splits sub-words based on two concatenation styles in naming identifiers: camel case (e.g., 'codeQualityIssueReporter') and underscore (e.g., 'code质量 issue_reporter'). The last issue (J08) occurs when
<table><caption>**Table 2.** Selected Python code quality issues.</caption><thead><tr><td>ID</td><td>Issu</td></tr></thead><tbody><tr><td>P01</td><td>Overly short comment</td></tr><tr><td>P02</td><td>Words in comment not meaningful</td></tr><tr><td>P03</td><td>Misspelled words in comment</td></tr><tr><td>P04</td><td>No comment before or after the first line of syntax block</td></tr><tr><td>P05</td><td>Overly short identifier</td></tr><tr><td>P06</td><td>Words in identifier not meaningful</td></tr><tr><td>P07</td><td>Misspelled words in identifier</td></tr><tr><td>P08</td><td>Inconsistent naming style for identifiers</td></tr><tr><td>P09</td><td>Inappropriate indentation</td></tr><tr><td>P10</td><td>No space after comment prefix (#)</td></tr><tr><td>P11</td><td>Overly long line</td></tr><tr><td>P12</td><td>Multiple statements in one line</td></tr><tr><td>P13</td><td>Unnecessary semicolon</td></tr><tr><td>P14</td><td>Compilation error</td></tr><tr><td>P15</td><td>Unused module</td></tr><tr><td>P16</td><td>Improper use of 'break' or 'continue'</td></tr><tr><td>P17</td><td>Improper use of 'return'</td></tr><tr><td>P18</td><td>Locally undefined identifier</td></tr><tr><td>P19</td><td>Duplicate function parameter</td></tr><tr><td>P20</td><td>Unused local variable</td></tr></tbody></table>

both concatenation styles exist in identifiers and one is more common than another.

J09-J32 are adapted from checkstyle, a general-purpose tool to report Java code quality issues. However, these issues are specifically curated for academia, and undergraduate students in particular. According to our analysis of 15,323 program files from seven distinct programming courses [15], all of these issues except J25 (overly deep nested try-catch) occur in student programs. However, we have still included J25 to accompany the similar issues with other syntax constructs (J23 with looping and J24 with branching).

Selected Python code quality issues can be seen in Table 2. P01-P08 are similar to J01-J08, but applied to Python rather than Java. The others (P09-P20) are adapted from flake8, another general-purpose tool dealing with code quality, but this time for Python. P09-P20 are considered relevant for academia; all of them except P16 (improper use of 'break' or 'continue') are found in the student programs from seven distinct programming courses [15]. P16 is still included, for its similarity with P17.
Two important remarks about CQIS. First, CQIS allows instructors to choose which code quality issues the system will report. Some of them might be subjective and considered acceptable in certain cases. Second, an early version of CQIS that only covers code quality issues in comments and identifier names (J01-J08, P01-P08) has been published elsewhere [16].

# 4 Evaluation

## 4.1 Addressing RQ1: Fewer Code Quality Issues

RQ1 asks whether students with CQIS have fewer code quality issues. This is measured by way of four quasi-experiments each of which compares student code quality issues from two academic semesters of a particular course.

Details of the courses used in the experiments can be seen in Table 3; the control group consists of students without CQIS, and the intervention group of students with CQIS. The first course was offered to third-year undergraduates of an information systems degree while the remaining courses were offered to undergraduates of an information technology degree (third-year, second-year, and first-year respectively). A large proportion of the involved students are males aged 17 to 22. While it would be interesting to analyse the findings according to student demographics, such analysis is not permitted by our ethics approval.

In the first two experiments, the assessments are about developing GUI applications with Java, and code quality is considered as part of the marking. For the next two experiments, the assessments expect console-based programs, and code quality is encouraged but not considered in the marking. The data structures assessment tasks are about implementing and using various linear data structures, whereas the introductory programming assessments are about basic programming syntax and problem solving. Most of the courses have weekly assessments, except the control group's offering of the second experiment, in which each assessment is to be completed in three or four weeks.

<table><caption>**Table 3.** Courses involved in the experiments; the control and intervention columns report the number of students, classes, and assessment tasks.</caption><thead><tr><td>Experiment</td><td>Course</td><td>Language</td><td>Control</td><td>Intervention</td></tr></thead><tbody><tr><td>First</td><td>Business application programming</td><td>Java</td><td>34 stud, 1 cls, 16 asmt</td><td>19 stud, 1 cls, 25 asmt</td></tr><tr><td>Second</td><td>Advanced object oriented programming</td><td>Java</td><td>27 stud, 1 cls, 3 asmt</td><td>47 stud, 2 cls, 21 asmt</td></tr><tr><td>Third</td><td>Data structures</td><td>Python</td><td>46 stud, 2 cls, 14 asmt</td><td>33 stud, 1 cls, 8 asmt</td></tr><tr><td>Fourth</td><td>Introductory programming</td><td>Java & Python</td><td>33 stud, 1 cls 16 asmt</td><td>35 stud, 1 cls 25 asmt</td></tr></tbody></table>
In each experiment the comparison is conducted in three stages. First, student submissions are grouped based on their corresponding assessment task. In offerings that have two classes, one assessment offered to both classes is considered as two separate assessments. Second, for each assessment task, its language-specific code quality issues are identified and counted with a modified version of CQIS. J01-J08 and P01-P08 are not reported since their effectiveness have been discussed in our previous study [16]. Third, average numbers of code quality issues from the two offerings are compared for each issue and for all issues combined. The significance is validated with two-tailed unpaired t-test with 95% confidence level.

Table 4 shows that for the first experiment, students with CQIS have fewer code quality issues in their programs. The assessments are about developing GUI applications; Given that the solutions are usually long and complex, student submissions naturally have many code quality issues that can be reminded by CQIS. Further, students are more encouraged to read the report given that code quality is part of the marking.

<table><caption>**Table 4.** Reported code quality issues with significant and substantial differences for four quasi-experiments; for each experiment, the intervention group used CQIS and the control group did not.</caption><thead><tr><th>Experiment</th><th>Issue</th><th>Mean of control</th><th>Mean of intervention</th></tr></thead><tbody><tr><td rowspan="8">First</td><td>All</td><td>5363</td><td>155</td></tr><tr><td>J10</td><td>27</td><td>0</td></tr><tr><td>J13</td><td>12</td><td>0</td></tr><tr><td>J14</td><td>2345</td><td>11</td></tr><tr><td>J16</td><td>24</td><td>0</td></tr><tr><td>J17</td><td>567</td><td>79</td></tr><tr><td>J19</td><td>43</td><td>0</td></tr><tr><td>J20</td><td>2141</td><td>29</td></tr><tr><td rowspan="3">Second</td><td>J20</td><td>16</td><td>39</td></tr><tr><td>J22</td><td>2</td><td>18</td></tr><tr><td>J31</td><td>17</td><td>2</td></tr><tr><td rowspan="2">Third</td><td>All</td><td>433</td><td>256</td></tr><tr><td>P10</td><td>179</td><td>93</td></tr><tr><td>Fourth</td><td>P10</td><td>23</td><td>6</td></tr></tbody></table>

When analysed per issue, students with CQIS appear to be specifically aware of the expected Java declaration order (J14) where static variables should be declared first, followed by attributes, constructors, and other methods. The number of reported issues is significantly and substantially lowered. They also learn not to put too much code in one line (J20). Students might see both issues reported a lot given that the arise quite frequently.
Five other issues arise less frequently among students using CQIS. They involve brackets after variable name in array declaration (J10), lowercased constant name (J13), empty catch block (J16), missing or unnecessary line separator (J17), and too many statements in one method (J19). All are relatively easy to fix. J19 falls into this category since it occurs mostly in GUI-related methods, where most of the statements do not rely on in-method results.

J09, J23, J25, and J27 do not arise in either the control or intervention groups. Students are already aware that the use of the 'abstract' prefix should be consistent with the use of abstract classes (J09), and that non-static attributes should be accessed with the keyword 'this' (J27). Overly-deep nested looping and try-catch (J23 and J25) are also not expected in these assessment tasks.

For the second experiment, students using CQIS have comparable code quality issues with those not using it: the difference is neither significant nor substantial as it is not reported in Table 4. Although students using CQIS might have greater awareness of code quality, the impact might be diminished by the fact that assessments of the control group have a longer completion period (three or four weeks per assessment), so students in this group have more time to reread and check the quality of their own code prior submission.

Although there is no significant reduction overall, students using CQIS have fewer issues related to unnecessary parentheses (J31). They appear to understand that expressions with unnecessary parentheses are prone to misinterpretation.

It is worth noting that J20 (overly long line) and J22 (missing braces in branching or looping statements) are the other two issues with substantial differences; but these differences are in favor of the control group, students without CQIS. With more limited time in completing assessments, students with CQIS appear not to prioritise fixing these issues; long lines and missing braces do not substantially affect code readability.

Overly-deep nested looping (J23) is the only issue that occurs in neither control nor intervention groups. The assessments do not expect such complex loops as they are mostly about developing business applications.

In the third experiment, students with CQIS generally have fewer code quality issues (see Table 4). This is interesting given that code quality is not specifically considered in the marking of the intervention group. Students with CQIS appear to understand that getting used to writing high quality code might be useful in the future.

When analysed separately, P10 (missing space after comment opening) is the only issue with substantial reduction. This is expected as the issue is common and easy to fix.

In both control and intervention groups from the third experiment, three code quality issues never occur: compilation error (P14), as Python mainly relies on an interpreter; improper use of 'break' and 'continue' (P16), as these keywords are not specifically taught; and duplicate function parameter (P19), as function parameters seldom have similar names.

For the fourth experiment, students using CQIS have comparable code quality issues overall, as Table 4 does not report substantial and significant difference
at the general level. The assessments expect short solutions, and code quality is not considered in the marking. P10 (no space after comment prefix) is the only issue with substantial reduction on students with CQIS. The issue can be fixed easily even without programming knowledge.

There are nine code quality issues that do not arise with either the control or the intervention group. Six of them are related to Java: inconsistency in abstract class naming (J09), inline conditional (J11), lowercased constant attribute name (J13), overly deep nested looping (J23), overly deep nested try-catch (J25), and access to non-static attributes without 'this' (J27). The concepts underlying these issues are not covered in the course materials (J09, J11, J13, J25, J27) or the assessment tasks are too simple (J23).

The other three code quality issues that do not arise are related to Python: compilation error (P14), as that kind of error rarely happens; improper use of 'return' (P17), as functions and methods are seldom used in these assessment tasks; and duplicate function parameter (P19), as each function parameter is expected to have a unique name.

## 4.2 Addressing RQ2: Explicit Awareness of Some Aspects in Code Quality

RQ2 asks whether students are explicitly aware of some aspects of code quality. This is addressed by asking students to list up to three remarkable pieces of information that they learned while using CQIS. The survey was performed at the end of semester in all intervention groups described in Table 3. One hundred and seven students responded to the survey, and together mentioned 18 distinct remarkable pieces of information.

Among the provided information, general knowledge about high quality code is noted by most students (47). This is expected, given that CQIS aims to improve the quality of submitted programs. The second most remarkable piece of information is that programmers should use declarative identifier names (34), variable names in particular. Mistyped names sometimes occur, and while CQIS does not always provide good spelling correction, it can at least report the issue.

The next two remarkable pieces of information are not to write too much code in one line (22) and to use white space for readability (17). While completing assessments, many students focus on how to complete the assessments, and they can easily overlook putting appropriate white space (including line breaks) to enhance readability.

Where to put comments (13), more declarative comments (12), and more efficient syntax (11) are three other remarkable pieces of information that were listed. These are mostly noted by students of business application programming and advanced object oriented programming since in these courses, the use of comments is explicitly encouraged and is considered as part of the marking. Further, due to the advanced level of their course materials, it is more likely that some students will begin by writing poor quality code and will then revise it to improve the quality.
The remaining remarkable pieces of information are noted by only a few students. They are about adequate use of braces and/or parentheses (7); removing unused code (4); expected order of declarations in Java (3); adequate use of semicolons (3); clarity in inline conditionals (2); clarity in complex boolean expressions (2); consistent identifier naming style (2); difficulty of expanding code with low quality (1); need for space after comment prefix (1); need for braces in branching or looping statements (1); and writing one statement per line (1).

We have grouped and analysed the responses for each course. However, since the findings are fairly comparable to the general finding described above, we choose not to report those results.

## 4.3 Discussion

Students using CQIS have fewer code quality issues (RQ1) when the assessments expect long solutions (first and third experiments). The impact can be further improved by designing more advanced assessments and explicitly including code quality in the marking scheme (first experiment). CQIS might be less useful for assessments with long completion periods, given that students have enough time to check the quality of their own code prior to submission (second experiment).

Students who use CQIS observe that their knowledge of code quality is improved (RQ2). They are particularly aware of the need to writing declarative names, to avoid writing too much code in one line, to use white space for readability, to write declarative comments in appropriate locations, and to write more efficient syntax.

# 5 Conclusion and Future Work

This paper presents CQIS, an automated tool to report a selection of code quality issues. The tool can be integrated in many teaching environments by being embedded in some assessment submission systems. It also does not require students to use a specific IDE, or require instructors to provide historical data. According to our evaluation, students using CQIS are likely to have fewer code quality issues in their submissions if the assessments expect long and complex solutions and code quality is considered as part of the marking. Students using CQIS also become more aware of aspects of code quality, mostly pertaining to readability.

The study has two limitations that can be addressed in the future work. First, the evaluation was performed only at one institution, with a moderate number of students. Replicating the study at other institutions with more students might strengthen the findings. Second, student awareness of code quality was addressed only by asking students to list up to three remarkable pieces of relevant information. Some lessons that they have learned might not be reported as they are not in the top three. Future work might list all knowledge aspects of code quality and ask students which aspects they believe have improved.
From the technical perspective, we plan to improve the quality of spelling checking as some of the suggested words do not make sense for students. We also plan to introduce gamification for further student engagement, especially in courses whose assessments do not consider code quality in the marking.

## References

1. Lethbridge, T.C., Leblanc, R.J., Jr., Kelley Sobel, A.E., Hilburn, T.B., Diaz-herrera, J.L.: SE2004: recommendations for undergraduate software engineering curricula. IEEE Softw. **23**(6), 19–25 (2006)

2. Keuning, H., Heeren, B., Jeuring, J.: A tutoring system to learn code refactoring. In: 52nd ACM Technical Symposium on Computer Science Education, USA, pp. 562–568. ACM (2021)

3. Blau, H., Moss, J.E.B.: FrenchPress gives students automated feedback on Java program flaws. In: ACM Conference on Innovation and Technology in Computer Science Education, Lithuania, pp. 15–20. ACM (2015)

4. Ureel II, L.C., Wallace, C.: Automated critique of early programming antipatterns. In: 50th ACM Technical Symposium on Computer Science Education, USA, pp. 738–744. ACM (2019)

5. Ala-Mutka, K., Uimonen, T., Jarvinen, H.-M.: Supporting students in C++ programming courses with automatic program style assessment. J. Inf. Technol. Educ. **3**, 245–262 (2004)

6. Roy Choudhury, R., Yin, H., Fox, A.: Scale-driven automatic hint generation for coding style. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 122–132. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39583-8_12

7. Keuning, H., Heeren, B., Jeuring, J.: Code quality issues in student programs. In: ACM Conference on Innovation and Technology in Computer Science Education, Italy, pp. 110–115. ACM (2017)

8. Aivaloglou, E., Hermans, F.: How kids code and how we know: an exploratory study on the Scratch repository. In: ACM Conference on International Computing Education Research, Australia, pp. 53–61. ACM (2016)

9. De Ruvo, G., Tempero, E., Luxton-Reilly, A., Rowe, G.B., Giacaman, N.: Understanding semantic style by analysing student code. In: 20th Australasian Computing Education Conference, Australia, pp. 73–82. ACM (2018)

10. Keuning, H., Heeren, B., Jeuring, J.: Student refactoring behaviour in a programming tutor. In: 20th Koli Calling International Conference on Computing Education Research, Finland, pp. 4:1–4:10. ACM (2020)

11. Pettit, R., Homer, J., Gee, R., Mengel, S., Starbuck, A.: An empirical study of iterative improvement in programming assignments. In: 46th ACM Technical Symposium on Computer Science Education, USA, pp. 410–415. ACM (2015)

12. Breuker, D.M., Derriks, J., Brunekreef, J.: Measuring static quality of student code. In: 16th Annual Joint Conference on Innovation and Technology in Computer Science Education, Germany, pp. 13–17. ACM (2011)

13. Tempero, E., Denny, P., Luxton-Reilly, A., Ralph, P.: Objects count so count objects! In: ACM Conference on International Computing Education Research, Finland, pp. 187–195. ACM (2018)

14. Karnalim, O., Mandala, R.: Java archives search engine using byte code as information source. In: International Conference on Data and Software Engineering, Indonesia, pp. 1–6. IEEE (2014)
15. Karnalim, O., Simon: Work-in-progress: code quality issues of computing under-graduates. In: IEEE Global Engineering Education Conference, Austria, pp. 1734–1736. IEEE (2022)

16. Karnalim, O., Simon: Promoting code quality via automated feedback on student submissions. In: IEEE Frontiers in Education, USA, pp. 1–5. IEEE (2021)