# Mitigating Accidental Code Plagiarism in a Programming Course Through Code Referencing

Muftah Afrizal Pangestu, Unknown Simon, Oscar Karnalim

► To cite this version:

Muftah Afrizal Pangestu, Unknown Simon, Oscar Karnalim. Mitigating Accidental Code Plagiarism in a Programming Course Through Code Referencing. IFIP World Conference on Computers in Education (WCCE), Aug 2022, Hiroshima, Japan. pp.615-626, 10.1007/978-3-031-43393-1_55. hal-05533316

HAL Id: hal-05533316

https://inria.hal.science/hal-05533316v1

Submitted on 2 Mar 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Mitigating Accidental Code Plagiarism in a Programming Course Through Code Referencing

Muftah Afrizal Pangestu<fnref n="1" />[✉]®, Simon<fnref n="2" />[ ID ], and Oscar Karnalim<fnref n="1" />,<fnref n="3" />[ ID ]

**Abstract.** Code plagiarism – taking code from external sources and using it without reference in one's own programs – can be a serious issue for programming students, depending on the policies being applied by their instructors. However, plagiarism can be inadvertent, due to a lack of knowledge among students. Our research shows varied understand- ings of correct code reuse, suggesting that students are not provided with appropriate guidelines. Our goal is to introduce good code referenc- ing practice to students, to help raise students' awareness of academic integrity and reduce the possibility of accidental plagiarism. We present Corona, a code referencing system that can assist students in creating references for their code while simultaneously educating them about eth- ical code reuse. Technical evaluation of the system shows that Corona can successfully generate references for code taken from 20 of 24 distinct programming assistance websites, and that it can find matches between students' code and instructors' example code and generate appropriate references. Our research in a small-scale environment suggests that the use of Corona as a demonstration tool in a lecture about code referencing increases student awareness of correct referencing practice. To improve our intervention, we also show steps that lecturers can take to further elevate students' engagement in code referencing.

**Keywords:** code referencing · code similarity detection · comment generation · academic integrity · programming · computing education

## 1 Introduction

Much research has been conducted into detecting similarities in programming students' code [1], with the intention that human checking will then evaluate the similarities and determine whether it constitutes plagiarism [2].

Code reuse is an important aspect of programming education. Lecturers may actively encourage students to find and reuse appropriate code as this is a practice that is both common [3] and helpful [4] in software development. However,

<footnote n="1">¹ University of Newcastle, University Dr Callaghan, Newcastle, NSW 2308, Australia
{muftah.pangestu,oscar.karnalim}@uon.edu.au</footnote>

<footnote n="2">² Newcastle, Australia</footnote>

<footnote n="3">³ Maranatha Christian University, Bandung, Jawa Barat 40164, Indonesia</footnote>
the legitimacy of code reuse in the learning process may not carry over to the
assessment process: unacknowledged code reuse in an assessment task may con-
stitute plagiarism [5]. Simply finding and copying the code of others can actually
impede the student's learning process [6], and when marking an assessment task,
educators are required to assess the students' own work.

Preventing plagiarism is a step beyond detecting it [2]. Just as with written
assignments in other disciplines of study, some plagiarism can be averted by
teaching students correct referencing practice and asking them to implement it.
But what is a correct referencing standard for program code? We have found
two proposed standards, but neither appears to be in broad use [7].

Both of the referencing standards we have found take the form of inline code comments; they include several fields of information, making them error-prone and tedious to apply. This process can potentially be eased by automating the comment creation process.

This research proposes an approach that helps to raise students’ awareness
of accidental plagiarism through the explicit introduction of code referencing
practice. We introduce Corona (COde Reference autONomous Assistance), a
largely automated referencing system for programming students to help them
avoid accidental plagiarism and adopt good code referencing habits following
the standard proposed by Simon et al. [7]. We also explain how to help to build
code referencing habits by encouraging students to participate in code reuse
practice in a way that requires them to create references for externally sourced
code and for collaborative assistance.

Corona can help students by creating a standardised reference for any assis-
tance received on their assignments, thus reducing the likelihood of accidental
plagiarism. Furthermore, information included by Corona in the reference, such
as the purpose, modifications to copied code, and type of assistance given, can
also help to make clear the students’ own contribution and understanding when
completing their assignments.

## 2 Related Work

Despite the wide acceptance of code reuse in programming, there are no gen-
erally accepted standards for referencing the sources of reused code, and there
are calls for that to change [7]. One code referencing standard is proposed by
Massachusetts Institute of Technology in its academic integrity handbook [8]
and has been adopted by several other universities. The guideline specifies when
and how to reference externally sourced code, making use of inline comments
in the code. Another standard, proposed by Simon et al. [7], addresses not only
references for external code (such as from websites or books) but also other
forms of assistance received while programming. Again, the information is to be
included in an inline comment. When referencing external code, it requires the
programmer to specify purpose, date, source, author, URL and modifications
made to the reused code, if any.

Various works have addressed the automatic generation of comments for code
[9], mainly because of the importance of code comments and the effort required to
make complete and helpful comments. Yang et al. [10] have classified a number of approaches to achieving this goal, one of which, template based methods, generally involves two instances of code: the user's code and the template code. A code similarity detection method is employed to match the user's code with the relevant part of the template code, and a predefined comment in the template code is then applied to its paired user code.

Rahman et al. [11] and Wong et al. [12] use code from the answers in Stack Overflow<fnref n="1" /> as the template code, and the description and title given by the code's author as the comment. This information is mined in advance from Stack Overflow and stored in a repository, and a code similarity detection approach is used to compare the user's code with the code in the repository.

In academic computing, code similarity detection is geared toward detecting collusion among students by finding similar code passages in their assignments. One approach is text and token matching, which compares the structures of code segments in either raw or tokenised form. Cordy and Roy [13] apply this approach, using code normalisation as a preprocessing step and longest common subsequences (LCS) for similarity counting.

## 3 Corona

We present Corona, a system that helps users to create references for externally sourced code or assistance. Corona uses the code referencing standard of Simon et al. [7] because it provides more detailed information than that of MIT [8] and because it includes a standard for referencing assistance received. Corona provides two different ways to generate a reference in the form of an inline comment: manually, by providing a form that gathers all the required information; and automatically, by using a template based code similarity detection approach on the code at a URL provided by the user. By introducing Corona to students, we can facilitate the code referencing process for them and also help them to learn correct ethical practice for using externally sourced code or assistance, thus helping to reduce plagiarism.

Corona is built as a standalone web app, so it can be used by all students regardless of the IDE that they use. However, it also provides a web service, which will enable future improvements such as a desktop app or IDE plugins. Both the user website and the web service are built using the Python Flask framework<fnref n="2" />. The way the system works can be divided into three cases, which will be explained in following paragraphs.

When creating a reference for externally sourced code, users first input their own code, either as raw text or as a file upload. They are then asked to indicate the external ‘target’ code. They can copy and paste the target code – for example, if it comes from a book or their previous assignments – or they can input the URL of the website where they found the target code. If they provide a URL, Corona will gather all code from the website, along with further selected information,

<footnote n="1">¹ www stackoverflow com.</footnote>

<footnote n="2">² https://flask.palletsprojects.com/en/2.0.x/.</footnote>
using either web scraping or an API request. API requests are employed for
particular websites that are popular sources for code reuse, such as Github and
Stack Overflow [14], while web scraping is a generalised approach that aims to
extract code and supporting information from any website. At the next step,
Corona shows a number of snippets from the target code, ranked by how similar
they are to the user's code. The user can then choose which code snippet they
want to use as the target code. This selection process is helpful when creating a
reference for code found on a site such as Stack Overflow, where a single page
usually has many code snippets provided by different authors.

Next, users are asked to indicate which part of their own code they copied
from the target. Code similarity detection is used again, this time to rank the
user's code snippets according to how similar they are to the selected target
code. We adapted the approach of NiCad [13], which employs longest common
subsequences (LCS). After both user's and target code have been selected, the
system generates a reference based on the information taken from the target
website. Users can fine-tune the reference by modifying it or adding further
information. For example, users will need to add information such as the reason
for copying the code and any modifications they have made to the code. Finally,
Corona displays the complete reference, which the user can copy and paste into
their code.

When creating a reference for code that is not drawn from a website, users
manually input the information in a form provided by Corona. There is still
some benefit, in that Corona makes it clear what information is required, and
forms that information into a valid inline code comment.

Using a similar form, users can also create a reference for any assistance they
have received in their work.

If users want to quickly create a reference for external code without going
through the process of selecting the most appropriate matching code segments,
there is also an option for them to manually complete all required fields, after
which Corona generates the reference for them to copy and paste into their code.

Because it uses a text-based method to compare code, Corona is not limited
to code written in a specific programming language, and has been tested on sev-
eral commonly used languages. Figure 1 shows an example of a Python reference
produced by the system and ready for copying to the user's code.

4 Educating Students in Use of the System

One of the biggest problems of code referencing is that it generally has to be
done voluntarily, at the students' own initiative, so students' participation can
be minimal. In this section, we suggest steps to actively encourage students to
participate in code referencing practice.

The lecturer can point out that code reuse is a common practice in profes-
sional software development [3] and proven to be helpful [4]. The same applies to
collaborative work, which is common in software development. These are there-
fore good habits for programming students to acquire. Lecturers can also argue
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** An example of a reference generated by Corona and ready for copying to the user's code</figcaption></figure>

for the importance of code referencing as it helps students to create good documentation for their code, which is a necessary skill for a programmer. With this in mind, the lecturer can design an assignment where students are required to adapt a set amount of external code to solve a certain problem. The lecturer can even provide the external code – perhaps for a standard algorithm such as sorting or searching. Students should quickly come to see that the reference helps both them and the lecturer remain aware of which code they have adapted from elsewhere and which code they have written themselves.

To stimulate the creation of references for assistance given in students' coding, we can encourage students to interact with one another in limited and clearly specified ways when working on an assignment. References for this activity will clearly indicate the nature and extent of assistance given by other students.

It is important to give students a clear guideline of acceptable code reuse practice in each course. This may include such topics as: what are acceptable sources of external code; when is a reference required; how much modification is required to the copied code; and how much assistance is allowed to be accepted from another person. Our initial evaluation shows that students are still confused about these questions, as seen in Sect. 5.

## 5 Evaluation

This section explains how Corona has been and will continue to be evaluated, and presents the findings of the evaluation that has so far been conducted. It is separated into several sections based on different aspects of the functionality. These sections will include evaluation methods that we have already applied and those that we plan to apply in the future.
## 5.1 Evaluating the Web Scraping

To evaluate our web scraping method, we selected several websites that we have confirmed to contain programming code. We used Google to search for “code tutorial” along with other words such as Python, Java, Javascript, C#, and C++. These searches led us to 24 freely accessible code tutorial websites. We then used our system to scrape all code from a page of each website and manually compared the scraped result with our own inspection of the page in question. We found that our web scraping system successfully scraped all code snippets from 20 of the 24 websites, a success rate of 83%. Table 1 shows the 24 sites, along with an indication of which were successfully scraped. We conclude that our system works well with most tutorial websites that students may use, but that there are a few websites that use distinct ways of displaying program code that are not covered by our generalised web scraping approach.

<table><caption>**Table 1.** Websites on which we tested our web scraping approach, with an indication of whether the approach was successful</caption><thead><tr><th>Website</th><th></th><th>Website</th><th></th><th>Website</th><th></th></tr></thead><tbody><tr><td>pypi.org</td><td>✓</td><td>javatpoint.com</td><td>✗</td><td>developer.mozilla.org</td><td>✓</td></tr><tr><td>w3schools.com</td><td>✓</td><td>codecademy.com</td><td>✓</td><td>javascripttutorial.net</td><td>✓</td></tr><tr><td>docs.python.org</td><td>✓</td><td>docs.microsoft.com</td><td>✓</td><td>datacamp.com</td><td>✗</td></tr><tr><td>learnpython.org</td><td>✓</td><td>csharp.net-tutorials.com</td><td>✗</td><td>kaggle.com</td><td>✗</td></tr><tr><td>tutorialspoint.com</td><td>✓</td><td>cplusplus.com</td><td>✓</td><td>flask.palletsproject.com</td><td>✓</td></tr><tr><td>programiz.com</td><td>✓</td><td>learncpp.com</td><td>✓</td><td>realpython.com</td><td>✓</td></tr><tr><td>guru99.com</td><td>✓</td><td>beginnersbook.com</td><td>✓</td><td>fullstackpython.com</td><td>✓</td></tr><tr><td>docs.oracle.com</td><td>✓</td><td>javascript.info</td><td>✓</td><td>laravel.com</td><td>✓</td></tr></tbody></table>

## 5.2 Evaluating the Code Similarity Detection

To evaluate the code similarity detection we checked for similarities between students' code assignments and the code examples given by lecturers. We chose an Algorithms and Data Structures course, in which students are taught how to implement data structures such as queues and stacks using Python lists. The learning materials provided by lecturers include code for creating and working with those data structures. Students are then given assignments that require them to extend those methods to fulfil a specified requirement. We evaluate our LCS similarity detection approach by looking for code blocks in the students' code that match examples in the lecturer's code.

Our first step was to copy the lecturer's example code from the provided lecture slides, and to group the students' submitted programs according to the examples they are expected to be based upon. Next, we defined 'blocks' in the
students' code as sequences of lines of code separated by blank lines. For each of the lecturer's examples, we then used LCS to measure the similarity between each block of the student's code and the lecturer's example. If the similarity percentage exceeds a predetermined threshold, we count that code block as similar to the example. We then calculate an average similarity for the assignment, being the total number of similar blocks divided by the number of student submissions. These assignments are quite tightly constrained, so we would expect the average similarity to be close to 100%, indicating that every student had copied the lecturer's example reasonably closely.

For this evaluation we selected three related assignments, all dealing with stacks. The first was based on a lecturer's example showing how to implement a stack; the second, how to push to the stack; the third, how to pop from the stack. The results can be seen in Table 2. The code similarity detector works relatively well at a similarity threshold of 50%. Higher thresholds do not come close to finding the expected level of similarity. However, the 134% value for code example 1 shows that our method is being overly sensitive in detecting similarity: effectively, it detects more than one match with the example code in some students' assignments even though the nature of the assignment suggests that it should only detect one copy. Based on our observations, the length of the code and the uniqueness of the code greatly affect the result. As Corona permits and expects users to select the appropriate piece of code from all segments found to match the target, this is not a problem: users will simply not select any incorrect matches that are found.

<table><caption>**Table 2.** Average of student similarities to lecturer's example code for three different code examples; the percentage greater than 100 is explained in the text</caption><thead><tr><td>Similarity threshold</td><td>Example 1: implement</td><td>Example 2: push</td><td>Example 3: pop</td></tr></thead><tbody><tr><td>50%</td><td>134%</td><td>93%</td><td>93%</td></tr><tr><td>60%</td><td>69%</td><td>38%</td><td>52%</td></tr><tr><td>70%</td><td>24%</td><td>14%</td><td>17%</td></tr><tr><td>80%</td><td>3%</td><td>7%</td><td>7%</td></tr><tr><td>90%</td><td>3%</td><td>7%</td><td>7%</td></tr><tr><td>100%</td><td>3%</td><td>0%</td><td>3%</td></tr></tbody></table>

We also compared LCS with other commonly used character-level similarity detection approaches (string alignment and RKRGST (running Karp-Rabin greedy string tiling) [15]) and several token-level approaches (surface, syntactic and pseudo-semantic token) adapted from the similarity detection method accuracy comparison conducted by Karnalim et al. [16]. The IR-Plag dataset [17] was employed for the comparison, and the results are shown in Table 3. The top-K precision of each approach was calculated on IR-Plag's six different groups of code, which correspond to differing amounts of disguise that were applied to
an original program. The execution time of each approach was also calculated, by embedding a time-calculating program in the code for each approach, and running all tests on the same computer. The comparison of execution times of the code similarity detection approaches can be found in Table 4. This is not necessarily a reliable measure, but it is sufficient to show that this method does not impose an unreasonable time burden.

<table><caption>**Table 3.** Top-K precision of several code similarity detection approaches at different levels of disguise</caption><thead><tr><th>Disguise level</th><th>LCS</th><th>String alignment</th><th>RKRGST</th><th>Surface</th><th>Syntatic token</th><th>Pseudo-semantic token</th></tr></thead><tbody><tr><td>L1</td><td>16%</td><td>5%</td><td>3%</td><td>8%</td><td>13%</td><td>12%</td></tr><tr><td>L2</td><td>6%</td><td>3%</td><td>0%</td><td>9%</td><td>15%</td><td>14%</td></tr><tr><td>L3</td><td>8%</td><td>0%</td><td>0%</td><td>12%</td><td>21%</td><td>13%</td></tr><tr><td>L4</td><td>0%</td><td>0%</td><td>0%</td><td>3%</td><td>3%</td><td>2%</td></tr><tr><td>L5</td><td>0%</td><td>0%</td><td>0%</td><td>2%</td><td>0%</td><td>0%</td></tr><tr><td>L6</td><td>5%</td><td>1%</td><td>1%</td><td>6%</td><td>9%</td><td>7%</td></tr><tr><td>Overall</td><td>5%</td><td>1%</td><td>1%</td><td>6%</td><td>9%</td><td>7%</td></tr></tbody></table>

As seen in Table 3, LCS generally outperforms other character-level approaches with a significant gap ($p = 0.01$ when compared with string alignment and $p < 0.01$ when compared with RKRGST). When observed for each disguise level, LCS results in higher top-K precision than string alignment and RKRGST at the first three levels. However, the differences are statistically significant only on level 3 disguises ($p = 0.03$), showing that LCS performs especially better on component declaration relocation disguises. As shown in Table 4, LCS also outperforms other character-level approaches in execution time, being 28 s faster than string alignment and 27 s faster than RKRGST.

When compared with token-level approaches, LCS appears to have a lower top-K precision; however, a two-tailed t-test with 95% confidence rate shows that the differences are not statistically significant. This illustrates that LCS is comparable to token-level approaches despite not requiring any language-specific tokenization when dealing with cross-language code similarity detection. However, LCS is slower than token-level approaches, given that LCS conducts the comparison on smaller code segments. For the pseudo-semantic token approach we used a web-based code similarity detection tool, JPlag [18], which made it appear slower as JPlag requires time to generate similarity reports.

## 5.3 Evaluating Student Learning

Evaluation of student learning entails conducting student surveys at the start and end of each semester in question. This shows us students' initial knowledge of acceptable code reuse practice, and the impact of our system after it is introduced to them through the semester. For the pilot experiment we selected a course that has not yet been optimized for adapting code referencing (as opposed to
<table><caption>**Table 4.** Execution time comparison of several code similarity detection approaches</caption><thead><tr><td>Similarity detection approach</td><td>Execution time (seconds)</td></tr></thead><tbody><tr><td>LCS</td><td>83</td></tr><tr><td>String alignment</td><td>112</td></tr><tr><td>RKRGST</td><td>111</td></tr><tr><td>Surface token</td><td>60</td></tr><tr><td>Syntatic token</td><td>65</td></tr><tr><td>Pseudo-semantic token</td><td>90</td></tr></tbody></table>

the approach that we propose in Sect. 4) as we wanted to see how effective our system is in a course that has not been tailored to its use. For the survey we have adapted the scenarios of Simon et al. [19], as shown in Table 5. Those scenarios were designed to find respondents’ opinions on whether certain actions are acceptable and on whether they constitute plagiarism or collusion. As used in our survey, they are used to ask whether students believe that the actions require a reference.

For each scenario, students are asked to choose one of three options: a reference is required; a reference is not required; I don’t know whether a reference is required. The answers to these questions are not set in stone, and are likely to vary across institutions and indeed across courses within the same institution. In Table 5 we show the pre-intervention results from 24 respondents, along with the answers selected by their lecturers. We can see that the students have varying knowledge of when a code reference is required, showing confusion in selecting the appropriate response for several scenarios. We hoped to see this improve following introduction to the students of our system and the corresponding code referencing guidelines.

We then conducted a similar survey with students at the end of semester to see how our system and the lecture on code referencing affected their knowledge regarding correct code reuse practices (Table 5).

In scenario C1, we can see a decrease in the number of students who think that references are necessary when reusing code from the internet, which is not an ideal result. This may be due to the fact that in the project course we selected, students are under the supervision of different lecturers, each of whom has their own acceptable code reuse expectations due to varying complexity of their projects: the type of system they are building, and their research topics, will affect the code reuse tolerance level. We believe that when students think that a scenario is unacceptable behaviour, they will not indicate that it requires a reference. They may think that code reuse should not take place, and that if it does, it should be hidden. The same also seems to apply to scenarios C4 (modifying another student’s code to make it look different from the original) and C9 (asking another student to fix your code), which in most cases will be considered as collusion. Students will clearly be uncertain whether a reference is needed for an action if they are not sure whether that action is acceptable, as we
<table><caption>**Table 5.** Knowledge of students before (N = 24) and after (N = 22) introduction of our system as to whether references are required; lecturers' expected answers are shown in bold; answer columns are Required, Don't know, and Not required</caption><thead><tr><th rowspan="2">Scenario</th><th colspan="3">Before</th><th colspan="3">After</th></tr><tr><th>Rqd</th><th>DKn</th><th>Not</th><th>Rqd</th><th>DKn</th><th>Not</th></tr></thead><tbody><tr><td>C1: Using a freely available internet source as the basis for your work</td><td><strong>96%</strong></td><td>0%</td><td>4%</td><td><strong>91%</strong></td><td>9%</td><td>4%</td></tr><tr><td>C2: Basing an assessment largely on work that you wrote and submitted for a previous course</td><td><strong>50%</strong></td><td>25%</td><td>25%</td><td><strong>55%</strong></td><td>18%</td><td>27%</td></tr><tr><td>C3: Posting a question to an online help board (e.g. stack overflow) and using the answer in your work</td><td><strong>37%</strong></td><td>21%</td><td>42%</td><td><strong>63%</strong></td><td>5%</td><td>32%</td></tr><tr><td>C4: Using another student's code and changing it so that it looks different from the original</td><td><strong>25%</strong></td><td>25%</td><td>50%</td><td><strong>23%</strong></td><td>18%</td><td>59%</td></tr><tr><td>C5: Copying an early draft of another student's work and developing it into your own</td><td><strong>63%</strong></td><td>17%</td><td>21%</td><td><strong>54%</strong></td><td>41%</td><td>5%</td></tr><tr><td>C6: Discussing with another student how to approach a task and what resources to use, then developing the solution independently</td><td>79%</td><td>0%</td><td><strong>21%</strong></td><td>64%</td><td>4%</td><td><strong>32%</strong></td></tr><tr><td>C7: Discussing the detail of your code with another student while working on it</td><td>71%</td><td>8%</td><td><strong>21%</strong></td><td>68%</td><td>9%</td><td><strong>23%</strong></td></tr><tr><td>C8: Showing your code to another student and asking them for advice about a coding problem</td><td><strong>71%</strong></td><td>4%</td><td>25%</td><td><strong>68%</strong></td><td>9%</td><td>23%</td></tr><tr><td>C9: Asking another student to fix your code so that it runs correctly</td><td><strong>50%</strong></td><td>17%</td><td>33%</td><td><strong>27%</strong></td><td>23%</td><td>50%</td></tr><tr><td>C10: After completing an assessment, adding features that you noticed when looking at another student's work</td><td>46%</td><td>12%</td><td><strong>42%</strong></td><td>45%</td><td>14%</td><td><strong>41%</strong></td></tr></tbody></table>

can see in scenario C5 (using another student's early draft) where the number of students who answered “Don't know” increased post-intervention. The same may also apply for scenarios C6 (discussing the approach with another student) and C7 (discussing your code with another student while working on it), as in both cases more students respond post-intervention that a reference is not required.

There is also a slight increase in the number of students who think that a reference is required when they are reusing a previously submitted code assignment (C2), which is a desirable result but still leaves room for improvement. Scenario
C3 shows a substantial increase (37% to 63%) in the number of students who think that a reference is needed when using an answer to a question they post on an online help board. This may be due to the fact that referencing code from an online help board is a substantial part of Corona's functionality.

Very minor changes can be observed in C8 (showing code to another student for advice) and C10 (adding features after looking at another student's code), but these may due to the difference in numbers of participants for the pre and post surveys, both of which were fully voluntary.

# 6 Conclusion

We have presented Corona, an automated code referencing system to assist students in creating references for externally sourced code or assistance, which is part of our initiative to raise students' awareness of correct code reuse practice through code referencing and thus to help mitigate inadvertent code plagiarism. A pre-intervention survey with a small pilot class shows that students have varying knowledge regarding acceptable code reuse practice and would benefit from a standardised baseline. By introducing Corona through a code referencing lecture and demonstration, and making the system available to students, we introduced a referencing standard to students and helped them to implement it. We employ a template based automatic code comment generator to assist students in creating references. Our evaluations of two components of our automatic code comment generation system, the web scraping method and the code similarity detection method, demonstrate that our system performs well in automatically generating code references. A post-intervention repeat of the pre-intervention survey suggests that Corona has had some positive impact on students' knowledge of correct code reuse practice. Further, we have suggested steps for lecturers to further improve students' participation in code referencing and help to raise their awareness of unintentional code plagiarism.

# References

1. Novak, M., Joy, M., Kermek, D.: Source-code similarity detection and detection tools used in academia: a systematic review. ACM Trans. Comput. Educ. **19**(3), 1–37 (2019)

2. Lukashenko, R., Graudina, V., Grundspenkis, J.: Computer-based plagiarism detection methods and tools: an overview. In: International Conference on Computer Systems and Technologies, pp. 1–6. CompSysTech 2007 (2007)

3. Haefliger, S., von Krogh, G., Spaeth, S.: Code reuse in open source software. Manag. Sci. **54**(1), 180–193 (2008)

4. Huang, H., Youssef, A.M., Debbabi, M.: Binsequence: fast, accurate and scalable binary code reuse detection. In: ACM on Asia Conference on Computer and Communications Security, pp. 155–166. ASIA CCS ’17 (2017)

5. Flores, E., Barrón-Cedeño, A., Rosso, P., Moreno, L.: Towards the detection of cross-language source code reuse. In: Muñoz, R., Montoyo, A., Métais, E. (eds.) NLDB 2011. LNCS, vol. 6716, pp. 250–253. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22327-3_31
6. Zander, C., Eckerdal, A., McCartney, R., Mostrom, J.E., Sanders, K., Thomas, L.: Copying can be good: how instructors use imitation in teaching programming. In: 24th Conference on Innovation and Technology in Computer Science Education, pp. 450–456. ITiCSE 2019 (2019)

7. Simon, Sheard, J., Morgan, M., Petersen, A., Settle, A., Sinclair, J.: Informing students about academic integrity in programming. In: 20th Australasian Computing Education Conference, pp. 113–122. ACM Press, New York, New York, USA (2018)

8. Academic integrity at MIT. https://integrity.mit.edu/handbook/writing-code. Accessed 20 Aug 2020

9. Zhao, L., Zhang, L., Yan, S.: A survey on research of code comment auto generation. J. Phys. Conf. Ser. **1345**(3), 250–253 (2019)

10. Yang, B., Liping, Z., Fengrong, Z.: A survey on research of code comment. In: Third International Conference on Management Engineering, Software Engineering and Service Sciences, pp. 45–51 (2019)

11. Rahman, M.M., Roy, C.K., Keivanloo, I.: Recommending insightful comments for source code using crowdsourced knowledge. In: 15th International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 81–90. IEEE (2015)

12. Wong, E., Yang, J., Tan, L.: Autocomment: mining question and answer sites for automatic comment generation. In: 28th International Conference on Automated Software Engineering (ASE), pp. 562–567. IEEE (2013)

13. Cordy, J.R., Roy, C.K.: The NiCad clone detector. In: 2011 IEEE 19th International Conference on Program Comprehension, pp. 219–220. IEEE (2011)

14. Hata, H., Treude, C., Kula, R.G., Ishio, T.: 9.6 million links in source code comments: purpose, evolution, and decay. In: 41st International Conference on Software Engineering, pp. 1211–1221. ICSE '19, IEEE Press (2019)

15. Wise, M.J.: YAP3: improved detection of similarities in computer program and other texts. In: 27th SIGCSE Technical Symposium on Computer Science Education, pp. 130–134. SIGCSE 1996 (1996)

16. Karnalim, O., Simon, Chivers, W.: Layered similarity detection for programming plagiarism and collusion on weekly assessments. Comput. Appl. Eng. Educ. **30**(6), 1739–1752 (2022)

17. Karnalim, O.: Source code plagiarism detection with low-level structural representation and information retrieval. Int. J. Comput. Appl. 566–576 (2019)

18. Prechelt, L., Malpohl, G.: Finding plagiarisms among a set of programs with JPlag. J Univers. Comput. Sci. **8**, 1016–1038 (2003)

19. Simon, Cook, B., Sheard, J., Carbone, A., Johnson, C.: Academic integrity perceptions regarding computing assessments and essays. In: 10th Annual Conference on International Computing Education Research, pp. 107–114. ACM Press, New York, New York, USA (2014)