# Evaluation of Different Group Formation Methods in the Context of Distributed Pair Programming: Design and Experiment in Higher Education

José Colin, Sébastien Hoarau, Julien Broisin

► To cite this version:

José Colin, Sébastien Hoarau, Julien Broisin. Evaluation of Different Group Formation Methods in the Context of Distributed Pair Programming: Design and Experiment in Higher Education. ITiCSE 2025: Innovation and Technology in Computer Science Education, Jun 2025, Nijmegen Netherlands, Netherlands. pp.569-575, 10.1145/3724363.3729029 . hal-05247202

HAL Id: hal-05247202

https://hal.science/hal-05247202v1

Submitted on 9 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Evaluation of Different Group Formation Methods in the Context of Distributed Pair Programming: Design and Experiment in Higher Education

José Colin

jose.colin@irit.fr

IRIT, Université de Toulouse

Toulouse, France

Sébastien Hoarau

seb.hoarau@univ-reunion.fr

Laboratoire d'Informatique et de

Mathématiques, Université de la

Réunion

Saint-Denis, France

Julien Broisin

julien.broisin@irit.fr

IRIT, Université de Toulouse

Toulouse, France

## Abstract

Group formation is an important issue in collaborative learning, as properly formed groups can lead to better interactions between students and enhance their learning. This issue also applies to pair programming when it is used as a learning activity. When the number of students is important and/or students and teachers do not know each other very well, manual group formation, either by students or teachers, can become difficult. Many of the existing automatic methods used for automatic group formation in pair programming require a certain amount of personal information about the students, which can make the implementation of this kind of system difficult. Hence, in this paper we present an automatic pair formation method integrated to a distributed pair programming application. Our approach is based on personal data from the students collected with questionnaires within our platform, but also on trace data automatically collected from the pair programming activities completed by students. It has been experimented during an introductory programming course in higher education. The objective was to identify, among programming experience, self-efficacy, gender and past engagement in pair programming activities, the formation criteria that led to the best perceived compatibility and work distribution as well as performance. The results show that homogeneous pairs in self-efficacy are significantly better than heterogeneous pairs, both in terms of perceived compatibility and perceived work distribution.

## CCS Concepts

* Software and its engineering → Pair programming; Social and professional topics → CS1.

## Keywords

Computer Science Education, Distributed Pair Programming, Tool Design and Evaluation, Group Formation

## ACM Reference Format:

José Colin, Sébastien Hoarau, and Julien Broisin. 2025. Evaluation of Different Group Formation Methods in the Context of Distributed Pair Programming: Design and Experiment in Higher Education. In *Proceedings of*

<figure><img src="image_2.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License.

ITiCSE 2025, Nijmegen, Netherlands

© 2025 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-1567-9/2025/06

https://doi.org/10.1145/3724363.3729029

the 30th ACM Conference on Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2025), June 27-July 2, 2025, Nijmegen, Netherlands.
ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3724363.3729029

## 1 Introduction

Pair programming (PP) is a development method involving two programmers: the Driver, who actively works on the programming task and writes the code, and the Navigator, who observes and assists the Driver by suggesting improvements. Applied in education, PP has multiple positive impacts on students, including greater confidence in code quality, improved problem-solving skills [7], and higher perceived satisfaction among students [17]. In a context where the two programmers are geographically distant, distributed pair programming (DPP) allows for remote PP activities. DPP takes place in a virtual environment where a shared code editor ensures that only the programmer in the Driver role can modify the source code. The environment also supports code execution and role-switching functionality between Driver and Navigator.

For both PP and DPP, as well as collaborative learning in general, forming pairs is a critical issue as it significantly impacts learning and participation [9]. A simple solution, examined in various studies [10, 22], is to allow students to choose their own partners. These studies show that students tend to select partners based on personal affinity [22], resulting in a high perceived compatibility rate. Alternatively, the teacher can form pairs based on their knowledge of the students, aligning pairs with learning objectives [12].

However, these solutions are not always feasible. In higher education, first-year students—often large in number—may not know each other, making it difficult to form pairs based on affinity. The significant number of students also makes this task complex and time-consuming for instructors. Similar challenges arise in MOOCs, where students have even fewer opportunities to get to know one another, and teachers are unlikely to know their students. Yet, pair formation remains a major challenge for effective PP.

In this paper, we propose an automatic method for forming pairs in the DPP context. This method seeks to match students based on their profiles when the student population is large. Specifically, we consider programming experience, self-efficacy, gender, and prior engagement in PP activities. The goal is to study and compare the impact of these characteristics on students’ perceived compatibility with their partners, perceived work distribution within the pair, as well as their performance and engagement in DPP activities.
Our approach involves creating homogeneous and heterogeneous groups for the same characteristic in students' profiles.

Following a review of the state of the art on group formation in computer-supported collaborative learning (CSCL) and PP, we present our pair formation system and the details of its algorithm within an existing DPP application [4]. This system was then used in a distributed pair programming experiment conducted in higher education, where students used the system in face-to-face classes. The experiment and the results are discussed. Finally, we provide the main conclusions of this work and future research directions to improve our group formation system.

# 2 State of the art

## 2.1 Group Formation in CSCL

Group formation in CSCL refers to the strategies, techniques, and methods implemented to group individuals based on criteria to create teams that promote better interactions and maximize learning [1]. In their literature review, Borges et al. [1] identified that in 106 studies on group formation in CSCL, the main criteria used were related to student profiles (knowledge level, skills, gender, etc.) and preferences of students and/or teachers. They also showed that these criteria are employed to form groups using two approaches: homogeneous and heterogeneous.

The homogeneous approach groups students who are similar based on one or more criteria, while the heterogeneous approach groups students who differ. The heterogeneous approach has been studied more extensively (84 out of 106 studies) than the homogeneous one (36 out of 106 studies).

When the number of students or criteria to consider is high, group formation can become a complex task without computational support. To address this challenge, researchers have proposed moving from manual approaches, where teachers and/or students are responsible for forming groups, to automatic methods. Many studies have suggested various algorithmic methods [5], with probabilistic models being the most widely used, as they can handle a large number of variables.

## 2.2 Existing Studies on Pair Formation in Pair Programming

In pair programming, as in CSCL, existing studies on pair formation either use manual methods—allowing students or teachers to select pairs [12, 22]—or automatic methods [3, 23]. Criteria such as students’ knowledge levels and skills are widely used for pair formation [6, 20, 21, 23], as well as homogeneous and heterogeneous approaches.

Several studies [3, 8, 12, 21] have used students’ personality traits as criteria for forming pairs, often relying on the Myers-Briggs Type Indicator (MBTI) model [14]. Choi et al. [3] demonstrated that pairs with diverse personalities show greater productivity compared to pairs with similar or opposing traits. However, measuring personality traits requires administering long questionnaires, such as the MBTI survey, which may not be feasible in all contexts.

To evaluate the effectiveness of a pair formation method, several indicators are used in the literature. Perceived compatibility

between the two members of a pair is often assessed using post-experimentation questionnaires[6, 12, 21, 23]. The pairs’ performance in programming tasks is evaluated by measuring the time required to complete the task [8] or by directly assessing the quality of the code produced by the pair [8, 23].

Among existing research, no studies have been found that incorporate interaction traces or learning analytics as criteria for pair formation or as measures of method effectiveness. For instance, the use of engagement metrics derived from user interactions for pair formation has not been explored in the literature. Utilizing interaction traces could provide more objective measures of students’ actual behavior in pair programming compared to self-reported data.

In this article, we aim to answer the following research questions (RQs):

* **RQ1:** How does the automatic formation of homogeneous and heterogeneous groups, based on learners' self-efficacy, programming experience, prior engagement in pair programming activities, and gender, affect perceived compatibility, perceived work distribution, engagement, and performance in DPP?

* **RQ2:** Does the automatic formation of groups according to these four criteria reveal significant differences in perceived compatibility, perceived work distribution, engagement, and performance between homogeneous and heterogeneous groups?

# 3 Pair Formation System

## 3.1 The Distributed Pair Programming Application

The automatic pair formation system we propose is directly integrated into our distributed pair programming (DPP) application [4]. This web application enables practical work in Python. For students, the application interface provides all the essential features for implementing DPP: a shared code editor, a role system to differentiate the Driver and Navigator roles (and allow role switching), a Python console for executing code, and an instant text messaging system. For instructors, the application includes an interface to create and modify educational activities.

Various interaction traces are collected by the application: user login and logout events, role switch requests and their responses, snapshots of the Python source code produced, interactions with the Python console, and messages exchanged in the text communication channel. These traces are timestamped and formatted in JSON.

The application has previously been used in experiments aimed at evaluating its usability [4]. It is accessible via the LTI protocol [13], allowing students to connect to it from a learning platform such as Moodle.

## 3.2 Overview of the System

To begin a pair programming session using the pair formation system, the instructor first creates an activity in the application. Students then connect simultaneously to this activity and are placed in a waiting queue, which is emptied as pairs are formed and redirected to the DPP environment.
The system collects the necessary information about students' profiles to calculate pairing scores. At regular intervals, the system considers all possible combinations of students in the queue and calculates a pairing score for each combination based on a chosen calculation method. The combination with the highest score is selected to form a pair. The two students in that pair leave the queue and are redirected to the DPP activity. This operation is repeated until the queue is empty (or until only one student remains in the queue). The implementation of this system is described in Algorithm 1.

<caption>**Algorithm 1 Pair Formation Algorithm**</caption>

students ← queue ▷ The set of students in the waiting queue
while size(students) ≥ 2 do
    maxScore ← 0
    bestCombinaison ← 0
    for (student1, student2) ∈ students × students do
        if student1 ≠ student2 then
            pairingScore ← pairingScore(student1, student2)
            if pairingScore > maxScore then
                maxScore ← pairingScore
                bestCombinaison ← {student1, student2}
            formPair(bestCombinaison)
            students ← students \ bestCombinaison

It is also possible to introduce a threshold for the pairing score in this system. In such cases, if no combination exceeds the threshold among all possible pairs, no pair is formed, and the system waits for a predefined duration before attempting to form pairs again. This waiting period prevents pairs from being formed too quickly and allows the queue to fill up to find better combinations of students. If no combination exceeds the threshold after several retries, the threshold is gradually lowered until pairs can be formed. This addition is particularly useful when students do not all connect to the application simultaneously. For the experiment presented in this study, the threshold system was not used.

## 3.3 Pairing Score Calculation

The calculation of the pairing score depends on the selected method. Each method is associated with a pairing criterion and has an objective: to maximize homogeneity or heterogeneity between the two students for that criterion.

For a method that maximizes homogeneity, the pairing score is calculated as follows:

$$pairingScore = \frac{1}{\max(0.01, |criterionStudent1 - criterionStudent2|)}$$

For a method that maximizes heterogeneity, the pairing score is calculated as follows:

$$pairingScore = |criterionStudent1 - criterionStudent2|$$

# 4 Methods

## 4.1 Experimental Setup

The study was conducted using our DPP application (described in Section 3.1) as part of an introductory Python programming course at Université de la Réunion. The experiment involved a cohort of first-year computer science students and a cohort of first-year mathematics students. Each cohort used the platform during two face-to-face lecture sessions by the same instructor.

During the first lecture session, the instructor introduced the platform and its main features. Students then had 20 minutes to complete a pair programming activity designed by the instructor. During the second lecture session, students participated in two pair programming activities, each lasting 40 minutes.

Each pair programming activity focused on topics previously covered in the Python programming course and was accessible through the Moodle learning platform of [Anonymous University]. Thus, students were automatically logged into the DPP platform via the LTI protocol.

During their first login to the DPP platform (i.e., while completing the first activity), students were required to fill out a pre-experimental questionnaire (Q1) about their programming experience and self-efficacy. After completing each of the three activities, students were asked to complete a post-experimental questionnaire (Q2) about perceived work distribution and compatibility with their partner.

At the start of each activity, pairs were formed using the automatic pairing system described in Section 3.2, using one of the pairing criteria offered by the system (i.e., programming experience, self-efficacy, gender, or prior engagement in pair programming activities). The system aimed to balance the number of pairs formed based on each criterion, depending on the available data about the students. Consequently, students were assigned a partner located anywhere in the lecture hall and communicated via the platform's text channel.

During the first lecture session, the pairing criteria used were programming experience, self-efficacy, and gender. For the second session, the criteria used were prior engagement, self-efficacy, and gender.

## 4.2 Collected Data

### 4.2.1 Self-Reported Data.
The pre-experimental questionnaire (Q1) was administered through the DPP platform during the students' first login. It aimed to measure three criteria used in the pairing system: programming experience, self-efficacy, and gender.

* Programming experience was measured using a questionnaire by Siegmund et al. [18], translated into Anonymous-Language. This questionnaire included 15 questions, with 8 single-choice questions and 7 Likert-scale questions (5 levels). The programming experience score was calculated by averaging the responses. Each single-choice question was assigned a scoring method (details available at recherche.data.gouv.fr).

* Self-efficacy was measured using a questionnaire by Tsai et al. [19], translated into French. This questionnaire consisted of 16 Likert-scale questions (6 levels), and the self-efficacy
score was calculated as the average of the respondent's answers.

* Gender was determined through a single-choice question with 3 options: man, woman, other.

The post-experimental questionnaire (Q2) was also administered via the DPP platform at the end of each pair programming activity. It measured students' perceptions of work distribution and compatibility with their partner. The questionnaire was based on a translated version of Williams et al.'s instrument [21] for pair programming contexts, which assessed:

* Perceived work distribution: Measured using three Likert-scale questions (5 levels).

* Perceived compatibility: Measured using a single Likert-scale question (3 levels).

The perceived work distribution score for a pair was calculated as the average of the responses from both members to the three questions. The perceived compatibility score was calculated as the average response from both members to the compatibility question.

### 4.2.2 Quantitative Data.
As mentioned in Section 3.1, the DPP application collects various interaction traces. These data were used to calculate two indicators: academic performance and engagement in the pair programming activity. **Academic performance** corresponds to a score based on the final source code produced by the pair, evaluated against the activity designed by the instructor. The evaluation followed a predefined rubric established by the instructor, and all coding tasks were graded by a single evaluator to avoid bias. Each pair received a performance score between 0 and 1 based on the grade they received for each of the three activities.

**Engagement in the pair programming activity** was calculated by combining several variables for each pair member:

* The number of characters inserted or deleted (nbChar): This value was obtained by comparing successive differences in the code editor content using Myers' algorithm [15].

* The number of code executions and console commands (nbCom).

* The number of role switch requests and responses (nbRoleChange).

* The number of messages sent in the text communication channel (nbMess).

Each variable was normalized by dividing it by its maximum observed value during the first session of the experiment (nbCharMax = 920, nbCommMax = 19, nbRoleChangeMax = 5, nbMessMax = 31). Engagement scores for each student ranged from 0 to 1, calculated as follows:

$$ engagement = \frac{\left( \frac{nbChar}{920} + \frac{nbComm}{19} + \frac{nbRoleChange}{5} + \frac{nbMess}{31} \right)}{4} $$

The pair's engagement score was then calculated as the average of the individual engagement scores of both members. During the second session, this individual engagement score was also used as a criterion for prior engagement.

## 4.3 Data Analysis

To address RQ1 regarding the impact of homogeneous and heterogeneous pairs (based on self-efficacy, programming experience, gender, and prior engagement) on perceived compatibility, work distribution, engagement, and performance, averages, standard deviations, and variances were reported.

For RQ2, which aims to identify significant differences between homogeneous and heterogeneous pairs for each pairing criterion, Mann-Whitney U tests were conducted for four variables: perceived work distribution, perceived compatibility, academic performance, and engagement. We used Mann-Whitney U tests instead of t-tests as our data was not normally distributed.

Pairs were classified into homogeneous and heterogeneous groups for analysis based on the selected pairing criterion. For most criteria, the pairs with the smallest differences were classified as homogeneous, while the rest were classified as heterogeneous. For gender, only 8 heterogeneous pairs (male/female) were formed, resulting in unequal group sizes.

# 5 Results

A total of 133 distributed pair programming (DPP) sessions were included in this study. Among these, 51 sessions were conducted during the first lecture session, and 82 during the second. Table 1 shows the number of pairs created during the experiment for each pairing criterion.

<table><caption>Table 1: Number of pairs per pairing criterion</caption><thead><tr><th></th><th>Programming Experience</th><th>Self-Efficacy</th><th>Gender</th><th>Prior Engagement</th></tr></thead><tbody><tr><th>Number of pairs</th><td>21</td><td>50</td><td>35</td><td>27</td></tr></tbody></table>

The number of pairs formed for each criterion varied. For instance, the "programming experience" criterion was used only during the first lecture session, which included fewer sessions. Similarly, the "prior engagement" criterion was used only during the second lecture session because this criterion could only be calculated automatically once a student had completed at least one pair programming activity on the platform.

Figures 1 through 4 present the average results for perceived compatibility, perceived work distribution, academic performance, and engagement based on the different group formation methods used. Figure 1 shows no significant differences in the average perceived compatibility scores for homogeneous and heterogeneous groups based on programming experience and gender. However, for self-efficacy, homogeneous groups show significantly higher perceived compatibility, while for engagement, heterogeneous groups perform better. Regarding perceived work distribution, Figure 2 shows slight differences for gender and engagement, but a notable difference is observed for self-efficacy, favoring homogeneous groups. As with perceived compatibility, homogeneous groups based on self-efficacy perform better. As seen in Figure 3, a significant difference is observed in academic performance for programming experience, favoring heterogeneous groups. However, no significant differences are found for self-efficacy, gender, or engagement. Finally, Figure
4 shows significant differences for engagement and programming experience, but no differences are seen for self-efficacy or gender in the average engagement scores of pairs.

<figure><img src="image_3.png" /><figcaption>**Figure 1:** Average perceived compatibility of pairs based on the different pairing methods.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Figure 2:** Average perceived work distribution in pairs based on the different pairing methods.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 3:** Average academic performance of pairs based on the different pairing methods.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Figure 4:** Average engagement of pairs based on the different pairing methods.</figcaption></figure>

Table 2 presents the results of the Mann-Whitney U tests comparing homogeneous and heterogeneous groups for each training criterion, as well as details of means, standard deviations and median for indicators of perceived compatibility, perceived work distribution, engagement and learner performance. This table reveals three significant differences between homogeneous and heterogeneous groups: pairs with homogeneous self-efficacy have significantly greater perceived compatibility and perceived work distribution than heterogeneous pairs, and pairs with heterogeneous programming experience have significantly better academic performance than homogeneous pairs.

# 6 Discussion

We observed that pairs with homogeneous self-efficacy showed significantly higher perceived compatibility compared to pairs with heterogeneous self-efficacy (p = 0.01). A similar result was found for perceived work distribution (p = 0.02). This finding aligns with a study by Katira et al. [12], which showed that pairs with homogeneous perceived technical competence were more compatible than heterogeneous pairs.

This result can also be connected to studies that demonstrated better perceived compatibility for pairs with homogeneous programming skills [11, 20, 23]. However, our study did not show a significant difference in perceived compatibility for the programming experience criterion. A possible limitation leading to this result is discussed in Section 7.

Regarding engagement, although there were notable differences in average scores between homogeneous and heterogeneous groups for two of the pairing criteria, the results of the Mann-Whitney U tests did not reveal significant differences. This lack of significance may be explained by the high standard deviation observed for this indicator.

For academic performance, heterogeneous pairs based on programming experience significantly outperformed homogeneous pairs (p = 0.01). This result is consistent with studies suggesting that pairing individuals with diverse experiences can lead to better outcomes [16].

The results of this study provide a better understanding of the impact of pairing criteria on various factors such as compatibility, work distribution, engagement, and performance in distributed pair programming (DPP). These insights can inform the design of
<table><caption>Table 2: Results of Mann-Whitney U tests comparing homogeneous and heterogeneous groups for each pairing criterion</caption><thead><tr><th rowspan="2">Criterion</th><th colspan="5">Perceived Compatibility</th><th colspan="5">Perceived Work Distribution</th><th colspan="5">Academic Performance</th><th colspan="5">Engagement</th></tr><tr><th>M</th><th>SD</th><th>Mdn</th><th>U</th><th>p</th><th>M</th><th>SD</th><th>Mdn</th><th>U</th><th>p</th><th>M</th><th>SD</th><th>Mdn</th><th>U</th><th>p</th><th>M</th><th>SD</th><th>Mdn</th><th>U</th><th>p</th></tr></thead><tbody><tr><td>Homogeneous Programming Experience</td><td>0.79</td><td>0.25</td><td>1</td><td>210</td><td>0.77</td><td>0.69</td><td>0.66</td><td>1</td><td>248.5</td><td>0.47</td><td>0.52</td><td>0.41</td><td>0.50</td><td>127.5</td><td><strong>0.01*</strong></td><td>0.203</td><td>0.205</td><td>0.106</td><td>250</td><td>0.46</td></tr><tr><td>Heterogeneous Programming Experience</td><td>0.81</td><td>0.25</td><td>1</td><td></td><td></td><td>0.66</td><td>0.36</td><td>0.83</td><td></td><td></td><td>0.84</td><td>0.30</td><td>1</td><td></td><td></td><td>0.120</td><td>0.085</td><td>0.132</td><td></td><td></td></tr><tr><td>Homogeneous Self-Efficacy</td><td>0.73</td><td>0.31</td><td>1</td><td>1604</td><td><strong>0.01*</strong></td><td>0.74</td><td>0.32</td><td>0.83</td><td>1634.5</td><td><strong>0.01*</strong></td><td>0.47</td><td>0.42</td><td>0.33</td><td>1210</td><td>0.78</td><td>0.111</td><td>0.117</td><td>0.082</td><td>1148.5</td><td>0.48</td></tr><tr><td>Heterogeneous Self-Efficacy</td><td>0.55</td><td>0.34</td><td>0.5</td><td></td><td></td><td>0.57</td><td>0.34</td><td>0.58</td><td></td><td></td><td>0.50</td><td>0.42</td><td>0.5</td><td></td><td></td><td>0.123</td><td>0.117</td><td>0.113</td><td></td><td></td></tr><tr><td>Homogeneous Gender</td><td>0.65</td><td>0.29</td><td>0.5</td><td>424.5</td><td>0.91</td><td>0.68</td><td>0.34</td><td>0.75</td><td>490</td><td>0.41</td><td>0.55</td><td>0.42</td><td>0.67</td><td>436</td><td>0.96</td><td>0.118</td><td>0.087</td><td>0.104</td><td>387</td><td>0.53</td></tr><tr><td>Heterogeneous Gender</td><td>0.66</td><td>0.30</td><td>0.5</td><td></td><td></td><td>0.59</td><td>0.37</td><td>0.71</td><td></td><td></td><td>0.58</td><td>0.31</td><td>0.58</td><td></td><td></td><td>0.125</td><td>0.087</td><td>0.115</td><td></td><td></td></tr><tr><td>Homogeneous Engagement</td><td>0.39</td><td>0.29</td><td>0.5</td><td>282</td><td>0.13</td><td>0.60</td><td>0.39</td><td>0.75</td><td>319.5</td><td>0.43</td><td>0.56</td><td>0.36</td><td>0.5</td><td>389.5</td><td>0.66</td><td>0.123</td><td>0.110</td><td>0.110</td><td>280</td><td>0.15</td></tr><tr><td>Heterogeneous Engagement</td><td>0.56</td><td>0.45</td><td>0.5</td><td></td><td></td><td>0.71</td><td>0.31</td><td>0.67</td><td></td><td></td><td>0.52</td><td>0.34</td><td>0.5</td><td></td><td></td><td>0.182</td><td>0.145</td><td>0.179</td><td></td><td></td></tr></tbody></table>

$M=\text{Mean} \quad SD=\text{standard deviation} \quad Mdn=\text{Median}$

$*p < .05$

future pairing systems to optimize group dynamics and learning outcomes.

## 7 Limitations

In this study, we aimed to replicate the conditions of distributed pair programming (DPP) by physically separating students in the same pair. However, although the students were not seated next to each other, being in the same lecture hall may have led some to communicate directly with their partners, potentially disrupting the intended experimental conditions.

The programming experience criterion was measured using responses to a questionnaire about students' profiles. Since many students were of the same age and had similar programming experience, primarily from high school, the standard deviation for this indicator was very low. As a result, groups considered homogeneous or heterogeneous for this criterion in our study were only marginally different. Using a more diverse cohort of students would have likely yielded more pronounced differences.

The engagement indicator was calculated using an arbitrary approach, particularly in terms of the weights assigned to each variable. It would be beneficial to adopt a more scientifically grounded method to determine the appropriate weights for each variable in the engagement calculation formula.

We measured the academic performance for pairs of students, not individually. This causes a problem for heterogeneous pairs in programming experience, because the 'stronger' student of the group can have a major influence on the academic performance of the pair, without necessarily increasing the other student's learning [2]. A better measure would be the individual learning of each student, because it would better detect a significant increase in students' learning for both members of the pair.

Additionally, while the system was designed to test pairing based on individual criteria, the integration of hybrid approaches (i.e., considering multiple criteria simultaneously) was not explored in this study. Such approaches could provide a more comprehensive understanding of pair formation dynamics and their impact on various outcomes.

## 8 Conclusion

In this study, we proposed a pair formation system exploring various methods that aim to maximize homogeneity or heterogeneity based on students' personal characteristics. This system was tested through an experiment involving 133 distributed pair programming (DPP) sessions. Among the pairing methods used in this study, no single method emerged as superior across all indicators (perceived compatibility, perceived work distribution, academic performance, and engagement). However, pairs that were homogeneous in terms of self-efficacy demonstrated significantly better results in two indicators: perceived compatibility and perceived work distribution.

This study only utilized methods based on single criteria for pair formation. It would be interesting to extend this work by employing hybrid approaches that combine multiple criteria simultaneously [1]. Such approaches could provide more nuanced insights into how various combinations of characteristics impact pair performance and dynamics in DPP.

All the data from this experiment is made publicly available on recherche.data.gouv.fr.

## References

[1] Simone Borges, Riichiro Mizoguchi, Ig Ibert Bittencourt, and Seiji Isotani. 2018. Group Formation in CSCL: A Review of the State of the Art. In *Higher Education for All. From Challenges to Novel Technology-Enhanced Solutions*, Alexandra Ioana Cristea, Ig Ibert Bittencourt, and Fernanda Lima (Eds.). Springer International Publishing, Cham, 71–88.

[2] Nicholas A. Bowman, Lindsay Jarratt, K.C. Culver, and Alberto Maria Segre. 2019. How Prior Programming Experience Affects Students' Pair Programming Experiences and Outcomes. In *Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education* (Aberdeen, Scotland Uk) (ITiCSE '19). Association for Computing Machinery, New York, NY, USA, 170–175. https://doi.org/10.1145/3304221.3319781

[3] Kyungsub S. Choi, Fadi P. Deek, and Il Im. 2008. Exploring the underlying aspects of pair programming: The impact of personality. *Information and Software Technology* 50, 11 (2008), 1114–1126. https://doi.org/10.1016/j.infsof.2007.11.002

[4] José Colin, Sébastien Hoarau, Christophe Declercq, and Julien Broisin. 2024. Design and Evaluation of a Web-based Distributed Pair Programming Tool for Novice Programmers. In *Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1* (Milan, Italy) (ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 527–533. https://doi.org/10.1145/3649217.3653571

[5] Wilma Marreiro Cruz and Seiji Isotani. 2014. Group Formation Algorithms in Collaborative Learning Contexts: A Systematic Mapping of the Literature. In
*Collaboration and Technology*, Nelson Baloian, Frada Burstein, Hiroaki Ogata, Flavia Santoro, and Gustavo Zurita (Eds.). Springer International Publishing, Cham, 199-214.

[6] Wanfeng Dou and Wei He. 2010. Compatibility and Requirements Analysis of Distributed Pair Programming. In *2010 Second International Workshop on Education Technology and Computer Science*, Vol. 1. 467-470. https://doi.org/10.1109/ETCS.2010.367

[7] Silvana Faja. 2011. Pair Programming as a Team Based Learning Activity: A review of research. *Issues in Information Systems* XII (01 2011), 207-216.

[8] Jo E. Hannay, Erik Arisholm, Harald Engvik, and Dag I.K. Sjoberg. 2010. Effects of Personality on Pair Programming. *IEEE Transactions on Software Engineering* 36, 1 (2010), 61-80. https://doi.org/10.1109/TSE.2009.41

[9] Seiji Isotani, Akiko Inaba, Mitsuru Ikeda, and Riichiro Mizoguchi. 2009. An ontology engineering approach to the realization of theory-driven group formation. *International Journal of Computer-Supported Collaborative Learning* 4 (2009), 445-478.

[10] Norman Jacobson and Suzanne K. Schaefer. 2008. Pair programming in CS1: overcoming objections to its adoption. *SIGCSE Bull.* 40, 2 (June 2008), 93-96. https://doi.org/10.1145/1383602.1383643

[11] Randall Jensen. 2003. A pair programming experience. *The Journal of Defense Software Engineering* 16, 3 (2003), 22-24. http://www.stsc.hill.af.mil/crosstalk/2003/03/jensen.html

[12] Neha Katira, Laurie Williams, Eric Wiebe, Carol Miller, Suzanne Balik, and Ed Gehringer. 2004. On understanding compatibility of student pair programmers. *SIGCSE Bull.* 36, 1 (March 2004), 7-11. https://doi.org/10.1145/1028174.971307

[13] LTI 2024. LTI. https://www.1edtech.org/standards/lti.

[14] Briggs Isabel Meyers and Peter B Meyers. 1995. Gifts Differing Understanding Personality type.

[15] Eugene W. Myers. 1986. AnO(ND) difference algorithm and its variations. *Algorithmica* 1, 1 (01 Nov 1986), 251-266. https://doi.org/10.1007/BF01840446

[16] Evelia Y. Salinas, Adrienne E. Williams, and Christine E. King. 2019. Effect of controlling group heterogeneity on student performance in a graphical programming course. In *2019 IEEE Frontiers in Education Conference (FIE)*. 1-8.

https://doi.org/10.1109/FIE43999.2019.9028593

[17] Norsaremah Salleh, Emilia Mendes, and John Grundy. 2011. Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review. *IEEE Transactions on Software Engineering* 37, 4 (2011), 509-525.

[18] Janet Siegmund, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. 2014. Measuring and modeling programming experience. *Empirical Software Engineering* 19, 5 (01 Oct 2014), 1299-1334. https://doi.org/10.1007/s10664-013-9286-4

[19] Meng-Jung Tsai, Ching-Yeh Wang, and Po-Fen Hsu. 2019. Developing the Computer Programming Self-Efficacy Scale for Computer Literacy Education. *Journal of Educational Computing Research* 56, 8 (2019), 1345-1360. https://doi.org/10.1177/0735633117746747

[20] Theodore Van Toll III, Roger Lee, and Tom Ahlswede. 2007. Evaluating the Usefulness of Pair Programming in a Classroom Setting. In *6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007)*. 302-308. https://doi.org/10.1109/ICIS.2007.96

[21] L. Williams, L. Layman, J. Osborne, and N. Katira. 2006. Examining the compatibility of student pair programmers. In *AGILE 2006 (AGILE'06)*. 10 pp.-420. https://doi.org/10.1109/AGILE.2006.25

[22] Stelios Xinogalos, Maya Satratzemi, Alexander Chatzigeorgiou, and Despina Tsompanoudi. 2017. Student perceptions on the benefits and shortcomings of distributed pair programming assignments. In *2017 IEEE Global Engineering Education Conference (EDUCON)*. 1513-1521. https://doi.org/10.1109/EDUCON.2017.7943050

[23] Ömer Demir and Süleyman Sadi Seferoglu. 2021. The Effect of Determining Pair Programming Groups According to Various Individual Difference Variables on Group Compatibility, Flow, and Coding Performance. *Journal of Educational Computing Research* 59, 1 (2021), 41-70. https://doi.org/10.1177/0735633120949787 arXiv:https://doi.org/10.1177/0735633120949787