## 5 Implementation and Experiments
### Assessment through Experiments
#### 5.3.1 Evaluating the Effectiveness of Learning  
This criterion evaluates the effectiveness of learning based on feedback from users. We created a questionnaire with 10 questions by considering the objectives of the system as shown in Table 3.  
The first three questions, Q1, Q2, and Q3, evaluate the content coverage of a target programming language's constructs. The next three questions, Q4, Q5, and Q6, evaluate the extent to which the foundational content, examples, and coding exercises are personalized for each student. The next three questions, Q7, Q8, and Q9, evaluate the quality of evaluation, i.e., grading of the submitted exercise solutions. The last question, Q10, evaluates the usability of the systems. The main user groups of the system are identified as faculty, students, and industry developers. Faculty members would use the system to generate the content for teaching and coding exercises, students would use the system to learn programming skills, and industry developers would use the system to learn the language from scratch or improve their language proficiency. Accordingly, we formed three distinct participant groups for the experiment: the faculty group, the student group, and the industry developer group. Each group used tailored evaluation criteria. The faculty group assessed the system from an instructor's perspective, comparing the provided learning content to their personal lecture materials and teaching experiences. The student group evaluated the system from a learner's perspective, reflecting on the effectiveness and productivity of their learning experiences. Meanwhile,  
Figure 16: Generated coding exercises for a student in finance. the developer group judged from a practitioner's standpoint, focusing on the practicality of the learning experience with the eLearning system. The experiment was conducted with a total of 73 participants. The student group consisted of 42 freshman and sophomore undergraduate students in Computer Science from our universities in California and a university in Korea. The faculty group consisted of six university professors who are currently teaching or have taught programming courses. The industry group consisted of 25 software developers and programmers from software development companies. Each participant took at least one course, mostly Java or Python. Each participant engaged in all learning activities including attending online classes and completing the exercise problems. Participants allocated an average duration of approximately 3.5 months for course completion. Consequently, the experiment required considerable time and effort from the participants. This requirement made the recruitment of additional subjects a practical challenge. Additionally, it imposed a significant logistical burden on the research team. The questionnaire items were rated on a scale of 1 to 5, with '5' indicating the highest level of satisfaction and '1' indicating significant dissatisfaction. The results of experiments using the survey form are shown in Figure 17.  
From the evaluation results, we make the following observations:  
Figure 17: Evaluation of learning effectiveness.