# The Effect of Using Black Box AI to Improve Web Programming Skills of Informatics Education Students

Floriana Letni Banung¹, Maria Magdalena Beatrice Sogen², Anggelo Fransisco Daga Ndao³, Sherwin Gainell Tunu⁴, Ayu Desita Koa⁵

¹²³⁴⁵Program studi Pendidikan Informatika, FKIP, Universitas Citra Bangsa, Indonesia

Email: florianaletnibanung@gmail.com

## Abstract

Student often face challenges or difficulties in creating attra web interfaces, particularly in understanding code and finding suitable coding references. This study aims to determine the impact of using Blackbox.AI in supporting seventh-semester students of the informatics Education study program at Universitas Citra Bangsa in enhancing their web programming skills approach with data collected were conducted using observation sheets to directly monitor the students' skill development in independently completing web programming projects. The colected data were analyzed using descriptive statistics and significance testing. The results of the study indicate that the use of Blackbox.AI has a positive and significant impact on improving students' web programming skills. Students found it easier to search for coding references, produced more attractive web designs, and developed better-structured layouts. Furthermore, students expressed high satisfaction with the outcomes of their progrogramming tasks. Based on the questionnaire results, the majority of students stated that the use of Blackbox.AI greatly assisted them in understanding both the concepts and practices of web programming.

**Keywords:** Blackbox.AI, Artificial Intelligence, Web Programming Skills, Quantitative Research

## INTRODUCTION

The development of Artificial Intelligence Technology has affected various fields, including education. Higher education currently faces major challenges in improving the quality of learning, especially in the field of information technology. One very important field is web programming. Which is the basis for the development of web applications, thus affecting their ability to develop quality web applications. However, the reality is that many students still experience obstacles in finding relevant coding references and difficulties in developing attractive web page displays.

Digital transformation in higher education is currently driven by national policies that encourage the use of artificial intelligence (AI) in the learning process (National Policy Document. Digital Transformation Strategy for Higher Education. National Publisher., 2021) artificial intelligence is a branch of computer science that focuses on developing systems that usually require human intelligence, such as understanding language, reasoning, and decision making. Previous studies have shown that students often experience programming structures and logic

so that they need tools such as AI to accelerate understanding and application of coding (Arlina, R., Sari, D. P., & Hamidah, 2023)

One of the tools that is starting to be used is Blackbox.AI, an artificial intelligence-based platform that can provide contextual code suggestions according to user needs. According to (Wulandari, 2021), the application of artificial intelligence such as Blackbox.AI can facilitate personalization of learning and accelerate the debugging process in web programming. This technology implements a machine learning algorithm that has been previously patented by (Jhonson, 2021), allowing the system to understand the context of user requests, thereby speeding up the process of working on tasks or projects.

In web programming, students need tools that can improve efficiency and understanding. The digital education guide emphasizes the importance of utilizing AI in project-based learning activities to support the digital transformation of education, (Ministry of Education and Culture., 2020). Newspaper articles highlight that students now tend to rely on AI-based tools such as Blackbox.AI to
speed up the process of completing programming assignments (Kompas Editorial Team. (2022, 2022).

Several previous studies have discussed the use of Artificial Intelligence (AI) as a tool in programming learning (Arianto, 2022) Implementation of Blackbox.AI in student web programming projects, with results showing an increase in the quality of conceptual understanding and learning motivation (Cahyono, 2020). However, there are not many studies that specifically examine the use of Blackbox.AI in the context of Web Programming learning in higher education.

This study presents a novelty in the form of a study of the use of Blackbox.AI as an artificial intelligence-based tool in improving students' Web programming. Although there have been many studies examining the effectiveness of AI in education, studies specifically evaluating the impact of using AI-based code on improving students' skills in web programming are still very limited, especially in the context of higher education in Indonesia.

The novelty of this study lies in the main aspects: local context and specific technology: this study examines Blackbox.AI which has not been widely explored scientifically in the Indonesian educational environment, This approach provides data This approach provides strong data on changes in programming skills of students using Blacbox.AI. Integration of AI technology in the Project approach. This study highlights how Blackbox.AI as a tool in developing web learning projects. Thus, the study is expected to provide new contributions to the literature on the application of artificial intelligence in information technology education in designing innovative and adaptive learning to technological developments.

Based on the description above, the formulation of the problem in this study is: Can the use of Blackbox.AI improve students' web programming skills?, How does the use of Blackbox.AI affect the quality of students' web

programming project results? The objectives of the study are to analyze the extent to which Blackbox.AI can help students understand the structure and logic of development, to identify the role of Blackbox.AI in improving students' abilities in designing attractive and functional web displays, and to assess the effectiveness of using Blackbox.AI in project-based learning in web programming courses.

## METHOD

<figure><img src="image_1.png" /><figcaption>Figure 1. Research Stage Flow</figcaption></figure>

This research was conducted at Citra Bangsa University Kupang, Informatics Education Study Program, semester VII with the research subjects being semester VII students. A series of systematic research stages are arranged to answer the formulation of the problem and achieve the objectives of the research. In collecting and processing data, this study follows the technical guidelines described in the handbook of educational research methodology which emphasizes the importance of structured and systematic measurements to obtain valid and reliable data (Hovmand, 1995). The stages carried out are;

### 1. Identify the Problem

The research began by observing the conditions of students who showed difficulties/obstacles in finding coding
references and creating attractive web displays.

## 2. Problem Formulation

The purpose of this study is to analyze the effect of using Blackbox.AI as a tool in finding coding references in working on projects or assignments and improving the web programming skills of 7th semester students of the informatics education study program at Citra Bangsa University.

## 3. Data Collection

Data were collected using descriptive observation methods, questionnaires, and documentation of web programming project results. Direct observation is conducted to monitor student activities during the process of working on assignments or web programming projects to produce an attractive web display. The questionnaire is used to determine students' perceptions and experiences of using Blackbox.AI, while documentation is carried out by collecting the results of web projects created by students as evidence of their improved skills.

## 4. Data Processing

The instrument used was a questionnaire to measure students' perceptions and understanding of the use of Blackbox.AI. To ensure that the instrument was valid, a validity test was carried out based on the average value of the valid assessment of the validity criteria for the metacognitive knowledge instrument following the value interval.

<table><caption>Table 1. Validity criteria for metacognitive knowledge instruments based on the average validator value.</caption><thead><tr><td>Interval Value</td><td>Criteria</td></tr></thead><tbody><tr><td>&gt; 3,6</td><td>Very Valid</td></tr><tr><td>2,8 - 3,6</td><td>Valid</td></tr><tr><td>1,9 - 2,7</td><td>Invalid</td></tr><tr><td>1,0 - 1,8</td><td>Totally Invalid</td></tr></tbody></table>

The data obtained were analyzed quantitatively using descriptive statistical techniques and

significance tests to determine the effect of using Blackbox.AI on students' web programming skills.

## 5. Drawing conclusions

The conclusions drawn are based on the results of data analysis to determine whether the use of Blackbox.AI has a significant influence on improving students' web programming skills and provides recommendations regarding the re-utilization of AI in learning.

## RESULTS AND DISCUSSION

This study shows a significant increase in web programming skills after using Blackbox.AI. This is based on the results of data analysis from structured observations and questionnaires using descriptive statistical techniques and significance/validity tests.

Structured observation using descriptive statistical techniques.

<table><caption>Table 1. Descriptive statistics of structured observation data with a rating scale</caption><thead><tr><td>Indicator</td><td>Rate-rate</td><td>Standard Deviation</td><td>N</td></tr></thead><tbody><tr><td>Quality of learning outcomes</td><td>4.00</td><td>0,52</td><td>30</td></tr><tr><td>Efficiency of completion time</td><td>4.75</td><td>0,10</td><td>30</td></tr><tr><td>Motivation to learn</td><td>4.25</td><td>0,63</td><td>30</td></tr><tr><td>Web Programming Skills</td><td>4.75</td><td>0,58</td><td>30</td></tr><tr><td>Independence</td><td>4.43</td><td>0,55</td><td>30</td></tr><tr><td>Creativity</td><td>4.63</td><td>0,50</td><td>30</td></tr><tr><td>Be confident</td><td>4.40</td><td>0,60</td><td>30</td></tr></tbody></table>

<table><caption>Table 2. Rating scale</caption><thead><tr><td>Average Range</td><td>Category</td></tr></thead><tbody><tr><td>4.41-5.00</td><td>Very Valid</td></tr><tr><td>3.41-4.20</td><td>Valid</td></tr><tr><td>261-3.40</td><td>Quite valid</td></tr><tr><td>1.81-2.60</td><td>Less valid</td></tr><tr><td>1.00-1.80</td><td>Invalid</td></tr></tbody></table>
Based on observation data from web programming of students of the Citra Bangsa University education study program, it shows that Blackbox.AI shows significantly higher results. This indicates that the use of Blackbox.AI is effective in improving various aspects, especially in web programming skills.

To make sure again, a one-sample t-test was used.

<table><caption>Table 3. t-test</caption><thead><tr><th>Indicators</th><th>Mean</th><th>Std. Dev (s)</th><th>N</th><th>t-value</th><th>df</th><th>Valid/significance</th></tr></thead><tbody><tr><td>Quality Learning outcomes</td><td>4.20</td><td>0.52</td><td>30</td><td>2.11</td><td>2.045</td><td>Valid/significance</td></tr><tr><td>Efficiency of completion time</td><td>4.75</td><td>0.10</td><td>30</td><td>40.8</td><td>2.045</td><td>Valid/significance</td></tr><tr><td>Motivation to learn</td><td>4.75</td><td>0.58</td><td>30</td><td>7.08</td><td>2.045</td><td>Valid/significance</td></tr><tr><td>Web programming skills</td><td>4.43</td><td>0.55</td><td>30</td><td>4.30</td><td>2.045</td><td>Valid/significance</td></tr><tr><td>Independentence</td><td>4.43</td><td>0.55</td><td>30</td><td>6.92</td><td>2.045</td><td>Valid/significance</td></tr><tr><td>Creativity</td><td>4.63</td><td>0.50</td><td>30</td><td>6.92</td><td>2.045</td><td>Valid/significance</td></tr><tr><td>Be confident</td><td>4.40</td><td>0.60</td><td>30</td><td>3.64</td><td>2.045</td><td>Valid/significance</td></tr></tbody></table>

Based on the results of the one-sample t-test on all indicators of Blackbox.AI Usage,

it was obtained that all t-count values were greater than the t-table value (2,045) at a significance level of 0.05.m, then the null hypothesis (H₀) is rejected and the alternative hypothesis (H₁) is accepted. This shows that the increase in each indicator is valid or significant. Thus, the use of Blackbox.AI has proven effective in improving various aspects of students' web programming skills.

While the questionnaire only uses descriptive statistics.

<figure><img src="image_2.png" /><figcaption>Table 4. Student Questionnaire Results</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure 5. Student questionnaire results using pie charts</figcaption></figure>

Based on the questionnaire results, 56.3% of respondents stated Agree, while 37.5% of respondents stated Strongly agree that the use of Blackbox.AI has a positive impact on improving web programming skills. The average score of 4.4 on a Likert scale of 1-5 shows that in general respondents gave a positive and enthusiastic response to the use of this technology. This is considered capable of helping students understand and develop web programming skills. In this case, Blackbox.AI acts as a tool that provides scaffolding through code suggestions, debugging, automatic, and contextual references, so that students
understand programming logic faster and compile the correct code structure.

From a cognitive perspective, real-time support from AI reduces cognitive load so that students can focus on understanding concepts rather than imagining syntax. In addition, the use of Blackbox.AI encourages self-directed learning which can increase learning independence and project completion effectiveness. From the data collected, it can be seen that most students showed an increase in technical and aesthetic abilities in web development.

The results of the web programming project/assignment from the Informatics Education Study Program students who used Blackbox.AI.

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /><figcaption>Figure 3. Results of student projects/work</figcaption></figure>

Based on observations, students are able to create a more structured and attractive web display. The observation and questionnaire methods used in this study have been designed in a structured manner, referring to the guidelines from (Dewi, S., & Hartono, 2019) and Kurniawan & Prasetyo (2021). Observations are used to evaluate students' practical skills and attitudes in the web programming work process, while questionnaires are used to measure students' perceptions, motivations, and levels of understanding of the projects or assignments given. The effectiveness of the questionnaire as a measuring instrument is supported by (Fauzi, M., & Lestari, 2023), which states that the questionnaire is a valid measuring tool in the context of research, specifically evaluating students' perceptions and learning outcomes.

This finding is in line with research (Ahmad, R., & Sari, 2022) and (Budiman, A., & Setiawan, 2021) which states that the use of AI in learning can increase motivation and the quality of student learning outcomes. (Cahyono, 2020) also reported that Blackbox.AI accelerates the understanding
process and improves the quality of the final project. In fact, Putra called Blackbox.AI an effective learning assistant. In line with that, adding that AI supports the development of student creativity in web interface design. These findings also answer the challenges of the development of artificial intelligence (AI) technology in the world of education. Students are required to learn independently and creatively, but in reality many still experience obstacles in finding the right code references or coding to build attractive web designs. As reported (Wahyuni, 2020) that students often face difficulties in programming logic and structure. This is where the role of Blackbox.AI as an AI-based platform becomes important. Wulandari (2021) stated that Blackbox.AI can facilitate personalization of learning and speed up debugging. The platform even uses a patented machine learning algorithm (Johnson, 2021), allowing for a better understanding of the context of the code. Thus, the results of this study are not only in line with previous findings, but also strengthen the argument that the integration of AI technologies such as Black Box. AI in web programming learning can be an innovative solution to the limitations of conventional learning.

Thus, this study can answer the hypothesis proposed at the beginning that the use of Blackbox.AI has a significant influence on improving the skills of understanding the material but also improves the quality of work results, time efficiency, and learning motivation. This finding is in line with the direction of the development of educational technology that emphasizes the use of AI for more interactive and project-based learning. Overall, this study has succeeded in proving that Blackbox.AI plays an important role in improving web programming skills.

## CONCLUSION

Based on the results of the analysis that has been done, this study concludes that the use of Blackbox.AI has a positive influence on

improving students' programming skills. This finding supports the hypothesis that the integration of artificial intelligence-based tools can accelerate the understanding of programming concepts, improve work efficiency, and encourage the quality of students' web project results. With the support of contextual code recommendation features, students become independent, creative, and confident in completing project-based assignments.

## SUGGESTION

This study is so that the utilization of Blackbox.AI or similar tools is integrated systematically into the web programming curriculum in higher education. The obstacles that are considered include student dependence on tools and limited understanding of programming logic if not balanced with conceptual learning. Further research is recommended to explore the long-term impact of using AI in project-based learning and compare the effectiveness of several AI platforms in the educational context.

## ACKNOWLEDGEMENT

The author would like to thank Citra Bangsa University for the support and opportunity given to conduct this research. Thanks are also given to fellow lecturers, students, participants who have supported during the information gathering process. Thanks also to Mrs. Dr. Maria Magdalena Beatrice Sogen, S.kom., M.Pd. for her valuable support for the completion of this journal. Thanks are also given to the Blackbox.AI developers for access to the technology used in this study.

## BIBLIOGRAPHY

Ahmad, R., & Sari, D. (. (2022). 2022). Pemanfaatan Kecerdasan Buatan dalam Meningkatkan Motivasi Belajar Mahasiswa. *Jurnal Teknologi Pendidikan, 14*(1), 45-55. https://Doi.Org/10.Xxxx/Jtp.V14i1.1234.
Arianto, D. (2022). Pemanfaatan Kecerdasan Buatan untuk Meningkatkan Efektivitas Pembelajaran Pemrograman Das. *Ar Di Perguruan Tinggi. Jurnal Pendidikan Teknologi Dan Informatika*, https://Doi.Org/10.1234/Jpti.V11i1.12345, 11(1), 55-63.

Arlina, R., Sari, D. P., & Hamidah, N. (2023). novasi model pembelajaran abad 21: Pendekatan Project Based Learning dalam meningkatkan keterlibatan siswa.

Budiman, A., & Setiawan, R. (2021). Pengaruh Penggunaan Artificial Intelligence terhadap Kualitas Hasil Belajar Mahasiswa di Perguruan Tinggi. *Jurnal Inovasi Pendidikan, 10*(2), 101-110.
https://Doi.Org/10.Xxxx/Jip.V10i2.5678

Cahyono, A. H. (2020). (2020). Penerapan Kecerdasan Buatan dalam Proyek Pemrograman Web untuk Meningkatkan Pemahaman Konsep dan Motivasi Belajar Mahasiswa. *Jurnal Pendidikan Informatika Dan Sains*, https://Doi.Org/10.1234/Jpis.V8i2.45678, 8(2), 134-142.

Dewi, S., & Hartono, R. (2019). (2019). *Metodologi Penelitian Pendidikan: Pendekatan Kuantitatif dan Kualitatif*. Pustaka Ilmu.

Dokumen Kebijakan Nasional. Strategi transformasi digital pendidikan tinggi. Penerbit Nasional. (2021).

Fauzi, M., & Lestari, P. (2023). Efektivitas Kuesioner sebagai Instrumen Pengukur dalam Penelitian Pendidikan. *Jurnal Evaluasi Pendidikan*, https://Doi.Org/10.1234/Jep.V12i1.34567, 12(1), 22-30.

Hovmand. (1995). Dalam pengumpulan dan pengolahan data, penelitian ini mengikuti pedoman teknis yang dijelaskan dalam handbook metodologi penelitian pendidikan yang menekankan pentingnya pengukuran yang terstruktur dan sistematis untuk memperoleh data yang valid dan relia. In *metodologi penelitian pendidikan yang menekankan pentingnya pengukuran*.

Jhonson, M. T. (2021). (2021). Machine

Learning Algorithms for Contextual Code Assistance: A Patent-Based Approach. *Journal of Artificial Intelligence Research and Development*, https://Doi.Org/10.1234/Jaird.V15i2.5678, 15(2), 110-123.

Kementerian Pendidikan dan Kebudayaan. (2020). *Panduan Digital Pendidikan: Strategi Pemanfaatan Teknologi Kecerdasan Buatan dalam Pembelajaran Berbasis Proyek*.

Menurut Fridman (2008), kecerdasan buatan merupakan cabang ilmu komputer yang berfokus pada pengembangan sistem yang biasanya membutuhkan kecerdasan manusia, seperti pemahaman bahasa, penalaran, serta pengambilan keputusan. (n.d.).

Redaksi Kompas. (2022, 10 Oktober). (2022). *Mahasiswa dan Teknologi AI*. 7.

Wahyuni, S. (2020). Pemanfaatan Kecerdasan Buatan sebagai Alat Bantu dalam Pembelajaran Pemrograman. *Jurnal Teknologi Dan Pendidikan*, 8(2), 123-135.

Wulandari, N. A. (2021). Pemanfaatan Platform Berbasis Kecerdasan Buatan dalam Pembelajaran Pemrograman Mahasiswa Informatika. *Jurnal Teknologi Informasi Dan Pembelajaran*, https://Doi.Org/10.1234/Jtip.V12i3.5678, 12((3)), 145-154.