# Comparative Analysis of Chatbots Using Large Language Models for Web Development Tasks
## 3 Materials and Methods
### Evaluation Criteria  
The chatbot outputs were evaluated based on the following criteria:  
1. HTML and CSS Syntax Rules: Evaluated using the Markup/CSS Validation Service. The number of syntax errors was scored from 1 to 5.
2. Understanding of Prompts: assessed whether the chatbot fully understood and completed the prompt, scored from 1 to 5.
3. Conversational Memory: evaluated the chatbot's ability to maintain context and build on previous outputs, scored from 1 to 3.
4. Original and Diverse Content: assessed the creativity and diversity of the content, including grammar and sentence composition, scored from 1 to 5.
5. Appearance and User-Friendliness: evaluated the design's responsiveness, clarity, and visual appeal, scored from 1 to 5.
6. Functioning of PHP Form and Database Linking: checked if the PHP form correctly connected to the database and displayed data on the page, scored from 1 to 5.
7. SQL Query Correctness: assessed using the SQL Validator, scored from 1 to 3.  
Each criterion had specific guidelines to ensure consistency across evaluators. For example, for the \"HTML and CSS Syntax Rules\", a score of 1 was given when the output contained more than 10 syntax errors, while a score of 5 represented flawless execution with no errors. The decision to use different scoring scales was based on the complexity of the criteria. For example, subjective measures like usability and creativity require broader scales (1-5) to capture subtle differences, while binary or objective measures like SQL query correctness were better suited to a 1-3 scale.  
The seven evaluation criteria were selected by the authors of the study to provide a comprehensive assessment of common challenges in web development [58]. These criteria collectively address the essential technical, functional, and user-centric challenges. The selection is rooted in the need to evaluate both the front-end and back-end components, as well as the overall user experience and content quality.  
The final assessment was determined by a weighted average of the scores for each criterion. Weights were assigned to each criterion based on their importance in web development, aligning with industry best practices [59]. The highest weight, 30%, was given to Functioning of PHP Form and Database Linking, as these are crucial for dynamic web applications and essential for security and user experience. Significant weights of 20% were allocated to both Original and Diverse Content and Appearance and User-Friendliness. While technical functionality is vital, unique content and a user-friendly interface greatly enhance user engagement and satisfaction. Criteria like HTML and CSSSyntax Rules and Understanding of Prompts received 10% each. Proper syntax ensures code efficiency and compatibility, and understanding prompts reflects the ability to meet client requirements accurately. Lesser weights of 5% were assigned to Conversational Memory and SQL Query Correctness. Though important, they were considered less critical within the overall scope, with SQL correctness partly encompassed by the higher-weighted PHP and database functionality.  
By assigning weights this way, the evaluation prioritizes both technical functionality and user-centric aspects, ensuring a comprehensive and objective assessment that mirrors real-world web development priorities.