## 2 Materials and Methods
### Scenario-Based Design
#### 2.1.3 Evaluation  
The evaluation phase focused on the evaluation of different prototypes of the planning poker activity, which were developed iteratively in a number of stages of scenario development and refinement. With the NAO robot taking the role of the facilitator of the collaborative activity of planning poker, most of the prototyping was focused on ensuring smooth dialogue and interaction. Earlier tests with the NAO robot showed that reliable pronunciation and a moderate speaking rate are crucial for effective interaction with students [27; 28]. Therefore, dialogue and interaction design were gradually improved based on feedback from these iterative phases:  
1. The first iteration focused on creating a first prototype based on the draft of the problem scenario from the analysis stage, which was derived from hands-on experiences without the possibility to rely on any prior research in the application of a robotic facilitator of planning poker. In order to evaluate the first prototype, a pretest with a small group of three students was conducted to obtain early feedback related to the overall design of the game and the specific aspects such as the robot's speech quality and the comprehensiveness of the explanation. The evaluation used a Likert-scale-based grading system for different rating criteria, such as comprehensibility of language and information and the robot's speech quality. Based on the results from this first pretest, the prototype was iterated further.
2. The second iteration focused on improving the speech quality, the explanation of the planning poker, and the explanation of interaction during agile estimation in the small group. The speaking rate of the robot was decreased and some ambiguities in the explanations were addressed. The information part was better tuned in to students without prior experience in planning poker and agile estimation. The improved version was tested with 15 undergraduate students.
3. The third iteration was based on the evaluation results from the previous test and incorporated a number of changes, such as (i) an improved dialog design with more questions and presentation of information in smaller chunks; (ii) further improvement of the explanation by shortening the texts spoken by the robot; (iii) reducing the number of tasks to be estimated from eight to six; (iv) reformulating the tasks to be estimated by adding more information about each task; and (v) introducing more variation in the responses of the robot to make the interaction less predictable and more exciting. The new version was tested with 14 graduate students. Following SBD by [24], the improved design elaborated in more detail how students will perform estimation in teams and how the robot will support group collaboration.
4. The fourth iteration resulted in the final version of the game \"Planning poker with NAO\", which was applied in the study presented in this paper. The narrative script was transformed into the robot application using the Choregraphe software for NAO (Version 2.8.6) and additional Python code for the creation of a PDF document with the results of each estimation.