# EduTVA: Metadata Schema for Educational Audiovisual Contents in Digital Television Environments
## 4 Evaluation
### Acceptance of Educational Metadata  
For evaluating the acceptance of the educational metadata proposed in EduTVA, first, we require a metadata authoring tool to mark up audiovisual content. Although different metadata authoring tools enable marking up content with TV-Anytime, such as M-Tool [44] and ETRI's Tool [45], they lack the educational metadata that this evaluation focuses on. Therefore, we implemented EduTVAtool, a Web-based metadata authoring tool that implements the EduTVA metadata schema (see Section 3) to enable marking up content with the proposed educational metadata. We resolved implementing a Web tool to promote collaborative marking up audiovisual content in future work. EduTVATool allows metadata authors interacting with a Web interface to select an audiovisual resource, to visualize the resource, and to describe the resource (or a segment of the resource) by entering the general and educational metadata. Figure 2, for example, depicts the educational metadata form of EduTVATool for marking up the selected resource. We achieved the last version of the tool after conducting two preliminary evaluations with two experts on usability. Each preliminary evaluation followed the concurrent think-aloud technique [46], that is, users interact with EduTVATool while expressing their thoughts about their interaction aloud; if conducting the evaluation remotely, they write their thoughts down [47]. Both evaluations ran independently and at different moments, using different versions of EduTVATool. Continuing with the evaluation of the EduTVA metadata, a group of 15 participants (different from the preliminary evaluations) used EduTVATool to mark up a given educational audiovisual resource. Then, each participant completed a survey about the educational metadata. This survey also included a questionnaire about the usability of EduTVATool for validating that its Web interface meets the usability criteria. Such a usability assessment allows detecting any bias in the metadata acceptance survey due to usage misunderstanding of the tool. The whole process was under the supervision of an expert. Following the details of the evaluation. * Purpose: the evaluation aims at measuring the level of acceptance of the EduTVA metadata for marking up educational characteristics of the audiovisual resource. This evaluation also validates if participants understand how to describe the educational resource using the Web interface of EduTVATool. * Participants: professors and teachers that incorporate technological and pedagogical competences into the education process. Technological competence refers to the capacity of selecting and understanding a variety of technological tools, and combining them into the education process in a proper and responsible way. Pedagogical competence refers to the capability of using technologies for strengthening the education process while recognizing their benefits and limitations in the professional development of the students [48]. Figure 2: Educational description form of the EduTVATool Web interface. Number of participants: 15, nine university professors and six high school teachers. The number of participants is based on the number of experts and tutors that evaluated SugarTube [49]. Age range: from 28 to 60 years. * Participants use EduTVATool, a Web-based metadata authoring tool that allows describing an audiovisual resource from an educational perspective. - The resource to describe corresponds to the six episode of \"Proyecto G\", a science television program of the educational and cultural channel Encuentro from Argentina [50]. All the episodes of this educational program are freely available for academic use [51]. - EduTVATool provides a default mark up for the general basic description of the resource (e.g., title, genre, actors). - During the test, a supervisor observes the participants' behavior and registers the time for marking up the resource.