# Learn-CIAM: A Model-Driven Approach for the Development of Collaborative Learning Tools
## 5 Validation
### Results of the First Experiment
Regarding the questions about the subjective perception of the potential users with respect to the suitability and completeness of the modelling elements related to awareness and pedagogical usability supported by the proposal, the answers were positive (**RQ7**), with the majority of the participants considering the number of awareness and pedagogical usability components as adequate and sufficient. In relation to the technological framework, regarding the difficulty of applying the modeling, validation and generation stages of collaborative learning tools supported by Learn-CIAT (**RQ8**), a greater disparity in the results can be seen, denoting certain doubts in this regard. Doubts arise specially on the self-explanatory nature of the Learn-CIAT user interface (TEC8), CS experts being most critical about it. However, it is noteworthy that 50% of the ED experts were in full agreement with this statement. Furthermore, with regard to the quality of the products generated as a result of applying Learn-CIAM (**RQ9**), the results were positive in general. None of the CS experts complained about its quality, and only half of the ED experts disagreed on aspects about the user interface (TEC4) and the mechanisms to regulate the shared workspaces (TEC7). This may suggest, again, that experts with a more technical profile are more familiar with these kinds of tools. In view of the results highlighted above, it is noteworthy how both profiles show interest in the methodological approach (MET1) and consider the Learn-CIAN notation adequate (CON2), the latter being the most important conceptual part of Learn-CIAM. At the same time, we observe a greater variation of the results of the capacity of the users with a less technical profile to use this proposal, denoting their doubts, but without presenting bad results (variables MET6 and TEC8) since most of them scored a three or higher. Another important aspect of this experience to consider, beyond the previous results, was the comments, complaints, recommendations, and improvements proposed by the experts. With respect to the methodological framework, two variables were referred to. Hence, in view of question MET3, two users agreed that limiting the solution to the Eclipse environment introduces a strong dependence, although they see it as reasonable. While it is true that the first phase of the methodological framework (the generation of graphical editors) is limited to the Eclipse platform by Eugenia technology, the second phase would be able to generate collaborative tools on other platforms besides Eclipse. Therefore, this limitation occurs only in the first phase of the proposed method. On the other hand, according to variable MET6, up to six people said that, despite agreeing with the statement, it might be necessary to support these types of users in some of the stages. In the section related to the conceptual framework, there were, again, two questions on which the experts made comment. In response to question CON4, one expert recommended adding a new widget that showed the statement of the task. With respect to question CON6, another expert indicated that some pedagogical usability criteria could be omitted or simplified. The latter has already been discussed in this paper: not all the criteria selected initially could be technologically supported. Nevertheless, despite being criteria that may not serve to guide users through the construction of collaborative tools, they are useful for the design of learning courses (imagine that a professor does not need to generate a collaborative tool but does need to design the flow of activities to be carried out in the course). The technological framework received the most comments from the experts. To illustrate, in response to question TEC2, one of the participants mentioned that it would be desirable that the validation process be fully automatic after a change has been made to the diagram. Currently, the chosen technology requires this small process to be carried out manually.