# Assessment of Knowledge and Competencies in 3D Virtual Worlds: A Proposal

María Blanca Ibáñez, Raquel M. Crespo, Carlos Delgado Kloos

► To cite this version:

María Blanca Ibáñez, Raquel M. Crespo, Carlos Delgado Kloos. Assessment of Knowledge and Competencies in 3D Virtual Worlds: A Proposal. IFIP TC 3 International Conference on Key Competencies in the Knowledge Society (KCKS) / Held as Part of World Computer Congress (WCC), Sep 2010, Brisbane, Australia. pp.165-176, 10.1007/978-3-642-15378-5_16. hal-01054702

HAL Id: hal-01054702

https://inria.hal.science/hal-01054702

Submitted on 8 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Assessment of Knowledge and Competencies in 3D Virtual Worlds: A Proposal

María Blanca Ibáñez, Raquel M. Crespo and Carlos Delgado Kloos

Departamento de Ingeniería Telemática, Universidad Carlos III de Madrid
Av. Universidad, 30. E-28911 Leganés, Madrid, Spain
{mbibanez, rcrespo, cdk}@it.uc3m.es

**Abstract.** Digital natives demand a more active approach to learning. Moreover, the acquisition and assessment of competencies, rather than the mere transmission of information, is becoming more relevant in the Knowledge Society. 3D virtual worlds are a promising environment to meet both of these requirements. In a 3D virtual world, learners are immersed in a rich environment that allows them to have an active experience through their avatars and interaction devices. The learning process in traditional learning management systems has been widely studied, but there is relatively little literature about the use of 3D virtual worlds for learning, although the expectations are high and the possibilities opened immense. This paper focuses on an important part of the teaching and learning process: the assessment. Our aim is to present a set of techniques adapted to this novel 3D medium that allows assessing knowledge, skills, and competencies by using the elements inherent to 3D virtual worlds ( avatars, synthetic characters, smart objects) and take advantage of the new dimension introduced.

**Keywords:** Assessment of knowledge. Assessment of competencies. 3D virtual learning environments.

## 1 Introduction

Learning is nowadays shifting from merely transmitting knowledge to an active, learner-based approach. Simultaneously, new requirements posed by the Knowledge Society and lifelong learning for professional development emphasize the need of higher level abilities beyond just memorizing facts or understanding concepts. In addition, the new generation of learners, the so-called Digital Natives [1] with different patterns of work, attention, and learning preferences [2] are demanding the introduction of new technologies in the teaching-learning process.

Despite this revolution in learning conditions and requirements, Learning Management Systems continue to be predominantly “flat”: contents are mostly delivered by means of texts, assessment questions and student responses are texts, etc. Recent advances have brought multimedia content into the scene (images, video, audio), which has brought more color and animation, but has not dramatically changed in particular how assessments are carried out and what can be assessed.
3D virtual worlds can be a solution to the motivation problem posed by the new generation of learners, but also to the activities that can be carried out to transmit knowledge and competencies. 3D worlds have been around for a long time, but thanks to advances in hardware performance and software development it has been only recently that they have evolved from systems with costly infrastructure to applications that can run on a personal computer either as stand-alone applications or even on top of web browsers. Relevant organizations have raised public interest in 3D virtual worlds for learning and teaching. From the business world, Gartner Research Group [3], a technology-related research and consulting firm, predicted that by 2011, 80% of internet users (and Fortune 500 enterprises) will be involved in virtual worlds. From the world of education, EDUCAUSE [4], a nonprofit association whose mission is "to advance higher education by promoting the intelligent use of information technology", claims that these virtual worlds will have a large impact on teaching and learning in a near future.

The advent of 3D virtual worlds has introduced new challenges for e-learning researchers such as how to use these environments effectively to transmit knowledge and promote cooperation among students. This area of research has been identified as central to building effective virtual learning systems for education and training [5]. Platforms such as Second Life [6], Croquet [7], Wonderland [8] have been successfully used for deploying 3D virtual worlds for learning [9], [10], [11], [12]. These initial experiences focus on simulation of real world environments, collaborative activities and even some aspects of learning management. In general they however lack from assessment mechanisms suited to 3D virtual worlds.

A side-project of SLOODLE 10 called quizHUD [13] can be considered as a pioneer to assess the acquisition of knowledge using not only classical interfaces in multiple choice questions, but also mechanisms inherent to 3D virtual worlds such as immersive navigation and choice of 3D objects.

Nevertheless, a rich medium such as 3D virtual worlds is expected to be useful not only to transmit knowledge, but also to promote and evaluate higher level competencies. 3D virtual worlds have been used for promoting collaborative activities, for providing students with a rich set of resources that can be used in non-standard ways and now there is a great expectation in relation to how these 3D worlds can be used for promoting and assessing competencies. Competencies have been treated in classical e-learning environments [14], for example as part of the TEN Competence Project [15] to detect the learner's competence proficiency level and suggest a set of activities the learner had to practice in order to acquire the needed competencies. Thus, this is a promising area of research but, to our knowledge, there have been no attempts to bring this competencies-based approach to 3D learning and assessment.

This paper is organized as follows. After this introduction, the terminology for learning outcomes is discussed in Section 2, differentiating knowledge, skills and competencies. Section 3 describes the elements that distinguish 3D virtual worlds from standard environments. New ways of using 3D virtual world elements such as avatars, non-player characters (NPCs), and smart objects for assessing knowledge, skills and competencies are outlined in Sections 4, 5 and 6 respectively. An illustrative case study is presented in Section 7. Finally, conclusions and future work are discussed in Section 8.
## 2 What do we Assess? Knowledge, Skills and Competencies

Two big efforts have been carried out in order to provide a formalized and homogeneous framework for describing learner's abilities. The European Commission for Education & Training (ECT) [16] has developed a reference framework with eight reference levels describing for each one the *knowledge, skills and competences* that a learner must have. On the other hand, the U.S. Departments of Labor and Education formed the Secretary's Commission on Achieving Necessary Skills (SCANS) to study the kinds of competencies and skills that workers must have to succeed in today's workplace [17]. The know-how identified by SCANS is made up of five competencies that effective workers can productively use: resources, interpersonal skills, information, systems and technology.

The European Qualifications Framework (henceforth EQF) [18] was born in pursue of a twofold aim: promoting citizens' mobility between countries as well as facilitating lifelong learning. The EQF is meant to provide a unified and homogeneous reference framework for relating and comparing qualifications. To accomplish this goal and assuming the distance between the different national qualifications systems implemented in Europe, it applies a learning outcome based approach, emphasizing the results of learning rather than focusing on inputs related to the learning process itself (such as length of study).

According to the EQF [18], *learning outcomes* means statements of what a learner knows, understands, and is able to do on completion of a learning process. They are defined in terms of knowledge, skills and competence, where:

* *knowledge* means the outcome of the assimilation of information through learning. Knowledge is the body of facts, principles, theories and practices that is related to a field of work or study. In the context of the European Qualifications Framework, knowledge is described as theoretical and/or factual [18];

* *skills* means the ability to apply knowledge and use know-how to complete tasks and solve problems. In the context of the European Qualifications Framework, skills are described as cognitive (involving the use of logical, intuitive and creative thinking) or practical (involving manual dexterity and the use of methods, materials, tools and instruments) [18];

* *competence* means the proven ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development. In the context of the European Qualifications Framework, competence is described in terms of responsibility and autonomy [18].

Whereas the EQF initiative provides a generic reference framework and clarifies the terminology, SCANS focuses on identifying concrete competencies which are particularly relevant for professional activity.

At the request of the U.S. Departments of Labor and Education, the SCANS commission conducted a study to determine the competencies and skills that workers must attain to succeed in today's workplace [17]. Although the commission completed its work in 1992, its findings and recommendations continue to be a
valuable source of information for individuals and organizations involved in education and workforce development [22]. The results of this study are summarized in **Table 1**.

The SCANS study claims that these five competencies requires basic skills (reading, writing, arithmetic and mathematics, speaking and listening), thinking skills (thinking creatively, making decisions, solving problems, seeing things in the mind's eyes, knowing how to learn, and reasoning) and personal qualities (individual responsibility, self-esteem, sociability, self-management and integrity).

<table><caption>**Table 1** Excerpted from What Work Requires of Schools: A SCANS Report for America 2000, U.S. Department of Labor, June 1991, pp. xvii-xviii.</caption><thead><tr><th>Competency</th><th>Meaning</th><th>Components</th></tr></thead><tbody><tr><td>Resources</td><td>Identifies, organizes, plans, and allocates resources</td><td>Time<br>Money<br>Materials and facilities<br>Human resources</td></tr><tr><td>Interpersonal</td><td>Works with others</td><td>Participates as a member of a team<br>Teaches others new skills<br>Serves clients/customers<br>Exercises leadership<br>Negotiates</td></tr><tr><td>Information</td><td>Acquires and evaluates information</td><td>Acquires and evaluates information<br>Organizes and maintains information<br>Interprets and communicates information<br>Uses computers to process information</td></tr><tr><td>Systems</td><td>Understand complex interrelationships</td><td>Understand systems<br>Monitors and correct performance<br>Improves or design systems</td></tr><tr><td>Technology</td><td>Works with a variety of technologies</td><td>Selects technology<br>Applies technology to task<br>Maintains and troubleshoots equipment</td></tr></tbody></table>

Although the SCANS study deals with competencies and skills for work, the proposals can be applied to the core curriculum area: Language, Mathematics, Science, Social Studies/Geography and History [23]. The competencies and skills discussed can thus be promoted and assessed in any course. Detailed information on assignments that integrate the SCANS competencies into the core curriculum area can be found in [23].

Assessment should therefore evaluate the performance of the learner for each of these aspects (knowledge, skills, and competencies). Considering the intrinsic differences of such diverse categories of learning outcomes, distinct assessment methods would apply. Although knowledge can be properly appraised using traditional exams, assessing skills and competencies poses more difficult challenges and requires complex activities. 3D worlds can help to accomplish this objective as they allow a cost-effective simulation of real-world scenarios and situations, including not only physical objects, but interaction and collaboration with colleagues too. As will be discussed in following sections, additional advantages appear related to the technical facilities provided by virtual worlds.
In this paper, the EQF terminology is followed to distinguish the different
categories of learning outcomes defined there<fnref n="1" />. Since the SCANS document defines
the set of competencies more precisely than the ECT document, the SCANS proposals
are nevertheless used as illustrative cases for presenting new ways of using 3D virtual
worlds to assess these competencies.

3 3D Virtual Worlds Contributions to the Assessment Process

3D virtual worlds have a set of elements –also present in video games– that may help to seduce students into learning. The use of avatars as representations of students (and instructors) and the possibilities for customizing avatars enhance the degree of realism and permit the subject to immerse him/herself in the 3D virtual environment for learning. The 3D scenarios also help to the sense of immersion and open up options of discovering new knowledge by exploring virtual worlds. The ability of avatars to interact with virtual 3D objects in collaborative environments offers new possibilities for the use of learning theories such as experiential learning, discovery learning and constructivism [19] and opens new possibilities for assessment.

3D virtual scenarios can be filled with information through text, audio, and images and also through 3D scenarios, smart objects, and NPCs. Technical elements used in video-games such as small-map, cursor indication and NPCs playing dialogues are useful to aid in orientation, to show objects with extra information and to transmit knowledge respectively. These elements are particularly useful in the assessment stages in which information should be given to students. Besides the aforementioned, one feature that distinguishes 3D virtual environments is the possibility of synchronous or asynchronous interaction of one or more users through 3D objects; this is suitable for monitoring student responses.

Non-player characters (NPCs) can improve *immersiveness* by acting in plausible ways; its activities can be programmed using artificial intelligence (AI) methods. Deterministic AI techniques such as finite states machines, decision trees or even fuzzy logic are currently being used in video games and non-deterministic techniques such as Bayesian networks, neural networks and genetic algorithms are starting to be exploited in these games [20]. 3D virtual learning environments should find which kind of synthetic characters could be used and what AI technique can be applied for programming their behaviors. The role of NPCs in the assessment process is especially important when the question is stated; when the student requires some help to answer it and also when feedback is provided.

In summary, 3D virtual worlds constitute a novel user interface, characterized by their high potential for immersiveness. They provide the learner with new interaction mechanisms with the rest of elements in the learning process: content, peer students and instructors. Such new mechanisms can be applied not only for teaching and training but also for deploying more effective methods of learning assessment. The characteristics that raise this potential can be categorized into two key features:

<footnote n="1">¹ Please note that although SCANS uses the term “competencies” while “competences” is used in EQF, both terms are meant to refer the same underlying concept and can thus be considered synonymous in this context for the purpose of this paper.</footnote>
* *Novel user interface:* the learner can interact with 3D objects and scenarios that allow a realistic simulation of real world objects and situations. The experience can be further improved by means of haptic interfaces which provide an even more realistic interaction. New interaction mechanisms allow new kinds of tasks and activities, facilitating the assessment of practical skills involving manual dexterity or the use of (simulated) tools and instruments.

* *Monitored environment:* all actions, interactions and events in the virtual world can be monitored and logged for future analysis or assessment<fnref n="2" />. This functionality provides a huge advantage compared to the physical, real world, where such information is lost and unavailable for the instructor, and a powerful assessment resource. Students themselves can base on the logged data for supporting assertions on their performance and achievements. The assessment of interpersonal competencies is an obvious application of these logs. But they will also allow the instructor to distinguish individual contributions to team work (providing a fairer evaluation). Or, beyond the analysis of peer interactions, they can provide invaluable information about the student evolution, allowing the assessment of his/her planning and organizational competencies.

Therefore, the challenge is how to use these unique features of 3D virtual worlds to develop more effective assessment techniques useful to the learning process.

# 4 Assessment of Knowledge

An assessment item typically consists of the following elements:

* A prompt

* A response template (which can consist of choices to choose from, some part to be filled in, etc.)

* A correct response or response model

* Feedback and grades for different response options

Now, most of these parts are typically text. Prompt, correct response and feedback can be directly text. A multiple choice test contains several possible texts, from which the student has to choose the correct one or ones, a fill-in-the-blank response template is a text with a hole.

A richer kind of assessment complements or replaces text with images. So, we can have a multiple-choice question with image hot spot rendering, where the student has to click on the right part of the image to respond the question. Analogously, the prompt or the response can be in audio or video form. These improvements provide a much richer way to assess the student's learning. But all these improvements are essentially directed towards assessing the knowledge in classical environments; 3D virtual worlds offer new possibilities that are worth being explored.

<footnote n="2">² Of course, privacy considerations should be taken into account and the students must be previously warned and agree to such recording.</footnote>
The prompt of the assessment item can be provided by an NPC stating a question orally and students can take all the opportunities they need to hear it again. Even more, the NPC might point at elements of the environment; it might interact with objects or other NPCs in the world to state questions.

A response template used to request a response can be composed by a set of places that the student should visit, or a set of 3D objects to point to, take, move or modify in order to answer the question. Thus, essentially what is necessary is to have mechanisms to highlight the areas and objects of the world which correspond to the different options given to the students.

In 3D virtual worlds, students can interact with the environment, the objects, NPCs and fellows through their avatars. Any interaction that occurs is likely to be monitored in the virtual world and thus can be used to evaluate the student. Among the interactions that can be detected by an assessment engine we can mention any mouse or keyboard event, the path followed by avatars, changes in the placement, size, color of 3D objects, and any transformation allowed by learning objects created ad hoc such as whiteboards, forums etc.

Providing feedback to individual students can enhance student learning, and in 3D virtual worlds each student can receive feedback through the mechanisms mentioned before for response template. Besides, the student might also receive information about his/her performance in the peer group, or receive recommendations from other students to improve his/her performance; all this is related with social learning aspects of these environments [21].

Therefore, the new interface and elements provided by 3D virtual worlds can be applied to change the way the assessment is carried out. More natural ways of presenting information to learners, new possibilities to test their knowledge and new techniques for monitoring their progress are now available in these 3D environments that are not present in 2D systems.

# 5 Assessment of Skills

As explained in Section 2, *skills* are defined, according to the EQF, as the ability to apply knowledge and use know-how to complete tasks and solve problems, and can be classified into cognitive and practical. Thus, skills cover a wide variety of learning outcomes. The assessment of cognitive skills, involving mental processes, logically requires different activities to be done by the learner than the evaluation of practical ones, which are related to physical activities.

The SCANS study reports a set of skills required to fulfill the fundamental competencies identified, ranging from basic skills (reading, writing, arithmetic and mathematics, speaking and listening) to thinking skills (thinking creatively, making decisions, solving problems, seeing things in the mind's eyes, knowing how to learn, and reasoning) and personal qualities (individual responsibility, self-esteem, sociability, self-management and integrity).

The requirements for the assessment of these skills depend on their concrete nature. For some of them, mainly basic skills, assessment methods similar to the ones applied for knowledge evaluation can be used. For example, reading comprehension
is traditionally assessed by means of an exam consisting on multiple-choice questions enquiring about the meaning of the text. Analogous considerations could apply to other skills of similar kind, such as listening or arithmetic and mathematics skills. In these cases, the ideas discussed in Section 4 for knowledge assessment apply.

There however exist other kinds of skills where such assessment methods are no longer suitable. Thinking skills such as thinking creatively or personal qualities such as self-management are not easily evaluated and require complex activities and processes to be completed by the learner in order to provide indicators of his/her performance.

These types of skills are closely related to competencies (in fact, the EQF defines competencies in terms of responsibility and autonomy) and pose similar requirements for their assessment. Therefore, assessment strategies discussed in Section 6 for evaluation of competencies fit to skills assessment as well.

Additionally, practical skills involving manual dexterity and the use of methods, materials, tools and instruments can benefit from the introduction of haptic interfaces. Simulators are a solution for assessing (as well as acquiring) skills related to the use of tools and instruments that may be expensive or even dangerous (aviation or spaceships are typical examples). Frequently, these simulators are however expensive themselves. The combination of the physical interaction provided by haptic devices with the inmersiveness provided by the 3D worlds facilitate the deployment of realistic virtual simulations that allows the evaluation of this kind of skills.

# 6 Assessment of Competencies

A 3D virtual world typically has a scenario filled with 3D objects that can be explored, manipulated by avatars populating the world. When the world is used for deploying a learning sequence, there are physical resources on it that must be used by students following a set of constraints such as limited time to carry on the activity, amount of money to spend on it, or the quality and quantity of human resources. Time can be easily controlled by the simulation of the learning experience. Resources such as money can be introduced following video-games techniques. Other organizational aspects as how to assign people to tasks can be done through NPCs previously created to play simulation roles. Monitoring the use of these resources does not pose technical problems.

3D virtual worlds have been used extensively as places for meeting and they offer a closed laboratory where students, represented by their avatars, can be fully observed, their interactions can be traced by a monitoring system coupled to the 3D virtual world. Competencies related with interpersonal relationships are subject of study of the area Computer-Supported Collaborative Learning and there are already CSCL scripting patterns [24] and authoring systems [25] suitable for Web environments that can be adapted to 3D virtual environments.

Acquiring, evaluating, communicating and handling information in general, are basic competencies inherent to any human activity. The natural workflow establishes mechanisms to acquire and evaluate information determining the accuracy of data collected from appropriate sources, store and maintain the information, and
communicate to others using oral, written, graphic or multi-media methods. In our opinion, the way to assess the quality of sources consulted and how the student interprets, organizes and presents the information in a 3D virtual environment, does not differ to the techniques used in other learning environments.

Probably, understanding of systems is the most complex mental process; it involves realizing the relationships among its components and predicting eventual consequences of any change in its constituent parts. This understanding can be materialized as a physical model that can be built by the student within the 3D virtual world using 3D objects. All building activities and the overall system performance are likely to be assessed in the environments analyzed.

No matter what curriculum area is explored, it is always necessary to know how to handle materials, technology. Probably the best way to teach the most suitable equipment or tools to use, and the best technology to apply for a given task, is by doing. A 3D virtual environment provides a cheap and safe space to deploy simulations where students can be taught to use and experiment with any kind of tools. The introduction of techniques of augmented reality to virtual worlds can facilitate the teaching process [26]. To automate the assessment task, it is necessary to couple to the simulation the logic to measure how well the tool manipulation is done.

Although the assessment of competencies suffers from the difficulties inherent of monitoring the mental process of human beings, 3D virtual environments provide a unique medium to isolate students from information or stimulus irrelevant to the subject of study. Also, they allow monitoring any possible interaction.

# 7 Case Study: Recreating the Industrial Revolution in England

In this section we describe a case study that illustrates how knowledge and the five SCANS competencies can be assessed.

A high school History teacher gives his students a problem: “Recreate the Industrial Revolution in England”. Students work together in small groups; each group tackle a different aspect of the Industrial Revolution: agriculture, manufacturing, mining, transport. They design scenes that include elements such as housing, food, clothing, and simulation of major inventions; they also include NPCs who dramatize situations reflecting how people lived and worked. When students need more information they consult websites provided by their teacher or discovered by themselves. As result of their work, students must organize, apply, produce, and communicate knowledge accurately and creatively, taking into account constraints inherent to the historical period.

First, it is convenient that students pass an assessment of knowledge about the Industrial Revolution, before beginning the activity designed for the assessment of the five SCANS competencies. The test may include questions to detect anachronisms. For instance, ask students to identify 3D objects, NPCs or actions deployed that do not belong to the historical period studied. Another assessment activity may be to dress up NPCs as characters of the 18th or the 19th century at England or to interact with an NPC in a role playing activity.
Once students have passed their knowledge test, we can proceed to assess skills and competencies. Interpersonal competencies are the focus of this educational experience, how students organize themselves into groups to work on their topic, how they distribute the work and finally how they integrate it. These activities can be carried out through student's avatars with the help of social tools embedded in the virtual world and orchestrated by the 3D virtual learning environment via collaborative patterns. A log of avatars conversations/interactions will allow the students to reflect on their work and constitute evidences supporting their performance, facilitating a fair assessment by the teacher of their team-work competencies.

The main resource to manage is the duration of the activity. To help students with this task, the 3D virtual learning environment may establish milestones in the instructional sequence. Besides, it may also provide the students with information about the advances of peer groups. These mechanisms will not only help the teacher to assess students' progress but also help students to self-assess themselves by comparison with their peers.

Information competencies can be assessed by tracking the links accessed, observing how the information is organized in files or how 3D objects included have been tagged. Object tagging can become a social activity where a peer assessment can be done.

Students plot the events that occurred in the Industrial Revolution through the deployment of 3D objects and NPCs dramatizations, avoiding anachronisms and modeling the social forces that led to this historical period. This setting of the scenes is evaluated by the teacher by observing the scene as a whole. In order to assess the individual contribution of each student to the final product, the teacher will have a log provided by the virtual learning environment with the activities that each avatar did.

There are two different aspects tied to technology competencies, the first is students' use of tools provided by the 3D virtual learning environment and the last is the deployment of simulation of inventions during the Industrial Revolution, which needs understanding the way these inventions worked. In this case study, students provide the simulation of a technology thus, the assessment must be based on qualitative criteria, observing the invention simulation.

# 8 Conclusions and Future Work

This paper is mainly focused on presenting the potential use of elements of 3D virtual worlds – avatars, 3D objects, 3D scenarios, NPCs–, besides elements such as text, audio, video, hypertext, widely used in current Learning Management Systems, for assessment in 3D virtual learning environments.

We have presented a set of techniques to assess not only knowledge, but also skills and competencies. The techniques used to assess knowledge are based on the use of 3D elements to represent the four components of any assessment item: prompt response template, response model and feedback. They intend to imitate an ideal teaching-learning-environment where students can interact with a 3D environment to prove their knowledge about a given topic.
Assessment of competencies has not been treated deeply enough even in 2D learning management systems. We have followed SCAN recommendation about what are the set of competencies and skills that are needed for workers to succeed in today's workplace and presented a set of ways of using 3D objects to this particular case of assessment. The key aspect to achieve our goal has been two characteristics of the 3D words, first the possibility of include into the world only the elements that students need for learning, practice and test his/her skills and second the possibility that we have in these environments to monitor any action or interaction students do in the 3D world.

Finally, we have illustrated the use of the techniques presented in a case study taken from a History course to show the feasibility of our proposal. We are currently developing an assessment 3D environment in the Wonderland platform using the ideas presented in this article.

**Acknowledgments.** This research is supported by the following projects: The Spanish project "Learn3: Towards Learning of the Third Kind" (TIN2008-05163/TSI) within the Spanish "Plan Nacional de I+D+I", the Madrid regional project "eMadrid: Investigación y Desarrollo de tecnologías para el e-learning en la Comunidad de Madrid" (S2009/TIC-1650), the European eContentPlus Project "iCoper: Interoperable Content for Performance in a Competency-driven Society" (PPI-2008-A-12). We also wish to thank the Gradient Lab group members for stimulating discussions of the ideas presented herein.

## References

1. Prensky, M.: Digital game-based learning. McGraw-Hill (2001)

2. Vassileva, J.: Toward social learning environments. IEEE Transactions on Learning Technologies 1, 199-214, October 2008. (2008)

3. Gartner, Inc. Gartner says 80 percent of active Internet users will have a "Second Life" in the virtual world by the end of 2011. In: Business Wire, http://www.businesswire.com/portal/site/google/index.jsp?ndmViewId=news_view&newsId=20070424006287&newsLang=en. (2007)

4. New Media Consortium and EDUCAUSE Learning Initiative: The horizon report. http://www.nmc.org/pdf/2007_Horizon_Report.pdf (2007)

5. Lombardi, J. and McCahill, M.: Enabling Social Dimensions of Learning Through a Persistent, Unified, Massively Multi-User, and Self-Organizing Virtual Environment. In: Proceedings of the Second International Conference on Creating, Connecting and Collaborating through Computing (2004).

6. Linden Lab, http://lindenlab.com/

7. The Croquet Consortium, http://www.opencroquet.org/index.php/Main_Page

8. Sun Microsystems, Project Wonderland: Toolkit for Building 3D Virtual Worlds, http://www.projectwonderland.com/

9. Jarmon, L., Traphagan, T., Mayrath, M., Trivedi, A.: Virtual world teaching, experiential learning, and assessment: An interdisciplinary communication course in Second Life. Computers & Education, 53 (1), 169-182 (2009)

10. Livingstone, D., Kemp, J.: Integrating Web-Based and 3D Learning Environments: Second Life Meets Moodle. UPGRADE (European Journal for the Informatics Professional) 9 (3), 8-14. (2008)
11. Open Cobalt Edusim, 3D virtual learning worlds for the interactive whiteboard, http://edusim3d.com/

12. Callaghan, V., Gardner M., Horan B., Scott, J., Shen L., Wang, M.: A Mixed Reality Teaching and Learning Environment. In: ICHL, pp. 54-65 (2008)

13. Bloomfield, P., Livingstone, D.: Immersive learning and assessment with quizHUD. Computing and Information System Journal 13(1) (2009).

14. Santos, P., Llobert, W., Hernández-Leo, D., Blat, J.: QTI for self-assessment and embedded-assessment in competence oriented scenarios: The Agora Case. In: International Conference on Intelligent Networking and Collaborative Systems, pp. 39-45 (2009)

15. TENCompetence, Building the European network for lifelong competence development, http://www.tencompetence.org (2009)

16. European Commission for Education and Training: The European Qualifications Framework, http://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htm

17. The Secretary's Commission on Achieving Necessary Skills U.S. Department of Labor: What Work Requires of Schools. A SCANS report for America 2000. June 1991. (1991)

18. Education and Culture DG (Education and Training): The European Qualifications Framework for Lifelong Learning (EQF). Luxembourg: Office for Official Publications of the European Communities (2008)

19. Kebritchi, M., Hirumi, A.: Examining the pedagogical foundation of modern educational computer games. Computers & Education 51, 1729-1743 (2008)

20. Bourg, D.M., Seeman, G.: AI for Game Developers. O'Reilly Media (2004)

21. Vassileva, J.: Toward Social Learning Environments. IEEE Transactions on Learning Technologies 1(4), October-December, 199--214 (2008)

22. United States Department of Labor. Employment and Training Administration. Secretary's Commission on Achieving Necessary Skills: Background. http://wdr.doleta.gov/SCANS/

23. United States Department of Labor. Employment and Training Administration. Secretary's Commission on Achieving Necessary Skills: Teaching the SCANS Competencies http://wdr.doleta.gov/SCANS/teaching/teaching.pdf (1993)

24. Hernández-Leo, D., Villasclaras-Fernández, E., Asensio-Pérez, J., Dimitriadis, Y., Retalis, S.: CSCL Scripting Patterns: Hierarchical Relationships and Applicability. In: Proceedings of the 6th Int. Conference on Advanced Learning Technologies, pp. 388-392 (2006)

25. Hernández-Leo, D., Villasclaras-Fernández, E., Ascensio-Pérez, J., Dimitriadis, Y., Jorrín-Abellán, I., Ruiz-Requiques, I., Rubia-Avi, B.: COLLAGE: A collaborative Learning Design editor based on patterns. Educational Technology & Society 9(1), 58-71 (2006)

26. Slocum, M.: Augmented reality and the ultimate user manual. The ARMAR project shows how augmented reality can revolutionize learning. O'Reilly Radar. http://radar.oreilly.com/2010/02/augmented-reality-and-the-ulti.html.