Indexing Learning Objects: Vocabularies and Empirical Investigation of Consistency

In addition to the LOM standard and instructional design specifications, as well as domain specific indexing vocabularies, a structured indexing vocabulary for the more elementary learning objects is advisable in order to support retrieval tasks of developers. Furthermore, because semantic indexing is seen as a difficult task, three issues concerning consistency in indexing learning objects were empirically investigated: 1) the extent to which different indexers annotate in the same way; 2) the extent to which structure in value lists supports consistent indexing; and 3) different degrees of consistency in annotating various media types and attributes. The results show that a standardized value list does not necessarily lead to a consistent application to learning objects. Differences occur, especially for more abstract attributes and media types. Ontologies can contribute to a higher consistency in indexing and could improve retrieval by making concepts that are more abstract available. ********** The Internet is becoming an important source of e-learning material, as is evident from the steadily growing number of learning object repositories and an increasing market interest. To fully realize the opportunities that are available for distribution and re-use of these resources, several problems must be resolved. First, there should be agreement about what constitutes a learning object. In addition, a knowledge-rich indexing structure for the more elementary learning objects, such as a paragraph of text or a single image, may help developers of learning material to retrieve these fragments more easily. As Polsani (2003) points out: "It is important that the developers agree on a set of specifications for development of learning objects covering such areas as technology, editorial requirements, and stylistic considerations. A commonly agreed standard will enable genuinely sharable and reusable content objects. " Mohan and Greer (2003) essentially stress the same issue: "There is yet no standard way to specify content in a learning object using XML. Moreover, the problem does not end with a set of commonly understood content tags. Content for different domains will need different markup tags. " However, even if there is a standard, this does not guarantee that the application of this standard to the same learning objects by different people will yield the same outcomes. If there are differences, this will be detrimental to the effectiveness and efficiency of retrieval for re-use. By combining theoretical and empirical research, this paper addresses these three issues. The first part is a theoretical discussion about what constitutes a learning object and what kind of annotation structure is required to classify learning objects of an elementary grain size. An annotation structure is defined as a collection of attributes (i.e., data elements) and attribute values. The possible attribute values are defined in an indexing vocabulary (i.e., vocabulary value spaces). Several indexing vocabularies that are available to classify learning objects of an elementary grain size are reviewed. In addition, an example of a knowledge-rich vocabulary, which is based on ontologies for fragments of technical manuals that are used for developing lesson material, is provided. Although a knowledge-rich indexing vocabulary for document fragments can improve retrieval performance and to some extent lead to a better product created from the retrieved material (Kabel, Wielinga, de Hoog & Anjewierden, 2003), there is a downside in the effort needed to annotate material. Automatic indexing with ontologies can be achieved to some degree (Anjewierden & Kabel, 2001), but most semantic tagging still must be done manually. As is stressed by the ARIADNE foundation, one of the practical problems that arises when a metadata system is widely used is indexation (i.e., the creation of the metadata by humans), which should be as easy as possible. …