DOCTORAL ABSTRACT

Two Information Retrieval Learning Environments:
Their Design and Evaluation
Kai Halttunen
Department of Information Studies
University of Tampere, Finland
kai.halttunen@uta.fi

The design and evaluation of two information retrieval (IR) learning environments took place in a basic
course of IR (6 ECTS credits) at the Department of Information Studies at the University of Tampere. The
course consisted of lectures, web exercises and tutored search exercises. Tutored exercises were carriedout in two learning environments, in a traditional and in an experimental one. The participants in thestudy (n=57), were undergraduate students of information studies at the university.
The pedagogical design of the experimental learning environment was based on the ideas from anchoredinstruction and intentional scaffolding. Anchoring related the search exercises to a simulated journalistic
work-task situation. Scaffolding, i.e. various ways of supporting learners in proceeding with their task
was provided by an instructional tool, the IR Game, or by a teacher. The IR Game provides query
performance feedback to the learner, based on predefined topics and a recall-base of known relevantdocuments for the topics. The pedagogical design of the traditional learning environment consisted ofsearch exercises on several operational IR systems with unintentional scaffolding.
The effect of the learning environments on the students' learning experiences, performance and learning
outcomes was evaluated. The evaluation was based on seven different datasets gathered in the basic
course on IR. The datasets consist of two essays, answers to a questionnaire, results of a learning styleinventory, log-files from two search sessions, stories describing learning experiences and answers to acourse feedback questionnaire. The data were analyzed by qualitative and quantitative methods.
The results indicate that anchoring and scaffolding are promising strategies in IR instruction. Participants
studying in the experimental learning environment took the view that anchored instruction increased
meaningfulness of learning tasks. The overall effectiveness of queries in the search exercises was slightly
better in the experimental environment and the students made far fewer semantic knowledge errors than
the students in the traditional learning environment. Students from both environments made quite thesame number of syntactic knowledge errors. The change in students' conceptual IR know-how was also
larger in the experimental learning environment. The results of the study, in terms of the benefits ofanchoring and scaffolding, are not categorical because of the range of intervening variables and thedifficulty of setting up a field experiment which tried to be naturalistic but at the same time tried to focus
on a specific aspect – the differences between the IR learning environments.

ACM SIGIR Forum59

Vol. 39 No. 1 June 2005

Publications
Halttunen, Kai: Two information retrieval learning environments : their design and evaluation. Ph.DDissertation. Acta Universitatis Tamperensis, vol. 1020. 2004. Dissertation consists of introduction andsummary of studies and four original research articles. Available: http://acta.uta.fi/
Halttunen, Kai & Sormunen, Eero. Learning information retrieval through an educational game : isgaming sufficient for learning? Education for Information (18) 2000:4, 289-311.
Halttunen, Kai. Students' conceptions of information retrieval : implications for the design of learning
environments. Library and Information Science Research (25) 2003:3, 307-332.
Halttunen, Kai. Scaffolding performance in IR instruction : exploring learning experiences andperformance in two learning environments. Journal of Information Science (29) 2003:5, 375-390.
Halttunen, Kai & Järvelin, Kalervo. Assessing learning outcomes in two information retrieval learning
environments. Information Processing and Management (41) 2005:4, 949-972.

ACM SIGIR Forum60Vol. 39 No. 1 June 2005