British Library Research and Development Report No. 5547

University of Aston, Computer Centre
Technical Report No. TR80002

OFDESIGN

STUDY

FORAN

ANOMALOUSKNOWLEDGEBASEDINFORMATIONSTATERETRIEVALSYSTEM

Final Report on Grant Sl/SG/09
2 May 1978 - 15 June 1979

N.J. Belkin
Centre for Information Science
The City University

R.N. Oddy
Computer Centre
University of Aston in Birmingham

September 1979

All opinions and observations are those of the authors, and notnecessarily those of the British Library.

-2TABLE OF CONTENTS

Page

ABSTRACT
Chapter 1.

INTRODUCTION

1.1

Project origins

51.2

Background

61.3

Project specification

71.4

Aims of the Design Study

8

Chapter 2.

THEORETICAL BASIS OF THE ASK

11

IR SYSTEM

Chapter 3.

METHODS

3.1

General experimental outline

163.2

Problem statements

16

3.3

Abstracts

173.4

Evaluation procedures

183.5

Text analysis - principles

183.6

Text analysis - procedure

21

Chapter 4.

RESULTS

4.1

Problem statements - general characteristics

314.2

Problem statements - evaluation

35

4.3

Abstracts - general characteristics

40

4.4

Abstracts - evaluation

40

Chapter 5.

DATA ANALYSIS

5.1

Classification of ASKs

445.2

Retrieval strategies

45

Chapter 6.

DISCUSSION

6.1

Design problems

52

6.2

Text characteristics

53

6.3

Evaluation of representations

53

6.4

Text analysis

54

-3Page
Chapter 7.

CONCLUSIONSANDFURTHERRESEARCH

56

ACKNOWLEDGEMENTS

57

REFERENCES

58

Appendix A

Evaluation package for problem
statements

60

Appendix B

Evaluation package for abstracts

65

Appendix C

Summaries of problem statements

68

Appendix D

Association map representationsof problem statements

87

Appendix E

Single-link clustering representationof problem statements

123

Appendix F

Abstracts

159

Appendix G

Association map representations ofabstracts

195

FIGURES
1.

An ASK-based IR system

92.

A model of IR

123.

The transcript of a problem statement

204.

Token processing

235.

Calculation of association strengths

24

6.

The strongest associations from theproblem statement in Figure 325

7.

Association map for problem statementin Figure 327

8.

Association clusters for problem
statement in Figure 328

9.

An example

29

10.

Association map for abstract in Figure 930

11.

Condensed p.-oblem statement network - 148

12.

Condensed problem statement network - 249

I.

Token - token and type - token ratios fororal problem statements.

32

2.

Association strengths and number of types
for oral problem statements

33

3.

Token-token and type-token ratios for written
problem statements

34

abstract

TABLES

-4-4.

Association strengths and numberof types for written problem statements

34

5.

Subject areas of interviewees

36

6.

Association map evaluation(Problem statement)

37

7.

Association cluster evaluation(Problem statement)

38

8.

Format comparison

39

9.

Token-token and type-token ratios
for abstracts

41

10.

Association strengths and number of types
for abstracts

42

11.

Abstract representation evaluation

43

12.

Summary of ASK types

46

ABBREVIATIONS
ASK

Anomalous state of knowledge

CIS

Central Information Services, University of London

IRInformation retrieval

PS

Problem structure