British Library Research and Development Report No. 5547

University of Aston, Computer Centre Technical Report No. TR80002

DESIGN OF KNOWLEDGE

STUDY BASED

FOR

AN

ANOMALOUS

STATE SYSTEM

INFORMATION

RETRIEVAL

Final Report on Grant Sl/SG/09 2 May 1978 - 15 June 1979

N.J. Belkin Centre for Information Science The City University

R.N. Oddy Computer Centre University of Aston in Birmingham

September 1979

All opinions and observations are those of the authors, and not necessarily those of the British Library.

-2TABLE OF CONTENTS ABSTRACT Chapter 1. 1.1 1.2 1.3 1.4 INTRODUCTION Project origins Background Project specification Aims of the Design Study 5 6 7 8

Page

Chapter 2.

THEORETICAL BASIS OF THE ASK IR SYSTEM

11

Chapter 3. 3.1 3.2 3.3 3.4 3.5 3.6

METHODS General experimental outline Problem statements Abstracts Evaluation procedures Text analysis - principles Text analysis - procedure 16 16 17 18 18 21

Chapter 4. 4.1 4.2 4.3 4.4

RESULTS Problem statements - general characteristics Problem statements - evaluation Abstracts - general characteristics Abstracts - evaluation 31 35 40 40

Chapter 5. 5.1 5.2

DATA ANALYSIS Classification of ASKs Retrieval strategies 44 45

Chapter 6. 6.1 6.2 6.3 6.4

DISCUSSION Design problems Text characteristics Evaluation of representations Text analysis 52 53 53 54

-3Page Chapter 7. CONCLUSIONS AND FURTHER RESEARCH

56
57 58 60

ACKNOWLEDGEMENTS REFERENCES Appendix A Evaluation package for problem statements Evaluation package for abstracts Summaries of problem statements Association map representations of problem statements Single-link clustering representation of problem statements Abstracts Association map representations of abstracts

Appendix B Appendix C Appendix D

65 68 87

Appendix E

123

Appendix F Appendix G

159 195

FIGURES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. An ASK-based IR system A model of IR The transcript of a problem statement Token processing Calculation of association strengths The strongest associations from the problem statement in Figure 3 Association map for problem statement in Figure 3 Association clusters for problem statement in Figure 3 An example abstract 9 12 20 23 24 25 27 28 29 30 48 49

Association map for abstract in Figure 9 Condensed p.-oblem statement network - 1 Condensed problem statement network - 2

TABLES I. 2. 3. Token - token and type - token ratios for oral problem statements. Association strengths and number of types for oral problem statements Token-token and type-token ratios for written problem statements 32 33 34

-4-

4. 5. 6. 7. 8. 9. 10. 11. 12.

Association strengths and number of types for written problem statements Subject areas of interviewees Association map evaluation (Problem statement) Association cluster evaluation (Problem statement) Format comparison Token-token and type-token ratios for abstracts Association strengths and number of types for abstracts Abstract representation evaluation Summary of ASK types

34 36 37 38 39 41 42 43 46

ABBREVIATIONS ASK CIS IR PS Anomalous state of knowledge Central Information Services, University of London Information retrieval Problem structure