Chapter 7 Future directions
7.1 Transaction logging and enhancements

The main finding of the project has been to establish that a basic screen logging facility is not an adequate tool for evaluating user performance at the library catalogue. Even though the entire searching activity at the terminal can be captured, it can only partly inform on user searching behaviour. Hence transaction logs can only be used as an effective evaluative method with the support of other means of eliciting information from users. The enhancement of logs for use in the evaluation of online catalogues can be achieved in several ways: a) Logged searches can be replayed allowing for the user's perception of the retrieval task and interaction with the system to be scrutinized in a simulated environment thus eliminating the risk of interfering with or influencing the search. b) Online pre-search or post-search questionnaires can be introduced to inform on the user's overall objectives in carrying out a search, as well as to provide predictive information on the the searching activity at the shelves. c) Online interactive questionnaires can be formulated to examine specific types of interaction during the searching process which can then be correlated with other elements of the information seeking behaviour. Each of these enhanced screen logging facilities can be applied in different evaluative environments either for monitoring or diagnostic purposes. Where as the replays require intensive experimental resources and can only be carried out on a small scale for qualitative data collection, the online methods can be applied on a much wider scale to collect more comparative quantitative data.

7.2

Evaluation in a holistic context

Whatever method or however they are applied, the evaluation of online library catalogues requires, by the very nature of the interactive system, that the user is taken
63

fully into account (Belkin &: Vickery, 1985). This implies the assessment of system performance as being dependent on user performance. User performance must also be considered in the context of the whole information seeking activity and not just at the point of intersection with the catalogue, (Hancock-Beaulieu, 1989). Moreover the holistic approach also encompasses the need to explore the user's internal cognitions. It is hoped that future research into online library catalogues will take up the call for a user driven paradigm, (Dervin & Nilan, 1986).

64