H p p e? m cil ± >< F I L B 1 E5 T R L J C T L J R E E 5 Bibliographic fiLes The r u n - t i m e database c o n s i s t s of records each c o n t a i n i n g a d i r e c t o r y , one f i x e d f i e l d and seven v a r i a b l e f i e l d s . The d i r e c t o r y c o n t a i n s the Length of each of the v a r i a b l e f i e l d s . The f i x e d f i e l d c o n t a i n s an encoded c o n t r o l number, language, d a t e , Dewey e d i t i o n number and h o l d i n g s i n f o r m a t i o n . The v a r i a b l e f i e l d s are Ruthor(s) and other names extracted from M R 100/110/111/700/ 710/711. RC Personal names are held as surname and i n i t i a l s . Personal and corporate names are distinguishable. Main t i t l e and subtitle from 245, or occasionally 240 (uniform t i t l e } . This is the only f i e l d apart from the directory and fixed f i e l d which is always non-empty. Publisher and edition from 260 and 250. Series and part titles i n the following order: 440, 490, 248, 250, 260. For records w i t h a n a l y t i c a l entries any contents note C H C 5053 i s incLuded MR here. Dewey numbers from MRRC 082 or a local field. The first Dewey number is followed by a spine label from a local field Cthe structure does not cater for different copies of the same work having different shelfmarks]. Sites and number of copies at each site Subject Headings from 600, 610, 511, 650, 651, 083. (083 in UK MRRC contains verbal feature headings derived from PRECI5 fields.) Indexes to bibliographic files R fairly conventional inverted index system is used. Each index consists of separate primary and secondary indexes, and a postings file. Secondary indexes are blocked files (block size is four kilobytes], with each block containing a variable number of variable length index term records. Each index term record includes a term (which may be a word, a stem, a phrase or a token representing a class of "synonymous" terms], a pointer to the Location of this term's postings in the postings file, pointers to the next and previous index term record, and some information about the roleCs] of this term in bibliographic records and its number of postings. CWhen a term has only a few postings these are directly stored in the secondary index instead of in the postings file. This is not of much significance in a subject search system, but saves disk access when searching, for example, a titLe phrase index, where the great majority of index terms occur in only one bibliographic record.] fl primary index is an indexed sequential file whose records are the last index term in each block of the corresponding secondary index. It is normally held in fast internal memory while a search program is running. Postings are only 32 bits long. The first 24 bits represent the position -83- Hpp 1 File structures of a bibliographic record, and the remaining eight give some indication of the role or position of the index term in this record. The exact function of the role bits varies from index to index. Rn index search proceeds as follows. The sought term is Looked up in the primary index using a modified binary search. Usually it wilL not be found, but what is returned by the primary search is the number of the first primary index record which is greater than or equal to the sought term. Thus if two successive primary index records are i cat i and "dog" and the sought term is "catalogue" the primary search will return the record number of "dog". This will be the number of the secondary index block which will contain "catalogue" if it occurs in the index. Following the primary search the appropriate secondary index block is read from disk. This is scanned sequentially record by record until either an index term record for "catalogue" is found or an index term greater than the sought term is reached. The procedure returns "success" or "failure" and the address of the index term record which has been found. This very simple structure does not lend itself to easy updating, but it is quite fast, requiring one disk access and a mean of about a hundred string comparisons for each term search. Rn otherwise idle machine (Sun 3/50 D can Look up about 30 terms a second. -84- R p p> e? m cl ± >< z I N P U T 2 P R O C E S S I N G This is a more detailed description of the procedures applied to user input outlined in 3.5. Much the same processing is done on bibliographic input during the generation of index terms. Input is put into Lower case and punctuation and spacing are normalized. Repeated blanks are reduced to single blanks. Punctuation not followed by a blank is replaced by a blank Cunless it is in a number, or part of an apparent "initiaLism" Like "u.s.a." in which case it is removed). Initialisrns containing blanks are also preserved Cnu s a" --> "usa"). Rfter this stage the search statement consists only of alphanumerics, blanks, hyphens and apostrophes, the processing of hyphens is fairly elaborate, although they are not very common in user input. Most hyphens are either removed or replaced by a single blank. The sequences " - ", " -" and •- " are all replaced by a blank except when they are apparently part of a range of dates, in which case they are replaced by a single hyphen. Some hyphens are dealt with by look-up Cso the arabic "al-" can be removed, and the system also knows some common prefixes like "post", "micro" and "neo"). Two hyphens representing a dash are replaced by a blank. In other cases hyphens are replaced by blanks if there is more than one of them in a "word" or if a single hyphen occupies position five or greater in a word. RLL other hyphens are removed. Rpostrophes are removed unless they are in the second or the penultimate position in a word. Rmpersand with a blank on each side becomes " and ". RLl other characters are removed. If at this stage the search is "empty" the input box clears and the system awaits further input. The text now consists only of lower case letters, digits, blanks and the occasional hyphen Cin date ranges) and apostrophe. In searching, the resulting pre-processed string is what will be shown to the user as the search. Following the pre-processing stage the text undergoes "weak" stemming and spelling standardization. Weak stemming removes regular english plural and "ingB endings with reasonable reliability. Spelling standardization copes with many of the orthographical differences between British and Rmerican english - for details see CWRLK87b, p653. The additional "strong" stemming - removing suffixes such as 8 Lizlation", "ous" and "ly" - discussed in the earlier report is not used in the current systems, as we concluded that it was rarely beneficial and sometimes detrimental. Strong stemming certainly has a place in searching small or highly specialised databases. One further point about search processing is worth making. In previous Dkapi systems we have always checked searches for duplicate terms ["history of spain and history of portugal" was treated as "history of spain and portugal"). This gave rise to complications in the [admittedly rare) cases when two members of the same synonym class occurred in the same search. When processing "haiti and haitian politics" the system had to display '"haitian" incLuded under "haiti"'. The present systems do not look for duplication. This results in a term which occurs more than once being given Cat Least) twice the weight it would have if it only appeared once. This only applies to the initial search, not to expanded queries, which never contain repeated terms. -85- R p p e n d ± x 3 Q U E 5 T I D N N R I R E 5 1 1. Questions asked before session on dumb system Rt which college are you a student? 2. Which course are you studying? 3. Which year are you in? 4. Do you use a library in connection with your college work? YE5/N0 I f YE5 5. Which library/ies do you use? If MORE T H m ONE 6. Which library do you use most frequently? 7. Does this library have a computer for you to use to look up books? YES/NO If NO 8. What do you use to look up books? If YE5 9. Do you ever use it? YES/NO If YES -86- Rpp 3 10. How often do you use i t ? Questionnaires a. once a term b. c. d. e. once a month once a week a few times a week every day 11. What sort of books do you look up on the computer? a. specific books b. books on a topic c. both of these 12. Have you ever used a computer in any other Library to look up books? YES/NO If YE5 13. Which library was it? 14. Do you ever use any other type of library catalogue? a. b. c. d. YE5/N0 card microfiche printed book ... other 2 1. Questions asked a f t e r s e s s i o n o n dumb system About what proportion of the time did you feel that the computer was useful i n helping you to find bocks which 'were relevant to the essay t i t l e s you were given? a. b. c. d. e. 0% 25% 50% 75% 100% Please give reasons for your answer 2. Did you find it helpful to be able to ask the computer to save details of books for you to look at later? YE5/N0 Comments/if NO please explain why not 3. Did you have any problems using the computer? If YE5 4. Please describe the problems you had. YE5/N0 -87- Rpp 3 Questionnaires 5. Can you think of any improvements which could be made to the system? YE5/N0 If YES 6. Please describe them 3 1. Questions asked a f t e r s e s s i o n on qe system Rbout what proportion of the time did you f e e l that the computer was useful i n helping you to f i n d books which were relevant to the essay t i t l e s you were given? a. 0% b. 25% c. 50% d. 75% e. 100% Please give reasons for your answer. 2. Did you find it helpful to be able to ask the computer to save details of books for you to look at later? YES/hU Comments/if NO please explain why not 3. Did you use the •MORE1 option? If YES YE5/N0 4. Did it help ycu to find more useful books? Comments/if NO please explain why not YES/NO 5. Did you have any problems using the computer? If YE5 6. Please describe the problems you had. YES/ND 7. Can you think of any improvements which could be made to the system? YE5/N0 If YES 8. Please describe them. -88- Ppp 3 4 Questionnaires Questions asked a f t e r session on f u l l system 1. About what proportion of the time did you feel that the computer was useful in helping you to find books which were relevant to the essay titles you were given? a. b. c. d. e. 0% 25% 50% 75% 100% Please give reasons for you answer 2. Did you find it helpful to be able to ask the computer to save details of books for you to look at later? YE5/N0 Comments/if NO please explain why 3. Did you use the 'MORE' option? If YES YE5/N0 4. Did it help you to find more useful books? Comments/if NO please explain why not YE5/N0 5. Did you opt to look at books shelved near the one you had chosen? If YES 6. Did this help you to find more useful books? Comments/if M ] please explain why YE5/N0 YE5/N0 7. Did you have any problems using the computer? If YES 8. Please describe the problems you had. YES/NO 9. Can you think of any improvements which could be made to the system? YES/NO 10. Please describe them. -83- Flpp 3 5 1. Comparison q u e s t i o n s asked a f t e r Questionnaires s u b j e c t had used two systems Which system did you find the easiest to use? a. First b. Second Why was this? 2. Which system did you find the most helpful in finding books which were relevant to the essay titles you were given? a. First b. Second Why was this? -30- H p> p> ee m c::l ± >< T HSK Rrt 1 1. 2. -4 S H E E T 5 Comment on the contribution of art to political propaganda. Did British pop art differ significantly from its American counterpart? What kinds of Gothic were revived in the 19th century? Describe the development of the decorative arts in Britain since 1800. Assess the contribution women have made to art. 3. 4. 5. Hrt 2 1. 2. 3. Was it possible for surrealist painting to fulfil surrealist theory? How do the mass media affect popular arts and culture? Discuss the development of parish church architecture in Britain during the middle ages. Discuss painting as a vehicle for social comment through the ages. Hew important was classical Reman architecture for Bruneileschi? 4. 5. Computing 1 1. 2. 3. 4. Describe how files are organised under the CP/M operating system. Describe the main components of a digital computer system highlighting the main connections between them. What are the strengths and weaknesses of COBOL compared with other structured programming languages? What factors should be taken into consideration in dialogue design if human-computer interaction is to be problem-free? Describe the role of computers in medicine. 5. Computing 2 1. Describe a model of a typical microcomputer indicating which levels use compilation and which use interpretation as their method of program translation. Describe some computer architectures which differ from the von Neumann model. 2. -31- Rpp 4 Task sheets 3. 4. How could the installation of a local area network be of benefit to an organisation? What factors should be taken into account when designing a management information system? How can computer data be transmitted through the telephone network? 5. Engineering 1 1. 2. What is the function of a comparator and what design criteria should it meet? What precautions should be taken when casting metal items to ensure a good quality product? What is the difference between "static pressure" and "stagnation pressure" applied to a fluid and how do they help determine velocity? What equipment is used to manufacture electronic circuit boards? Describe the principles of strengthening processes in metals. 3. 4. 5. Engineering 2 1. What properties must a cutting tool material exhibit in order to remain effective during the metal-cutting process? 2. What factors should the engineer be aware of to ensure safety in the machine workshop and with what legislation should he comply? Describe the high pressure polymerisation of polyethylene. Describe the Frank-Read mechanism of dislocation generation. 3. 4. 5 , When are radioactive isotopes used in engineering? . Life sciences 1 1. 2. 3. Describe the effects insecticides can have on the environment. Compare the relative influence of heredity and culture on intelligence. Discuss the processes by which water, ions etc. move across cell membranes. In what ways has man been seen to adapt to his enviroment? Outline the mechanisms and principles of plant nutrition. 4. 5. Life sciences 2 1. Outline the main issues in the biological control of disease -92- Rpp 4 in plants. Task sheets 2. 3. How accurate are personality tests as predictors of behaviour? Discuss the relationship between host and parasite with specific reference to man. Describe the effects of pollution on the ecosystem of freshwater resources. Discuss the molecular and cellular basis of the immune system. 4. 5. Social sciences 1 1. 2. Discuss the role of multinational companies in the economics of the third world. Were the Germans right to criticise the peace settlement of Versailles in 1319? Outline the role of social factors in illness. What affects the human's ability to remember? Would the introduction of an electoral system based on proportional representation Lead to a more representative government in Britain? 3. 4. 5. Social sciences 2 1. Discuss the role of the mother in the development of the child. 2. How do sociological perspectives shed light on the role education in society? Would perfectly competitive markets ensure maximisation of social welfare? How widespread was the Slump by 1332? national recovery programmes. Compare the various 3. 4. 5. Rssess the significance of the Alliance as a "third force" in British party politics. General 1 CThe general question sheets were used by library and information science students. They are made up from the other question sheets.3 1. When are radioactive isotopes used in engineering? 2. Outline the main issues in the biological control of disease in plants. -33- Rpp 4 3. 4. Task sheets Discuss painting as a vehicle for social comment through the ages. Discuss the role of the mother in the development of the child. What factors should be taken into account when designing a management information system? 5. General 2 1. Describe the role of computers in medicine. 2. 3. 4. Outline the role of social factors in illness. What kinds of Gothic were revived in the 19th century? Compare the relative influence of heredity and cuLture on intelligence. Describe the principles of strengthening processes in metals. 5. -34- H (ZD p e? m cd ± >< 5 E X T R R C T 5 FRCDM F) L O G F I L E R N N D T R T E D Key Lines from the Log are in roman, comments and annotations in italic. The first character in each Line identifies the type of data: I T C S U E L Identification Date and time to nearest second Comment to improve readability Screen identification User searches and commands Elapsed time to nearest second since Last "E" or "Tu Line. Lookup. Contains enough information to reconstruct what was displayed on the screen. 0 Only used here to show number of items with maximum possible weight, minimum good weight and minimum acceptable weight. These figures can be used to deduce how the result of a search was presented to the user. The log I 22732 - May 25 1988 Log number and date T 880525GS5930 Times tamp. Date and time to nearest second. C 15 f Subject number and system identification - inserted by the experimenters. C C ** start T 880525100135 C ** new topic art 1.2 Question reference (see Appendix * ) . The question is 'Did British pop art differ significantly from its American counterpart?'. 3.*J. and presses Return. 5 SCR7.1 Screen is new subject Subject enters search, search input screen (Fig U poo aryt bntish making one correction, timestamp. the class of terms treated as E +62 62 seconds has elapsed since the Lookup results. The 'token' L pop (10), art C1822), §0227 C12666), §0227 represents ynonymous with "british" ("Britain", "Great Britain', : +1 Lookup and merge took a second. C start 0 NMPW = 1 Ntl = 10 NRW = 10 Gt "GB" etc). One record contains all the terms and there are W of 'good' weight. The system has retrieved all the records for 'pop'. Six of them also contain 'art'. The last four records contain 'pop' only, and are all false drops (the Last is about a computer programming language). System reports "1 book matches your search well HO books found altogether)". Ud -95- Rpp5 User chooses 'Display* o p t i o n . 5 BRIEFS Extracts from a transaction Log Brief record display. U1 E +41 5 FULL Having spent nearly 41 seconds looking at the brief display, the user chooses to see the first record in full. It is George Melly's 'Revolt into style: the pop arts in Britain'. Uy Yes, it is relevant. Uy 'Yes, I would like to see books classified near this one.' E +7 Seven seconds looking at full record and answering the two guestions. 5 BR0W5EJEWEY at 700.341 5 BRIEFS Ud E +33 User spends 33 seconds looking at the first screen of brief records, then goes forward one screen (nine records) in classified order. Unfortunately 700.341 is a very general number representing art in the British Isles. S BRIEFS Ud E +4 S BRIEF5 Ud E +4 S BRIEFS Ub Having gone forward two more screens the user reguests 'Back' and returns to the first screen of the records originally found. Reading time for each screen is probablv about 3 112 seconds 14 seconds less display time). E +7 5 BRIEFS U 4 E +7 Looks at fourth original record in full ... 5 FULL Uy ... and judges it relevant. Uy E +6 5 BR0W5EJBEY at 703.046 Despite previous experience again chooses to see classified list ... 5 BRIEFS Ub E +14 ... but after looking at it for 14 seconds goes back to the original list; although some of the records could welt have been useful this is not particularly evident from the one-line displays. 5 BRIEFS U5 E +10 S FULL Uy Un E +14 Chooses another record, but not to see books classified near it. -96- Rpp5 Extracts from a transaction Log 5 BRIEF5 Um E t9 5 MORE E +2 0 NMPW = 0 NOW = 0 NRW -- 6 Tries the "Nope* (query expansion) option. The search takes 2 seconds. It finds six records, but they are apparently not very similar to any of the three records chosen so far. Ud 5 BRIEFS Ub E +33 S BRIEF5 Up E +4 5 RESTRRT Uq E +6 E finish T 880525100553 E ** Finish - duration 2G4 sec(s) C ** start T 880525100553 E ** new topic art 1.3 'What kinds of Gothic wepe revived in the 19th centupy?" 5 5ER7.1 U 13th century gothic E +34 L §0013 (738), centuri 12302), gothic (49), §0013 peppesents "13', '13th', 'XIX' etc. 'centuri' is what the stemming procedure makes of "centupy' OP 'centuries' (it is not shown to the usep like this). E +1 C start 0 NMPW = 2 NGW = 2 NOW = 43 '2 books match your search well (43 books found altogether)" l d i 5 BRIEFS U1 E +22 5 FULL Uy u v record and looks at books classified neap it. Chooses first E +10 5 BRCWSEJEWEY at 720.342 5 BRIEFS U1 E +15 5 FULL Uy E +6 S BRIEFS Ub Having chosen one book from classified display petupns to original list. E +3 S BRIEFS -97- RppS Extracts from a transaction Log U 2 E +6 5 FULL U (RETxDEDy Here user makes two inappropriate keystrokes. Since neither of these keys is used except during input of a search it is possible that the user was trying to decline to answer the relevance question, an unimplemented feature. Uy E +7 5 BROIIEEJEWEY at 726.50342 5 BRIEF5 Ub Has chosen another of the original list, then looked at classified display without choosing any. E +19 5 BRIEF5 Um E +21 5 MORE E +3 0 N P = 0 N W = 3 NflUI = 12 MW G 'More' finds 12 books, three of them probably matching quite w e l l . The system only reports 'Found some more books, 12 altogether*. Ud 5 BRIEFS U6 E +34 5 FULL Uy U n Chooses the s i x t h book from the query expansion search. It i s 'The High Victorian movement in architecture'. E +5 5 BRIEFS Up E +6 5 RESTART Uq E +4 C finish T 880525100915 E * * Finish - duration 196 secCs3 T h i s was a b r i s k a n d d e c i s i v e s e a r c h e r who seems t o f a v o u r s h o r t r e a d i n g l i s t s . He c o m p l e t e d n i n e o f t h e t e n q u e s t i o n s d u r i n g h i s t w o s e s s i o n s , a f e a t e q u a l l e d by o n l y one o t h e r s u b j e c t and e x c e e d e d by n o n e . -98- H pD p> 6B m cd ± >< E3 I N S T R U C T I O N S n 5 5 E 5 5 D R 5 ' Instructions First of all, thank you very much for agreeing to take part in this study. THE BACKGROUND Over the past month we have been running experiments to assess the efficiency of a new version of Okapi, the online public access catalogue. The experiments have consisted of a number of students performing subject searches to produce Lists of books to assist in answering hypothetical essay questions 'which we gave them. We now need the help of people with some reference experience to look through these lists of references and decide to what extent each item would be useful to the person writing the essay. WHAT TO DO Ycu have been given one cr more lists of bibliographic references. Each list represents the combined results of all the searches conducted on the online catalogue on one topic. The appropriate essay question is printed at the head of each list. Please read the question carefully. Then, far each reference: a!) Use your experience as 3 Librarian to make a quick judgment about the usefulness Cor otherwise) of each book to a student writing the essay. b) On the scale printed immediately underneath each reference, tick the option which best describes how you feel about the book Cwe have found that most people take about 10-20 seconds per reference}. When you have finished please return your reference lists to: Rachel de Vere, Polytechnic of Central London, 74 Gt Portland 5t, W1. Rpp 6 (Assessors ' instructions Example r e c o r d sent t o assessors Page 1 Outline the mechanisms and principles of plant nutrition CD [16515713 RUTHORCS): SMITH H TITLEC5D: Regulation of enzyme synthesis and activity in higher plants : proceedings of the Phytochemical Society symposium, Oxford, April 1976. Rnnual proceedings of the Phytochemical Society. Symposia series. PUBLICATION: Rcademic Press, 1977. 5UBJECTC53: Plant proteins - Congresses. Enzymes - Congresses. Plants. Biochemistry. Shelved at : 581.1925 REG — > Very useful/quite useful/slightly useful/not useful/other--please specify -100- H p FD 6=? m c:l ± >< ~7 C L R 5 5 BROWSING RESULTS This i s 5.6 an e x p a n d e d v e r s i o n o f some o f t h e d a t a summarized i n Table Question i s the t o p i c reference CRppendix typed 43. Search statement i s what the user in. by t h e user Number c h o s e n i s t h e number o f from the c l a s s i f i e d d i s p l a y . Rssessment is r e c o r d s chosen r e l e v a n t the experimenter's o p i n i o n of the d i s p l a y Csee 5.5.23. Question 5earch statement Number chosen Dewey number total 1st scr Rssessment comp1.5 comp1.3 comp1.4 comp2.5 0 0 Programming languages cobol 0 0 Human computers [ i e humanComputers i n medicine computer i n t e r a c t i o n ] computer data i n telephone network 0 0 0 D 0 0 0 0 0 0 G I 0 0 I 362.172.. remote 610.2854.. good CO1.6424 scattered 001.64 scattered 621.38 025.002.. 001.54044 621.38 001.64 001.64404 658.40388 001.64 001.54 610.285.. 001.6425 703.22 scattered remote remote scattered scattered scattered good scattered 2 0 1 comp2.2 ccmp2.4 computer architectures 0 G 0 comp1.2 i 0 Management information systemO 1 Computer a r c h i t e c t u r e and interfaces 0 ccmp1.5 compl. 1 art1.5 computers i n medicine cpm operating system women art 1 1 0 1 art1.1 art's contribution to political propaganda contemporary political propaganda 0 0 G 0 0 1 0 2 0 0 G 1 G 0 0 0 0 1 0 1 scattered good scattered possible possible 759 possible 703.22 possible 753 possible 704.042 763.4394.. possible 940.5488 303.375 possible good SGC1.1 sod.4 sad.5 sod.5 multinational companies third world 7 memory proportional representation 0 proportional representation 6 electoral systems 0 0 4 0 3 769.4994.. possible 303.375.. possible 309.097.. remote 338.891.. possible 153.12 324.63 324.21 good good possible 101 Ppp 7 soc2.2 CLass browsing resuLts sociological perspectives education 0 G 3 G 0 G 1 0 1 1 1 1 G 1 G G 1 7 G 0 1 G 1 1 G 0 2 370.1934.. possible 370.1901 370.19 700.941 709.046 720.942 726.50942 769.499.. 747.05 747.213 745.03 good good remote good good possible possible possible possible good possible possible possible possible good possible possible possible possible possible possible possible possible good art1.2 art 1.3 art1.1 art1.4 3 7 pop art british 0 0 1 19th century gothic 0 1 art politics propaganda decorative arts britain 1800 1 1 1 women art treaty of Versailles 1913 art1.5 sod.2 sod.4 sod.5 human memory electoral reform soc2.1 influence of the mother on the child 0 1 G G 1 1 7 0 0 1 0 2 1 0 1 2 759 943.085 943.085 943.085 616.85232 153.12 324.630.. 324.210.. 324.25 324.21 401.9 155.422 155.422 155.418 324.53 sod.5 sod.4 sod.3 sod.1 electoral system proportional representation british government memory in humans illness social factors third world multinational companies economic development 4 2 G 2 0 4 2 G 2 G 153.12 153.12 616.895 306.46 338.888.. good good remote possible possible 0 0 soc2.1 soc2.3 comp1.1 comp1.2 life1.3 role of mother 1 eng2.5 0 welfare economics 0 G cp/m file 0 0 operating systems 1 computer architecture semi [permeable] membranes 5 G ion transport 1 0 7 engineering and radioactive 0 isotopes 0 G 1 0 G G G 0 1 2 G • 338.880.. 338.880.. 155.422 155.422 330.126 332 001.6425 001.6425 001.64 574.875 541.3724 574.875 581.875 574.876 620.0044 541.388 632.96 632.96 G 3 0 1 2 0 1 possible possible possible possible possible remote scattered scattered scattered good remote good possible good remote possible good good possible life2.1 biological control and plants and disease plants and disease 1 2 0 1 632 -102- Rpp 7 comp2.4 management information system design Class browsing 658.403 results possible soc2.1 child developement [sic] and mother 0 0 0 0 2 3 3 3 0 0 0 0 0 1 001.64 658.403 658.403 658.403 155.422 155.422 155.418 155.418 001.64 001.64 001.64044 001.64 001.64 001.64 001.64404 324.63 324.630.. 354.410.. 324.63 324.21 remote possible possible possible possible possible possible possible scattered scattered scattered scattered scattered scattered possible good possible remote good possible comp2.2 computers parallel architecture 1 3 3 0 0 0 0 0 0 1 2 1 0 1 2 comp2.5 sod. 5 0 0 0 computer architecture 0 computer architecture 0 Ian 1 proportional representation 2 britain 1 electoral reform 0 british government proportional representation 1 2 -103- R f 3 £D E2 m ClI ± >< E X A M P L E S 5 E R R C H E 5 1 Extracts f r o m a L i v e s e a r c h on t h e q u e r y e x p a n s i o n system LI5T OF BOOKS Search: "slump 1932" Books 1 to 9 of 19 No. 1 2 3 4 5 6 7 B 9 Title Author Classmark Date Slump c i t y : the p o l i t i c s of mass unemp. . FRIEND R The second stump : a Marxist analysis .. MRNDEL E The o r i g i n and nature of the Great Slum. . FERRQN P The slump. STEVENSON J The slump : society and p o l i tics during. . STEVENSON J How to survive the slump : a guide to .. BGNNOCK G P o l i t i c i a n s and the slump : the Labour . . SKIDELSKY R Money i n boom and slump : an empirical . . WRLTER5 P P P o l i t i c i a n s and the slump : the Labour . . SKIDELSKY R 331.137341 330.3047 338.54034 341.083 341.083 330.3410.. 941.083 332.4342 941.083 1981 1980 1979 1979 1977 1975 1970 1970 1967 Type i t s number to see i f a book i s relevant 3 or type Down ( n e x t ) , Restart/new search/quit Numbers 3 , 4 and 5 , and p o s s i b l y o t h e r s , l o o k r e l e v a n t . books a r e n o t r e l e v a n t , c o n t a i n i n g o n l y " 1 9 3 2 " . The other FULL DI5PLRY Search: "slump 1332" Book 3 of 13 RUTH0RCS): FEflRDN P TITLEC5): The origin and nature of the Great Slump, 1323-32. Studies in economic and social history. PUBLICATION: Macmillan, June 1373. 5UBJECTC5): Depressions - 1323 - Europe. Depressions - 1323 - United States. Economic depressions, 1323-1338. Related to economic depressions in United States. Related to economic depressions in Europe. Not in this branch No. of copies in other PCL libraries : RHS C23 Shelved at : 338.54034 FER 338.540373 Is this at all the sort of book you are looking for? Cy/n3 YES This brings in useful terms such as "depression", "1929 1 and "1329-32", as well as unhelpful o n e s . "economic", Uni -104- Rpp 8 S y s t e m has g o n e b a c k f o u n d two b o o k s . to b r i e f Example display, searches a n d u s e r has c h o s e n •More", which LIST OF BOOKS s i m i l a r to the one yau chase (Original search: "slump 1332"3 Books 1 to 2 of 2 No. Title 1 2 The Great Crash, 1329. The great crash, 1929. * * END OF LI5T * * Ruthor GRLBRRITH J K GRLBRRITH J K Classmark Date 338.540973 1980 338.540973 1978 Type i t s number to see i f a book i s relevant 1 Type Back to r e t u r n to the books you o r i g i n a l l y found or type Restart/new search/quit FULL DISPLAY of books similar to the one you chose (Original search: "slump 1932') Book 1 of 2 FUTHORCS): GRLBRRITH J K TITLECS3: The Great Crash, 1929. PUBLICATION: 50th anniversary ed. Deutsch, 1980. SUBJECT(S3: Depressions - 1929 - United States. Economic depressions. Not in this branch No. of copies in other PCL libraries : RHS (23 Shelved at : 338.540973 GRL Is this at all the sort of book you are looking for? Cy/n) YES We n o w have the term "crash" as w e l l . -105- Rpp 8 User has gone b a c k Example list searches and c h o s e n a n o t h e r record. to the o r i g i n a l FULL DISPLAY Search: 'slump 1332" Book 5 of 13 RUTHORCSh STEVENSON J ; C D K C CO D TITLEC5D: The slump : society and p o l i t i c s during the depression. PUBLICATION: Cape, 1377. SUBJECTS): Great B r i t a i n - History - George V, 1310-1336. Great B r i t a i n History - Edward V I I I , 1336. Great B r i t a i n - History - George V I , 1336-1352. Great B r i t a i n , 1330-1333. No. of copies i n t h i s branch : C13 No. of copies i n other PCL l i b r a r i e s : MRd CD RHS (4) Shelved at : 341.083 5TE Is t h i s at alL the sort of book you are looking for? Cy/n) YE5 T h i s has b r o u g h t i n some s i n g u l a r l y u s e l e s s LC s u b j e c t a l m o s t u s e l e s s PRECIS v e r b a l f e a t u r e h e a d i n g . User does " M o r e " a g a i n , w h i c h t h i s t i m e f i n d s t h r e e w h i c h t h e u s e r h a s now s e l e c t e d } . five headings, a n d an books Cbased on t h e LIST DF BOOKS s i m i l a r to the ones you chose (Original search: 'slump 1332') Books 1 to 5 of 5 No. T i t l e 1 2 The g r e a t c r a s h , 1323. The g r e a t c r a s h , 1323. Ruthor GRLBRRITH J K GRLBRRITH J K Classmark Date 338.540373 1378 338.540373 1375 338.540373 1370 3 3 8 . 5 4 0 3 . . 1383 731.430373 1372 3 4 5 Rmerica's greatest depression, 1323-1341. CHRNDLER L V The world economic and social c r i s i s : . . CRSTRO F We're i n t h e money : d e p r e s s i o n R m e r i c a . . BERGMFM R * * END OF LIST * * Type i t s number t o see i f a book i s r e l e v a n t Type Back t o r e t u r n t o t h e books you o r i g i n a l l y found or type Restart/new search/quit W now have two more e d i t i o n s of G a l b r a i t h . One was not p r e v i o u s l y e r e t r i e v e d because i t had a d i f f e r e n t s u b j e c t h e a d i n g . The book by Chandler i s p r o b a b l y r e l e v a n t , and the user s e l e c t s i t . The C a s t r o i s a f a l s e d r o p , and t h e l a s t book i s about making f i l m s i n t h e d e p r e s s i o n . 106- Rpp 8 Example searches The user now has what is probably a reasonable reading list, given the Limitations of the database. LIST OF BOOKS YOU HRVE CHOSEN [Original search: 'slump 1932') Books 1 to 4 of 4 No. Title Ruthor CLassmark Date 338.54094 1979 338.540973 1980 941.083 1977 338.540973 1970 1** The origin and nature of the Great Slum.. FERRON P 2** The Great Crash, 1S29. GRLBRRITH J K 3** The slump : society and politics during.. STEVENSON J 4** America's greatest depression, 1923-1941. CHRNBLER L V ** END OF LIST ** Type its number to see further details of one book Type Back to go back to where you were before viewing the 'chosen' List or type Restart/new search/quit QUIT 5ome books which were not retrieved are yet another edition of the Galbraith with no subject headings at all, and Kindleberger's "The world in depression, 1329-1939". The latter would be retrieved if a somewhat lower cutoff were used. In fact there are a number of possibly relevant books. Most of them can be retrieved by sifting through the 87 books indexed under "depression" or "slump", but many Library databases would be too large for this to be feasible. 2 Notes on a search of the "full" system The subject, a student of social sciences at a London college, was tackling the question "How do sociological perspectives throw light on the role of education in society?". She entered "perspective role of education in society". This finds one book matching "well", 135 books altogether. The first book is "The sociology of education: introductory analytical perspectives", with a subject heading "Society - role of education". CIt is worth noting that this is the only book which would have been found by a system which combined words with an implicit END.) The "noisy" words "role" and "perspectives" are in some ways helpful here: a search for "education in society" yields many false drops, with the above mentioned book at about position 100 in the sequence, because it is fairly old and items of equal weight display in reverse publication date order. Dewey classifies "Social aspects of education" at 370.19, and the first book retrieved is classified at 370.123 ("Relations between school and society"3. It is interesting that only eight in the first 100 books retrieved by this search are at 370.19. Some clearly relevant books are classified at 373.41 (Secondary education in the United Kingdom). The 107- Flpp 8 Example searches user chose the first book, and then Looked at books "shelved near". Several on the first screen were clearly relevant, and the subject chose two of them. These brought in the terms "sociology", "sociological" and "educational". Unfortunately, because of the date of publication sequence, they were as old as the initially chosen book. To see more recent works it would have been necessary to go back several screens of 370.193. To see books, equally likely to be relevant, at the more general number 370.19, one would need to go back several more screens. This would have been a more serious problem in a bigger library, but even so, the user, understandably, selected from what was before her eyes. This raises the question of whether it is better to use random ordering in this type of browsing display, or to start the display at the beginning of the sequence for the selected class number. The disadvantage of the latter is that it is impossible to show the "pivot" item in context. Having chosen two books classified near the one originally chosen, the subject then requested "More". This gives a List of 48 books, mostly at 370.193 or 370.19, with one book at 373.41. She chose number five from the first screen, "The sociology of educational ideas" (indexed by only one of the terms of the original search), and requested books classified near. She chose three books from the first screen and the succeeding one, then went down four screens (she was now back at 370.193) where she chose "Rn introduction to the sociology of education" and then again requested "More". This gave a list of 95 books, mostly at 370.19 and 370.1. Many of them were, as would be expected, included in the previous query expansion List Cof which the subject had only looked at the first screenful and selected one). Near the bottom of this List, and not seen by the user, there were two alarming faLse drops ("What's Left for immunology" and "The management of climatic resources"). These were unfortunately indexed under "inaugural", "delivered" and "February", as was Banks on "Sociology and education", one of the books previously chosen. Qn the whole, though, "noise11 words have remarkably Little effect on the results of query expansion searches. Without Looking at the rest of the List, the user chose the first record (Morrish on "Sociology of education"), then looked at records shelved nearby C370.13 again). Before going beyond the first screen she checked on what she had selected using the "View" option - nine records, all with "sociology" and "education" in the title. 5he then went rapidly forward no less than 48 screens, spending between three and eight seconds looking at each screen. The whole session took about 11 minutes, with nine records chosen, all of which were judged relevant by the assessors. The 14 subjects who searched for this topic chose a total of 67 distinct titles, and it is not difficult to find more than this. One subject, apparently short of time, chose no records. The other 13 chose an average of 12 books each, with a mean assessed precisian of 87% and about 10 brief records seen per choice. The comparative figures for all topics were 6.9 books chosen, mean precision 65% and 9.8 brief records per choice. This is one of the three (out of 50) topics which we rated as easy. Rny of the six combinations of a term from "society", "sociology", "sociological" with "education" or "educational" retrieves a List from which one can select some relevant books. These two-term combinations find between 15 and 130 records. Rs we have seen, the addition of a few woolly qualifiers may not be harmful. It might seem that expansion facilities are not helpful for topics like this one. However, the Lists obtained by expanding from one or two good records, -108- Rpp 8 Example searches or even the shelf list, are usually very much more "concentrated" than the original Lists. Rn example is the list obtained by the first expansion in the session described above. It retrieved 48 records, of which 16 of the first 18 Ctwo screens] are worth considering. In contrast, the original sequence of records retrieved by "Perspective role of education in society" contains only four possibly-relevant items in the first 18. It is easy to obtain these more concentrated Lists, so it is likely that the use of query expansion often reduces the effort involved in even an "easy" search Like this one. It is difficult to see what additional information would help users to choose between one title and another. Dur subjects were of course acting more as search intermediaries than as end users. Perhaps an experienced intermediary wouLd have tried to choose a selection of titles which Look as though they may treat the subject from different viewpoints - one might choose, for example, texts such as "Schooling and capitalism", "Education and the state", "Family, work and education" as well as some standard-looking books on educational sociology. In contrast, users Looking for books to satisfy barely-verbaLized personal needs will be influenced in their choices by subjective factors such as knowledge of the author, previous acquaintance, and other, less tangible, aspects, none of which is very useful as feedback information for the system. -109-