Statistical problems in the application of

probabilistic models to information retrieval

S.E. Robertson J.D. Bovey

Final report to the British Library Research and Development Department on Project SI/G/517 (October 1981 - September 1982)

November

1982

Centre for Information Science City University Northampton Square, London ECIV OHB.

<ii)

Abstract

Some statistical problems arising in the use of probabilistic models in retrieval are analysed. An attempt is made to identify

statistical methods from other fields which may be useful in retrieval. The logistic model, or class of models, which was developed for medical diagnostics, is identified as a promising approach. Retrieval methods based on the logistic model are developed and tested on three test collections; but the results indicate that the logistic models do not achieve the performance levels of traditional methods. Reasons for this discussed.

The logistic approach remains an attractive one for research purposes, particularly or removed at will. because it allows parameters to be added

An attempt is made to derive a formal rule

to determine when a new parameter (representing say an interaction between two terms) should be added. A single applicable rule is

not reached, but the analysis throws some light on the problem.

(iii)

Contents

Abstract Contents Page

1.

Introduction

1.1 1.2

Background The state of the art

1.2.1 1.2.2 1.2.3

The Robertson/Sparck Jones Model Collection frequency Term dependence and other statistical properties

1.3 1.4

Related work The present project

2.

Logistic Models

2.1 2.2

General description Mathematical description

2.2.1 2.2.2 2.2.3 2.2.4

Notation and the likelihood function Estimation Existence of estimates Discussion

2.3 2.4

Example : the independent logistic model Comparison of the present model with previous approaches

2.4.1 2.4.2

Possible application of the independent logistic model Main points of comparison

Civ)

3.

Experiments

3.1 3.2

The test collections The experiments

3.2.1 3.2.2 3.2.3

Half-collection experiments Continued searching experiments Evaluation

3.3

Logistic weights

3.3.1 3.3.2 3.3.3

The evaulation of

^ and #

The prior distribution The inclusion of interactions

3.4

Results of the experiments

3.4.1 3.4.2

Continued searching experiments Half-collection experiments

3.5

Survey of the results

3.5.1 3.5.2 3.5.3

Continued searching experiments Half-collection experiments Other experiments

4.

Further formal analysis

4.1 4.2 4.3 4.4

Definitions The "variance" The "bias" An example

5.

Final discussion

Acknowledgements

References