how to find everything relevant to some topic or question
needle-in-a-haystack test of directed search
read-everything test of directed search
why does directed search succeed for Thesa
Barber performed an informal test of directed search in October 1989 with several subjects, and presented the results in a research seminar. The experiment should be redone in a formal setting.
The results indicate that directed search is better than keyword search or read-everything search.
One person selected and wrote 10 questions for passages ("needles") in (QuoteRef: akscRM7_1988). The questions did not use keywords from the passages. The subjects were computer science graduate students. They were given a short introduction to Thesa and to the task. The subjects searched for relevant material for each question. There was three conditions: reading a preprint of the article, using a keyword index, or using directed search in Thesa. Read-everything Search
See notes below for further details.
|needles||8 of 9
Thesa Directed Search
|needles||4 of 8
|needles||8 of 8
Found measures the number of relevant quotations found. Missed is the number of quotations found in some other search. Searched is the number of quotations viewed. Recall is 'found/(found+missed)' . Precision is 'found/searched'. Needles is the number of needles (i.e., pre-selected passages) found.
The passage for question #5 was from a figure of the preprint. It was not in the electronic copy of the article.
The subjects decided that the passage for question #10 was not relevant to the question. So the directed and keyword searches had eight needles to locate, and the read-everything searches had nine needles to locate.
In the read-everything case, Searched is the total number of quotations identified for the directed and keyword searches plus the number of tables and figures. Subjects were explicitly asked about the relevance of unfound needles. Other quotations were deemed relevant if one of the subjects deemed it relevant.
ThesaHelp: how to find everything relevant to some topic or question
ThesaHelp: needle-in-a-haystack test of directed search
ThesaHelp: read-everything test of directed search
ThesaHelp: why does directed search succeed for Thesa