• Aninja Helene Grøn
  • Jessica Blanca Johansen
Speciale, Kandidatuddannelsen
Title in English: A Royal experiment - Usability test and IR evaluation for the Royal Danish Library of their search system REX/Primo

In our thesis we conduct an experiment of the Royal Danish Library's search system, REX/Primo. We test the usability of the user interface by observing the behahaviour of first time users and analyzing them with Usability Engineering Tools, and we examine the system’s ranking algorithm by analyzing the test subjects' relevance assessments of bibliographic entries, using tools from Information Retrieval Evaluation. REX is the name of the interface and Primo is the name of the underlying engine, an integrated search system.
We measure how the search system REX/Primo performs as an integrated search-system, and our focus is on the users. A total of 13 test persons were used for the experiment. The usability test consisted of six search tasks that the test persons had to solve inside REX/Primo. The IR evaluation consisted of three Works Tasks and a total of 90 search results. The test persons assessed the relevance of the documents according to the Work Tasks. Relevance assessments from IR evaluation were analysed by using Cumulated Gain (CG) Mean Reciprocal Rank (MRR).
We conclude that REX/Primo is a well functioning system but still has room for improvement. Our usability test found two major usability problems and 22 minor problems and take them into concideration when creating a usability design update of REX’s graphical user interface (in paper mock ups). The relevance algorithm works quite well. We conclude from our CG curves that the vast majority of the relevant documents are found within the first fifteen search results, just as there is not much noise within the first fifteen results.
SprogDansk
Udgivelsesdato1 jul. 2010
Antal sider157
Udgivende institutionIVA - København

    Emneord

  • usability test, Information Retrieval, Information Retrieval Evaluation, Interactive Information Retrieval, Det Kongelige Bibliotek, relevans, morae, REX / Primo, HCI, speciale

Dokumenter