Center for Studies of Information Resources, Wuhan University, Wuhan, China

---

Abstract—For a long time, recall is a key indicator in evaluating the retrieval performance of search engines. However, with the fact that the total documents on the internet are hard to access completely and users always focus merely on the first few pages, we believe that the traditional recalls can’t undertake the function of evaluation effectively any more. As a result, this paper proposes a new modified recall algorithm named as R-W(n), which not only focuses on the top N relevant results judged by users but also brings in different weights for different rankings. Meanwhile, we develop an experimental system, which is similar to a meta-search engine, to gather users' feedback and related data. And then, based on the criteria of measuring recall algorithm's effectiveness we propose and the experimental data gathered, the results between R-W(n) and traditional recall algorithms are compared. Finally, we draw a conclusion that the R-W(n) is superior to traditional recall algorithms, for it solves the weakness presented before and performs better in discerning good search engines from bad ones.

Index Terms—relative recall, weights, relevant feedback, AHP

[PDF]

Cite: Juncheng Wang, Zhenzhen Fu, Jun Cheng, and Feicheng Ma, " The Measurement of Relative Recall with Weights: a Perspective of User Feedback," Journal of Computers vol. 5, no. 11, pp. 1686-1692, 2010.