Many people get information from Baidu、Google、Bing and so on. OK, now we will consider how to test the performance of a search system.
The testing is based on an annotation file,annotation file consist of some query,every query has a query word and many reference URL. For example,“MM xxoo.com ooxx.com xoxo.com”,“MM”is the query word and “xxoo.com ooxx.com xoxo.com”is the reference URL.
If we search the query word from search system, we can also get a result list, and then we can test the search performance by the result list and annotation file. You job is calculate the MAP of the search performance. The definition of MAP is:
Rank: Position of a retrieved URL in the list of retrieved list.
Precision at a given cut-off rank r for a single query: P(r) = (number of relevant URL of rank r or less) / r
Average precision: defined as follows
Where N is the number of retrieved URL, R is the number of relevant URL, and rel(r) = 1 if URL at rank r is relevant, 0 otherwise.
Mean average precision: average precision for a set of queries is defined as follows:
Where Q is the number of queries.