Evaluation The following table (Table 1) shows the search results on the first p

Question

Evaluation The following table (Table 1) shows the search results on the first page of a Web search engine along with binary relevance judgments where a “+" (or "-") indicates that the corresponding document is relevant (or non-relevant). Suppose there are 20 relevant documents in total in the whole collection Table 1 Document ID Dl D2 D3 D4 D5 D6 D7 D8 D9 D10 Relevance Judgment Compute the following measures for this list of retrieval results (no need to reduce an expression to a decimal or the simplest form of a fraction): [10 points] (a) Precision = (b) Recall (c) Mean Average Precision (d) Precision @5- (e) F1=

Explanation / Answer

(A) The fraction of relevant instances among the retrieved instances is known as precision. Therefore

precision = 4/10

(B) The fraction of relevant instances that have been retrieved over the total amount of relevant instances is known as recall. Therefore,

recall = 4/20

(C) Mean average precision for a set of queries is the mean of the average precision scores for each query. Therefore,

MAP = 34/200

(D) Precision @5 = 0

(E) The traditional F-measure or balanced F-score (F1 score) is the harmonic mean of precision and recall:

F1 = [ 2 * ( 4/10 ) * ( 4/20 ) ] / [ ( 4/10 ) + ( 4/20 ) ]

Related Questions

Navigate

• Browse (All)
• Browse E
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.