• +1-617-874-1011 (US)
  • +44-117-230-1145 (UK)
Live Chat
Follow Us:

IR evaluation

Question 2 :IR Evaluation

In this exercise, firstly I have to evaluate the performance of different search engines.

Primarily, I have to choose two search engines I am familiar with such as Yahoo, Google, Bing!.

Second, I need to choose a target among the groups and design two queries to search in both search engines.

The target is chosen by the last number of your student ID. For example, if your student ID ends with the number is 1, please choose target 1; if it is 0, please choose target 10.

  • Target 1: obtain the unit guide of SIT771.
  • Target 2: obtain the unit guide of SIT772.
  • Target 3: obtain the unit guide of SIT773.
  • Target 4: obtain the unit guide of SIT774.
  • Target 5: obtain the price of the new Macbook.
  • Target 6: obtain the price of the new iPhone.
  • Target 7: obtain the price of a Lenovo Laptop.
  • Target 8: obtain the install document of MongoDB.
  • Target 9: obtain the manual of MongoDB.
  • Target 10: obtain the operation guide of MongoDB.

Thirdly, select the first 20 results in both search engines, if they return the target, then mark them as relevant documents, otherwise, they are irrelevant. The following exercises are based on your search results.

  1. List your target and designed search queries (you can use any keywords you think are related to the target). For Search Engine 1, plot the precision versus recall curves for Query 1 and Query 2, interpolated to the 11 standard recall levels. Also plot the average precision versus recall curve for Search Engine 1 (all three curves should be on a single chart).
  2. For Search Engine 2, plot the precision versus recall curves for Query 1 and Query 2, interpolated to the 11 standard recall levels. Also plot the average precision versus recall curve for Search Engine 2 (all three curves should be on a single chart, but a separate chart from that used in part (a)).
  3. Plot the averages for Search Engine 1 and Search Engine 2 on a separate chart, and compare the algorithms in terms of precision and recall. Which search engine do you think is superior? Why?

SOLUTION:

Search Engines: Google/Bing

  • Target 5: obtain the price of the new Macbook.

Query 1 on Google search engine: Price+New+Macbook

Precision versus recall curves for Query 1 and interpolated to the 11 standard recall levels

Actual Recall & Precision Table

Interpolated Recall & Precision Table

Rank

Relevant

N

Precision

Recall

Precision

Recall

1

R

1

1.00

0.10

1

0.0

2

R

2

1.00

0.20

1

0.1

3

2

0.67

0.20

1

0.2

4

2

0.50

0.20

0.71

0.3

5

R

3

0.60

0.30

0.71

0.4

6

R

4

0.67

0.40

0.71

0.5

7

R

5

0.71

0.50

0.67

0.6

8

5

0.63

0.50

0.64

0.7

9

R

6

0.67

0.60

0.64

0.8

10

6

0.60

0.60

0.64

0.9

11

R

7

0.64

0.70

0.56

1.0

12

7

0.58

0.70

13

R

8

0.62

0.80

14

R

8

0.64

0.80

15

8

0.60

0.80

16

8

0.56

0.80

17

8

0.53

0.80

18

R

9

0.56

0.90

19

9

0.53

0.90

20

9

0.50

0.90
IR evaluation Image 1

Query 2 on Google search engine: MacBook + Price

Precision versus recall curves for Query 2 andinterpolated to the 11 standard recall levels

Actual Recall & Precision Table

Interpolated Recall & Precision Table:

Rank

Relevant

N

Precision

Recall

Precision

Recall

1

R

1

1.00

0.08

1

0.0

2

1

0.50

0.08

0.82

0.1

3

1

0.33

0.08

0.82

0.2

4

R

2

0.50

0.17

0.82

0.3

5

R

3

0.60

0.25

0.82

0.4

6

R

4

0.67

0.33

0.82

0.5

7

R

5

0.71

0.42

0.82

0.6

8

R

6

0.75

0.50

0.82

0.7

9

R

7

0.78

0.58

0.77

0.8

10

R

8

0.80

0.67

0.69

0.9

11

R

9

0.82

0.75

0.6

1.0

12

9

0.75

0.75

13

R

10

0.77

0.83

14

10

0.71

0.83

15

10

0.67

0.83

16

R

11

0.69

0.92

17

11

0.65

0.92

18

11

0.61

0.92

19

11

0.58

0.92

20

R

12

0.60

1.00
IR evaluation Image 2

Average on Google search engine

Interpolated Recall & Precision Table:

Precision

Recall

Precision Query 1

Precision Query 2

Average Precision

1

0.0

1

1

1

0.82

0.1

1

0.82

0.91

0.82

0.2

1

0.82

0.91

0.82

0.3

0.71

0.82

0.77

0.82

0.4

0.71

0.82

0.77

0.82

0.5

0.71

0.82

0.77

0.82

0.6

0.67

0.82

0.75

0.82

0.7

0.64

0.82

0.73

0.77

0.8

0.64

0.77

0.71

0.69

0.9

0.64

0.69

0.67

0.6

1.0

0.56

0.6

0.58

Average precision versus recall curve for Search Engine 1 i.e. Google Search engine

IR evaluation Image 3

Query 1 on BING search engine: Price New Macbook

Precision versus recall curves for Query 1 andinterpolated to the 11 standard recall levels

Actual Recall & Precision Table

Interpolated Recall & Precision Table:

Rank

Relevant

#

Precision

Recall

Precision

Recall

1

R

1

1.00

0.11

1

0.0

2

1

0.50

0.11

1

0.1

3

R

2

0.67

0.22

0.75

0.2

4

R

3

0.75

0.33

0.75

0.3

5

3

0.60

0.33

0.57

0.4

6

3

0.50

0.33

0.56

0.5

7

R

4

0.57

0.44

0.5

0.6

8

4

0.50

0.44

0.5

0.7

9

R

5

0.56

0.56

0.5

0.8

10

5

0.50

0.56

0.5

0.9

11

5

0.45

0.56

0.5

1.0

12

5

0.42

0.56

13

R

6

0.46

0.67

14

6

0.43

0.67

15

R

7

0.47

0.78

16

7

0.44

0.78

17

R

8

0.47

0.89

18

R

9

0.50

1.00

19

9

0.47

1.00

20

9

0.45

1.00
IR evaluation Image 4

Query 2 on BING search engine: Macbook Price

Precision versus recall curves for Query 2 andinterpolated to the 11 standard recall levels

Actual Recall & Precision Table

Interpolated Recall & Precision Table:

Rank

Relevant

N

Precision

Recall

Precision

Recall

1

R

1

1.00

0.08

1

0.0

2

1

0.50

0.08

0.67

0.1

3

1

0.33

0.08

0.67

0.2

4

1

0.25

0.08

0.67

0.3

5

1

0.20

0.08

0.67

0.4

6

R

2

0.33

0.17

0.67

0.5

7

R

3

0.43

0.25

0.67

0.6

8

R

4

0.50

0.33

0.67

0.7

9

R

5

0.56

0.42

0.67

0.8

10

R

6

0.60

0.50

0.63

0.9

11

R

7

0.64

0.58

0.63

1.0

12

7

0.58

0.58

13

R

8

0.62

0.67

14

R

9

0.64

0.75

15

R

10

0.67

0.83

16

10

0.63

0.83

17

10

0.59

0.83

18

R

11

0.61

0.92

19

R

12

0.63

1.00

20

12

0.60

1.00
IR evaluation Image 5

Average on Bing search engine

Average precision versus recall curve for Search Engine 2 i.e. Bing search engine

Interpolated Recall & Precision Table:

Precision

Recall

Precision Query 1

Precision Query 2

Average Precision

1

0.0

1

1

1

0.82

0.1

1

0.67

0.84

0.82

0.2

0.75

0.67

0.71

0.82

0.3

0.75

0.67

0.71

0.82

0.4

0.57

0.67

0.62

0.82

0.5

0.56

0.67

0.62

0.82

0.6

0.5

0.67

0.59

0.82

0.7

0.5

0.67

0.59

0.77

0.8

0.5

0.67

0.59

0.69

0.9

0.5

0.63

0.57

0.6

1.0

0.5

0.63

0.57
IR evaluation Image 6

Resources

  • 24 x 7 Availability.
  • Trained and Certified Experts.
  • Deadline Guaranteed.
  • Plagiarism Free.
  • Privacy Guaranteed.
  • Free download.
  • Online help for all project.
  • Homework Help Services

Testimonials

Urgenthomework helped me with finance homework problems and taught math portion of my course as well. Initially, I used a tutor that taught me math course I felt that as if I was not getting the help I needed. With the help of Urgenthomework, I got precisely where I was weak: Sheryl. Read More