View source

From ATIRE

You do not have permission to edit this page, for the following reasons:

• The action you have requested is limited to users in the group: Users.
• You must confirm your e-mail address before editing pages. Please set and validate your e-mail address through your user preferences.

You can view and copy the source of this page:

This page documents some of the expected results when searching with the ATIRE search engine. The documented commands assume that the default filenames are used and all binaries from the project are in the PATH. The exact output may not match due to parallel indexing, but should match to 3 or 4 decimal points. == Wall Street Journal == The Wall Street Journal topics and assessments are contained within the evaluation folder of the ATIRE search engine repository. These are not necessarily the best results possible, but are used to verify that changes made to the search engine have not affected the results <pre>atire -a evaluation/WSJ.qrels -q evaluation/51-100.titles</pre> Expected output: <pre>Processed 50 topics: MAP: 0.221737</pre> '' The following was generated 2014-02-11 using ATIRE changeset 1847:118b625a87ce on Mac OS X 10.9.1 .'' <h3>WSJ</h3> <table><tr><th>Ranking Function</th><th>MAP</th></tr> <tr><td>-RBM25:0.9:0.4</td><td>0.222400</td></tr> <tr><td>-Rlmds:2500</td><td>0.224130</td></tr> <tr><td>-RDPH</td><td>0.214867</td></tr> <tr><td>-RDFRee</td><td>0.195040</td></tr> </table> == Wikipedia 2009 == The Wikipedia 2009 queries and assessments are available from the INEX website. The queries first need to be run through the topic extractor found in the evaluation directory like so: <pre>INEXtopics_to_ANTquery -clean 2009-topics.xml > 2009.topics INEXtopics_to_ANTquery -clean 2010-topics.xml > 2010.topics</pre> Then after removing topics that have no assessments: <pre>atire -a inex2009.qrels -q 2009.topics atire -a inex2010-article.qrels -q 2010.topics</pre> Expected output: <pre>Processed 68 topics: MAP: 0.344935 Processed 52 topics: MAP: 0.351517</pre> == TREC Collections == See [[TREC Results]]

Return to Expected Results.

Copyright © 2011 ATIRE.ORG. ALL Rights Reserved