WebSciences/Panda-information-retrieval-package — pull request #1 by yiweichen (2014-04-27 13:39:25+00:00)
Title: Adding feature for search result diversity re-ranking and evaluation - submission for the IRDM group project

**Adding feature for search result diversity re-ranking and evaluation - submission for the IRDM group project.**

Group 6: Yiwei Chen, Yifei Rong, Tianxiang Li

This is a submission of code for our group project of COMPGI15 Information Retrieval and Data Mining. We forked the original repository and added two features onto the Panda platform:
- The `-r` mode: re-ranking evaluation, format: `-r `. We have implemented two re-ranking approaches, MMR and Portfolio, for our comparison experiment. But it is not limited to this particular two approaches. As long as fitting the Rerank interface, more re-ranking approaches can be added into Panda. Also, `-m` can be used together to specify the underlying document scoring model.
- The `-br` mode: batch re-ranking evaluation, format: `-br :a:i:b`. As in some re-ranking approaches, a parameter (e.g. λ in MMR or b in Portfolio) can be tuned, this mode is used to run evaluations over various parameters, similar to the `-b` mode. Same as `-r`, the `-m` option can be used together to specify the underlying document scoring model.

**Usage Examples**
- `-r mmr -m 2` runs a evaluation using MMR as the re-ranking approach and BM25 (numbered 2) as the underlying document scoring model.
- `-br portfolio:0:0.1:1 -m 2` runs 11 evaluations with b = 0, 0.1, 0.2, ... , 0.9, 1 using Portfolio as the re-ranking approach and BM25 (numbered 2) as the underlying document scoring model.

About the details of implementation and our comparison experiments, please see our report submitted.