Charles L. A. Clarke

Academic and Professional Experience
2009-present
2002-2009
1999-2002

Professor
Associate Professor
Assistant Professor
David R. Cheriton School of Computer Science, University of Waterloo
Associate Dean, Innovation and Entrepreneurship, 2021Software Engineer, Facebook, 2016-18 (while on leave)
Director of Software Engineering, University of Waterloo, 2007-2012
Visiting Researcher, Microsoft, 2006 (while on leave)

2000-2005

Co-Founderisagn Inc., Waterloo, Canada

1996-1999

Assistant Professor
Department of Electrical and Computer Engineering, University of Toronto

1992-1993

Technical Architect
SHL Systemhouse, Toronto/Los Angeles

1985-1988

Programmer
Newfoundland Oceans Research and Development Corporation
St. John’s, Newfoundland

Education
Doctor of Philosophy, October 1996
Computer Science, University of Waterloo
Thesis: An Algebra for Structured Text Search
Supervisor: G. V. Cormack
Master of Mathematics, October 1990
Computer Science, University of Waterloo
Thesis: Language and Compiler Support for Synchronous Message Passing Architectures
Supervisor: G. V. Cormack
Bachelor of Science (Honours), May 1986
Mathematics/Computer Science, Memorial University of Newfoundland.
Awarded University Medal for Academic Excellence in Mathematics and Statistics

Other Professional Distinctions
SIGIR Academy, 2022
ACM Distinguished Scientist, 2015
Professional Engineer, licenced in the Province of Ontario, October 2006

RESEARCH AND SCHOLARSHIP
Areas of Interest
search, information retrieval, question answering, data science, data mining, software tools

Publications
Book
B1. Stefan Büttcher, Charles L. A. Clarke, and Gordon V. Cormack. Information Retrieval: Implementing
and Evaluating Search Engines, MIT Press, 606 pages, 2010. Honorable Mention, 2010 American
Publishers Award.
Edited Conference Proceedings
P1. Makoto P. Kato, Yiqun Liu, Noriko Kando, and Charles L. A. Clarke (editors). NII Testbeds andCommunity for Information Access Research — 14th International Conference, NTCIR 2019, Tokyo,
June 2019. Springer LNCS 11966, 2019.
P2. Shlomo Geva, Andrew Trotman, Peter Bruza, Charles L. A. Clarke, and Kalervo Järvelin (editors). Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information
Retrieval (SIGIR), Gold Coast, Australia, July 2014.
P3. Charles L. A. Clarke, Norbert Fuhr, Noriko Kando, Wessel Kraaij and Arjen P. de Vries (editors).
Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development inInformation Retrieval, Amsterdam, July 2007.
P4. Jamie Callan, Charles Clarke, Gordon Cormack, David Hawking and Alan Smeaton (editors). Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development inInformation Retrieval, Toronto, July 2003.
Journal Articles and Refereed Book Chapters
J1. Negar Arabzadeh, Alexandra Vtyurina, Xinyi Yan, and Charles L. A. Clarke. Shallow pooling for sparcelabels. Information Retrieval Journal, 2022.
J2. Charles L. A. Clarke, Alexandra Vtyurina, and Mark D. Smucker. Assessing top-k preferences. ACM
Transactions on Information Systems, 39(3), July 2021.
J3. Carolyn Lamb, Daniel G. Brown, and Charles L. A. Clarke. Evaluating Computational Creativity: An
Interdisciplinary Tutorial. ACM Computing Surveys 51(2), March 2018.
J4. Yongli Ren, Martin Tomko, Flora Dilys Salim, Jeffrey Chan, Charles L. A. Clarke, and Mark Sanderson.
A Location-Query-Browse Graph for Contextual Recommendation. IEEE Transactions on Knowledge
and Data Engineering 30(2):204-218, 2018.
J5. Roberto Konow, Gonzalo Navarro, and Charles L. A. Clarke, and Alejandro López-Ortiz. Inverted
Treaps. ACM Transactions on Information Systems 35(3), June 2017.
J6. Carolyn Lamb, Daniel G. Brown, and Charles L.A. Clarke. A Taxonomy of Generative Poetry Techniques. Journal of Mathematics and the Arts, 11(3):159-179, 2017.
J7. Charles L. A. Clarke, J. Shane Culpepper, and Alistair Moffat. Assessing Efficiency-Effectiveness Tradeoffs in Multi-Stage Retrieval Systems Without Using Relevance Judgments. Information Retrieval Journal 19(4):351-377, August 2016.

2

J8. Luchen Tan and Charles L. A. Clarke. A Family of Rank Similarity Measures Based on Maximized
Effectiveness Difference. IEEE Transactions on Knowledge and Data Engineering, 27(11):2865-2877,
November 2015.
J9. Azin Ashkan and Charles L.A. Clarke. Location and Query Aware Modeling of Browsing and Click
Behavior in Sponsored Search. ACM Transactions on Intelligent Systems and Technology, 5(4):32 pages,
2014.
J10. Peter B. Golbus, Javed A. Aslam, and Charles L. A. Clarke. Increasing Evaluation Sensitivity toDiversity. Information Retrieval, 16(4):530-555, 2013
J11. John A. Akinyemi and Charles L.A. Clarke. Fast and Effective Soft Links. Software Practice andExperience,43(5):577-593, 2013.
J12. Azin Ashkan and Charles L A. Clarke. Impact of Query Intent and Search Context on ClickthroughBehavior in Sponsored Search. Knowledge and Information Systems, 34(2):425-452, 2013.
J13. John S. Whissell and Charles L. A. Clarke. Improving Document Clustering using Okapi BM25 Feature
Weighting. Information Retrieval, 14(5):466-487, 2011.
J14. Gordon V. Cormack, Mark D. Smucker, Charles L. A. Clarke. Efficient and Effective Spam Filteringand Re-Ranking for Large Web Datasets. Information Retrieval, 14(5):441-465, 2011.
J15. Charles L. A. Clarke, Gordon V. Cormack, Thomas R. Lynam, Chris Buckley and Donna Harman.
Swapping Documents and Terms. Information Retrieval, 12(6):680-694, December 2009. Condensed
version of paper C53 for special issue.
J16. Stefan Büttcher and Charles L. A. Clarke. Hybrid Index Maintenance for Contiguous Inverted Lists.
Information Retrieval, 11(3):175-207, June 2008.
J17. Charles L. A. Clarke, Gordon V. Cormack, Thomas R. Lynam and Egidio L. Terra. Question Answeringby Passage Selection. In Advances in Open Domain Question Answering, Tomek Strzalkowski (editor),
Springer, 2006.
J18. Charles L. A. Clarke and Robert C. Good. Schema-Independent Retrieval from Heterogeneous Structured Text. In Mind Technologies: Humanities Computing and the Canadian Academic Community,
University of Calgary Press, 2006.
J19. Stefan Büttcher and Charles L. A. Clarke. Adding Full-Text Filesystem Search to Linux. ;login: The
USENIX Magazine, 31(3), June 2006.
J20. G. V. Cormack, C. L. A. Clarke and C. R. Palmer. MultiText Experiments for TREC. In TREC:
Experiments and Evaluation in Information Retrieval, Donna Harman and Ellen Voorhees (editors),
MIT Press, 2005.
J21. Anthony Cox and Charles Clarke. Multi-Layered Data Modelling. Electronic Notes in TheoreticalComputer Science, 94:71-79, Elsevier Science, 2004. A preliminary version of this paper was presented
at the First International Workshop on Meta-Models and Schemas for Reverse Engineering, Victoria,
B.C., November, 2003.
J22. Charles L. A. Clarke and Gordon V. Cormack. Shortest Substring Retrieval and Ranking. ACM Transactions on Information Systems, 18(1):44-78, January 2000.
J23. Charles L. A. Clarke, Gordon V. Cormack and Elizabeth A. Tudhope. Relevance Ranking for One toThree Term Queries. Information Processing and Management, 36(2):291-311, 2000. Invited submission.
A preliminary version of this paper appears in Proceeding of the Fifth RIAO Conference, Montreal, June,
1997.
3

J24. Gordon V. Cormack, Charles L. A. Clarke, Christopher R. Palmer and Samuel S. L. To. Passage-Based
Query Refinement. Information Processing and Management, 36(1):133-153, 2000. Invited submission. A preliminary version of this paper appears in Proceedings of the Sixth Text REtrieval Conference(TREC-6), Gaithersburg, Maryland, November 1997.
J25. Charles L. A. Clarke and Gordon V. Cormack. On the use of Regular Expressions for Searching Structured Text. ACM Transactions on Programming Languages and Systems, 19(3):413-426, 1997.
J26. Charles L. A. Clarke and David V. Mason. Compacting Garbage Collection can be Fast and Simple.
Software — Practice and Experience, 26(2):177-194, 1996.
J27. Charles L. A. Clarke, G. V. Cormack and F. J. Burkowski. An Algebra for Structured Text Search anda Framework for its Implementation. The Computer Journal, 38(1):43-56, 1995.
Refereed Full Conference Papers
C1. Negar Arabzadeh and Charles Clarke. Friéchet Distance for Offline Evaluation of Information Retrieval
System with Sparse Labels. In 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024), Malta, March 2024.
C2. Negar Arabzadeh, Amin Bigdeli, and Charles Clarke. Adapting Standard Retrieval Benchmarks to
Evaluate Generated Answers. Findings of 46th European Conference on Information Retrieval, Glasgow,
March 2024.
C3. Siqing Huo, Negar Arabzadeh, and Charles L. A. Clarke. Retrieving Supporting Evidence for Generative
Question Answering. In 1st International ACM SIGIR Conference on Information Retrieval in the Asia
Pacific, November 2023.
C4. Guglielmo Faggioli, Laura Dietz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Claudia
Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein and Henning Wachsmuth.
Perspectives on Large Language Models for Relevance Judgment. In ACM SIGIR International Conference on the Theory of Information Retrieval, July 2023. Best paper award.
C5. Negar Arabzadeh, Oleksandra Kmet, Ben Carterette, Charles L. A. Clarke, Claudia Hauff and Praveen
Chandar. A is for Adele: An Offline Evaluation Metric for Instant Search. In ACM SIGIR International
Conference on the Theory of Information Retrieval, July 2023.
C6. Ehsan Kamalloo, Nouha Dziri, Charles L. A. Clarke and Davood Rafiei. Evaluating Open-Domain
Question Answering in the Era of Large Language Models. In 61st Annual Meeting of the Association
for Computational Linguistics, July 2023.
C7. Xinyi Yan, Chengxi Luo, Charles L. A. Clarke, Nick Craswell, Ellen M. Voorhees, and Pablo Castells.
Human Preferences as Dueling Bandits. In 45th International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, July 2022.
C8. Hao Xue, Yongli Ren, Flora Salim, and Charles Clarke. Translating Human Mobility Forecasting
through Natural Language Generation. In 15th ACM International Conference on Web Search andData Mining, Phoenix, Arizona, February 2022.
C9. Chengxi Luo, Charles L. A. Clarke, and Mark Smucker. Evaluation measures based on preferencegraphs. In 44th International ACM SIGIR Conference on Research and Development in Information
Retrieval, July 2021.
C10. Charles L. A. Clarke and Mark D. Smucker and Alexandra Vtyurina. Offline evaluation by maximum
similarity to an ideal ranking. In 29th ACM Conference on Information and Knowledge Management,
October 2020.
4

C11. Charles L. A. Clarke and Alexandra Vtyurina and Mark D. Smucker. Offline Evaluation Without Gain.
In ACM SIGIR International Conference on the Theory of Information Retrieval, September 2020.
C12. Alexandra Vtyurina, Charles Clarke, Edith Law, Johanne R Trippas and Horaţiu Bota. A MixedMethod Analysis of Text and Audio Search Interfaces with Varying Task Complexity. In ACM SIGIR
International Conference on the Theory of Information Retrieval, September 2020.
C13. Amira Ghenai, Mark D. Smucker, and Charles L. A. Clarke. A Think-Aloud Study to Understand
Factors Affecting Online Health Search. In ACM Conference on Human Information Interaction and
Retrieval, pages 273,282, Vancouver, March 2020.
C14. Joel Mackenzie, J. Shane Culpepper, Roi Blanco, Matt Crane, Charles L. A. Clarke, and Jimmy Lin.
Query Driven Algorithm Selection in Early Stage Retrieval. In 11th ACM International Conference on
Web Search and Data Mining, pages 396-404, Los Angeles, February 2018.
C15. Frances A. Pogacar, Amira Ghenai, Mark D. Smucker, and Charles L.A. Clarke. The Positive and
Negative Influence of Search Results on People’s Decisions about the Efficacy of Medical Treatments. In
ACM SIGIR International Conference on Theory of Information Retrieval, pages 209-216, Amsterdam,
October 2017.
C16. Charles L. A. Clarke, Gordon V. Cormack, Jimmy Lin, and Adam Roegiest. Ten Blue Links on Mars.
In 26th World Wide Web Conference, pages 273-281, Perth, Australia, April 2017.
C17. J. Shane Culpepper, Charles L. A. Clarke and Jimmy Lin. Dynamic Cutoff Prediction in Multi-Stage
Retrieval Systems. In 21st Australasian Document Computing Symposium, pages 17-24, Melbourne,
Australia, December 2016. Best paper award.
C18. Carolyn Lamb, Daniel G. Brown, and Charles L.A. Clarke. Evaluating Digital Poetry: Insights fromthe CAT. In 7th International Conference on Computational Creativity, pages 60-67, Paris, June 2016
C19. Carolyn Lamb, Daniel G. Brown and Charlie L.A. Clarke A Taxonomy of Generative Poetry Techniques.
In Bridges Conference, pages 195-202, Jyväskylä, Finland, August 2016.
C20. Adam Roegiest, Gordon V. Cormack, Charles L. A. Clarke and Maura R. Grossman. Impact of Surrogate
Assessments on High-Recall Retrieval. In 38th International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, pages 555-564, Santiago, Chile, August 2015.
C21. Gaurav Baruah, Mark D. Smucker and Charles L. A. Clarke. Evaluating Streams of Evolving News
Events. In 38th International ACM SIGIR Conference on Research and Development in Information
Retrieval, pages 675-684, Santiago, Chile, August 2015.
C22. Carolyn Lamb, Daniel G. Brown and Charles Clarke Can Human Assistance Improve a Computational
Poet? In Bridges Conference, pages 37-44, Baltimore, July 2015.
C23. Carolyn Lamb, Daniel G. Brown and Charles L. A. Clarke. Human Competence in Creativity Evaluation. In 6th International Conference on Computational Creativity, pages 102-109, Park City, Utah,
June 2015.
C24. Charles L. A. Clarke and Mark D. Smucker. Time Well Spent. In 5th Information Interaction in Context
Symposium (IIiX), pages 205-214, Regensburg, Germany, August 2014.
C25. John S. Whissell and Charles L. A. Clarke. Effective Measures for Inter-document Similarity. In 22nd
ACM Conference on Information and Knowledge Management, pages 1361-1370, San Francisco, October
2013.
C26. Roberto Konow, Gonzalo Navarro, Charles L. A. Clarke, and Alejandro Lòpez-Ortiz. Faster and Smaller
Inverted Indices with Treaps. In 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 193-202, Dublin, July 2013.
5C27. Mark Smucker and Charles L. A. Clarke. Modeling User Variance in Time-Biased Gain. In 6th Symposium on Human-Computer Interaction and Information Retrieval, Cambridge, Massachusetts, October
2012.
C28. Mark D. Smucker and Charles L. A. Clarke. Time-Based Calibration of Effectiveness Measures. In 35th
International ACM SIGIR Conference on Research and Development in Information Retrieval, pages
95-104, Portland, August, 2012. Best paper award.
C29. John S. Whissell and Charles L. A. Clarke. Clustering for Semi-Supervised Spam Filtering. In Proceedings of the 8th Annual Collaboration, Electronic Messaging, Anti-Abuse and Spam Conference, pages
125-134, Perth, Australia, September 2011.
C30. Charles L. A. Clarke, Nick Craswell, Ian Soboroff, and Azin Ashkan. A Comparative Analysis of Cascade
Measures for Novelty and Diversity. In Proceedings of the 4th ACM International Conference on Web
Search and Data Mining pages 75-84, Hong Kong, February 2011.
C31. Azin Ashkan and Charles L. A. Clarke. On the Informativeness of Cascade and Intent-Aware Effectiveness Measures. In Proceedings of the 20th International World Wide Web Conference, pages 407-416,
Hyderabad, India, March 2011.
C32. Benjamin Lafreniere, Andrea Bunt, John S. Whissell, Charles L. A. Clarke, and Michael Terry. Characterizing Large-Scale Use of a Direct Manipulation Application in the Wild, In Proceedings of the 36thGraphics Interface Conference, pages 11-18, May 2010.
C33. Azin Ashkan and Charles L. A. Clarke. Characterizing Commercial Intent. In Proceedings of theEighteenth ACM Conference on Information and Knowledge Management, pages 67-76, Hong Kong,
November 2009.
C34. John Whissell, Charles L. A. Clarke, and Azin Ashkan. Clustering Web Queries In Proceedings of theEighteenth ACM Conference on Information and Knowledge Management, pages 899-908, Hong Kong,
2009.
C35. Azin Ashkan, Charles Clarke, Eugene Agichtein, and Qi Guo. Estimating Ad Clickthrough Rate throughQuery Intent Analysis. In IEEE/WIC/ACM International Conference on Web Intelligence, pages 222229, Milan, September 2009.
C36. Charles L. A. Clarke, Maheedhar Kolla and Olga Vechtomova. An Effectiveness Measure for Ambiguousand Underspecified Queries. In 2nd International Conference on the Theory of Information Retrieval,
pages 188-199, Cambridge, England, September 2009.
C37. Charles L. A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, StefanBüttcher and Ian MacKinnon. Novelty and Diversity in Information Retrieval Evaluation. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development inInformation Retrieval, pages 659-666, Singapore, 2008. SIGIR 2019 Test of time award.
C38. Stefan Büttcher and Charles L. A. Clarke. Index Compression is Good, Especially for Random Access.
In Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, pages
761-770, Lisbon, 2007.
C39. Charles L. A. Clarke, Eugene Agichtein, Susan Dumais and Ryen W. White. The Influence of Caption
Features on Clickthrough Patterns in Web Search. In Proceedings of the 30th Annual International ACM
SIGIR Conference on Research and Development in Information Retrieval, pages 135-142, Amsterdam,
2007.

6

C40. Stefan Büttcher, Charles L. A. Clarke, Peter C. K. Yeung and Ian Soboroff. Reliable Information
Retrieval Evaluation with Incomplete and Biased Judgements. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 63-70,
Amsterdam, 2007.
C41. Stefan Büttcher and Charles L. A. Clarke. A Document-Centric Approach to Static Index Pruning In
Text Retrieval Systems. In 15th ACM Conference on Information and Knowledge Management (CIKM),
pages 182-189, Arlington, Virginia, November 2006.
C42. Luanne Freund, Charles Clarke and Elaine Toms. Towards Genre classification for IR in the Workplace.
In First Symposium on Information Interaction in Context (IIiX), Copenhagen, October 2006.
C43. Stefan Büttcher, Charles L. A. Clarke and Brad Lushman. Hybrid Index Maintenance for Growing Text
Collections. In 29th Annual International ACM SIGIR Conference on Research and Development inInformation Retrieval, pages 175-207, Seattle, August 2006.
C44. Celine Latulipe, Ian Bell, Charles L.A. Clarke and Craig S. Kaplan. symTone: Two-Handed Manipulation of Tone Reproduction Curves. In Graphics Interface 2006 (GI2006), Québec City, June 2006.
Best student paper award.
C45. Stefan Büttcher and Charles L. A. Clarke. A Hybrid Approach to Index Maintenance in Dynamic Text
Retrieval Systems. In 28th European Conference on Information Retrieval (ECIR 2006), London, UK,
April 2006. Best student paper award.
C46. Celine Latulipe, Stephen Mann, Charles L.A. Clarke and Craig S.Kaplan. symSpline: Bimanual Symmetric Spline Manipulation. In 2006 Conference on Human Factors in Computing Systems (CHI 2006),
Montréal, April 2006.
C47. Stefan Büttcher and Charles L. A. Clarke. A Security Model for Full-Text Search in Multi-User Environments. In 4th USENIX Conference on File and Storage Technologies (USENIX FAST), San Francisco,December 2005.
C48. Celine Latulipe, Craig S. Kaplan and Charles L.A. Clarke. Bimanual and Unimanual Image Alignment:
An Evaluation of Mouse-Based Techniques. In 18th Annual ACM Symposium on User Interface Software
and Technology (UIST), Seattle, October 2005.
C49. Celine Latulipe, Craig S. Kaplan and Charles L.A. Clarke. Simultaneous Rotation and Translation. In
19th British HCI Group Annual Conference, British Computer Society, Edinburgh, September 2005.
C50. Charles L. A. Clarke. Controlling Overlap in Content-Oriented XML Retrieval. In 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador,
Brazil, August 2005.
C51. Luanne S. Freund, Elaine G. Toms and Charles L.A. Clarke. Modeling Task-Genre Relationships for IR
in the Workplace. In 28th Annual International ACM SIGIR Conference on Research and Development
in Information Retrieval, Salvador, Brazil, August 2005.
C52. Charles L. A. Clarke and Egidio L. Terra. Approximating the Top-m Passages in a Parallel Question
Answering System. In 13th ACM Conference on Information and Knowledge Management (CIKM),
Washington DC, November 2004.
C53. Thomas R. Lynam, Chris Buckley, Charles L. A. Clarke and Gordon V. Cormack. A Multi-System
Analysis of Document and Term Selection for Blind Feedback. In 13th ACM Conference on Information
and Knowledge Management (CIKM), Washington DC, November 2004.

7

C54. Egidio Terra and Charles L. A. Clarke. Scoring Missing Terms in Information Retrieval Tasks. In 13th
ACM Conference on Information and Knowledge Management (CIKM), Washington DC, November
2004.
C55. Egidio Terra and Charles L.A. Clarke. Fast Computation of Lexical Affinity Models. In 20th International Conference on Computational Linguistics, pages 50-58, Geneva, Switzerland, July 2004.
C56. Christopher A. Mennie and Charles L. A. Clarke. Giving Meaning to Macros. In 12th International
Workshop on Program Comprehension (IWPC), pages 79-88, Bari, Italy, June 2004.
C57. Charles L. A. Clarke, Philip L. Tilker, Allen Quoc-Luan Tran, Kevin Harris, and Antonio S. Cheng.
A Reliable Storage Management Layer for Distributed Information Retrieval Systems. In 12th ACM
Conference on Information and Knowledge Management (CIKM), New Orleans, November 2003.
C58. E. L. Terra and Charles Clarke. Frequency Estimates for Statistical Word Similarity Measures. In 2003
Human Language Technology Conference (HLTC), Edmonton, Canada, May 2003.
C59. Anthony Cox and Charles Clarke. Syntactic Approximation Using Iterative Lexical Analysis. In 11th
International Workshop on Program Comprehension (IWPC), Portland, Oregon, May 2003.
C60. Chiasen Chung and Charles L. A. Clarke. Topic-Oriented Collaborative Crawling. In 11th ACM Conference on Information and Knowledge Management (CIKM), McLean, Virginia, November 2002.
C61. Anthony Cox and Charles Clarke. Representing and Accessing Extracted Information. In IEEE International Conference on Software Maintenance (ICSM), Florence, Italy, November, 2001.
C62. Charles L. A. Clarke, Gordon V. Cormack and Thomas R. Lynam Exploiting Redundancy in Question
Answering. In 24th Annual International ACM SIGIR Conference on Research and Development inInformation Retrieval, New Orleans, September, 2001.
C63. Susan E. Sim, Charles L. A. Clarke, Ric C. Holt, and Anthony M. Cox. Browsing and Searching Software
Architectures. In IEEE International Conference on Software Maintenance (ICSM), Oxford, England,
September 1999.
C64. Gordon V. Cormack, Christopher R. Palmer and Charles L. A. Clarke. Efficient Construction of Large
Test Collections. In 21st Annual International ACM SIGIR Conference on Research and Development
in Information Retrieval, Melbourne, August, 1998.
C65. Susan E. Sim, Charles L. A. Clarke and Ric C. Holt. Archetypal Source Code Searching. In 6th IEEE
International Workshop on Program Comprehension (IWPC), Ischia, Italy, June 1998.
C66. Charles L. A. Clarke, Gordon V. Cormack and Forbes J. Burkowski. Schema-Independent Retrieval
from Heterogeneous Structured Text. In 4th Annual Symposium on Document Analysis and Information
Retrieval (SDAIR), Las Vegas, Nevada, April 1995.
C67. F. J. Burkowski, C. L. A. Clarke, S. C. Cowan and G. J. Vreugdenhil. Performance Evaluation of theSylvan Multiprocessor Architecture. In Symposium on Experiences with Distributed and Multiprocessor
Systems, Atlanta, March 1991.
Refereed Short Papers and Posters
S1. Negar Arabzadeh, Kiarash Golzadeh, Christopher Risi, Charles Clarke, and Jian Zhao. KnowFIRES: a
Knowledge-graph Framework for Interpreting Retrieved Entities from Search. Demonstration abstract
in 46th European Conference on Information Retrieval, Glasgow, March 2024.
S2. Ehsan Kamalloo, Charles L. A. Clarke, and Davood Rafiei. Limitations of Open-Domain Question
Answering Benchmarks for Document-level Reasoning. Short paper in 46th International ACM SIGIR
Conference on Research and Development in Information Retrieval, July 2023.
8

S3. Mahsa Seifikar, Linh Nhi Phan Minh, Negar Arabzadeh, Charles L. A. Clarke, and Mark D. Smucker. A
Preference Judgment Tool for Authoritative Assessment. Demonstration abstract in 46th International
ACM SIGIR Conference on Research and Development in Information Retrieval, July 2023.
S4. Dahlia Shehata, Negar Arabzadeh, and Charles L. A. Clarke. Early Stage Sparse Retrieval with Entity
Linking. Short paper in 31st ACM International Conference on Information and Knowledge Management, October 2022.
S5. Negar Arabzadeh, Mahsa Seifikar, and Charles L. A. Clarke. Unsupervised Question Clarity Prediction
Through Retrieved Item Coherency, Short paper in 31st ACM International Conference on Information
and Knowledge Management, October 2022.
S6. Negar Arabzadeh, Xinyi Yan, and Charles L. A. Clarke. Predicting Efficiency/Effectiveness Trade-Offs
for Dense vs. Sparse Retrieval Strategy Selection. Short paper in 30th ACM International Conferenceon Information and Knowledge Management, November 2021.
S7. Mustafa Abualsaud, Mark Smucker, and Charles Clarke. Visualizing Searcher Gaze Patterns. Shortpaper in 6th ACM SIGIR Conference on Human Information Interaction and Retrieval, March 2021.
S8. Zeynep Akkalyoncu Yilmaz, Charles Clarke, and Jimmy Lin. A Lightweight Environment for Learning Experimental IR Research Practices. Demonstration abstract in 43rd International ACM SIGIR
Conference on Research and Development in Information Retrieval, July 2020.
S9. Charles L. A. Clarke. Length Normalization in the Era of Neural Rankers. Short paper in 9th International Workshop on Evaluating Information Access, Tokyo, June 2019.
S10. Alexandra Vtyurina, Denis Savenkov, Eugene Agichtein, and Charles L. A. Clarke. Exploring Conversational Search With Humans, Assistants, and Wizards. In CHI Extended Abstracts, pages 2187-2193,
Denver, May 2017.
S11. Mark D. Smucker and Charles L. A. Clarke. Modeling Optimal Switching Behavior. Short paper in 1st
ACM SIGIR Conference on Human Information Interaction and Retrieval, pages 317-320, Chapel Hill,
North Carolina, March 2016.
S12. Charles L. A. Clarke, Gordon V. Cormack, Jimmy J. Lin, and Adam Roegiest. Total Recall: BlueSky on Mars. Short paper in 2nd ACM SIGIR International Conference on the Theory of Information
Retrieval, pages 45-48, Newark, Delaware, September 2016.
S13. Luchen Tan, Adam Roegiest, Jimmy Lin, and Charles L. A. Clarke. An Exploration of Evaluation
Metrics for Mobile Push Notifications. Short paper in 39th International ACM SIGIR Conference on
Research and Development in Information Retrieval, pages 741-744, Pisa, Italy, July 2016.
S14. Luchen Tan, Adam Roegiest, Charles L. A. Clarke and Jimmy Lin. Simple Dynamic Emission Strategies
for Microblog Filtering. Short paper in 39th International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, pages 1009-1012, Pisa, Italy, July 2016.
S15. Adam Roegiest, Luchen Tan, Jimmy J. Lin, and Charles L. A. Clarke. A Platform for Streaming
Push Notifications to Mobile Assessors. Short paper in 39th International ACM SIGIR Conference on
Research and Development in Information Retrieval, pages 1077-1080, Pisa, Italy, July 2016.
S16. Luchen Tan, Haotian Zhang, Charles L. A. Clarke and Mark D. Smucker. Lexical Comparison Between
Wikipedia and Twitter Corpora by Using Word Embeddings. Short paper in 53rd Annual Meeting of
the Association for Computational Linguistics, pages 657-661, Beijing, July, 2015.
S17. Seyyed Hadi Hashemi, Charles L. A. Clarke, Adriel Dean-Hall, Jaap Kamps and Julia Kiseleva. On the
Reusability of Open Test Collections. Short paper in 38th International ACM SIGIR Conference on
Research and Development in Information Retrieval, pages 827-830, Santiago, Chile, August 2015.
9S18. Adriel Dean-Hall and Charles L.A. Clarke. The Power of Contextual Suggestion. Short paper in 37th
European Conference on Information Retrieval Research (ECIR), pages 352-357, Vienna, April 2015.
S19. Luchen Tan and Charles L.A. Clarke. Succinct Queries for Linking and Tracking News in Social Media. Short paper in 23rd ACM International Conference on Conference on Information and Knowledge
Management (CIKM), pages 1883-1886, Shanghai, November 2014.
S20. Adriel Dean-Hall and Charles L. A. Clarke. Assessing Contextual Suggestion. Short paper in 6thInternational Workshop on Evaluating Information Access (EVIA), Tokyo, December 2014.
S21. John Whissell, and Charles Clarke Classification-Based Clustering Evaluation. Short paper in IEEE
International Conference on Data Mining, Dallas, December 2013.
S22. Tetsuya Sakai, Zhicheng Dou and Charles L. A. Clarke. The impact of intent selection on diversified search evaluation. Poster paper in 36th International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, pages 921-924, Dublin, July 2013.
S23. Mark D. Smucker and Charles L. A. Clarke. Stochastic Simulation of Time-Biased Gain. Short paperin 21st ACM International Conference on Information and Knowledge Management, pages 2040-2044,
October, 2012.
S24. Azin Ashkan and Charles L. A. Clarke. Modeling Browsing Behavior for Click Analysis in Sponsored
Search. Short paper in 21st ACM International Conference on Information and Knowledge Management,
pages 2015-2019, October, 2012.
S25. Gobaan Raveendran and Charles L. A. Clarke. Lightweight Contrastive Summarization for News Comment Mining. Poster paper in 35th International ACM SIGIR Conference on Research and Development
in Information Retrieval, pages 1103-1104, Portland, August, 2012.
S26. Kelly Y. Itakura, Charles L. A. Clarke, Shlomo Geva, Andrew Trotman, and Wei Chi Huang. Topical and
Structural Linkage in Wikipedia. Short paper in 33rd European Conference on Information Retrieval,
pages 460-465, 2011.
S27. John A. Akinyemi and Charles L. A. Clarke. Do Subtopic Judgments Reflect Diversity? Poster paper
presented at 3rd International Conference on the Theory of Information Retrieval, pages 309-312, 2011.
S28. John Akinlabi Akinyemi, Charles L. A. Clarke and Maheedhar Kolla. Towards a Collection-Based
Results Diversification. Short paper presented at 9th RIAO Conference, pages 202-205, Paris, April
2010.
S29. Kelly Y. Itakura and Charles L. A. Clarke. A Framework for BM25F-based XML Retrieval. Poster paperin 33rd Annual International ACM SIGIR Conference on Research and Development in Information
Retrieval, pages 843-844, Geneva, July 2010.
S30. Qi Guo, Eugene Agichtein, Charles Clarke, and Azin Ashkan. In the Mood to Click? Inferring Searcher
Receptiveness to Advertising via Modeling Contextualized Interactions. Short paper in IEEE/WIC/ACM
International Conference on Web Intelligence, pages 319-324, Milan, September 2009.
S31. Azin Ashkan and Charles Clarke. Term-Based Commercial Intent Analysis. Poster paper in Proceedingsof the 32nd Annual International ACM SIGIR Conference on Research and Development in Information
Retrieval, pages 800-801, Boston, July, 2009.
S32. Kelly Itakura and Charles Clarke. Using Dynamic Markov Compression to Detect Vandalism in the
Wikipedia. Poster paper in Proceedings of the 32nd Annual International ACM SIGIR Conference on
Research and Development in Information Retrieval, pages 822-823, Boston, July, 2009.

10S33. Gordon Cormack, Stefan Büttcher and Charles Clarke. Reciprocal Rank Fusion Outperforms Condorcet
and Individual Rank Learning Methods. Poster paper in Proceedings of the 32nd Annual International
ACM SIGIR Conference on Research and Development in Information Retrieval, pages 758-759, Boston,July, 2009.
S34. Azin Ashkan, Charles L. A. Clarke, Eugene Agichtein, and Qi Guo. Classifying and Characterizing QueryIntent. Short paper in 31st European Conference on Information Retrieval, pages 578-586, Toulouse,
France, April, 2009.
S35. Ryen W. White, Charles L. A. Clarke and Silviu Cucerzan. Comparing Query Logs and PseudoRelevance Feedback for Web-Search Query Refinement, Poster paper in Proceedings of the 30th Annual
International ACM SIGIR Conference on Research and Development in Information Retrieval, pages
831-832, Amsterdam, 2007.
S36. Peter C. K. Yeung, Charles L. A. Clarke and Stefan Büttcher. Improving Retrieval Accuracy by Weighting Document Types with Clickthrough Data. Poster paper in Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 759-760,
Amsterdam, 2007.
S37. Peter C.K. Yeung and Luanne Freund and Charles L.A. Clarke. X-Site: A Workplace Search Tool
for Software Engineers. Demonstration abstract in Proceedings of the 30th Annual International ACM
SIGIR Conference on Research and Development in Information Retrieval, page 900, Amsterdam, 2007.
S38. Peter C. K. Yeung, Stefan Büttcher, Charles L. A. Clarke and Maheedhar Kolla. A Bayesian approachfor Learning Document Type Relevance. In 29th European Conference on Information Retrieval, pages
753-756, Rome, April 2007.
S39. Stefan Büttcher, Charles L. A. Clarke and Brad Lushman. Term Proximity Scoring for Ad-Hoc Retrieval
on Very Large Text Collections. Poster paper in 29th Annual International ACM SIGIR Conference on
Research and Development in Information Retrieval, Seattle, August 2006.
S40. Stefan Büttcher and Charles L. A. Clarke. Indexing Time vs. Query Time Trade-offs in Dynamic
Information Retrieval Systems. Poster paper in 14th ACM Conference on Information and Knowledge
Management (CIKM), Bremen, Germany, October 2005.
S41. Egidio Terra, Kevyn Collins-Thompson, Jamie Callan and Charles L. A. Clarke. The Effect of Document
Retrieval Quality on Factoid Question Answering Performance. Poster paper in 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK,
July 2004.
S42. Charles L. A. Clarke and Egidio L. Terra. Passage Retrieval vs. Document Retrieval for Factoid
Question Answering. Poster paper in 26th Annual International ACM SIGIR Conference on Researchand Development in Information Retrieval, Toronto, Canada, July 2003.
S43. C. L. A. Clarke, G. V. Cormack, M. Laszlo, T. R. Lynam and E. L. Terra. The Impact of Corpus Size on
Question Answering Performance. Poster paper in 25th Annual International ACM SIGIR Conference
on Research and Development in Information Retrieval, Tampere, Finland, August 2002.
S44. T. R. Lynam and C. L. A. Clarke and G. V. Cormack. Information Extraction with Term Frequencies.
Short paper in 2001 Human Language Technology Conference (HLTC), San Diego, 2001.
S45. Charles Clarke, Anthony Cox and Susan Sim. Searching Program Source with a Structured Text Retrieval System. Poster paper in 22st Annual International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, Berkeley, August, 1999.
S46. Gordon V. Cormack, Charles L. A. Clarke, Christopher R. Palmer, and Robert C. Good. The MultiText
Retrieval System. Demonstration abstract in 22st Annual International ACM SIGIR Conference on
Research and Development in Information Retrieval, Berkeley, August, 1999.
11

Patents
P1. Aaron Jaech, Hetunandan Munisharan Kamichetty, Eric Ringger, and Charles Lester Clarke. Tensorbased deep relevance model for search on online social networks. US10268646B2. Granted April 23,
2019.
P2. Girish Kumar, Gaurav Sareen, Namita Gupta, Charles Lester Alexander Clarke, and Junhua Wang.
Using anchor text to provide context. US8458207B2. Granted June 6, 2013.
P3. Ariel Lazier, Hugh J. Williams, Nick Craswell, Charles Clarke, and Julia H. Farago. Content item query
formulation. US7647338B2. Granted September 1, 2010.
Selected Workshop Papers
W1. Siqing Huo, Negar Arabzadeh, and Charles Clarke. Retrieving Supporting Evidence for LLM Generated Answers. In 1st International Workshop on Retrieval-Enhanced Machine Learning, SIGIR 2023
Workshop, July 2023.
W2. Siqing Huo, Xinyi Yan, and Charles L. A. Clarke. WaterlooClarke at the TREC 2022 Conversational
Assistant Track. Presented at the 31st Text REtrieval Conference (TREC), Gaithersburg, Maryland,
November 2022.
W3. Charles L. A. Clarke, Maria Maistro, Mahsa Seifikar, and Mark D. Smucker. Overview of the TREC
2022 Health Misinformation Track. Presented at the 31st Text REtrieval Conference (TREC), Gaithersburg, Maryland, November 2022.
W4. Charles L. A. Clarke, Mark D. Smucker, and Maria Maistro. Overview of the TREC 2021 Health Misinformation Track. Presented at the 30th Text REtrieval Conference (TREC), Gaithersburg, Maryland,
November 2021.
W5. Xinyi Yan, Charles L. A. Clarke and Negar Arabzadeh. WaterlooClarke at the TREC 2021 Conversational Assistant Track. Presented at the 30th Text REtrieval Conference (TREC), Gaithersburg,
Maryland, November 2021.
W6. Charles L. A. Clarke, Saira Rizvi, Mark D. Smucker, Maria Maistro, and Guido Zuccon. Overview
of the TREC 2020 Health Misinformation Track. Presented at the 29th Text REtrieval Conference(TREC), Gaithersburg, Maryland, November 2020.
W7. Negar Arabzadeh and Charles L. A. Clarke. WaterlooClarke at the TREC 2020 Conversational Assistant Track. Presented at the 29th Text REtrieval Conference (TREC), Gaithersburg, Maryland,
November 2020.
W8. Charles L. A. Clarke. WaterlooClarke at the TREC 2019 Conversational Assistant Track. Presented
at the 28th Text REtrieval Conference (TREC), Gaithersburg, Maryland, November 2019.
W9. Amira Ghenai, Mark D. Smucker and Charles L. A. Clarke. A Think-Aloud Study about Medical
Misinformation in Search Results. In SIGIR Workshop on Reducing Online Misinformation Exposure,
Paris, July 2019.
W10. Seyyed Hadi Hashemi, Jaap Kamps, Julia Kiseleva, Charles L.A. Clarke, and Ellen Voorhees. Overview
of the TREC 2016 Contextual Suggestion Track. Presented at the 25th Text REtrieval Conference(TREC), Gaithersburg, Maryland, November 2016.
W11. Adam Roegiest, Gordon V. Cormack, Maura R. Grossman and Charles L.A. Clarke. TREC 2016
Total Recall Track Overview. Presented at the 25th Text REtrieval Conference (TREC), Gaithersburg,
Maryland, November 2016.

12W12. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps and Julia Kiseleva. Online Evaluation of PointOf-Interest Recommendation Systems. In First International Workshop on Supporting Complex Search
Tasks, ECIR workshop, Vienna, Austria, March 29, 2015.
W13. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Julia Kiseleva, and Ellen Voorhees. Overview
of the TREC 2015 Contextual Suggestion Track. Presented at the 24th Text REtrieval Conference(TREC), Gaithersburg, Maryland, November 2015.
W14. Adam Roegiest, Gordon V. Cormack, Charles L.A. Clarke, and Maura R. Grossman. TREC 2015
Total Recall Track Overview. Presented at the 24th Text REtrieval Conference (TREC), Gaithersburg,
Maryland, November 2015.
W15. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Paul Thomas, and Ellen Voorhees. Overview
of the TREC 2014 Contextual Suggestion Track. Presented at the 23nd Text REtrieval Conference(TREC), Gaithersburg, Maryland, November 2014.
W16. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Paul Thomas, Nicole Simone, and Ellen Voorhees.
Overview of the TREC 2013 Contextual Suggestion Track. Presented at the 22nd Text REtrieval
Conference, Gaithersburg, Maryland, November 2013.
W17. Kevyn Collins-Thompson, Paul Bennett, Fernando Diaz, Charles L. A. Clarke, and Ellen M. Voorhees.
TREC 2013 Web Track Overview. Presented at the 22nd Text REtrieval Conference, Gaithersburg,
Maryland, November 2013.
W18. Charles L. A. Clarke. Nugget-based computation of graded relevance. Presented at the 5th International
Workshop on Evaluating Information Access (EVIA), Tokyo, June 2013.
W19. Adriel Dean-Hall and Charles L. A. Clarke. Evaluating contextual suggestion. Presented at the 5thInternational Workshop on Evaluating Information Access (EVIA), Tokyo, June 2013.
W20. Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Paul Thomas and Ellen M. Voorhees. Overview of
the TREC 2012 Contextual Suggestion Track. Presented at 21st Text REtrieval Conference, November
2012.
W21. Charles L. A. Clarke, Nick Craswell and Ellen M. Voorhees. Overview of the TREC 2012 Web Track.
Presented at 21st Text REtrieval Conference, November 2012.
W22. Mark D. Smucker and Charles L. A. Clarke. The Fault, Dear Researchers, is not in Cranfield, But inour Metrics, that they are Unrealistic. Position paper in 2nd European Workshop on Human-Computer
Interaction and Information Retrieval, pages 11-12, Nijmegen, August, 2012.
W23. Charles L. A. Clarke, Nick Craswell, Ian Soboroff, and Ellen M Voorhees. Overview of the TREC 20l1
Web Track. Presented at 20th Text REtrieval Conference, November 2011.
W24. Azin Ashkan, John S. Whissell and Charles L.A. Clarke. Unsupervised Learning of Result Page Context
for Clickthrough Analysis in Sponsored Search. Presented at 5th Annual International Workshop on
Data Mining and Audience Intelligence for Online Advertising, San Diego, August 2011.
W25. Charles L. A. Clarke, Nick Craswell, Ian Soboroff, and Gordon V. Cormack. Overview of the TREC
2010 Web Track. Presented at 19th Text REtrieval Conference, November 2010.
W26. Kelly Y. Itakura and Charles L. A. Clarke. University of Waterloo at INEX 2009: Ad Hoc, Book, Entity
Ranking, and Link-the-Wiki Tracks. Presented at 8th Workshop of the Initiative for the Evaluation of
XML Retrieval, December 2009.
W27. Charles L. A. Clarke, Nick Craswell, and Ian Soboroff. Overview of the TREC 2009 Web Track.
Presented at 18th Text REtrieval Conference, November 2009.
13

W28. Kelly Y. Itakura and Charles L.A. Clarke. University of Waterloo at INEX2008: Adhoc, Book, and
Link-the-Wiki Tracks. In 7th Workshop of the Initiative for the Evaluation of XML Retrieval, December
2008.
W29. Azin Ashkan, Charles L. A. Clarke, Eugene Agichtein, and Qi Guo. Characterizing Query Intent From
Ad Clickthrough Data. In 1st Workshop on Information Retrieval in Advertising, SIGIR Workshop,
Singapore, July 2008.
W30. Maheedhar Kolla, Olga Vechtomova, and Charles L. A. Clarke. Nugget-based Framework to Support
Graded Relevance Assessments. In Beyond Binary Relevance: Preferences, Diversity, and Set-Level
Judgments, SIGIR Workshop, Singapore, July 2008
W31. S. Büttcher, C. L. A. Clarke, G. V. Cormack and T. R. Lynam. MultiText Legal Experiments at TREC
2007. Presented at 16th Text REtrieval Conference, November 2007.
W32. Kelly Itakura and Charles L. A. Clarke. From Passages into Elements in XML Retrieval. In SIGIR
2007 Workshop on Focused Retrieval, Amsterdam, 2007.
W33. Stefan Büttcher, Charles L. A. Clarke and Peter C. K. Yeung. Index Pruning and Result Reranking:
Effects on Adhoc Retrieval and Named Page Finding. Presented at the 15th Text REtrieval Conference(TREC 2006). Gaithersburg, Maryland, November 2006.
W34. Stefan Büttcher, Charles L. A. Clarke and Ian Soboroff. The TREC 2006 Terabyte Track. Presented
at the 15th Text REtrieval Conference (TREC 2006). Gaithersburg, Maryland, November 2006.
W35. Stefan Büttcher and Charles L. A. Clarke. Efficiency vs. Effectiveness in Terabyte-Scale Information
Retrieval. Presented at the 14th Text REtrieval Conference (TREC 2005). Gaithersburg, Maryland,
November 2005.
W36. Charles L. A. Clarke, Falk Scholer and Ian Soboroff. The TREC 2005 Terabyte Track. Presented at
the 14th Text REtrieval Conference (TREC 2005). Gaithersburg, Maryland, November 2005.
W37. Egidio Terra and Charles L. A. Clarke. Comparing Query Formulation and Lexical Affinity Replacements in Passage Retrieval. In ELECTRA: Methodologies and Evaluation of Lexical Cohesion Techniques in Real-World Applications, SIGIR Workshop, Salvador, Brazil, August 2005.
W38. Charles Clarke, Nick Craswell and Ian Soboroff. Overview of the TREC 2004 Terabyte Track, Presented
at the 2004 Text REtrieval Conference (TREC 2004), Gaithersburg, Maryland, November 2004.
W39. Charles L. A. Clarke and Philip L. Tilker. MultiText Experiments for INEX 2004. In Advances in XML
Information Retrieval, 3rd Workshop of the INitiative for the Evaluation of XML Retrieval, INEX 2004,
Schloss Dagstuhl, Germany, December 2004. Published as Lecture Notes in Computer Science, Volume3493, Springer, 2005.
W40. Stefan Büttcher, Charles L. A. Clarke and Gordon V. Cormack. Domain-Specific Synonym Expansion
and Validation for Biomedical Information Retrieval. Presented at the 2004 Text REtrieval Conference(TREC 2004), Gaithersburg, Maryland, November 2004.
W41. David L. Yeung, Charles L. A. Clarke, Gordon V. Cormack, Thomas R. Lynam and Egidio L. Terra.
Task-Specific Query Expansion. Presented at the 2003 Text REtrieval Conference (TREC 2003),
Gaithersburg, Maryland, November 2003.
W42. C. L. A. Clarke and G. V. Cormack and G. Kemkes and M. Laszlo and T. R. Lynam and E. L. Terra and
P. L. Tilker. Statistical Selection of Exact Answers. Presented at the 2002 Text REtrieval Conference(TREC 2002), Gaithersburg, Maryland, November 2002.

14

W43. Anthony Cox and Charles Clarke. Relocating XML Elements from Preprocessed to Unprocessed Code.
In Proceedings of the 10th International Workshop on Program Comprehension (IWPC), Paris, June
2002.
W44. C. L. A. Clarke, G. V. Cormack, T. R. Lynam, C. M. Li and G. L. McLearn. Web Reinforced Question
Answering. Presented at the 2001 Text REtrieval Conference (TREC 2001), Gaithersburg, Maryland,
November 2001.
W45. Anthony Cox and Charles Clarke. A Comparative Evaluation of Techniques for Syntactic Level Source
Code Analysis. In Proceedings of the Seventh Asia-Pacific Software Engineering Conference, Singapore,
December, 2000.
W46. C. L. A. Clarke, G. V. Cormack, D. I. E. Kisman and T. R. Lynam. Question Answering by Passage
Selection. Presented at the Ninth Text REtrieval Conference (TREC-9), Gaithersburg, Maryland,
November 2000.
W47. Anthony Cox and Charles Clarke. A Functional Approach to Complex Retrieval Tasks. In 9th International Workshop on Functional and Logic Programming, Benicassim, Spain, September 2000.
W48. G.V. Cormack, C.L.A. Clarke, C.R. Palmer and D.I.E. Kisman. Fast Automatic Passage Ranking.
Presented at 8th Text REtrieval Conference (TREC-8), Gaithersburg, Maryland, November 1999.
W49. Anthony Cox, Charles Clarke and Susan Sim. A Model Independent Source Code Repository. In
CASCON’99, Toronto, November 1999.
W50. Gordon V. Cormack, Christopher R. Palmer, Michael Van Biesbrouck and Charles L. A. Clarke. Deriving Very Short Queries for High Precision and Recall. Presented at the 7th Text REtrieval Conference(TREC-7), Gaithersburg, Maryland, November 1998.
W51. Gordon V. Cormack, Charles L. A. Clarke, Christopher R. Palmer and Samuel S. L. To. Passage-Based
Refinement. Presented at the 6th Text REtrieval Conference (TREC-6), Gaithersburg, Maryland,
November 1997.
W52. Charles L. A. Clarke and Gordon V. Cormack. Interactive Substring Retrieval. Presented at the 5th
Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1996.
W53. R. C. Good, G. V. Cormack, C. L. A. Clarke and D. J. Taylor. A Robust Storage Systems Architecture.
In 8th International Conference on Computing and Information (ICCI), June 1996.
W54. Charles L. A. Clarke, Gordon V. Cormack and Forbes J. Burkowski. Shortest Substring Ranking.
Presented at 4th Text REtrieval Conference (TREC-4), Gaithersburg, Maryland, November 1995.
Other Selected Contributions
X1. George Robert Buchanan, Dana McKay, and Charles L. A. Clarke. Made to Measure: A Workshop on
Human-Centred metrics for information seeking. Workshop at ACM Conference on Human Information
Interaction and Retrieval, pages 461–462, Austin, Texas, March 2020.
X2. Charles L. A. Clarke, Fernando Diaz, and Negar Arabzadeh. Preference-Based Offline Evaluation.
Tutorial at 16th ACM International Conference on Web Search and Data Mining, Singapore, Feb 2023.
X3. Makoto P. Kato, Yiqun Liu, Noriko Kando, and Charles L. A. Clarke. Report on the 15th round of NII
testbeds and community for information access research (NTCIR-15). SIGIR Forum 55(2), 2021.
X4. George Buchanan, Dana McKay, Charles L. A. Clarke, Leif Azzopardi, and Johanne R. Trippas. Madeto Measure: A Workshop on Human-centred Metrics for Information Seeking. Workshop at ACM
Conference on Human Information Interaction and Retrieval, pages 484-487, Vancouver, March 2020.
15

X5. Yiqun Liu, Makoto P. Kato, Charles L. A. Clarke, Noriko Kando, and Tetsuya Sakai. Report on NTCIR13: The Thirteenth Round of NII Testbeds and Community for Information Access Research. SIGIR
Forum 52(1):102-110, August 2018.
X6. Jimmy Lin, Charles L.A Clarke, and Gaurav Baruah. Searching from Mars. IEEE Internet Computing
20(1):78-82, 2016.
X7. Grace Hui Yang, Ian Soboroff, Li Xiong, Charles L. A. Clarke, and Simson L. Garfinkel. PrivacyPreserving IR 2016: Differential Privacy, Search, and Social Media. Workshop at 39th International
ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1247-1248, Pisa,
Italy, July 2016.
X8. Alessandro Moschitti, Lluı́s Màrquez, Preslav Nakov, Eugene Agichtein, Charles L. A. Clarke, and Idan
Szpektor. Web Question Answering: Beyond Factoids. Workshop at 39th International ACM SIGIR
Conference on Research and Development in Information Retrieval, pages 1251-1252, Pisa, Italy, July2016.
X9. Charles L. A. Clarke, Mark D. Smucker, and Emine Yilmaz. IR Evaluation: Modeling User Behavior
for Measuring Effectiveness. Tutorial at 38th International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, pages 1117-1120, Santiago, Chile, August 2015.
X10. Eugene Agichtein, David Carmel, Charles L. A. Clarke, Praveen Paritosh, Dan Pelleg and Idan Szpektor.
Web Question Answering: Beyond Factoids. Workshop at 38th International ACM SIGIR Conference
on Research and Development in Information Retrieval, pages 1143, Santiago, Chile, August 2015.
X11. M.-Dyaa Albakour, Craig Macdonald, Iadh Ounis, Charles L. A. Clarke, and Veli Bicer. Information
Access in Smart Cities (i-ASC). Workshop report in 36th European Conference on Information Retrieval
Research (ECIR),pages 810-814, Amsterdam, April 2014.
X12. Charles L. A. Clarke, Luanne Freund, Mark D. Smucker, and Emine Yilmaz. SIGIR 2013 workshop
on modeling user behavior for information retrieval evaluation. In 36th International ACM SIGIR
Conference on Research and Development in Information Retrieval, page 1134, Dublin, July 2013.
X13. Andrew Trotman, Charles L. A. Clarke, Iadh Ounis, Shane Culpepper, Marc-Allen Cartright and Shlomo
Geva. Open source information retrieval: A report on the SIGIR 2012 workshop. SIGIR Forum46(2):95-101, 2012.
X14. Craig Macdonald, Jun Wang and Charles L. A. Clarke. The 2nd International Workshop on Diversity
in Document Retrieval. Workshop report in 5th International Conference on Web Search and Web Data
Mining, Seattle, February 2012.
X15. Nicholas J. Belkin, Charles L. A. Clarke, Ning Gao, Jaap Kamps and Jussi Karlgren. Report on
the SIGIR Workshop On “entertain me”: Supporting Complex Search Tasks. ACM SIGIR Forum,45(2):51-59, December 2011.
X16. Craig Macdonald, Charles Clarke, and Jun Wang. The 1st International Workshop on Diversity inDocument Retrieval, ACM SIGIR Forum, 45(2):87-93, December 2011.
X17. Charles L. A. Clarke. Web Question Answering. In Encyclopedia of Database Systems, Ling Liu and
M. Tamer Özsu (editors), Springer, 2008.
X18. Jamie Callan, James Allan, Charles L. A. Clarke, Susan Dumais, David A. Evans, Mark Sanderson
and ChengXiang Zhai. Meeting of the MINDS: An information retrieval research agenda. ACM SIGIR
Forum, 41(2):25-34, December 2007.
X19. Charles L. A. Clarke, Nick Craswell and Ian Soboroff. The TREC Terabyte Retrieval Track SIGIR
Forum, 39(1):25, 2005.
16

X20. Charles L. A. Clarke, Gordon V. Cormack, Christopher R. Palmer. An Overview of MultiText, SIGIR
Forum, 32(2):14-15 (1998). Abstract of a presentation given at the SIGIR ’98 workshop on Information
Retrieval: Theory into Practice, August 1998.

Invited Talks and Presentations (partial list)
June 2024
March 2024
November 2023
November 2023
May 2023
October 2021
October 2021
January 2020
June 2019
May 2019
May 2019
August 2017
March 2016
December 2015
August 2015
February 2015
February 2014
September 2013
June 2013
February 2013
December 2012
June 2010
July 2009
December 2008
May 2008
April 2008
October 2007
June 2007
March 2006
May 2005
April 2005
February 2005
December 2003
May 2002
November 2001
June 2001
March 2001
May 2000Southeastern University, Nanjing, China
ECIR, Glasgow, Scotland (keynote)
Tsinghua University
Baidu, Beijing
University of Massachusetts Amherst
Spotify Research
University of Glasgow, Scotland
ACM SIGIR/SIGKDD Africa Summer School on ML for Data Mining and Search (invited tutorial)
NTCIR, National Institute of Informatics, Tokyo (keynote)
Computer Science, RMIT University, Melbourne, Australia
SEEK, Melbourne, Australia
3rd Asian Summer School in Information Access, Kyoto, Japan (invited tutorial)
1st Conference on Human Information Interaction and Retrieval, UNC (invited tutorial)
7th Forum for Information Retrieval Evaluation, Gandhinagar, India
9th Russian Summer School in Information Retrieval, St. Petersburg
WSDM tutorial on Dynamic Information Retrieval Modeling, Shanghai
LTI Colloquium, CMU
9th European Summer School in Information Retrieval, Granada, Spain
NTCIR, National Institute of Informatics, Tokyo (keynote)
School of Computing Science, University of Glasgow
17th Australasian Document Computing Symposium (keynote)
NTCIR, National Institute of Informatics, Tokyo
SIGIR Workshop on Redundancy, Diversity and Interdependent Document Relevance (keynote)
Microsoft Research, Cambridge
Department of Mathematics and Computer Science, Emory University
13th Search Engine Meeting, Boston (keynote)
UW-IEEE Workshop on Multimedia Mining and Knowledge Discovery
ORION Ontario R&D Summit
The Information School, University of Washington
IBM Toronto Software Lab
School of Computer Science, Dalhousie University
Microsoft Research, Redmond, Washington
4th Dutch-Belgian Information Retrieval Workshop (keynote)
Mind Technologies Session, Congress of the Social Sciences and Humanities, Toronto
Canadian Digital Libraries Symposium, Toronto
IFIP 2.4 working group meeting, San Miniato, Italy
CITO Research Review on Software Technology and Distributed Systems
IFIP 2.4 working group meeting, Delft, Netherlands

17

TEACHING ACTIVITIESGraduate Student Supervision
Current students
Amin Bigdeli, Ph.D. expected 2026
Topic: Adversarial Information Retrieval.
Dahlia Shehata, Ph.D. expected 2026
Topic: High Revall Information Retrieval.
Negar Arabzadeh, Ph.D. expected 2024
Topic: Conversational question answering.
Doaa Elfayoumi, M.Math. expected 2023
Topic: Recommendation.
Doctoral students, completed
Alexandra Vtyurina (Edith Law, co-supervisor), Ph.D. 2021
Supporting Voice-Based Natural Language Interactions for Information Seeking Tasks of
Various Complexity.
Amira Ghenai, (Mark Smucker, co-supervisor), Ph.D. 2019
Health Misinformation in Search and Social Media.
Carolyn Lamb (Dan Brown, co-supervisor), Ph.D. 2018
TwitSong: A Current Events Computer Poet and the Thorny Problem of Assessment.
Luchen Tan, Ph.D. 2016.
Tracking Events in Social Media.
Gaurav Baruah (Mark Smucker, co-supervisor), Ph.D. 2016.
Filtering News from Document Streams.
Azin Ashkan, Ph.D. 2013.
Characterizing User Search Intent and Behaviour for Click Analysis in Sponsored Search.
Maheedhar Kolla (Olga Vechtomova, co-supervisor), Ph.D. 2012.
Novelty and Diversity in Information Retrieval Evaluation.
John Whissell, M.Math. 2006, Ph.D. 2012.
Evaluating Clusterings by Estimating Clarity, Ph.D. thesis
Significant Feature Clustering, M.Math. thesis (Chrysanne DiMarco, co-supervisor)
John Akinyemi, Ph.D. 2012.
Similarity and Diversity in Information Retrieval.
Kelly Itakura, Ph.D. 2010.
Focused Retrieval.
Stefan Büttcher, Ph.D. 2007.
Multi-User File System Search- Faculty of Mathematics Award for Outstanding Achievement in Graduate Studies- University Award for Exceptional Teaching by a Student
18

Celine Latulipe (Craig Kaplan, co-supervisor), Ph.D. 2006.
A Symmetric Interaction Model for Bimanual Input- Faculty of Mathematics Award for Outstanding Achievement in Graduate Studies- Alain Fournier Ph.D. Thesis Award
Egidio L. Terra, Ph.D. 2004.
Lexical Affinities and Language Applications.
Tony Cox (G. V. Cormack, co-supervisor), Ph.D. 2002.
A Source-Based Approach to Representing and Managing Information Extracted
by Program Analysis.
Master’s students, completed
Siqing Huo, M.Math. 2023
Retrieving Supporting Evidence for Generative Question Answering.
Kamyar Ghajar (Mark Smucker, co-supervisor), M.Math. 2022
Evaluating the Ability of Commercial Search Engines to Help People Answer Health Questions.
Mahsa Seifikar, M.Math. 2023
A Preference Judgment Interface for Authoritative Assessment.
Nicole Yan (Mark Smucker, co-supervisor), M.Math. 2022
Offline Evaluation via Human Preference Judgments: A Dueling Bandits Problem.
Dahlia Shehata, M.Math. 2022
Information Retrieval with Entity Linking.
Chengxi Luo, M.Math. 2021
Information Retrieval Evaluation Measures Based on Preference Graphs.
Saira Rizvi, M.Math. 2021
Misinformation Retrieval.
Shaikh Quader, M.Math. 2017 (by essay).
Adriel Dean-Hall, M.Math. 2014
Evaluating Contextual Suggestion.
Younos Aboulnaga, M.Math. 2013
Efficient Temporal Synopsis of Social Media Streams.
Gobaan Raveendran, M.Math. 2013
Efficiently Crawling, Collecting, and Condensing News Comments.
Mohammed Alliheedi (Chrysanne DiMarco, co-supervisor), M.Math. 2012.
Multi-document Summarization Using Rhetorical Information
Hani Nikkhoo, M.Math. 2011.
The Impact of Near-Duplicate Documents on Information Retrieval Evaluation
Peter Yeung, M.Math. 2007.
Weighting Document Genre in Enterprise Search
19Dimitri Mostinski (Pascal Poupart, co-supervisor), M.Math. 2007.
Dynamic Factored Particle Filtering for Context-Specific Correlations
Archana Adma, M.Math. 2006.
iTalk — Index Your Talk.
Sandy Graham, M.Math. 2006 (by essay).
Philip Tilker, M.Math. 2005.
A Reliable Storage Management Layer for a Distributed Information Retrieval System.
Chris Mennie, M.Math. 2004.
Giving Meaning to Macros.
John Johansen, M.Math. 2004.
Delta Pointers.
Feng Zhao, M.Math. 2002.
Exploring Altruistic Links in a Web Crawler.
Chiasen Chung, M.Math. 2002.
Topic-oriented Collaborative Web Crawling.
Asim Elahi, M.Math. 2001 (by essay).
Alan Tran, University of Toronto M.Sc. 2000.
A Network Management Facility for a Fault-Tolerant Distributed Information
Retrieval System
Susan Sim (R. C. Holt, co-supervisor.), University of Toronto M.Sc. 1998.
Supporting Multiple Program Comprehension Strategies During Software Maintenance.
Tony Cheng, University of Toronto M.A.Sc. 1998.
A Low-Cost Robust Distributed Storage System for Large Information Retrieval Systems.

Thesis Committee Member (internal)
Oluwaseun Cardoso (M.Math. 2024), Aisha Khatun (M.Math. 2024), Houmaan Chamani (M.A.Sc.
2024) Mofetoluwa Adeyemi (M.Math. 2023), Luyun (Aileen) Lin (M.Math. 2023), Wei Zhong(Ph.D. 2023), Yuqing (Sherry) Xie (Ph.D. 2023), Shadi Ghasemitaheri (M.Math. 2023), Veronica Salm (M.Math. 2023), Odunayo Ogundepo (M.Math. 2023), Linh Nhi Phan Minh (M.Math.
2023), Ji (Jayen) Xin (Ph.D. 2023), Peng Shi (Ph.D. 2023), He (Richard) Bai (Ph.D. 2023),
Amir Vakili Tahami (M.Math. 2022), Alireza Vezvaei (M.Math. 2022), Kelechi Ogueji (M.Math.
2022), Solaiappan Alagappan (M.Math. 2022), Mohammad Dehghan (M.Math. 2022), Dake
Zhang (M.Math. 2022), Mustafa Absualsaud (Ph.D. 2021), Jean Wang (M.Math. 2021), Anup
Anand Deshmukh (M.Math. 2021), Kyle Robinson (M.Math. 2020), Bahar Sarrafzadeh (Ph.D.
2020), Nalin Chhibber (M.Math. 2019), Zeynep Akkalyoncu Yilmaz (M.Math. 2019), Daniel J.
Watson (M.Math. 2019), Abeer Khan (M.Math. 2019), Mohammed Alliheedi (Ph.D. 2019), Wei
Yang (M.Math. 2019), Adam Roegiest (Ph.D. 2017), Aiman L. Al-Harbi (Ph.D. 2016), Andrew
Kane (Ph.D. 2014), Rakesh Guttikonda (M.Math. 2014), Kaheer Suleman (M.Math. 2014), JackThomas (M.Math. 2014), Francisco Claude (Ph.D. 2013), Le Li (M.Math. 2013), Claus Strommer (Ph.D. 2011), Ting Liu (M.Math. 2011), C P Jethani (M.Math. 2011), Hussein Hirjee(M.Math. 2010), Rehan Rauf (M.Math. 2010), Jakub Gawryjolek (M.Math. 2009), Rana Alshaar (M.A.Sc. 2009), Yuan Lin (Ph.D. 2008), Radoslav Radoulov (M.Math. 2008), Jeff Dicker
20

(M.Math. 2008), Kun Cen (M.A.Sc. 2008), Brad Lushman (M.Math. 2002, Ph.D. 2007), Jingwei
Wu (Ph.D. 2006) Ahmed E Hassan (Ph.D. 2005), Jiongxiong Chen (Ph.D. 2005), Allan Menezes(M.Math. 2005), Luke Andrew Tanur (M.Math. 2005), Rick Bodner (Ph.D., Toronto, 2004),
Jiongxiong Chen (Ph.D. 2004), Jack S. Chi (M.Math. 2004), Rodolfo Gabriel Esteves Jaramillo(M.Math. 2004), Yuato Guo (M.Math. 2004), Xuhu Li (M.Math. 2004), Fuchun Peng (Ph.D.
2003), Yingwei Wang (Ph.D. 2003), Lijie Zou (M.Math. 2003), Vlado Keselj (Ph.D. 2002), Reem
Al-Halimi (Ph.D. 2002), Sunny K. S. Lam (M.Math. 2002), Thomas Lynam (M.Math. 2002),
Maryam Aamir Khan (M.Math. 2002), Greg McLearn (M.Math. 2002), Steven Engels (M.Math.
2002), Ian Bull (M.Math. 2002), Zarrin Langari (M.Math. 2001), Rayan Yahfoufi (M.Math.
2001), Stergios Anastasiadis (Ph.D., Toronto, 2001), Michal Ostrowski (M.Math. 2000), Michael
Van Biesbrouck (M.Math. 1999), Larry Baptiste (M.A.Sc., Toronto, 1998), Kevin Gior (M.A.Sc.,
Toronto, 1998), Stephen Green (Ph.D., Toronto, 1998), Bing He (M.Sc., Toronto, 1998), Tony
D. Lindsay (M.A.Sc., Toronto, 1998), Tomi Poutanen (M.A.Sc., Toronto, 1998), Nikos Koudas(Ph.D., Toronto, 1997), Dattatraya H. Kulkarni (Ph.D., Toronto, 1997), Sudarson Tandri (Ph.D.,
Toronto, 1997), Govindan Ravindran (Ph.D., Toronto, 1997), Kelvin Shek Yiu (M.A.Sc., Toronto,
1997), Gene Golovchinsky (Ph.D., Toronto, 1996), Mark Stephen Lawford (Ph.D., Toronto, 1996).

Undergraduate Student Supervision
Zahra Ahmed, Co-op Student Employee, May.-Aug. 2020
Nicole Simone, Co-op Student Employee, May.-Aug. 2013
Cat Mercer, Co-op Student Employee, Sept.-Dec. 2012
Michael Chong, Co-op Student Employee, Sept.-Dec. 2011
Ryan King, Co-op Student Employee, Sept.-Dec. 2011 and May-Aug. 2012
William Song, Undergraduate Research Associate, May-Aug. 2011
Trevor Fairey, Co-op Student Employee, May-Aug. 2010
Yubin Kim, Undergraduate Research Associate, May-Aug. 2010
Youngki Chang, Undergraduate Research Associate, May-Aug. 2007
James Rupke, Co-op Student Employee, May-Aug. 2007
Peter Yeung, Co-op Student Employee, May-Aug. 2005
While at the University of Toronto I supervised 26 undergraduate theses and final-year design
projects. In 1999, two of my students (Zhao Ping Tang and Chuan Zhang) received thedepartmental award for the best final-year design project.

21

Courses Taught
Term
Fall 2023
Winter 2023
Spring 2022
Winter 2022
Fall 2020
Fall 2019
Fall 2019
Fall 2018
Fall 2018
Spring 2015
Fall 2014
Fall 2013
Spring 2013
Fall 2011
Fall 2011
Spring 2011
Fall 2010
Fall 2010
Fall 2009
Fall 2008
Fall 2007
Winter 2007
Fall 2005
Fall 2005
Fall 2004
Spring 2004
Spring 2004
Fall 2003
Spring 2003
Winter 2003
Winter 2003
Spring 2002
Spring 2002
Winter 2002
Spring 2001
Winter 2001
Winter 2001
Spring 2000
Winter 2000
Spring 1999
Fall 1998
Fall 1998
Spring 1998
Spring 1998
Fall 1997
Spring 1997
Fall 1996

Title
CS135: Designing Functional Programs
CS497: CS Capstone design
CS497: CS Capstone design
CS497: CS Capstone design
CS135: Designing Functional Programs
CS135: Designing Functional Programs
CS135: Designing Functional Programs
CS135: Designing Functional Programs
CS135: Designing Functional Programs
CS848: Search and Recommendation
CS137: Programming Principles
CS137: Programming Principles
CS247: Software Abstraction and Specification
CS848: Information Retrieval
CS137: Programming Principles
SE490: Design Project I
CS848: Information Retrieval
CS137: Programming Principles
CS137: Programming Principles
CS137: Programming Principles
CS798: Information Retrieval
CS452: Real-Time Programming
CS133: Developing Programming Principles
CS133: Developing Programming Principles
CS452: Real-Time Programming
CS350/353: Operating Systems
CS350/353: Operating Systems
CS452: Real-Time Programming
CS848: Question Answering
CS342: Concurrent Programming
CS452: Real-Time Programming
CS342: Concurrent Programming
CS342: Concurrent Programming
CS444: Compiler Construction
CS452: Real-Time Programming
CS444: Compiler Construction
CS130: Developing Programming Principles
CS452: Real-Time Programming
CS748R: Information Retrieval
ECE1765: File Structures and Storage Systems
CSC442: Software Engineering
APS105: Computer Fundamentals
ECE1765: File Structures and Storage Systems
ECE544: Large Software Systems
CSC442: Software Engineering
ECE544: Large Software Systems
ECE544: Special Topics in Software Engineering

22

Level
UG
UG
UG
UG
UG
UG
UG
UG
UG
G
UG
UG
UG
G
UG
UG
G
UG
UG
UG
G
UG/G
UG
UG
UG/G
UG
UG
UG/G
G
UG
UG/G
UG
UG
UG/G
UG/G
UG/G
UG
UG/G
G
G
UG
UG
G
UG
UG
UG
UG

Class Size
104
12
8
6
online
86
85
83
86
15
138
152
94
9
127
75
23
136
116
121
31
11
114
82
42
74
77
22
30
101
7
105
97
40
30
34
80
28
24
18
138
140
11
12
104
13
3

(Toronto)(Toronto)(Toronto)(Toronto)(Toronto)(Toronto)(Toronto)(Toronto)

SERVICEInternal service
2021-present

Associate Dean, Innovation and Entrepreneurship, Faculty of Mathematics

2020-2021

Chair of School Advisory Committee on Appointments, School of Computer Science

2019-2020

Co-chair of School Advisory Committee on Appointments, School of Computer Science

2018-2019

Member of School Advisory Committee on Appointments, School of Computer Science

2015-2016

Director of Undergraduate Studies, School of Computer Science

2013-2014

Associate Director of Undergraduate Studies, School of Computer Science

2012-2013

Accreditation Representative for Software Engineering, Faculty of Engineering

2007-2012

Director of Software Engineering, University of Waterloo

2004-2005

Graduate Student Advocate, School of Computer Science

2004-2005

Member of Undergraduate Academic Plans Committee, School of Computer Science

2002-2004

Member of Advisory Committee, Computer Science Computing Facility

2001-2004

Member of Computing Advisory Committee, Faculty of Mathematics.

1998-1999

Member of Student-Staff Committee, Department of Electrical and Computer Engineering, University of Toronto.

1997-1999

Member of Recruiting Committee, Department of Electrical and Computer Engineering, University of Toronto.

1997-1999

Coach and faculty advisor for Toronto’s ACM International Programming Contest
teams. In 1999 one team advanced to the world finals in Eindhoven.

External service
General co-Chair, NTCIR Conference Series, Tokyo, Japan, 2016Past Chair, ACM Special Interest Group on Information Retrieval, 2016-2019
Chair, ACM Special Interest Group on Information Retrieval, 2013-2016
Co-editor-in-chief, Information Retrieval Journal, 2010-2018
Editorial board:
ACM Books, 2016Springer Informational Retrieval Series
Foundations and Trends in Information Retrieval, 2007-2018
Steering committee, Conference on Information and Knowledge Management, 2010-201623

Program co-chair, ACM SIGIR Conference, 2014
Program co-chair, ACM SIGIR Conference, 2007
General co-chair, ACM SIGIR Conference, 2003
Perspectives Track co-chair, ACM SIGIR Conference, 2024
Doctoral consortium chair:
ACM CHIIR Conference, 2023
ACM SIGIR Conference, 2018, 2019Best paper co-chair, ACM WSDM Conference, 2017, 2018
Poster chair, International World Wide Web Conference, 1999
Text REtrieval Conference, Track coordinator:
Lateral Reading Track Coordinator, 2024
Health Misinformation Track Coordinator, 2020, 2021, 2022
Total Recall Track Coordinator, 2015,2016
Contextual Suggestion Track Coordinator, 2012-2016
Web Track Coordinator, 2009-2013
Terabyte Track Coordinator, 2004-2006
Workshop organizer:
ECIR Workshop on Search Futures, 2024
CHIIR Workshop on Human-Centred Metrics for Information Seeking, 2020, 2023
SIGIR Workshop on Privacy-Preserving IR, 2016
SIGIR Workshop on Web Question Answering, 2015, 2016
ECIR Workshop on Information Access in Smart Cities, 2014
SIGIR Workshop on Modeling User Behavior for IR Evaluation, 2013
SIGIR Workshop on Open Source Information Retrieval, 2012
WSDM Workshop on Diversity in Document Retrieval, 2012
ECIR Workshop on Diversity in Document Retrieval, 2011
SIGIR Workshop on Complex Search Tasks, 2011
Meta-reviewer:
ACM SIGIR Conference, 2002, 2004-2006, 2009-2013, 2015-2016, 2019-2023
European Conference on Information Retrieval, 2022-2024
ACM WSDM Conference, 2009, 2012, 2014-2016, 2019-2021, 2024
ACM CIKM Conference, 2019-2023
International World Wide Web Conference, 2012, 2021, 2023, 2024
HLT-NAACL Human Language Technology Conference, 2003, 2004, 2009
Program committee member:
ACM WSDM Conference, 2022, 2023
International Conference on the Theory of Information Retrieval, 2013, 2023
International World Wide Web Conference, 2008, 2009, 2011, 2012, 2019-2020, 2022
String Processing and Information Retrieval Symposium, 2008, 2011, 2015, 2020
ACM CIKM, 2001-2005, 2007-2009, 2013, 2015
ACM KDD, 201524

Asia Information Retrieval Symposium, 2008, 2013, 2014
European Conference on Information Retrieval, 2007-2010
Recherche d’Information Assistée par Ordinateur, 2010
ACM SIGIR Conference, 2001, 2003, 2008
Network and Distributed System Security Symposium, 2007, 2008
Human Language Technology Conference/EMNLP, 2005
Pacific Association for Computational Linguistics Conference, 2005
International Joint Conference on Natural Language Processing, 2005

Reviewer:
ACM Transactions on Information Systems
Information Retrieval
Information Processing and Management
Journal of the American Society for Infomation Science and Technology
IEEE Transactions on Knowledge and Data Engineering
ACM Computing Surveys
ACM Transactions on the Web
VLBD Journal
Software Practice and Experience
Information Systems
Information Sciences
Computational Intelligence
Knowledge and Information Systems
Grant reviewer:
Swiss National Science Foundation
Netherlands Organisation for Scientific Research
Natural Sciences and Engineering Research Council, Canada
Mitacs, Canada
U.S. National Science Foundation
Hong Kong Research Grants Council
Dutch National Science Foundation
Israel Science Foundation
Ph.D. external examiner:
Valeriia Baranova, RMIT University, Melbourne, Australia, 2024
Hadi Hashemi, University of Amsterdam, 2021
Julia Kiseleva, University of Eindhoven, 2016
Matt Crane, University of Otago, 2015
Arbi Bouchoucha, University of Montreal, 2015
Peter Golbus, Northeastern, 2014
Falah Al-Akashi, University of Ottawa, 2014
Rodrygo Santos, University of Glasgow, 2013
Mir Sadek Ali, University of Toronto, 2012
William Webber, University of Melbourne, Australia, 2010
Yohannes Tsegay, RMIT University, Melbourne, Australia, 2009
Yang Liu, York University, Toronto, Canada, 2009.
Muath Alzghool, University of Ottawa, 2009
Thanh Tin Tang, Australian National University, 2006.
Dirk Bahle, RMIT University, Melbourne, Australia, 2003.

25

Christof Monz, University of Amsterdam, 2003.

26