CONFERENCE REPORT

Report on ECIR 2016: 38th European
Conference on Information Retrieval
Nicola Ferro
University of Padua, Italy
ferro@dei.unipd.it

Fabio Crestani
Università della Svizzera italiana, Switzerland
fabio.crestani@usi.chMarie-Francine Moens
KU Leuven, Belgiumsien.moens@cs.kuleuven.be
Fabrizio Silvestri
Yahoo! Labs, London
silvestr@yahoo-inc.com

Josiane Mothe
IRIT, Université de Toulouse, France
josiane.mothe@irit.fr
Jaana Kekäläinen
University of Tampere, Finlandjaana.kekalainen@uta.fi

Paolo Rosso
Universitat Politècnica de València, Spainprosso@dsic.upv.es
Gabriella Pasi
University of Milano Bicocca, Italy
pasi@disco.unimib.it

Paul Clough
University of Sheffield, UK
p.d.clough@sheffield.ac.uk

Christina Lioma
University of Copenhagen, Denmarkc.lioma@di.ku.dk

Stefano Mizzaro
University of Udine, Italy
mizzaro@uniud.it

Giorgio Maria Di Nunzio
University of Padua, Italy
dinunzio@dei.unipd.it

Claudia Hauff
TU Delft, The Netherlands
c.hauff@tudelft.nl

Omar Alonso
Microsoft Bing, USA
omar.alonso@microsoft.com

Pavel Serdyukov
Yandex, Russia
pavser@yandex-team.ru

Gianmaria Silvello
University of Padua, Italy
silvello@dei.unipd.it

ACM SIGIR Forum

12

Vol. 50 No. 1 June 2016

Abstract
The 38th European Conference on Information Retrieval took place from the 20th to the23rd of March 2016 in Padua, Italy. This report summarizes the conference in terms of thepresented keynotes, scientific and social programme, industry day, tutorials, workshops andstudent support.

1

Introduction

The European Conference on Information Retrieval (ECIR) is the main European forum for thepresentation of new research results in the field of information retrieval and cognate areas. Initiatedas a BCS Information Retrieval Specialist Group (IRSG) Colloquium for Information Retrieval in
1979, the event developed in a renowned international scientific meeting. To reflect its European
scale, since 1998 the conference was held alternately in the UK and continental Europe. From2012, this rule has been removed. The most recent ECIR editions were held in Glasgow, UK(2008); Toulouse, France (2009); Milton Keynes, UK (2010); Dublin, Ireland (2011); Barcelona,
Spain (2012), Moscow, Russia (2013), Amsterdam, The Netherlands (2014); and, Vienna, Austria(2015).
The 2016 edition of ECIR1 [4] was hosted by the Information Management Systems 2 (IMS)
research group at the Department of Information Engineering3 of the University of Padua4 , Italy,
from 20th to 23rd of March 2016. A total of 220 people attended ECIR 2016.

2

Organizing Committee, Venues and Social Program

2.1

Organizing Committee

ECIR 2016 featured a large and multidisciplinary organizing committee, almost perfectly gender balanced: General Chair, Nicola Ferro (University of Padua, Italy); Program Chairs, Fabio
Crestani Università della Svizzera Italiana (USI), Switzerland) and Marie-Francine Moens (KU
Leuven, Belgium); Short Paper Chairs, Josiane Mothe (ESPE, IRIT, Université de Toulouse,
France) and Fabrizio Silvestri (Yahoo! Labs, London); Student Mentorship Chairs, Jaana Kekäläinen(University of Tampere, Finland) and Paolo Rosso (Universitat Politècnica de València, Spain);Workshop Chairs, Paul Clough (University of Sheffield, UK) and Gabriella Pasi (University of
Milano Bicocca, Italy); Tutorial Chairs, Christina Lioma (University of Copenhagen, Denmark)and Stefano Mizzaro (University of Udine, Italy); Demo Chairs Giorgio Maria Di Nunzio (University of Padua, Italy) and Claudia Hauff (TU Delft, The Netherlands); Industry Day Chairs,
Omar Alonso (Microsoft Bing, USA) and Pavel Serdyukov (Yandex, Russia); Local Organization
Chair, Gianmaria Silvello (University of Padua, Italy).
1

http://ecir2016.dei.unipd.it/
http://ims.dei.unipd.it/
3
http://www.dei.unipd.it/en/4
http://www.unipd.it/en/
2

ACM SIGIR Forum

13

Vol. 50 No. 1 June 2016

(a) Botanical garden: plenary lecture room.

(b) Botanical garden: parallel lecture room.

(c) Bo Palace: plenary lecture room.

(d) Bo Palace: parallel lecture room.

Figure 1: ECIR 2016 main venues.

2.2

VenuesECIR 2016 has been organized in three separate venues, shown in Figure 1: workshops andtutorials have been hosted at the Department of Geosciences; the first day of the conference has
been hosted at the University of Padua Botanical Garden, which is the world’s oldest academic
botanical garden that is still in its original location; the other two days of the conference and theindustry day have been hosted at the Bo Palace, the main historical building of the University of
Padua.

2.3

Social Programme

ECIR 2016 featured a very rich social program. The Welcome Reception on Sunday evening offered
the merry environment of a little country festival, with typical local dishes, such as “porchetta”(Italian spit-roasted pork), hand made pizza, and draft beer. The poster session and the reception
on Monday evening were hold at Caffè Pedrocchi, one of the most important historical cafes

ACM SIGIR Forum

14

Vol. 50 No. 1 June 2016

(a) Welcome reception.

(b) Poster session and reception.

(c) Social dinner.

Figure 2: ECIR 2016 social events.

ACM SIGIR Forum

15

Vol. 50 No. 1 June 2016

in Italy, inaugurated in 1831, keeping a mixture of neoclassic and neo-Gothic styles. Since its
opening, the Caffè Pedrocchi welcomes eminent guests such as the writer Stendhal, who named
its desserts as some of the best he ever tried, as well as students who always spent hours in thereading room discussing all sorts of matters, sometimes producing innovative ideas, sometimes therevolution, as they did in 1848 during Asburgic domination. Here, in the Rossini room, dedicatedto the composer Rossini, after the Poster session, ECIR 2016 participants were greeted with thetypical north-eastern Italian wine “Prosecco” and a dinner with Italian cheeses, cured meats,shellfish, and other Italian culinary specialities. Finally, the social dinner was hosted at the “SanGaetano” Cultural Centre, built in 2008 renovating the old city courthouse. The Cultural Centre
is a modern five-storeyed structure, which preserves ancient signs of old city: the excavation
made in order to create the basement places revealed late medieval pavements and a considerable
amount of archaeological findings stratified from the Renaissance back to the Roman age, partiallyvisible in the foyer. The social dinner was prepared by one of the best caterers in town and gave
participants the opportunity to taste fine Italian ingredients, dishes, wines and dozens of different
types of “grappa”.

3

Keynotes

ECIR 2016 included three keynotes: two keynotes were the Karen Spärck Jones (KSJ) Award
ones – Jordan Boyd-Graber (University of Colorado, USA) and Emine Yilmaz (University College
London, UK) – while the third keynote, Domonkos Tikk (Gravity R&D, Hungary), was part ofthe industry day.

3.1

Machine Learning Shouldn’t be a Black Box

Jordan Boyd-Graber gave the first KSJ keynote, entitled “Machine Learning Shouldn’t be a BlackBox”. Machine learning is ubiquitous: detecting spam e-mails, flagging fraudulent purchases, andproviding the next movie in a Netflix binge. But few users at the mercy of machine learning
outputs know what’s happening behind the curtain. Jordan Boyd-Graber tried to demystify theblack box for non-experts by creating algorithms that can inform, collaborate with, compete with,and understand users in real-world settings, providing several examples ranging from topic models, simultaneous machine interpretation in translation, and language-based games in diplomacy,
politics, and trivia.

3.2

A Task-Based Perspective to Information Retrieval

Emine Yilmaz gave the second KSJ keynote, entitled “A Task-Based Perspective to Information
Retrieval”. Ideally, an information retrieval system should be able to understand the reason thatcaused the user to submit a query and it should help the user achieve the actual task by guiding
her through the steps (or subtasks) that need to be completed. Devising task based information
retrieval systems have several challenges that have to be tackled and Emine Yilmaz described them,in comparison and contrast with the traditional way in building information retrieval systems.
The main challenges to be faced are: (1) devising methodologies for accurately extracting andrepresenting tasks, (2) building and designing new interfaces for task based information retrieval

ACM SIGIR Forum

16

Vol. 50 No. 1 June 2016

systems, (3) devising methodologies for evaluating the quality of task based information retrieval
systems, and (4) task based personalization of information retrieval systems.

3.3

Lessons Learned at Building Recommendation Services in Industry Scale

Domonkos Tikk gave the industry keynote, entitled “Lessons learnt at building recommendation
services in industry scale”. He has shown how Gravity’s recommendation technology, top contender in Netflix, has evolved from the big pile of task specific program codes to scalable services
that serve billions of recommendation requests monthly. Having academic origin with strong research focus, the recommendation quality has always been the primary differentiating factor at
Gravity but, over times, they also learnt that machine learning competitions are different fromscalable and robust services. Therefore, Domonkos Tikk discussed and presented some lessons
learnt on this road to create a solution from a offspring of the Netflix competition.

4

Programme

ECIR 2016 received a total of 284 submissions in three categories: 201 full papers out of which 7
papers in the Reproducibility track, 66 short papers, and 17 demonstrations.
The geographical distribution of the submissions was as follows: 51% were from Europe, 21%from Asia, 19% from North and South America, 7% from North Africa and the Middle East, and
2% from Australasia.
All submissions were reviewed by at least three members of an international two-tier Program
Committee. Of the full papers submitted to the conference, 42 were accepted for oral presentation(22% of the submitted ones) and 8 as posters (4% of the submitted ones). Of the short papers
submitted to the conference, 20 were accepted for poster presentation (30% of the submittedones). In addition, 6 demonstrations (35% of the submitted ones) were accepted. The accepted
contributions represent the state of the art in information retrieval, cover a diverse range of
topics, propose novel applications, and indicate promising directions for future research. Figure 3
provides a break-down of the submitted and accepted papers and demos by country and highlights
the global reach of ECIR as a premier international conference.
ECIR 2016 continued the Reproducibility track introduced in ECIR 2015 which specificallyinvited the submission of papers reproducing a single paper or a group of papers from a third
party, where the authors were not directly involved in the original paper. Authors were requested
to emphasize the motivation for selecting the papers to be reproduced, the process of how results
were attempted to be reproduced (successfully or not), the communication that was necessary togather all information, the potential difficulties encountered, and the result of the process. Of the
7 papers submitted to this track, 4 were accepted (57% of the submitted ones).
The Programme Committee consisted of 183 members, 21 of which were Meta-Reviewers and162 of which were Reviewers. 59 Additional Reviewers entered some reviews on behalf of a
Reviewer. The full paper decisions were made based on a two-tier review process involving both
Reviewers and Meta-Reviewers, while the decisions on the short papers and demos were madebased only on reviews.

ACM SIGIR Forum

17

Vol. 50 No. 1 June 2016

(a) Full papers.

(b) Short papers.

(c) Demos.

Figure 3: Breakdown of submitted and accepted papers by country.
A panel on “Data-driven Information Retrieval” was organized at ECIR by Maristella Agosti(University of Padua, Italy) with Omar Alonso (Microsoft Bing, USA), Raffaele Perego, (ISTI
CNR, Italy), and Maarten de Rijke (University of Amsterdam, The Netherlands) as panelists.
The panel stems from the fact that IR has been always concerned with finding the “needle in ahaystack” to retrieve the most relevant information from huge amounts of data, able to best addressuser information needs. Nevertheless, nowadays we are facing a radical paradigm shift, common
also to many other research fields, and IR is becoming a more and more data-driven science due,for example, to recent developments in machine learning, crowd-sourcing, user interaction analysis,
and so on. The goal of the panel is to discuss the emergent trends in this area, their advantages,
their pitfalls, and their implications for the future of the field.
Finally, two moments have been dedicated to the “Women in IR” meeting and discussions –one during the coffe break in the afternoon of the first day of the conference and the other during

ACM SIGIR Forum

18

Vol. 50 No. 1 June 2016

the lunch of the second day of the conference. In order to give visibility to these events andsolicit awareness in the community, the participants met around an ECIR 2016 totem dedicatedto Elena Lucrezia Cornaro Piscopia, the first woman in the world to receive an academic degree
at the University of Padua in 1678.

5Awards

5.1

Test of Time Award

Considering the long history of ECIR, which is now at it 38th edition, ECIR 2016 introduced
a new award, the Test of Time (ToT) Award, to recognize research that has had long-lasting
influence, including impact on a subarea of information retrieval research, across subareas of
information retrieval research, and outside of the information retrieval research community (e.g.
non-information retrieval research or industry).
The ECIR ToT Award went to Cyril Goutte and Eric Gaussier for the paper entitled “A
Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation”
presented at ECIR 2005 [5].
There were also two honorable mentions: Giambattista Amati, Claudio Carpineto, Giovanni
Romano for the paper entitled “Query Difficulty, Robustness, and Selective Application of Query
Expansion” presented at ECIR 2004 [1]; and, Iadh Ounis, Gianni Amati, Vassilis Plachouras,
Ben He, Craig Macdonald, Douglas Johnson for the poster entitled “Terrier Information Retrieval
Platform” presented at ECIR 2005 [7].
The ToT Award Committe was chaired by Norbert Fuhr (University of Duisburg-Essen, Germany) and its members were: Maristella Agosti (University of Padua, Italy), Pia Borlund (University of Copenhagen, Denmark), Djoerd Hiemstra (University of Twente, The Netherlands),
Kalervo Järvelin (University of Tampere, Finland), Gabriella Kazai (Lumi.do, UK), Iadh Ounis(University of Glasgow, UK), and Jacques Savoy (University of Neuchatel, Switzerland). The ToTAward has been supported by Springer.

5.2

Best Paper Award

The Best Paper Award went to Mengdie Zhuang, Elaine G. Toms, and Gianluca Demartini forthe paper entitled “The Relationship between User Perception and User Behaviour in Interactive
Information Retrieval Evaluation” [10].
There were also three honorable mentions: Liora Braunstain, Oren Kurland, David Carmel,
Idan Szpektor, and Anna Shtok for the paper entitled “Supporting Human Answers for AdviceSeeking Questions in CQA Sites” [2]; Anjie Fang, Craig Macdonald, Iadh Ounis, and Philip Habel
for the paper entitled “Topics in Tweets: A User Study of Topic Coherence Metrics for Twitter
Data” [3]; and, Johannes Jurgovsky, Michael Granitzer, and Christin Seifert for the paper entitled“Evaluating Memory Efficiency and Robustness of Word Embeddings” [6].
The Best Paper Award Committee was chaired by Jaap Kamps (University of Amsterdam,
The Netherlands) and its members were: Charles L.A. Clarke (University of Waterloo, Canada)
and Jussi Karlgren (Gavagai, Sweden). The Best Paper Award has been supported by Yahoo!.

ACM SIGIR Forum

19

Vol. 50 No. 1 June 2016

During the poster and demo session, participants have been asked to vote for the best poster
and the best demo. The best poster was awarded to Martin Potthast, Sebastian Kpsel, Benno
Stein and Matthias Hagen for the poster entitled “Clickbait Detection” [8]; the best demo wasawarded to Pedro Saleiro, Jorge Teixeira, Carlos Soares and Eugénio Oliveira for the demo entitled“TimeMachine: Entity-centric Search and Visualization of News Archives” [9].

6Tutorials

6.1

Collaborative Information Retrieval: Concepts, Models and Evaluation

Recent work have shown the potential of collaboration for solving complex or exploratory searchtasks allowing to achieve synergistic effects with respect to individual search, which is the prevalent
information retrieval (IR) setting this last decade. An important challenge is related to thedesign of Collaborative Information Retrieval (CIR) models and their effectiveness evaluation
since individual IR frameworks and measures do not totally fit with the collaboration paradigms.
The CIR tutorial presented first a general overview of collaborative search introducing the mainunderlying notions. Then, it focused on related work dealing with collaborative ranking models
and their effectiveness evaluation.

6.2

Group Recommender Systems: State of the Art, Emerging Aspects and Techniques, and Research Challenges

A recommender system aims at suggesting to users items that might interest them and that they
have not considered yet. A class of systems, known as group recommendation, provides suggestions
in contexts in which more than one person is involved in the recommendation process. The tutorial
provided the audience with an overview on group recommendation, illustrating the recommendersystem principles. It then formally introduced the problem of producing recommendations togroups, and presented a survey based on the tasks performed by these systems. It also analyzed
challenging topics like their evaluation and present emerging aspects and techniques in this area.

6.3

Living Labs for Online Evaluation: From Theory to Practice (LiLa2016)

Experimental evaluation has always been central to Information Retrieval research. The field isincreasingly moving towards online evaluation, which involves experimenting with real, unsuspecting users in their natural task environments, a so-called living lab. Specifically, with the recent
introduction of the Living Labs for IR Evaluation initiative at CLEF and the OpenSearch trackat TREC, researchers can now have direct access to such labs. With these benchmarking platforms in place, online evaluation will be an exciting area to work on in the future. This half-day
tutorial aimed to provide a comprehensive overview of the underlying theory and complement itwith practical guidance.

ACM SIGIR Forum

20

Vol. 50 No. 1 June 2016

6.4

Real-Time Bidding based Display Advertising: Mechanisms andAlgorithms (RTBMA 2016)

Real-Time Bidding (RTB), which allows selling and buying in real-time one ad impression at atime. The ability of making impression level bid decision and targeting to an individual user inreal-time has fundamentally changed the landscape of the digital media. The further demand
for automation, integration and optimisation in RTB brings new research opportunities in the IR
fields, including information matching with economic constraints, CTR prediction, user behaviour
targeting and profiling, personalised advertising, and attribution and evaluation methodologies.
The tutorial teamed up presenters from both the industry and academia and it brought theinsightful knowledge from the real-world systems, by providing an overview of the fundamental
mechanism and algorithms with the focus on the IR context.

7

Workshops

7.1

3rd International Workshop on Bibliometric-enhanced Information Retrieval (BIR2016)

The BIR workshop brought together experts in Bibliometrics and Information Retrieval. First,
both communities only partly overlap, albeit sharing various interests. Second, it is profitable forboth sides to tackle some of the emerging problems that scholars face today when they have to
identify relevant and high quality literature in the fast growing number of electronic publications
available worldwide. Bibliometric techniques are not yet used widely to enhance retrieval processes
in digital libraries, although they offer value-added effects for users. Information professionals
working in libraries and archives, however, are increasingly confronted with applying bibliometrictechniques in their services. The third BIR workshop aimed to foster a common ground forthe incorporation of bibliometric-enhanced services into scholarly search engine interfaces. In
particular it addressed specific communities and studies on large, cross-domain collections like
Mendeley and ResearchGate.

7.2

1st International Workshop on Modeling, Learning and Mining
for Cross/Multilinguality (MultiLingMine 2016)

The increasing availability of text information coded in many different languages poses new challenges to modern information retrieval and mining systems in order to discover and exchange
knowledge at a larger world-wide scale. The 1st International Workshop on Modeling, Learning
and Mining for Cross/Multilinguality (dubbed MultiLingMine 2016) provided a venue to discussresearch advances in cross-/multilingual related topics, focusing on new multidisciplinary researchquestions that have not been deeply investigated so far (e.g., in CLEF and related events relevant
to CLIR). This included theoretical and experimental on-going works about novel representation models, learning algorithms, and knowledge-based methodologies for emerging trends andapplications, such as, e.g., cross-view cross-/multilingual information retrieval and document mining, (knowledge-based) translation-independent cross-/multilingual corpora, applications in social
network contexts, and more.
ACM SIGIR Forum

21

Vol. 50 No. 1 June 2016

7.3

ProActive Information Retrieval: Anticipating Users’ Information
Needs (ProAct IR)

Given the increasing popularity of smartphones and Internet enabled wearable devices, how can
the information retrieval systems use the additional data, and better interact with the user so
as to better understand, and even anticipate her precise information needs? Building such zeroquery or minimum user effort systems require research efforts from multiple disciplines covering
algorithmic aspects of retrieval models, user modeling and profiling, evaluation, context modeling,novel user interfaces design, etc. The ProAct IR workshop gathered together the researchers fromacademia and industry practitioners with these diverse backgrounds to share their experiences andopinions on challenges and possibilities of developing such proactive information retrieval systems.

7.4

1st International Workshop on Recent Trends in News Information Retrieval (NewsIR’16)

The news industry has gone through seismic shifts in the past decade with digital content andsocial media completely redefining how people consume news. Readers check for accurate freshnews from multiple sources throughout the day using dedicated apps or social media on theirsmartphones and tablets. At the same time, news publishers rely more and more on social networks
and citizen journalism as a frontline to breaking news. In this new era of fast-growing instantnews delivery and consumption, publishers and aggregators have to overcome a great number ofchallenges. These include the verification or assessment of a source’s reliability; the integrationof news with other sources of information; real-time processing of both news content and social
streams in multiple languages, in different formats and in high volumes; deduplication; entity
detection and disambiguation; automatic summarization; and news recommendation.
The NewsIR stimulated discussion around new and powerful uses of IR applied to news sources
and the intersection of multiple IR tasks to solve real user problems. To promote research efforts
in this area, NewsIR released a new dataset consisting of one million news articles to the researchcommunity and introduced a data challenge track as part of the workshop.

8

Industry Day

The goal of the Industry Day track was to bring an exciting programme that contained a mix ofinvited talks by industry leaders with presentations of novel and innovative ideas from the searchindustry. The final program consisted of invited talks and accepted proposal talks.
The invited invited talks were:• Domonkos Tikk (Gravity), see Section 3.3 above;• Etienne Sanson (Criteo), who presented an overview of the challenges R&D had to face overthe last 10 years at Criteo;• Debora Donato (StumbleUpon), who presented how to improve user engagement throughgenre diversification;

ACM SIGIR Forum

22

Vol. 50 No. 1 June 2016

• Nicola Montecchio (Spotify), who presented music search, personalization and discovery at
Spotify.
Four proposal talks were accepted but just three have been actually presented:• “Worst Practices for Designing Production Information Access Systems” by Fernando Diaz
(Microsoft Research NYC) who presented on the gap between studying and implementing
information access systems. Fernando’s presentation covered open problems in production
systems that academia is better suited for addressing than industry;• “Get on with it! Recommender system industry challenges move towards real-world, online
evaluation” by Roberto Turrin (Moviri/ContentWise), Martha Larson (Delft University ofTechnology), and Daniel Kohlsdorf (XING);• “Multilingual query categorization” by Michal Laclavik (Magnetic), Marek Ciglan (Magnetic), Sam Steingold (Magnetic), and Alex Dorman (Magnetic).

9

Student Support

ECIR has traditionally a strong focus on student participation. This year student support included
a mentoring program and student grants.
The student mentorship was aimed to assist young authors in preparing papers for submission
and improve quality of submissions. We invited about 30 researchers to participate as mentors
and 21 students applied for the programme. Students submitted six full papers and four shortpapers to the conference. Both mentors and mentees regarded the program as satisfactory. One
student paper was accepted to ECIR 2016 conference.
Student grant program included travel and accommodation grants supported by BCS andELIAS. We received 34 grant applications in total and 27 students were granted with travel andaccommodation support. Criteria for selection were: relevance of the research topic to ECIR;PhD year and track record (preference was given to students at an earlier stage of their career andwith evidence of an exceptionally strong track record); presentation of a full/short/demo paper;
strong letter of support. Preference was given to full-time students studying/working in developing
countries and presenting their work and to those who had no alternative travel support.
The Student Grant Committee was chaired by John Tait (JohnTait.net, UK) and its members
were: Jaime Arguello (UNC Chapel Hill, USA), Pavel Braslavski (Ural Federal University /Kontur Labs, Russian Federation), Jaap Kamps (University of Amsterdam, The Netherlands),
Diane Kelly (University of North Carolina, USA), Mihai Lupu (Vienna University of Technology,
Austria), and Nils Pharo (Oslo and Akershus University College of Applied Sciences, Norway).

10

Communication and MediaThe ECIR 2016 Web site has been launched at ECIR 2015 and provided all the necessary information about the conference, its program and its logistics.
Figure 4 summarizes about one year of activity of the ECIR 2016 Web site, from its launch at
ECIR 2015 in April 2015 to April 2016. The site has been visited by about 17,000 users, amounting
ACM SIGIR Forum

23

Vol. 50 No. 1 June 2016

(a) Web site access statistics.

(b) Geographic distribution of Web site accesses.

Figure 4: Participants survey results.

ACM SIGIR Forum

24

Vol. 50 No. 1 June 2016to around 33,000 session and 80,000 page views. The geographic distribution of the accesses shows
how ECIR has attracted interest from most of the countries in the world, confirming its global
reach.
ECIR 2016 had a presence on Twitter with the account @ecir2016 and the hashtag #ecir2016,
followed by 468 followers.
The main plenary sessions (opening, keynote talks, panel, and closing) were live streamed andare now available online at:• http://ecir2016.dei.unipd.it/video/opening• http://ecir2016.dei.unipd.it/video/ksj-keynote-jordan-boyd-graber• http://ecir2016.dei.unipd.it/video/ksj-keynote-emine-yilmaz• http://ecir2016.dei.unipd.it/video/industry-keynote-domonkos-tikk• http://ecir2016.dei.unipd.it/video/panel-data-driven-ir• http://ecir2016.dei.unipd.it/video/closing
Overall, in less than one month, since late March 2016 to mid April 2016, they have been watchedaround 2,000 times.
The photo stream of the conference is available at the following address: http://ecir2016.
dei.unipd.it/photo-stream.

11

Final Remarks

At the end of the conference, and in the subsequent days, we conducted a survey among theparticipants to ECIR 2016 to gather their feedback and opinion on different aspects of the conference and its organization. A total of 115 participants answered the survey and the results are
summarized in Figure 5. The overall impression on ECIR 2016 was positive or more than positivefor 96% of the participants and 90% or more of the participants were satisfied or very satisfied
with all the different aspects of the conference, detailed in Figure 5.
ECIR 2016 introduced, for the first time, the Test of Time Award in order assess the maturity
and impact of the scientific production of its community. Moreover, ECIR 2016 continued an
innovation introduced at ECIR 2015, namely the Reproducibility Track, in order to stress theimportance of the methodologies we adopt to conduct our experiment as well as to share andanalyze their results.
ECIR 20175 will be held in Aberdeen, UK, from 8th to 13th April 2017, hosted by RobertGordon University (RGU) and AECC (Aberdeen Exhibition and Conference Centre).
5

http://ecir2017.org/

ACM SIGIR Forum25

Vol. 50 No. 1 June 201690%
Very Poor
Poor

80%

Neutral
Good

70%

Very Good

60%

50%

40%

30%

20%

10%

0%
Overall
Impression

Scientiﬁc
Program

Registration
Desk

Web SiteCommunication Staff Support
and Media

VenuesSocial Events

Catering

Figure 5: Participants survey results.

Acknowledgements
We wish to express our gratitude to all the committee members, mentors, authors, and participants
who determined the real success of ECIR 2016.
We wish to greatly thanks all the people in the local organizing team who contributed tothe organization and running of ECIR 2016. Research staff: Maristella Agosti, Linda Cappellato, Sergio Canazza, Emanuele Di Buccio, Marco Ferrante, Maria Maistro, and Ivano Masiero;administrative staff: Silvia Bedin, Antonio Camporese, Alessandro Cardinale, Debora Leoncini,and Sabrina Michelotto; technical services staff: Alessandra Angarano, Federico Beccaro, Carlo
Calore, Marco Filippi, Vittorio Gallo, Andrea Graziani, Lorenzo Franceschin, Giorgio Paolucci,
Agostino Pinto, Elisabetta Piva, Dario Da Re, Lorenzo Sartoratti, Maria Daniela Toffan, andMaurizio Vedaldi; student volunteers: Eugenio Angriman, Giulio Busato, Andrea Carraro, Jessica
Mangano, Stefano Marchesin, Jennypher Mottola, Agata Rotondi, Daniel Zilio, Virginia Zorzi.
ECIR 2016 has been held under the patronage of: Regione del Veneto (Veneto Region), Comune
di Padova (Municipality of Padua), University of Padua, Department of Informantion Engineering,and Department of Mathematics.
Finally, ECIR 2016 would have not been possible without the generous financial support fromour sponsors: Google (gold level); Elsevier, Spotify, and Yahoo! Labs (palladium level); Springer(silver level); and, Yandex (bronze level). The conference was supported by the ELIAS Research
Network Program of the European Science Foundation, British Computer Society, University of
Padua, Department of Information Engineering, and Department of Mathematics.

References[1] G. Amati, C. Carpineto, and G. Romano. Query Difficulty, Robustness, and Selective Application of Query Expansion. In S. McDonald and J. Tait, editors, Advances in Information

ACM SIGIR Forum

26

Vol. 50 No. 1 June 2016

Retrieval. Proc. 26th European Conference on IR Research (ECIR 2004), pages 127–137.
Lecture Notes in Computer Science (LNCS) 2997, Springer, Heidelberg, Germany, 2004.
[2] L. Braunstain, O. Kurland, D. Carmel, I. Szpektor, and A: Shtok. Supporting Human Answers
for Advice-Seeking Questions in CQA Sites. In Ferro et al. [4], pages 129–141.
[3] A. Fang, C. Macdonald, I. Ounis, and P. Habel. Topics in Tweets: A User Study of Topic
Coherence Metrics for Twitter Data. In Ferro et al. [4], pages 492–504.
[4] N. Ferro, F. Crestani, M.-F. Moens, J. Mothe, F. Silvestri, G. M. Di Nunzio, C. Hauff,and G. Silvello, editors. Advances in Information Retrieval. Proc. 38th European Conference
on IR Research (ECIR 2016). Lecture Notes in Computer Science (LNCS) 9626, Springer,
Heidelberg, Germany, 2016.
[5] C. Goutte and E. Gaussier. A Probabilistic Interpretation of Precision, Recall and F-Score,with Implication for Evaluation. In D. E. Losada and J. M. Fernández-Luna, editors, Advances in Information Retrieval. Proc. 27th European Conference on IR Research (ECIR2005), pages 345–359. Lecture Notes in Computer Science (LNCS) 3408, Springer, Heidelberg, Germany, 2005.
[6] J. Jurgovsky, M. Granitzer, and C. Seifert. Evaluating Memory Efficiency and Robustness of
Word Embeddings. In Ferro et al. [4], pages 200–211.
[7] I. Ounis, G. Amati, V. Plachouras, B. He, C. Macdonald, and D. Johnson. Terrier Information Retrieval Platform. In D. E. Losada and J. M. Fernández-Luna, editors, Advances inInformation Retrieval. Proc. 27th European Conference on IR Research (ECIR 2005), pages
517–519. Lecture Notes in Computer Science (LNCS) 3408, Springer, Heidelberg, Germany,
2005.
[8] M. Potthast, S. Köpsel, B. Stein, and M. Hagen. Clickbait Detection. In Ferro et al. [4],pages 810–817.
[9] P. Saleiro, J. Teixeira, C. Soares, and E. Oliveira. TimeMachine: Entity-centric Search andVisualization of News Archives. In Ferro et al. [4], pages 845–848.
[10] M. Zhuang, E. Toms, and G. Demartini. The Relationship between User Perception and User
Behaviour in Interactive Information Retrieval Evaluation. In Ferro et al. [4], pages 293–305.

ACM SIGIR Forum

27

Vol. 50 No. 1 June 2016