# Meta-HCI: First Workshop on Meta-Research in HCI

Jonas Oppenlaender, Sylvain Malacria, Xinrui Fang, Niels van Berkel, Fanny Chevalier, Koji Yatani, Simo Hosio

<figure><img src="image_1.png" /></figure>

## To cite this version:

Jonas Oppenlaender, Sylvain Malacria, Xinrui Fang, Niels van Berkel, Fanny Chevalier, et al. Meta-HCI: First Workshop on Meta-Research in HCI. 2025, pp.1-8. 10.1145/3706599.3706723. hal-05457729

HAL Id: hal-05457729

https://inria.hal.science/hal-05457729v1

Submitted on 14 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Meta-HCI: First Workshop on Meta-Research in HCI

**Jonas Oppenlaender**

University of Oulu
Oulu, Finland
jonas.oppenlaender@oulu.fi

**Sylvain Malacria**

Univ. Lille, Inria, CNRS, Centrale Lille,
UMR 9189 CRISTAL
Lille, France
sylvain.malacria@inria.fr

**Xinrui Fang**

University of Tokyo
Tokyo, Japan
xinrui.fang@iis-lab.org

**Niels van Berkel**

Department of Computer Science
Aalborg University
Aalborg, Denmark
nielsvanberkel@cs.aau.dk

**Fanny Chevalier**

Department of Computer Science and
Statistical Sciences
University of Toronto
Toronto, Ontario, Canada
fanny@cs.toronto.edu

**Koji Yatani**

University of Tokyo
Tokyo, Japan
koji@iis-lab.org

**Simo Hosio**

Center for Ubiquitous Computing
University of Oulu
Oulu, Oulu, Finland
simo.hosio@oulu.fi

<figure><img src="image_3.png" /><figcaption>Figure 1: First Workshop on Meta-Research in HCI at ACM CHI '25 in Yokohama, Japan</figcaption></figure>

## Abstract

Human-Computer Interaction (HCI) is a rapidly evolving field. It has undergone many changes, and several current challenges deserve more attention from the community. Meta-research – the study of research practices – offers insights into how a field can refine its methodological frameworks, enhance rigor, and address its challenges. We believe CHI deserves a dedicated space for meta-research. This workshop establishes an open space for HCI scholars in the top conference of the field to explore and discuss meta-research

in HCI. We are equally focused on the past, present, and future: what we study, how we document it, how we evaluate, and how we distribute our work. Collateral effects such as mounting career pressures to publish always more are interesting, too. Short term results of this workshop include a research roadmap specifically for HCI meta-research. In the long term, we hope to see this workshop be the initial spark to establishing a permanent HCI meta-research community.

## CCS Concepts

* Human-centered computing → Human computer interaction (HCI).

## Keywords

meta research, meta science, science of science, CHI, meta-HCI
# 1 Motivation

Human-Computer Interaction (HCI) is a rapidly evolving field, marked by diverse methodologies, interdisciplinary approaches, and a growing global community of researchers. The HCI field has undergone topical changes, represented in at least three waves [13, 40]. These continuing topical changes occur against the backdrop of broader changes in academia and science, such as the exponential growth in publications [15] and the increasing career pressure and workload experienced by academics [27, 38]. The ACM CHI Conference is the flagship conference of HCI, and we believe it deserves a dedicated space for *meta-research* to document, discuss, and collaborate on how the community evolves and operates. We next discuss just a fraction of the meta-research aspects that deserve increased attention from such a meta-research perspective.

**The evolution of research topics in HCI:** HCI research involving human factors reflects the zeitgeist and ethos of its time [40]. In recent years, popular topics have been influenced by the rapid proliferation of large language models, conversational systems, and other generative systems. On the negative side of this development, the availability of these writing machines that can “delve” into any topic [51] and simulate participants [31, 81, 99] opens opportunities for bad actors to productionize their output at the expense of quality [29]. Further, some community members have voiced strong opinions recently about how a disproportionate number of HCI work is now thematizing or using generative AI [5]. Yet, generative AI also has transformative positive potential in how we conduct research at large, and going forward it will only make sense to shed light on those ways and try to identify the constructive and creative ways of using them ethically and productively.

**How we conduct research:** Generative AI aside, there are other fundamental issues to focus on in terms of how we conduct research. For instance, the “replication crisis” has been identified earlier [3, 23, 46, 72] but not necessarily fully addressed yet. Expectations of the field also steer the ways we work. For instance, the CHI Conference has seen an increase in the quality expectations for published works. Arguably, some of the research published ten years ago would no longer meet the minimum requirements of today’s CHI Conference. For instance, the number of references included in a CHI paper has increased with each year since CHI ’16 [74], and most likely this is now an implicit expectation by reviewers. From a pure knowledge-creation perspective, it is also fascinating to understand the citation behaviors of the community. For instance, Pohl and Mottelson [83] conducted a quantitative analysis of 6,578 CHI papers, investigating how authors write their papers and how factors such as readability and name dropping influence citation counts. It is critical to reflect on the consequences of (and the reasons behind) all these developments.

**How we evaluate research:** Peer review is an integral part of our work, in evaluating the work. While critically important, it

also faces a number of recognized limitations [2, 8, 24, 44, 56, 59, 87, 95, 101]. As the field evolves, peer review changes. Indeed, CHI has also introduced several changes over the years. Currently we are hearing the weak signals from the community in terms of how, for instance, the already mentioned generative AI is used, should be used, or should not be used in peer review [74]. Other aspects include the obsession with novelty and the lukewarm reception of negative but nonetheless interesting results. Unclear policies can result, for instance, in inconsistently enforced desk rejections and peer reviews [64]. Differing ethics policies among institutions is a constant topic of debate in peer reviewing. And the entire community now feels the increased workload in review duties, leading to difficulties in identifying the best reviewers for the research. There is a growing urgency to act on these issues. For instance, the CHI Steering Committee recently established a one-year working group on peer review [64]. The working group’s objectives include evaluating the potential for the introduction of a credit system for peer review [25] and the establishment of more consistent norms and policies [64].

**What we publish:** Similarly, the very nature of *what* we evaluate has evolved, with submissions now not only consisting of a simple PDF submission, but coming with various adjunct materials (video figure, data sets, submission history, etc.) that may influence the peer review process in many subjective ways. It is easy to envision how the current format is not the “final optimal” way of documenting research [34, 35, 94, 97], warranting constant attention and joint brainstorming among the community doing the practical work [65–67]. In addition to what we publish, there is also the consideration of *how much* we publish any of that. And there are high incentives to publish always more and more. These mounting career pressures to publish [27] also seem to affect the young academics disproportionately.

# Why Meta-Research?

As the field of Human-Computer Interaction continues to evolve, it is becoming increasingly important to thematize the growing pain points in HCI research, not only in steering and working groups, but also in rigorous scientific publications. Self-reflective [85] and meta-scientific contributions [30, 47] are critically important to advance the HCI field as a whole. Yet, little research and self-reflection is being published in HCI. It is time for the CHI community to pause and reflect on how things have progressed in the past decade and where we are headed.

Meta-research is the study of research practices [47]. Meta-research and the science of science [30] offer valuable insights into how HCI as a field can refine its methodological frameworks, enhance rigor, and address its challenges. Meta-research contributions on HCI currently have no dedicated venue at ACM conferences, and the CHI Conference currently has no fitting subcommittee for meta-scientific investigations. In the past, such investigations (e.g., [11, 26, 34, 83, 84]) have been relegated to poster contributions or adjunct proceedings (“alt.chi”) where the investigations might not get all the attention they deserve. This is suboptimal and also contrasts with the field’s critical and self-reflective tradition [85, 86, 88].
## 2 Meta-HCI Workshop

The first edition of this one-day workshop at CHI '25 in Yokohama, Japan, aims to bring together researchers, practitioners, and reviewers within the HCI community and beyond to explore and discuss meta-research. By critically evaluating the processes underlying HCI research, we can improve the ways in which studies are conducted, reported, and assessed, fostering a culture of transparency and methodological rigor. Given the cross-disciplinary nature of HCI, CHI — its flagship conference — serves as the ideal venue to discuss these topics, thanks to its diverse community in terms of background, practice, and perspectives.

This workshop will provide a forum for researchers to share, discuss, and brainstorm their ideas about improving the current state of meta-research in HCI. We aim to provide a common platform for meta-research in HCI, and to establish a research agenda for meta-scientific investigations in HCI.

### Relevance and Impact

This workshop is for anyone in the HCI community who cares about understanding, cataloging, and improving the ways we conduct research. By taking a step back and looking critically at how we design, conduct, and share our work, we can identify areas where we can improve. The goal is to create a space where we can safely and openly discuss the challenges of HCI research and develop practical ways to ensure that HCI research is as rigorous, transparent, and impactful as possible. Our workshop is intended to mark the starting point of a meta-research community in HCI that outlasts the workshop event.

The workshop will be a catalyst for change in the CHI and HCI community. The potential for meta-research to produce meaningful impact is manifold. The Meta-HCI workshop explores how the HCI field has evolved, its methodologies, and its impacts. This provides an opportunity for researchers to reflect on the direction HCI research is heading and what has worked or needs to be rethought. The workshop provides a platform for critical engagement with the field, encouraging researchers to question assumptions, frameworks, and biases in HCI research. This fosters a more mature, self-aware research community. Meta-research encourages researchers to scrutinize the relevance of the work being presented at CHI in the context of real-world HCI problems. This can help ensure that future CHI research addresses pressing societal and technological issues, making the research more impactful. By examining trends in meta-HCI, researchers can identify gaps and areas where HCI needs to evolve, creating a shared agenda for future research. This is particularly relevant in the fast-developing field of HCI which is continuously reshaped by new technologies. Meta-research is also an important source for policy and guideline development.

### Long-term objectives

Meta-HCI contributions need a dedicated venue. The long-term objective of this workshop is to offer this venue, potentially and hopefully until a dedicated track at CHI for meta-scientific contributions is created. We commit to hosting future iterations of the workshop, or finding and passing the torch to future organizers.

## 3 Workshop Structure

### 3.1 Pre-Workshop Activities

We will publicize the workshop on social media and in HCI and ACM distribution lists. We will also reach out to scholars from our network and other scholars to encourage them to submit their work. Materials and accepted submissions will be accessible to participants at least 1 week before the workshop through the workshop website as well as a private workspace we will set up in Slack. We will also prepare quantitative summaries of past CHI publication trends, citation trends, and a short presentation on how the community's publishing practices have evolved. Upon the potential acceptance of the workshop, we will find a senior academic to deliver a keynote to kick off the workshop with a brief and engaging talk on HCI meta-research. We have already secured the funds to sponsor the speaker's participation.

### 3.2 Workshop Topics

Meta-science provides a wealth of opportunities for research and discussions around conducting good research. The workshop topics include, but are by no means limited to, the topics listed in Table 1.

### 3.3 Workshop Format, Activities, and Structure

The workshop consists of two parts. The first part provides a traditional platform for participants to briefly present their contributions (submissions), followed by discussions and a collaborative decision on the topics that we address during the second part of the workshop. The second part engages in hands-on meta-research. Working in small groups, we will work on the topics identified in the first part. Some of our own ideas include activities such as:

(1) outline a research agenda for future meta-research in HCI,

(2) map existing resources we already have and will need to have for more impactful meta-research (for example, perhaps we could publish a dataset that is more accessible than the ACM DL?),

(3) seek to understand what we can learn from scientific fields that have a more established tradition of conducting meta-research, or

(4) design a preliminary survey to better understand our community's position on the topics listed in Table 1.

As a highly participatory workshop, our aim is to foster an active community of researchers interested in concretely conducting meta-research. To this end, making this community outlast the workshop is our top priority. The workshop will not focus on long presentations. We will instead focus on interaction and collaboration between participants. This workshop format allows for deep, focused conversations (with plenty of coffee) to build stronger professional networks around the topic of meta-research.

### 3.4 Expected outcomes

The outcomes of this workshop will be immediately actionable ideas for the community and input for future iterations of this workshop.

*Meta-HCI community survey.* We will design a survey questionnaire on one of the main questions that will attract the most interest during the second part of the workshop, and distribute this survey
<table><caption>Table 1: List of possible topics and examples or related literature</caption><thead><tr><th>Topic</th><th>Examples / related literature</th></tr></thead><tbody><tr><td><strong>What we study in HCI:</strong><br>Trends, topics, evolution, opportunities, and challenges in HCI</td><td>[21, 55, 61, 76, 77]</td></tr><tr><td><strong>How we conduct research in HCI:</strong><br>Meta-analysis of HCI research (e.g., WEIRD participants, method use, etc.)<br>The replication crisis in HCI and generalizability of HCI research<br>Biases in HCI research, inclusivity and intersectional inequalities in HCI research<br>Datasets and software re-use for meta-scientific HCI investigations<br>Norms, culture, behavior, practices, and motivation of HCI researchers (e.g., citations)</td><td>[19, 41, 42, 49, 52, 73, 75, 89, 90]<br>[3, 23, 46, 72]<br>[37, 54, 68]<br>[7, 14]<br>[53, 74, 82, 100]</td></tr><tr><td><strong>How we evaluate research:</strong><br>Issues and bias with peer review and AI-generated peer reviews in HCI venues<br>Issues resulting from use of generative AI in HCI studies</td><td>[2, 8, 24, 28, 36, 44, 56, 58, 59, 87, 95, 101]<br>[18, 29]</td></tr><tr><td><strong>What we publish in HCI:</strong><br>Language used in HCI studies (e.g., hedging and boosting)<br>How AI-generated writing has impact the HCI literature</td><td>[45, 45, 51, 96]<br>[1, 32, 51, 60]</td></tr><tr><td><strong>How HCI research affects us:</strong><br>Well-being of HCI researchers, hypercompetition, and hyper-prolific authors in HCI<br>How the CHI community and CHI Conference has developed over the past decade<br>Climate impact and carbon emissions of HCI research and the CHI Conference<br>The impact and implications of HCI research</td><td>[20, 27, 33]<br>[50, 74, 77]<br>[48]<br>[9, 62, 63, 92, 98]</td></tr><tr><td><strong>How contextual factors and learnings from broader science affect HCI research:</strong><br>The growth of science, such as the growth in publications, citations, and co-authorship<br>How lessons from meta-research in other disciplines (e.g. Physics) can apply to HCI<br>Issues with ACM-DL and influence of ACM on HCI research<br>Research on the opportunities and challenges of meta-research in HCI</td><td>[15, 38]<br>[12, 22, 30]<br>[17, 39, 57]<br>[43]</td></tr></tbody></table>

to the community during the conference and report its findings in an ArXiv submission and as part of the next iteration of the workshop.<fnref n="1" />

*Research agenda for meta-HCI investigations.* Based on the results of the workshop, and especially the second part of it (the group activities), the authors of this workshop will prepare a proposal for a meta-HCI research agenda for the coming years. We seek input from all workshop authors via email, and later on publish the roadmap in a suitable format online. At a minimum, it will be deployed on the workshop website and in the form of blog posts and social media posts. The most important function of this research agenda is to help steer the activities in the next *venue* of meta-HCI.

*Future Meta-HCI venue brainstorming.* Together, we will envision the description and operational model of a future HCI-focused venue for meta-research. This could be, among other options, an entire future subcommittee in CHI, or a dedicated journal for HCI meta-research, or simply ways to expand and make this workshop into a larger more impactful event in the future in some ways. At a minimum, this will include the collaborative development of the description and purpose of such a venue, including the related target audience, scope, research areas and topics, core contributions, methodological categories, example application areas, and ethical research practices.

## 3.5 Submissions, Acceptance, and Publication

We accept full papers (≥6 pages, references excluded), short papers (5–6 pages, references excluded), and position papers (2–5 pages), outlining the author’s view on the workshop theme and the reasons for the submitter’s interest in the topic. Accepted papers will be published as part of a proceedings volume with CEUR-WS. Submissions should follow the CEUR-WS formatting and licensing requirements. Specifically, submissions should be formatted according to the one-column CEURART template (see zip file with docx template and Overleaf). CEUR calculates 2,500 characters per page (≈380–400 words per page). Papers can be anonymized for single-blind review, although this is not required. All papers must be submitted as PDF documents to meta-hci@oulu.fi.

The workshop organizers will review submissions primarily based on their potential to stimulate engaging discussions. Submissions will be judged and selected against the acceptance criteria of relevance, provocativeness, novelty, and research quality.

## 3.6 Proposed Workshop Schedule

The proposed workshop schedule for the Meta-HCI workshop is as follows.

**Keynote (9:00 – 9:20):** An invited keynote speaker will discuss the state of HCI meta-research, along with key results.

**Welcome and introductions (9:20 – 9:45):** We will kick-off the event with an interactive introduction session for participants to get to talk and know each other.

<footnote n="1">¹We are currently in the process of filing IRB approval requests by anticipation, in order to be allowed to collect subjective anonymous information during the conference.</footnote>
**Lightning presentation rounds (9:45 – 10:30):** Participants will briefly present their work. Speaking time will be commensurate with submission length. We will aim not to make this a mini-conference. Instead, the focus is on getting to know the participants’ research interests in regard to meta-research.

**Coffee break (10:30 – 11:00)**

**Lightning presentation rounds (continued) (11:00 – 11:45)**

**Preparing afternoon sessions (11:45 – 12:00)** At the end of the morning sessions, we will discuss and prepare the afternoon group sessions, including group formation and topics, and an overview of the pre-distributed datasets that might help ideation.

**Lunch break (12:00 – 13:00):** We will propose to have lunch together. This will ensure continued discussions but also provide some levity and more social connection towards building a community.

**Group work sessions (13:00 – 14:30)**

**Break (14:30 – 15:00)**

**Group work sessions (15:00 – 16:30)**

**Closing (16:30 – 17:00):** We will together review and discuss the workshop outcomes and discuss next steps, future activities, events, and publications.

## 3.7 Attendance and Estimated Number of Participants

Workshop attendance is in-person only. This will avoid issues related to work-related stress [71] (e.g., the triple-peak day [69]), timezone scheduling issues [70], disconnectedness and "zone-outs" in remote meetings [16], and other issues surrounding virtual and hybrid group work. We will discuss going to lunch together at the workshop, which will facilitate further community-building.

We seek to attract anywhere between 20-30 participants (including the organizers). This will allow us to shape a comprehensive future research agenda, build collaborations, and form the foundations to an active community around meta-research in HCI. The workshop is open to a broad audience to stimulate the workshop participants by exposure to new points of views from different disciplines and methodological backgrounds. While the workshop topic will primarily be interesting for senior faculty members in HCI, we seek to attract junior faculty and PhD students as well.

## 3.8 Accessibility

The workshop does not require a special venue or hardware. Only a projector and WiFi connection are required. The workshop organizers will bring a set of materials (stationary items and grid sheets) to support group work. Participants will be encouraged to submit an accessible version of their papers. If there are additional accessibility requirements, we will work with the Accessibility Chairs to ensure a positive experience.

## 3.9 Post-Workshop Plans

We hope to build a persistent research area and community around HCI meta-research. Toward that goal, we plan to create a blog post for the general HCI audience. In addition, we want to create resources for teaching and researching in this emergent domain. Therefore, we hope to publish selected papers from the workshop in a summary article in a journal with all workshop attendees who wish to participate as co-authors.

# 4 Organizers

**Jonas Oppenlaender** is a Postdoctoral Researcher in the Center for Ubiquitous Computing at the University of Oulu, Finland. His research interests include supporting people in being more creative and applications of generative artificial intelligence for the future of work and science. He has previously served as an organizer of several workshops at CHI (REGROW '22 [4], 2VT '21 [80], DC²S² '19 [78]) and at Creativity & Cognition '19 [79]. He was co-organizer of CrowdCamp at AAAI HCOMP '23.

Website: https://www.jonaso.de

**Sylvain Malacria** is a research scientist at Inria, where he conducts his research in the Loki group. He is also an adjunct assistant professor at the University of Waterloo, Canada (2021-2024). His research interests is in the area of human-computer interaction (HCI), with additional focus on designing interactive systems and interaction techniques.

Website: https://www.malacria.com

**Xinrui Fang** is a 1st year PhD Student in the HCI field based in Tokyo, under the supervision of Prof. Koji Yatani. His passion is to combine his engineering skills and design smell to create cool stuffs. He has engineering experience and a master degree from Keio University. His research interests are human-computer and human-AI interaction and applied machine learning.

Website: https://xinrui.design

**Niels van Berkel** is an Associate Professor at Aalborg University. His work focuses on the design and evaluation of intelligent computing systems, particularly in real-world contexts, publishing in HCI, Social Computing, and Ubiquitous Computing. He has previously served as organizer of workshops at CHI (REGROW '22 [4], 2VT '21 [80], Emergent Interaction '21 [10]) and UbiComp (UbiTtention '20 [91], Mobile Human Contributions '18 [93], Sensors & Behaviour '18), and served on the editorial board for IJHCS (2019-present) and ACM TiiS Special Issue on Human-Centered Explainable AI.

Website: https://www.nielsvanberkel.com

**Fanny Chevalier** is an Associate Professor at the University of Toronto, and a Knight of the France's Order of Academic Palms. Her research focuses primarily on human-computer interaction for creativity, and data visualization. She has previously co-organized workshops at the IEEE VIS conference [6].

Website: https://fannychevalier.net

**Koji Yatani** is an Associate Researcher and the director of the Interactive Intelligent Systems Laboratory at the University of Tokyo. His current research focuses on Human-AI Interaction and human well-being support. He is serving as a Technical Program Chair for CHI 2025.

Website: https://iis-lab.org

**Simo Hosio** is a Professor and the leader of Crowd Computing Research Group at the University of Oulu, Finland. He has organized multiple workshops in CHI, Ubicomp and CSCW conferences, and
is interested in crowdsourcing and digital wellbeing.
Website: https://www.simohosio.com

# 5 Call for Participation

Human-Computer Interaction (HCI) is a rapidly evolving field, with a growing need for reflection on how research is conducted. The Meta-HCI workshop invites researchers and practitioners to discuss meta-research in HCI. Meta-research focuses on studying research practices and offers insights into how HCI can enhance its methodological frameworks, improve rigor, and address the field's growing challenges. This one-day workshop will serve as a platform for the HCI community to share thoughts and experiences with meta-research, and collectively examine the processes that shape the research in our field. The workshop themes include, but are not limited to, what we study and publish in HCI, how we conduct and evaluate research in HCI, how HCI research affects us, and how the broader context of science affects HCI research. Participants are invited to submit short papers (5–6 pages, references excluded) or full papers (min. 6 pages, references excluded) in the form of studies, experiments, bibliometric and scientometric investigations, or other meta-scientific research. Since meta-research is an emerging area in HCI, we also accept short position papers (2–5 pages, references excluded). Submissions should use the CEURART 1-column template. The workshop organizers will review submissions primarily based on their potential to stimulate engaging discussions. At least one author of each accepted paper must register for the workshop and attend the event. We look forward to your contributions and to welcoming you to an exciting and productive discussion on the future of HCI research practices! Please visit https://meta-hci.github.io to learn more about the workshop and how to participate.

# 6 Website and Important Dates

The website https://meta-hci.github.io provides further and updated information about the workshop.

Submission Deadline: March 14, 2025

Notification of acceptance: March 21, 2025

Workshop Date: April 26, 2025

# References

[1] Academ-ai.info. 2024. Suspected AI. https://www.academ-ai.info/

[2] Balazs Aczel, Barnabas Szaszi, and Alex O. Holcombe. 2021. A billion-dollar donation: Estimating the cost of researchers' time spent on peer review. *Research Integrity and Peer Review* 6 (2021), 14 pages. https://doi.org/10.1186/s41073-021-00118-2

[3] Herman Aguinis and Angelo M. Solarino. 2019. Transparency and replicability in qualitative research: The case of interviews with elite informants. *Strategic Management Journal* 40, 8 (2019), 1291–1315. https://doi.org/10.1002/smj.3015

[4] Andy Alorwu, Saiph Savage, Niels van Berkel, Dmitry Ustalov, Alexey Drutsa, Jonas Oppenlaender, Oliver Bates, Danula Hettiachchi, Ujwal Gadiraju, Jorge Goncalves, and Simo Hosio. 2022. REGROW: Reimagining Global Crowdsourcing for Better Human-AI Collaboration. In *Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22)*. Association for Computing Machinery, New York, NY, USA, Article 88, 7 pages. https://doi.org/10.1145/3491101.3503725

[5] Ian Arawjo. 2024. LLM Wrapper Papers are Hurting HCI Research. https://ianarawjo medium.com/llmwrapperpapers-are-hurting-hci-research-8ad416a5d59a

[6] Benjamin Bach, Fanny Chevalier, Helen-Nicole Kostis, Mark Subbaro, Yvonne Jansen, and Robert Soden. 2024. IEEE VIS Workshop on Visualization for Climate Action and Sustainability. arXiv:2404.02743 https://arxiv.org/abs/2404.02743

[7] Maria Teresa Baldassarre, Neil Ernst, Ben Hermann, Tim Menzies, and Rahul Yedida. 2023. (Re)Use of Research Results (Is Rampant). *Commun. ACM* 66, 2

(Jan. 2023), 75–81. https://doi.org/10.1145/3554976

[8] Eva Barlösius, Laura Paruschke, and Axel Philipps. 2023. Peer review's irremediable flaws: Scientists' perspectives on grant evaluation in Germany. *Research Evaluation* 32, 4 (Oct 2023), 623–634. https://doi.org/10.1093/reseval/rvad032

[9] Christoph Bartneck and Jun Hu. 2009. Scientometric Analysis of the CHI Proceedings. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09)*. Association for Computing Machinery, New York, NY, USA, 699–708. https://doi.org/10.1145/1518701.1518810

[10] Dan Bennett, Alan Dix, Parisa Eslambolchilar, Feng Feng, Tom Froese, Vassilis Kostakos, Sebastien Lerique, and Niels van Berkel. 2021. Emergent Interaction: Complexity, Dynamics, and Enaction in HCI. In *Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21)*. Association for Computing Machinery, New York, NY, USA, Article 119, 7 pages. https://doi.org/10.1145/3411763.3441321

[11] Lonni Besançon and Pierre Dragicevic. 2019. The Continued Prevalence of Dichotomous Inferences at CHI. In *Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19)*. Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290607.3310432

[12] Nicholas Bloom, Charles I. Jones, John Van Reenen, and Michael Webb. 2020. Are Ideas Getting Harder to Find? *American Economic Review* 110, 4 (Apr 2020), 1104–44. https://doi.org/10.1257/aer.20180338

[13] Susanne Bødker. 2015. Third-wave HCI, 10 years later—participation and sharing. *Interactions* 22, 5 (Aug 2015), 24–31. https://doi.org/10.1145/2804405

[14] Toine Bogers, Maria Gäde, Mark Michael Hall, Marijn Koolen, Vivien Petras, and Birger Larsen. 2023. How we Work, Share, and Re-use at CHIIR. In *Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (CHIIR '23)*. Association for Computing Machinery, New York, NY, USA, 351–356. https://doi.org/10.1145/3576840.3578305

[15] Lutz Bornmann, Robin Haunschild, and Rüdiger Mutz. 2021. Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases. *Humanities and Social Sciences Communications* 8, 1 (2021), 15 pages. https://doi.org/10.1057/s41599-021-00903-w

[16] Nigel Bosch and Sidney K. D'Mello. 2022. Can Computers Outperform Humans in Detecting User Zone-Outs? Implications for Intelligent Interfaces. *ACM Trans. Comput.-Hum. Interact.* 29, 2, Article 10 (Jan. 2022), 33 pages. https://doi.org/10.1145/3481889

[17] William J. Bowman. 2024. ACM Profits Considered Harmful. *Commun. ACM* 67, 9 (Aug 2024), 40–42. https://doi.org/10.1145/3663958

[18] Guillaume Cabanac, Cyril Labbé, and Alexander Magazinov. 2021. Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals. arXiv:2107.06751 https://arxiv.org/abs/2107.06751

[19] Kelly Caine. 2016. Local Standards for Sample Size at CHI. In *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16)*. Association for Computing Machinery, New York, NY, USA, 981–992. https://doi.org/10.1145/2858036.2858498

[20] Lydia Carson, Christoph Bartneck, and Kevin Voges. 2013. Over-Competitiveness in Academia: A Literature Review. *Disruptive Science and Technology* 1, 4 (2013), 183–190. https://doi.org/10.1089/dst.2013.0013

[21] Mark Chignell, Lu Wang, Atefeh Zare, and Jamy Li. 2023. The Evolution of HCI and Human Factors: Integrating Human and Artificial Intelligence. *ACM Trans. Comput.-Hum. Interact.* 30, 2, Article 17 (March 2023), 30 pages. https://doi.org/10.1145/3557891

[22] Johan S. G. Chu and James A. Evans. 2021. Slowed canonical progress in large fields of science. *Proceedings of the National Academy of Sciences* 118, 41 (2021), e2021636118. https://doi.org/10.1073/pnas.2021636118

[23] Andy Cockburn, Pierre Dragicevic, Lonni Besançon, and Carl Gutwin. 2020. Threats of a replication crisis in empirical computer science. *Commun. ACM* 63, 8 (2020), 70–79. https://doi.org/10.1145/3360311

[24] Corinna Cortes and Neil D. Lawrence. 2021. Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment. arXiv:2109.09774

[25] David Crotty. 2020. Revisiting: The Problem(s) With Credit for Peer Review. https://scholarlykitchen.sspnet.org/2020/08/17/revisiting-the-problems-with-credit-for-peer-review-2/

[26] Pierre Dragicevic, Fanny Chevalier, and Stephane Huot. 2014. Running an HCI experiment in multiple parallel universes. In *CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14)*. Association for Computing Machinery, New York, NY, USA, 607–618. https://doi.org/10.1145/2559206.2578881

[27] Marc A. Edwards and Siddhartha Roy. 2017. Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. *Environmental Engineering Science* 34, 1 (2017), 51–61. https://doi.org/10.1089/ees.2016.0223

[28] Michael B Eisen, Anna Akhmanova, Timothy E Behrens, Jörn Diedrichsen, Diane M Harper, Mihaela D Iordanova, Detlef Weigel, and Mone Zaidi. 2022. Scientific Publishing: Peer review without gatekeeping. *eLife* 11 (Oct 2022), e83889. https://doi.org/10.7554/eLife.83889
[29] Holly Else and Richard Van Noorden. 2021. The Battle Against Paper Mills. *Nature* 591, 7851 (2021), 516–519. https://doi.org/10.1038/d41586-021-00733-5

[30] Santo Fortunato, Carl T. Bergstrom, Katy Börner, James A. Evans, Dirk Helbing, Staša Milojević, Alexander M. Petersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, Alessandro Vespignani, Ludo Waltman, Dashun Wang, and Albert-László Barabási. 2018. Science of science. *Science* 359, 6379 (2018), eaa00185. https://doi.org/10.1126/science.aao0185

[31] Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. 2024. Scaling Synthetic Data Creation with 1,000,000,000 Personas. arXiv:2406.20094 https://arxiv.org/abs/2406.20094

[32] Andrew Gray. 2024. ChatGPT “contamination”: Estimating the prevalence of LLMs in the scholarly literature. arXiv:2403.16887 https://arxiv.org/abs/2403.16887

[33] Kevin Gross and Carl T. Bergstrom. 2019. Contest models highlight inherent inefficiencies of scientific funding competitions. *PLOS Biology* 17, 1 (Jan 2019), 1–15. https://doi.org/10.1371/journal.pbio.3000065

[34] Tovi Grossman, Fanny Chevalier, and Rubaiat Habib Kazi. 2015. Your Paper is Dead! Bringing Life to Research Articles with Animated Figures. In *Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15)*. Association for Computing Machinery, New York, NY, USA, 461–475. https://doi.org/10.1145/2702613.2732501

[35] Tovi Grossman, Fanny Chevalier, and Rubaiat Habib Kazi. 2016. Bringing research articles to life with animated figures. *Interactions* 23, 4 (Jun 2016), 52–57. https://doi.org/10.1145/2949762

[36] Yanzhu Guo, Guokan Shang, Virgile Rennard, Michalis Vazirgiannis, and Chloé Clavel. 2023. Automatic Analysis of Substantiation in Scientific Peer Reviews. In *Findings of the Association for Computational Linguistics: EMNLP 2023*, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 10198–10216. https://doi.org/10.18653/v1/2023.findings-emnlp2023.684

[37] Aniko Hannak, Kenneth Joseph, Andrei Cimpian, and Daniel B. Larremore. 2020. Explaining Gender Differences in Academics’ Career Trajectories. arXiv:2009.10830 https://arxiv.org/abs/2009.10830

[38] Mark A. Hanson, Pablo Gómez Barreiro, Paolo Crosetto, and Dan Brockington. 2024. The strain on scientific publishing. *Quantitative Science Studies* (Sept 2024), 1–29. https://doi.org/10.1162/qss_a_00327

[39] Vicki L. Hanson and Patricia Ryan. 2024. ACM’s Reserves Enable Its Mission. *Commun. ACM* 67, 9 (Aug 2024), 43–46. https://doi.org/10.1145/3673863

[40] S. Harrison, D. Tatar, and P. Sengers. 2007. The three paradigms of HCI. In *Alt. Chi. Session at the SIGCHI Conference on Human Factors in Computing Systems*. ACM, 1–18.

[41] Joseph Henrich, Steven J. Heine, and Ara Norenzayan. 2010. Most people are not WEIRD. *Nature* 466, 29 (2010). https://doi.org/10.1038/466029a

[42] Kasper Hornbæk, Aske Mottelson, Jarrod Knibbe, and Daniel Vogel. 2019. What Do We Mean by “Interaction”? An Analysis of 35 Years of CHI. *ACM Trans. Comput.-Hum. Interact.* 26, 4, Article 27 (Jul 2019), 30 pages. https://doi.org/10.1145/3325285

[43] Kasper Hornbæk. 2006. Current practice in measuring usability: Challenges to usability studies and research. *International Journal of Human-Computer Studies* 64, 2 (2006), 79–102. https://doi.org/10.1016/j.ijhcs.2005.06.002

[44] Jürgen Huber, Sabiou Inoua, Rudolf Kerschbamer, Christian König-Kersting, Stefan Palan, and Vernon L. Smith. 2022. Nobel and novice: Author prominence affects peer review. *Proceedings of the National Academy of Sciences* 119, 41 (2022), e2205779119. https://doi.org/10.1073/pnas.2205779119

[45] Ken Hyland. 1996. Writing Without Conviction? Hedging in Science Research Articles. *Applied Linguistics* 17, 4 (Dec 1996), 433–454. https://doi.org/10.1093/applin/17.4.433

[46] John P. A. Ioannidis. 2005. Why Most Published Research Findings Are False. *PLoS Med* 2, 8, Article e124 (2005). https://doi.org/10.1371/journal.pmed.0002012

[47] John P. A. Ioannidis, Daniele Fanelli, Debbie Drake Dunne, and Steven N. Goodman. 2015. Meta-research: Evaluation and Improvement of Research Methods and Practices. *PLOS Biology* 13, 10 (Oct 2015), 1–7. https://doi.org/10.1371/journal.pbio.1002264

[48] Jason T. Jacques. 2020. CHI 2020: Right Here, Right Now? A bottom-up approach to estimating the carbon emissions from more than twenty years of CHI conference travel. In *Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA ’20)*. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3334480.3381806

[49] Annika Kaltenhauser, Evropi Stefanidi, and Johannes Schöning. 2024. Playing with Perspectives and Unveiling the Autoethnographic Kaleidoscope in HCI – A Literature Review of Autoethnographies. In *Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24)*. Association for Computing Machinery, New York, NY, USA, Article 819, 20 pages. https://doi.org/10.1145/3613904.3642355

[50] Joseph 'Jofish' Kaye. 2009. Some Statistical Analyses of CHI. In *CHI '09 Extended Abstracts on Human Factors in Computing Systems (CHI EA '09)*. Association for Computing Machinery, New York, NY, USA, 2585–2594. https://doi.org/10.1145/1520340.1520364

[51] Dmitry Kobak, Rita González-Márquez, Emóke Ágnes Horvát, and Jan Lause. 2024. Delving into ChatGPT usage in academic writing through excess vocabulary. arXiv:2406.07016 https://arxiv.org/abs/2406.07016

[52] Lisa Koeman. 2018. How many participants do researchers recruit: A look at 678 UX/HCI studies. https://lisakoeman.nl/blog/how-many-participants-do-researchers-recruit-a-look-at-678-ux-hci-studies/

[53] Sadamori Kojaku, Giacomo Livan, and Naoki Masuda. 2021. Detecting anomalous citation groups in journal networks. *Scientific Reports* 11 (2021), 14524. https://doi.org/10.1038/s41598-021-93572-3

[54] Diego Kozlowski, Thema Monroe-White, Vincent Larivière, and Cassidy R. Sugimoto. 2024. The Howard-Harvard effect: Institutional reproduction of intersectional inequalities. *Journal of the Association for Information Science and Technology* 75, 8 (2024), 869–882. https://doi.org/10.1002/asi.24931

[55] Clement Lee, Andrew Garbett, Junyan Wang, Bingzhang Hu, and Dan Jackson. 2019. Weaving the Topics of CHI: Using Citation Network Analysis to Explore Emerging Trends. In *Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA ’19)*. Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312776

[56] Carole J. Lee, Cassidy R. Sugimoto, Guo Zhang, and Blaise Cronin. 2013. Bias in peer review. *Journal of the American Society for Information Science and Technology* 64, 1 (2013), 2–17. https://doi.org/10.1002/asi.22784

[57] Shaobo Liang, Daqing He, Dan Wu, and Haoge Hu. 2020. Challenges and Opportunities of ACM Digital Library: A Preliminary Survey on Different Users. In *Sustainable Digital Communities*, Anneli Sundqvist, Gerd Berget, Jan Nolin, and Kjell Ivar Skjeringstad (Eds.). Springer International Publishing, Cham, 278–287. https://doi.org/10.1007/978-3-030-43687-2_22

[58] Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuan-dong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel A. McFarland, and James Y. Zou. 2024. Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews. arXiv:2403.07183 https://arxiv.org/abs/2403.07183

[59] Michael L. Littman. 2021. Collusion rings threaten the integrity of computer science research. *Commun. ACM* 64, 6 (May 2021), 43–44. https://doi.org/10.1145/3429776

[60] Jialin Liu and Yi Bu. 2024. Towards the relationship between AIGC in manuscript writing and author profiles: evidence from preprints in LLMs. arXiv:2404.15799 https://arxiv.org/abs/2404.15799

[61] Yong Liu, Jorge Goncalves, Denzil Ferreira, Bei Xiao, Simo Hosio, and Vassilis Kostakos. 2014. CHI 1994-2013: Mapping Two Decades of Intellectual Progress through Co-Word Analysis. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14)*. Association for Computing Machinery, New York, NY, USA, 3553–3562. https://doi.org/10.1145/2556288.2556969

[62] Nabeil Maflahi and Mike Thelwall. 2021. Domestic researchers with longer careers generate higher average citation impact but it does not increase over time. *Quantitative Science Studies* 2, 2 (Jul 2021), 560–587. https://doi.org/10.1162/qss_a_00132

[63] Dmitry Malkov, Ohid Yaqub, and Josh Siepel. 2023. The spread of retracted research into policy literature. *Quantitative Science Studies* 4, 1 (Mar 2023), 68–90. https://doi.org/10.1162/qss_a_00243

[64] Regan Mandryk. 2024. Working Group on Peer Review of Full Papers at CHI: Call for Volunteers. https://medium.com/sigchi/working-group-on-peer-review-of-full-papers-at-chi-call-for-volunteers-b831f48fad5c

[65] Damien Masson, Sylvain Malacria, Géry Casiez, and Daniel Vogel. 2023. Character: Interactive Generation of Charts for Realtime Annotation of Data-Rich Paragraphs. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23)*. Association for Computing Machinery, New York, NY, USA, Article 146, 18 pages. https://doi.org/10.1145/3544548.3581091

[66] Damien Masson, Sylvain Malacria, Géry Casiez, and Daniel Vogel. 2023. Statistician: Interactive Translation of NHST and Estimation Statistics Reporting Styles in Scientific Documents. In *Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23)*. Association for Computing Machinery, New York, NY, USA, Article 91, 14 pages. https://doi.org/10.1145/3586183.3606762

[67] Damien Masson, Sylvain Malacria, Edward Lank, and Géry Casiez. 2020. Chameleon: Bringing Interactivity to Static Digital Documents. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20)*. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376559

[68] Robert K. Merton. 1968. The Matthew Effect in Science. *Science* 159, 3810 (1968), 56–63. https://doi.org/10.1126/science.159.3810.56

[69] Microsoft. 2021. The Rise of the Triple Peak Day. https://www.microsoft.com/en-us/worklab/triple-peak-day

[70] Lillio Mok, Lu Sun, Shilad Sen, and Bahareh Sarrafzadeh. 2023. Challenging but Connective: Large-Scale Characteristics of Synchronous Collaboration Across Time Zones. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23)*. Association for Computing Machinery, New York, NY, USA, Article 611, 17 pages. https://doi.org/10.1145/3544548.3581141
[71] Mehrab Bin Morshed, Javier Hernandez, Daniel McDuff, Jina Suh, Esther Howe, Kael Rowan, Marah Abdin, Gonzalo Ramos, Tracy Tran, and Mary Czerwinski. 2022. Advancing the Understanding and Measurement of Workplace Stress in Remote Information Workers from Passive Sensors and Behavioral Data. In *2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)*. 1–8. https://doi.org/10.1109/ACII55700.2022.9953824

[72] Kevin J. Mullinix, Thomas J. Leeper, James N. Druckman, and Jeremy Freese. 2015. The Generalizability of Survey Experiments. *Journal of Experimental Political Science* 2, 2 (2015), 109–138. https://doi.org/10.1017/XPS.2015.19

[73] Natalia Obukhova. 2021. A Meta-Analysis of Effect Sizes of CHI Typing Experiments. In *Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21)*. Association for Computing Machinery, New York, NY, USA, Article 476, 7 pages. https://doi.org/10.1145/3411763.3451520

[74] Jonas Oppenlaender. 2024. Past, Present, and Future of Citation Practices in HCI. https://doi.org/10.48550/arXiv.2405.16526

[75] Jonas Oppenlaender, Tahir Abbas, and Ujwal Gadiraju. 2024. The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines. *Proc. ACM Hum.-Comput. Interact.* 8, CSCW1, Article 184 (Apr 2024), 45 pages. https://doi.org/10.1145/3641023

[76] Jonas Oppenlaender and Simo Hosio. 2025. Keeping Score: A Quantitative Analysis of How the CHI Community Appreciates Its Milestones. https://doi.org/10.48550/arXiv.2501.02456

[77] Jonas Oppenlaender and Joonas Hämäläinen. 2023. Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT for Mining Insights at Scale. https://doi.org/10.48550/arXiv.2306.05036

[78] Jonas Oppenlaender, Maximilian Mackeprang, Abderrahmane Khati, Maja Vuković, Jorge Goncalves, and Simo Hosio. 2019. DC2S2: Designing Crowd-powered Creativity Support Systems. In *Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19)*. Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3290607.3299027

[79] Jonas Oppenlaender, Naghi Shireen, Maximilian Mackeprang, Hali Erhan, Jorge Goncalves, and Simo Hosio. 2019. Crowd-powered Interfaces for Creative Design Thinking. In *Proceedings of the 2019 ACM Conference on Creativity and Cognition (C&C '19)*. ACM, New York, NY, USA, 722–729. https://doi.org/10.1145/3325480.3326553

[80] Ville Paananen, Piia Markkanen, Jonas Oppenlaender, Lik Hang Lee, Haider Akmal, Ava Fatah gen. Schieck, John Dunham, Konstantinos Papangelis, Nicolas Lalone, Niels van Berkel, Jorge Goncalves, and Simo Hosio. 2021. 2VT: Visions, Technologies, and Visions of Technologies for Understanding Human Scale Spaces. In *Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21)*. Association for Computing Machinery, New York, NY, USA, Article 104, 5 pages. https://doi.org/10.1145/3411763.3441315

[81] Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. In *Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23)*. Association for Computing Machinery, New York, NY, USA, Article 2, 22 pages. https://doi.org/10.1145/3586183.3606763

[82] Victor Le Pochat. 2024. Reflecting on Research Practices. Commun. ACM 67, 5 (May 2024), 37–39. https://doi.org/10.1145/3651965

[83] Henning Pohl and Aske Mottelson. 2019. How we Guide, Write, and Cite at CHI. In *Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19)*. Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290607.3310429

[84] Raquel Breejon Robinson, Alberto Alvarez, and Elisa D. Mekler. 2024. How to write a CHI paper (asking for a friend). In *Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems (CHI EA '24)*. Association for Computing Machinery, New York, NY, USA, Article 558, 8 pages. https://doi.org/10.1145/3613905.3644051

[85] Donald A. Schön. 1992. *The Reflective Practitioner: How Professionals Think in Action*. Routledge. https://doi.org/10.4324/9781315237473

[86] Phoebe Sengers, Kirsten Boehner, Shay David, and Joseph 'Jofish' Kaye. 2005. Reflective design. In *Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility (CC '05)*. Association for Computing

Machinery, New York, NY, USA, 49–58. https://doi.org/10.1145/1094562.1094569

[87] Nihar B. Shah. 2022. Challenges, experiments, and computational solutions in peer review. Commun. ACM 65, 6 (2022), 76–87. https://doi.org/10.1145/3528086

[88] Jessica Soedirgo and Aarie Glas. 2020. Toward Active Reflexivity: Positionality and Practice in the Production of Knowledge. PS: Political Science & Politics 53, 3 (2020), 527–531. https://doi.org/10.1017/S1049096519002233

[89] Evropi Stefanidi, Marit Bentvelzen, Pawel W. Woźniak, Thomas Kosch, Mikolaj P. Woźniak, Thomas Mildner, Stefan Schneegass, Heiko Müller, and Jasmin Niess. 2023. Literature Reviews in HCI: A Review of Reviews. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)*. Association for Computing Machinery, New York, NY, USA, Article 509, 24 pages. https://doi.org/10.1145/3544548.3581332

[90] Christian Sturm, Alice Oh, Sebastian Linxen, Jose Abdelnour Nocera, Susan Dray, and Katharina Reinecke. 2015. How WEIRD is HCI? Extending HCI Principles to other Countries and Cultures. In *Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15)*. Association for Computing Machinery, New York, NY, USA, 2425–2428. https://doi.org/10.1145/2702613.2702656

[91] Niels van Berkel, Anja Exler, Martin Gjoreski, Tine Kolenik, Tadashi Okoshi, Veljko Pejovic, Aku Visuri, and Alexandra Voit. 2020. UbiTtention 2020: 5th International Workshop on Smart & Ambient Notification and Attention Management. In *Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers (UbiComp/ISWC '20 Adjunct)*. Association for Computing Machinery, New York, NY, USA, 431–435. https://doi.org/10.1145/3410530.3414616

[92] Niels van Berkel and Kasper Hornbæk. 2023. Implications of Human-Computer Interaction Research. *Interactions* 30, 4 (Jun 2023), 50–55. https://doi.org/10.1145/3600103

[93] Niels van Berkel, Simo Hosio, Jorge Goncalves, Katarzyna Wac, Vassilis Kostakos, and Anna Cox. 2018. MHC '18: International Workshop on Mobile Human Contributions: Opportunities and Challenges. In *Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers (UbiComp '18)*. Association for Computing Machinery, New York, NY, USA, 590–593. https://doi.org/10.1145/3267305.3274134

[94] Karen van Godtsenhoven, Mikael Karstensen Elbæk, Gert Schmeltz Pedersen, Barbara Sierman, Magchiel Bijsterbosch, Patrick Hochstenbach, Rosemary Russell, and Maurice Vanderfeesten. 2009. *Emerging Standards for Enhanced Publications and Repository Technology Survey on Technology*. Amsterdam University Press.

[95] Philippe Vincent-Lamarre and Vincent Larivière. 2021. Textual analysis of artificial intelligence manuscripts reveals features associated with peer review outcome. *Quantitative Science Studies* 2, 2 (Jul 2021), 662–677. https://doi.org/10.1162/qqs_a_00125

[96] Christiaan H Vinkers, Joeri K Tijdink, and Willem M Otte. 2015. Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. *BMJ* 351 (2015). https://doi.org/10.1136/bmj.h6467

[97] Niels-Oliver Walkowski. 2019. *Beyond the Flow: Scholarly Publications During and After the Digital*. Meson Press.

[98] Dashun Wang, Chaoming Song, and Albert-László Barabási. 2013. Quantifying Long-Term Scientific Impact. Science 342, 6154 (2013), 127–132. https://doi.org/10.1126/science.1237825

[99] Zhilin Wang, Yu Ying Chiu, and Yu Cheung Chiu. 2023. Humanoid Agents: Platform for Simulating Human-like Generative Agents. In *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*, Yansong Feng and Els Lefever (Eds.). Association for Computational Linguistics, Singapore, 167–176. https://doi.org/10.18653/v1/2023.emnlp-demo.15

[100] Seokkyun Woo and John P. Walsh. 2024. On the shoulders of fallen giants: What do references to retracted research tell us about citation behaviors? *Quantitative Science Studies* 5, 1 (Mar 2024), 1–30. https://doi.org/10.1162/qss_a_00303

[101] Wenqing Wu, Haixu Xi, and Chengzhi Zhang. 2024. Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI. *Scientometrics* 129 (2024), 4109–4135. https://doi.org/10.1007/s11192-024-05070-8