# “Should I choose a smaller model?”: Understanding ML Model Selection and Its Impact on Sustainability

Eya Ben chaaben

eya.ben-chaaben@inria.fr

Université Paris-Saclay, CNRS, Inria

ExSitu, LISN

Paris, France

Janin Koch

janin.koch@inria.fr

Univ. Lille, Inria, CNRS, Centrale Lille

UMR 9189 CRISTAL

Lille, France

Wendy E. Mackay

mackay@lri.fr

Université Paris-Saclay, CNRS, Inria

ExSitu, LISN

Paris, France

## ABSTRACT

The increasing accessibility of large machine learning (ML) models has resulted in their widespread adoption in everyday products, with a correspondingly negative environmental impact. Selecting more suitable ML models could not only improve training time and achievable accuracy, but also long-term sustainability. However, ML developers' model selection process remains underexplored, especially with respect to sustainability trade-offs. Interviews with 13 ML developers showed that participants select models mainly based on familiarity, accuracy and interpretability, but often overlook sustainability. We asked them to critically reflect on the current trends of large models and the lack of available information regarding model sustainability. We present implications for the ML and HCI communities, emphasizing the importance of critical reflection on model selection in education and practice. Based on our findings, we provide initial recommendations for promoting model sustainability evaluation and how the HCI community can assist in making sustainable model alternatives more accessible.

## CCS CONCEPTS

* **Social and professional topics → Sustainability; Human-centered computing → HCI theory, concepts and models.**

## KEYWORDS

Machine Learning Practice, Model Selection, Sustainable HCI

**ACM Reference Format:**

Eya Ben chaaben, Janin Koch, and Wendy E. Mackay. 2025. "Should I choose a smaller model?": Understanding ML Model Selection and Its Impact on Sustainability. In *CHI Conference on Human Factors in Computing Systems (CHI '25), April 26-May 1, 2025, Yokohama, Japan*. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3706598.3713240

# 1 INTRODUCTION

Rapid advances in machine learning (ML) have enabled the development of numerous powerful interactive applications. Large language models (LLM) such as ChatGPT have attracted significant attention, doubling weekly users to over 200 million in less than a year [3]. Public release of models such as LLama3 and API access to ChatGPT has led to widespread adoption of these tools for both research and everyday applications.

<figure><img src="image_1.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License.
CHI '25, April 26-May 1, 2025, Yokohama, Japan
© 2025 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-1394-1/25/04
https://doi.org/10.1145/3706598.3713240

The field of Human-Computer Interaction (HCI) has adopted many of these models, creating a surge of advanced interactive tools, with a corresponding rise in research on human-artificial intelligence (AI) collaboration. However, these developments come at an increasing environmental cost [32]. The HCI and ML communities have been studying sustainability for over two decades [27, 48], but today's increased complexity and commercial use of ML models has increased awareness of this pressing issue. Research on sustainability emphasizes the need to ensure that progress is sustainable by meeting the requirements of the current generation without compromising future generations' ability to meet their needs. The goal is to promote equity, preserve human rights, develop long-term economic well-being, and protect the natural environment through effective practices and policies [15]. A common argument is that ML applications can be a double-edged sword in this context, with the capacity to address sustainability challenges while at the same time contributing to the problem [53]. For example, Deep Learning, Generative AI, and similar technologies consume ever greater amounts of data storage, computation time, and environmental impact while an increasing amount of ML applications also supporting crisis management in the energy [38], water [7] and ecological [49] domains.

We need more research to better understand the sustainability trade-offs of ML models to improve rather than worsen these serious concerns. The HCI community has considered various approaches to addressing sustainability, from improving individual behavior [33], developing persuasive technologies [18] to suggesting a more "Green Policy informatics", which seeks to leverage a more traditional HCI skillset to address sustainability issues [10].

At the same time, the majority of work on sustainable ML and AI focuses primarily on individual algorithms, including how to improve data collection, power sources, and infrastructures [60], as well as how to quantify and lower the carbon footprint [53] associated with developing and fine-tuning an algorithm [35].

However, optimizing individual algorithms is clearly inadequate in the face of the widespread adoption of ML in everyday applications such as chatbots, wearables, and personal health apps. Research shows that the inference costs of large ML models, i.e. the everyday use of LLMs or Diffusion models, quickly overgrow their training costs, thus shifting the sustainability costs further down the machine learning lifecycle [39]. This results in an expected doubling of the energy consumption of data centers, AI, and cryptocurrency by 2026, which would result in an energy consumption equivalent to the energy consumed by all of Japan today [30]. We argue that a transition towards a more holistic view of the use of ML and AI is necessary. This includes a deeper reflection on the
ML models' requirements concerning the application domain, user needs, and the impact on the entire ML life-cycle and sustainability in general.

The initial problem definition and model selection in the ML life-cycle plays a critical role here. During this early phase, ML developers investigate the characteristics of the problem, consider several model alternatives, and establish prioritizing criteria such as interpretability and processing speed. The relevance of sustainability in this model trade-off evaluation, as well as how to enhance the reflection on alternatives in terms of sustainability and long-term environmental effects for ML model application, are still unclear.

We interviewed 13 ML developers to better understand their reflection process when selecting ML models and their awareness and considerations regarding ML sustainability. Our results provide first 1) insights into current ML practice regarding ML model trade-off and sustainability considerations; and 2) recommendations for supporting sustainability consideration in HCI and ML practice.

This paper briefly introduces work on enabling ML developers to assess ML model alternatives and sustainable ML and HCI research on the ML life-cycle. We then present insights from our interviews and engage in a discussion regarding the necessary change to increase the long-term sustainability of ML. We conclude with implications for HCI and ML to support more sustainable decision-making when developing or applying ML in practice.

# 2 RELATED WORK

We briefly introduce existing work on ML model selection as well as selected recent work on improving the sustainability impact of ML models discussed in the ML and HCI literature.

## 2.1 Understanding ML Model selection practice

Model selection is the first phase of the ML life-cycle in which developers translate user and model goals into ML model requirements. It is critical for the ML application as a well-chosen model to ensure that the ML application meets user expectations effectively and efficiently, setting a foundation for the overall success of the project. A sub-optimal model choice can result in unnecessary training, noisy findings, misleading conclusions, or poor performance [17]. In fact, the selection process profoundly affects the features available in the system [58], as well as its maintenance and re-usability [17].

The decision of what constitutes a suitable model is often made by the developer and relies on a variety of factors. Some developers prioritize models that have demonstrated good performance and strong outputs [59] using metrics such as accuracy and loss [43] to measure their effectiveness, depending on the specific task [28]. For others a good model is interpretable and explainable [52], one that uses less energy [42] or better fits the available data. Ultimately, the majority believe that the model's fit for the task at hand is crucial but the final decision always depends on the developer's needs, knowledge, and preferences [16]. Some researchers argue that ML model selection is ultimately exploratory and experimental rather than confirmatory [17], which means without experimental validation, developers cannot be sure a ML model is the most suitable for their goal. It often involves trial and error with training several models using different sets of hyper-parameter values on different

data samples to determine the best one [11] rather than confirming the decisions with full confidence from the beginning.

One critical aspect is the adequacy of data relative to the chosen model. The lack of high-quality features can significantly reduce a model's effectiveness, resulting in poor performance regardless of the model's capabilities [58]. To address this, many researchers strongly emphasize the importance of evaluating models based on robust performance metrics, selecting those that demonstrate optimal accuracy, precision, and generalization to unseen data [28] often resulting in more complex models than necessary. Beyond performance metrics, the computational cost associated with running and training models has become a significant criterion more recently, particularly in the context of energy efficiency. This has led some practitioners to prefer models that, while potentially less complex, achieve a balance between performance and computational needs [42]. The increase of Human-AI interaction introduced consideration that ML models should not just be efficient and high-performing, but also interpretable and explainable [52]. Explainability is increasingly recognized as essential, especially in high-risk domains where understanding the decision-making process of a model and its results is crucial [14].

Researchers have explored a variety of strategies to support model selection ranging from traditional statistical methods [4] to more advanced techniques like automated machine learning (AutoML) [5]. AutoML, a system that automates the selection and optimization of ML algorithms, has transformed model selection and hyper-parameter tuning, allowing these tasks to be automated without direct human oversight [54]. While AutoML offers efficiency and can simplify the model selection process, it has sparked debate among researchers as it frequently ignores the understanding and intuition that humans offer to the process often resulting in rather standard and too complex model suggestions. They suggest a "human-in-the-loop" approach where interactive, iterative model selection benefits from human prioritizing and context knowledge about the application domain and end users could improve model results [37].

There is a significant gap in understanding how ML experts resolve trade-offs including model accuracy against interpretability, balancing computational efficiency and resource consumption, and how well a model fits with the application, requirements, and limits. Understanding the role of human knowledge about context, needs, and preferences in the model selection procedure can, however, help to develop better model selection approaches that take advantage of this expertise.

## 2.2 Role of models selection in sustainable ML

While ML researchers have discussed related sustainability issues for several decades [6, 31], the increase of complex ML systems used daily has created awareness and concern within and outside the community [48].

Recent work defines sustainable ML as "a movement to foster change in the entire life-cycle of AI products (e.g. idea generation, training, re-tuning, implementation, governance) towards greater ecological integrity and social justice" [56] It represents a combination of its environmental footprint combined with ethical concerns [21], model reliability, and considerations of longevity [53].
This includes reducing model sizes [41], optimized use of hardware, as well as improvements in data protection and accessibility to maintain privacy and security while decreasing energy consumption [29].

Most work in the sustainable ML community focuses on the optimization of resource-intensive activities [6, 53] and the development of new models that consume less energy during training while achieving equivalent results [51]. It includes addressing trade-offs between accuracy and energy consumption in ML proposing novel sustainability indicator that allows assessing the feasibility of ML models early on [45], other showing how known approaches like transfer learning can be used to reduce overall energy consumption with comparable results [57].

Research shows that the inference costs of large ML models, e.g. ChatGPT and Dall•E, can quickly outgrow their training costs shifting sustainability costs further down the ML life cycle. A recent 2024 study that evaluated the impact of ML model type on energy consumption concluded that "using multi-purpose models for discriminative tasks is more energy-intensive compared to task-specific models for these same tasks" [39]. The authors highlight how this contradicts the current trend of using complex, readily available multi-purpose models such as ChatGPT as a one-for-all model and warn about the potential consequences of such trends. At the same time, a recent comparative analysis started quantifying the impact of LLMs on sustainability compared to human consumption. The authors highlight that the environmental and economic impacts of using LLMs to create content of a set text length could outperform human labor in terms of energy consumption, water usage, carbon emissions, and economic costs [46]. However, these estimates do not consider the cost of prompt engineering and contextualization, nor the need, relevance, or usefulness of a multi-purpose model for the results. This trend of using complex models leads to worrying forecasts of energy consumption of data centers, AI, and cryptocurrencies by the International Energy Agency, which implies that within three years consumption might exceed energy demands similar to Japan today [30].

To quantify this trend and help to increase awareness, recent tools such as CodeCarbon [22], a lightweight software package, calculate the carbon dioxide (CO2) emissions caused by the cloud or personal computing resources needed to run code. Also, Tracarbon[55], a Python library, and Green Algorithms [36] measure device energy use and calculate carbon emissions to enable users to estimate and report the carbon footprint.

A similar effort by Green Web Foundation [1] not only measures emissions and provides resources to identify greener hosting providers that rely on renewable energy, but also provides frameworks and data to articulate the importance of sustainable internet practices to movement leaders and policymakers. Their initiatives contribute to shaping the environmental impact of digital services in a way that emphasizes the need for systemic changes, such as encouraging data centers to adopt renewable energy sources and promoting digital equity.

Despite continuous progress, rarely ML developers consider sustainability impacts when selecting a model, nor do they consider how their model choices affect the overall ML life cycle.

## 2.3 HCI perspectives on ML model selection

Sustainable HCI began over two decades ago [27] and aims to make the interaction process between humans and machines less taxing and more efficient [34]. It emphasizes the sustainability of the computational process itself when real-time feedback is involved [13] and the multidimensional impact of HCI and its environmental, economic, and societal implications [20].

Various approaches have been proposed address sustainability, beginning with working on individual behavior [33], applying persuasive technologies [18], to a more general 'Green Policy informatics' aiming to leverage a more traditional HCI skillset in addressing sustainability issues [10]. Recent work also sought to increase awareness and provide guidance towards more sustainable behavior in a variety of domains. This includes establishing eco-design standards that, for example, highlight the use of renewable and biodegradable materials [26], as well as reflections on how HCI can support sustainable innovations [19]. Others have addressed how incorporating collaborative modeling into the design process can help system designers to reduce rebound effects and promote more sustainable HCI practices overall [8].

Early work on individual behavior and persuasive systems for sustainability considered users as rational consumers [10] that technology can (and should) support to make more sustainable decisions. Methodologies commonly used in HCI, such as rapid prototyping, user involvement, and iterative design, can rather allow for early and increasing feedback when developing interactive systems and increase awareness regarding sustainability impacts [23]. In a recent review of existing work on sustainable HCI and its critiques, the authors concluded that the HCI community "should challenge the narrative that we can rely on technology to save us, just as we challenged the narrative that climate change is an individual behavior change problem" [10].

In the context of ML, the impact of such work is reflected in the INTRPRT guidelines, which recommend formative user research and human-centered design principles to understand user demands and domain requirements to improve ML models [12].

The HCI community has also called for increased discussion on more responsible, transparent, accessible, and safe ML models transparency [50] which is echoed within the ML community and resulted in a requirement for high-risk applications in the recent European AI Act [14]. Efforts in HCI include making explainable AI more accessible using tangible and embodied interactions for improving the understanding of ML models [24] or a new technique that explains the predictions of classifier in an interpretable and more faithful manner [47].

While there is an increasing awareness of the need for explainability, interpretability, and safety of ML models for interactive systems, there is unfortunately little work on supporting more critical reflection on the necessity and scale of technology embedded in them, as well as how to support more sustainable decision making for developers of and with ML models.
## 2.4 Research Questions

To support more sustainable decision-making when selecting ML models, we first must step back to better understand how ML developers currently reflect on and consider model selection. This results in the following research questions:

RQ1. How do ML developers explore ML model alternatives?

RQ2. What trade-off factors do ML developers consider when selecting ML models in their workflow?

RQ3. How do ML developers consider sustainability when selecting ML models?

## 3 STUDY METHODOLOGY

The study explores how ML developers select ML models and their understanding of sustainability, specifically how it affects and is addressed in their workflow in their daily work.

We conducted semi-structured interviews to investigate how ML developers reflect on the model selection process and which trade-offs are considered relevant as a starting point for how to support more sustainable decision-making within this process.

**3.0.1 Participants:** We recruited thirteen ML practitioners (11 men, 2 women, avg. 8 years of experience [SD=3.1]) through emails to local research facilities or direct contact. All participants are ML developers (in research and industry) who specialize in a particular ML application, such as biology, or medicine (see Table 1). Participants were not paid and were informed about their rights and data usage via an informed consent form following GDPR.

**3.0.2 Procedure:** Each interview lasted approximately one hour. All interviews were conducted in English by one of the researchers, either in person or online. Participants first confirmed the consent form, before the interviewer briefly introduced the context. The story interviews[40] prompted to reflect on a recent experience searching for ML model for a specific task in their application fields such as physics, mechanics, or healthcare (see Table 1). The interviewer asked the participants to guide them through their approach to model selection, including how they initiated the process, the metrics they considered, how they evaluated their choice, and whether they referred to external resources. Additionally, we asked about their understanding of sustainability, how they define it, and whether they assess the sustainability aspects of their projects (see interview guide in Appendix A).

**3.0.3 Data collection and analysis:** The interview was audio recorded and transcribed using a local Whisper instance<fnref n="1" /> in line with GDPR. An investigator manually corrected the transcripts based on the audio recording.

We ran a mixed-approach reflective thematic analysis [9] where one author coded the participants' answers based on both top-down themes and themes that emerged (bottom-up).

The top-down themes included 'Starting dimensions', 'Sources of model exploration', 'Considered model trade-offs', 'Selection criteria' as well as 'Sustainability awareness', 'Sustainability Factors' and 'Impact on Sustainability' based on literature and interview questions. The author summarized relevant themes and discovered trends. To

verify coding reliability, a second author who was not involved in the interviews re-coded the data with the final set of themes.

## 4 UNDERSTANDING THE IMPACT OF SUSTAINABILITY ON ML PRACTICE

Our key results concerning current ML model-selection workflows show that developers understand sustainability goals better when they accompany problem definition with their choice of data formats and benchmarks. They often consult relevant literature about LLMs and AutoML to identify model alternatives and reflect on the trade-offs between performance and user understanding. We also found that developers rarely evaluate the model's impact on energy and infrastructural sustainability. Due to a lack of awareness of a model's energy and infrastructure impact, there is a trend toward large ML models, having a negative impact on sustainability. We saw only a few perspectives on machine learning and sustainability that go beyond environmental implications.

### 4.1 Defining problems by exploring problem structure, data formats and literature

The context of model selection largely varies greatly across projects: Most participants (12/13) said they need a clear starting point to select a suitable model: Ideally “the definition of the problem, the data, and also the final goal determines which model that you will use at the end” (P9). However, the starting point varied across developers and projects, making it difficult to create a robust structure to efficiently make decisions. Most began (10/13) by exploring the project goals; half (6/13) focused on the data; while others (5/13) explained situations where model selection was pre-determined. Understanding the problem is crucial: Most participants (11/13) said they needed to understand the problem before selecting a model selection because “if the definition of the problem is clear, then it’s very easy to select the model” (P8). This includes developing a clear understanding of the target audience and application domain (P4), which some address by framing the situation as a narrative and highlighting its origins, conditions, and limitations (P6, P8, P10). For example, P8 divided the problem into smaller steps and P1 sketched different input factors. P12 and P9 created presentations and diagrams, respectively, whereas P6 and P13 investigated the source problem further. The type of solution that best solves the situation is then based on this more complete problem description and often requires the use of diagrams and mathematical representation (P1), the definition of the priorities regarding the problem (P4, P3, P5), and an assessment of the adequacy of the solution given certain limitations (P10). Another common starting point for defining ML problems is to examine existing datasets and format considerations (6/13). This analysis consists primarily of analyzing the input “to be sure that our model was working just fine, we needed to be sure that we kept the right features” (P12), its format, and how it is linked to the output.

Focusing on goals and metrics: Some participants (P8, P1) underlined the need for setting a clear goal before jumping into coding. By contrast, highly experienced participants (P1, P5) who can clearly specify established statistics prefer to prioritize specific evaluation metrics. The fundamental metric, whether it is centered on accuracy

<footnote n="1">¹https://github.com/openai/whisper</footnote>
<table><caption>Table 1: Demographics of the interviewed ML experts</caption><thead><tr><th>ID</th><th>Age</th><th>Gender</th><th>Years of experience</th><th>Work Domain</th><th>Project domain</th></tr></thead><tbody><tr><td>1</td><td>30-40</td><td>M</td><td>10</td><td>ML, Robotics</td><td>AI algorithms</td></tr><tr><td>2</td><td>20-30</td><td>M</td><td>6.5</td><td>ML, Physics</td><td>Computational Physics</td></tr><tr><td>3</td><td>20-30</td><td>M</td><td>10</td><td>ML, Data Science</td><td>Brain Signal Processing</td></tr><tr><td>4</td><td>40-50</td><td>M</td><td>15</td><td>ML, Physics</td><td>Biology and Research in ML</td></tr><tr><td>5</td><td>20-30</td><td>M</td><td>6</td><td>ML</td><td>Environmental health</td></tr><tr><td>6</td><td>30-40</td><td>M</td><td>10</td><td>ML, Embedded intelligence</td><td>Human Activity Recognition</td></tr><tr><td>7</td><td>30-40</td><td>M</td><td>11</td><td>ML, Science communication</td><td>Learning from Noisy Labels</td></tr><tr><td>8</td><td>30-40</td><td>M</td><td>14</td><td>ML, AI</td><td>electrical vehicles</td></tr><tr><td>9</td><td>20-30</td><td>M</td><td>3</td><td>ML, Engineering</td><td>Electric Vehicle Routing Problems</td></tr><tr><td>10</td><td>20-30</td><td>F</td><td>3</td><td>ML, Engineering</td><td>Finance</td></tr><tr><td>11</td><td>20-30</td><td>F</td><td>3</td><td>ML</td><td>Cyber Security</td></tr><tr><td>12</td><td>20-30</td><td>M</td><td>3</td><td>ML, Computer vision</td><td>Trading</td></tr><tr><td>13</td><td>20-30</td><td>M</td><td>3</td><td>ML</td><td>Text generation</td></tr></tbody></table>

or the need for real-time processing, is determined from the start (P8). Meanwhile, P7 emphasizes the significance of calibrating the log-likelihood, a novel approach that values this specific strategy. P2 takes a different approach, which consists of constructing an initial test model. P9 on the other hand, examines his final goal before defining relevant evaluation metrics. One participant (P5) reported that model selections are sometimes predetermined by the clients, not necessarily for immediate outcomes or tasks, but to enable future work. This includes certain proprieties such as the type of model, including AI as a feature, as well as physical properties based on available hardware.

## 4.2 Developers consult relevant literature, LLM, and AutoML to identify model alternatives

Half of the participants (6/13) actively look for solutions or current achievements by referring to literature. Some participants (P2, P6, P9, P11), in particular, go into existing literature to find identical challenges and develop solutions “I wanted to look for different papers and different projects on the internet and also different related problems to better define my problem” (P9). For P13, he decided to start with literature to understand his model and find similar cases. P3, on the other hand, is primarily concerned with identifying impressive features through a literature review.

Approximately half of the participants (7/13), stated that they used literature and other tools to identify suitable models in their workflow. Some participants (P4, P7, P9, P10) prefer to base their work on pre-existing answers and beginning points obtained from the literature. Others use Twitter (P7) or model hosting platforms such as Huggingface (P6) to identify suitable candidates. Moreover, P4 highlighted the value of shared resources in extending and improving techniques to reach his final goal and find a good model. Finally, P7 mainly consults literature during his model selection as a means to better understand existing ML model approaches and to inspire potential means of improvements.

A few participants (3/13) displayed a considerable degree of confidence in AI tools, indicating a preference for relying on them to

make decisions. For example, P1 and P11 often consult ChatGPT, and trust its ability to provide insights or suggestions: “these days I rely a lot on chat GPT to provide valuable information” (P11). P2 also prefers using AutoML, a tool designed to automate tuning and the model selection process, demonstrating P2’s belief in the tool’s ability to identify optimal machine learning models and configurations for a given problem.

## 4.3 Developers reflect on the trade-offs between performance and user understanding

*Considering user understanding of model results:* When deciding which model to use, some participants (3/13) place a high value on the understanding of the models by the end-user “we really want our models not to be a black box and we really want to understand what is happening if not using any deep learning approach” (P10). Notably, P5 prefers models that have essential clarity in their decision-making systems. By using such models, he hopes to ensure that end-users understand the reasoning behind the model’s conclusions, making the results more transparent and trustworthy. Differently, P1 takes a strategic approach, beginning with the selection of a baseline model. This serves as a reference or a benchmark, allowing end users to compare the performance of potential models to this baseline.

*Selecting models that perform well with the data:* Participants (8/13) were mostly in agreement on the importance of data compatibility in the model selection process. For most of them, the most important criterion is that a model correlates well with their specific dataset. Several individuals, including P6, P7, P2, P3, P9, and P10 highlight the necessity of completely knowing the structure of their data. They feel that understanding the complexities and nuances of their data helps in selecting models that are able to use the knowledge contained within. Meanwhile, P1 takes an experimental position. This participant tries to find which models work best in practice by running several models on the same dataset. Meanwhile, P1 and P12 take an experimental position. These participants try to
find which models work best in practice by running several models on the same dataset. This repeated method provides insights into model performance under consistent settings, enabling improved decisions. P8 focuses on this concept by emphasizing the extraction of model selection parameters directly from data. This participant's approach indicates a data-driven methodology in which the properties of the data itself determine the model selection criteria.

*Identifying Selection Parameters:* Half of the participants agreed that this is a question that primarily concerns the identification and use of particular metrics or parameters to select their model. P8's approach is fundamental, with accuracy as the primary objective. This participant considered accuracy to be a critical metric. P1's solution, on the other hand, requires a delicate balancing act. The participant is looking for models that find an appropriate balance between the complexity (or number) of parameters and overall performance. This implies that for P1, both efficiency and effectiveness are critical. P2 wanted to create uniformity in the evaluation process by defining a standard set of parameters for each challenge. Finally, highlighting that different challenges may entail different evaluation measures, P3 is willing to recalibrate and redefine the selection criteria based on the specific needs of each problem. P11 wants to achieve the highest accuracy of the model. However, P12 searches explainability more than the performance and explained that “it didn’t matter if we had a poor model in terms of performance, as long as it gives an explanation to what’s going on”.

*Prioritizing Interpretability and Explainability for Model Viability*

The majority of participants (7/13) emphasize the importance of explainability and interpretability in the model selection process. For these people, it's not just about how effectively a model works, but also about how it reaches its conclusions. P2 and P12, for example, valued models that not only perform well but also provide good explanations and reasons for their results. In keeping with the interpretability theme, both P2 and P3 preferred models that developed justifications and explanations, particularly when it comes to decision-making. The goal is to create models that not only make correct decisions but also explain why they are correct. P10 tried to avoid black box models. For one participant (P5), simplicity and explainability go hand in hand. This method reinforces the concept that the simpler and less complicated a model is, the easier it is to understand and explain its operations and conclusions. P4 takes a different but related approach, focusing on explainability through incremental improvement. This implies that P4 may begin with a basic model and then make incremental changes, each time improving its explainability until an ideal balance of performance and transparency is obtained.

## 4.4 Developers rarely evaluate the energy and infrastructure impact of models

*Comparing Models Against Existing Solutions and Benchmarks:* The comparison of the selected models against existing benchmarks or solutions is a key activity among the participants (7/13). This analytical approach assures that their models not only perform well in isolation but also when compared to existing solutions. P5 and P10, for example, make comparisons by citing academic and professional

literature. This strategy emphasizes dependence on literature and previously reported results, providing a context for evaluating the effectiveness of their model. P3's strategy includes comparing the findings of their model to those of other existing solutions. He identified the relative strengths and weaknesses of their selected model by contrasting it with the results of existing models or solutions. Meanwhile, P6 focuses on real results in prediction, comparing the predicted results of his model to real values. P8, P9, and P13 begin with a foundational methodology, establishing a baseline model. This baseline serves as a standard or reference against which other models, including the one chosen by the researchers, are evaluated. Finally, P5 examined and compared the efficacy of each model in the same settings by running many models on the same dataset. This head-to-head comparison provides insight into each model's relative performance, ensuring a full evaluation.

*Defining standard evaluation metrics:* All participants agreed that model evaluation requires the use of specific metrics based on the problem and personal preferences. For example, P6 and P2 emphasize loss functions as an evaluation metric. P4, on the other hand, uses the Pearson correlation as an evaluation metric. P3's training time appears as a significant indicator, demonstrating the model's efficiency. P7 and P9 use the F1 score and the calibrated log probability as evaluation measures in a more holistic approach. P8 identified the evaluation metrics as an implication of several aspects of model performance at the same time. While admitting the relevance of different metrics, P1 believes that the precise strategy chosen should be intimately linked to the nature and requirements of the initial problem. P10 mentioned that the running time is more important than the accuracy. P11 focuses on the cost of the model like time consumption and the storage of the model.

*Insufficient training and evaluation data challenges model selection:* Some participants (4/13) faced challenges during their learning process due to a lack of data. Participants highlighted the problem of data availability issues, describing the difficulty in obtaining essential data for robust analysis or model training (P1, P7). This might be also due to the scarcity of available data in distinct domains, which is especially challenging for interdisciplinary collaborations (P4). One participant also highlighted the lack of data and support to evaluate the outcome of a model at the point of selecting it (P3).

## 4.5 Awareness of the energy and infrastructure impact of a model is limited

The majority of the senior participants (8 out of 13) were aware of the impact of machine learning on sustainability. However, this information does not often turn into considerations for sustainability in their daily work. P4 stated, "I know a lot, I think because I'm concerned with it," indicating a personal interest rather than continuous professional practice. Some participants have a basic knowledge of sustainability (P7), while others believe it is irrelevant to their work (P1, P2, P3). A few participants, such as P2, only consider sustainability after getting their results, rather than including it throughout the entire process. This shows a varied understanding and use of sustainability principles among ML practitioners, with space for further incorporation into routine workflows.
In fact, the scale of consideration varied across participants. Some actively work on reducing the impact of models (P4, P5, P6, P7), and some collect information about their consumption and report them (P3, P4) however others are aware of them but do not actively consider them in their work (P1, P2, P8). However, we found that younger ML engineers (P9, P10, P11, P12, and P13) had a significantly lower degree of understanding and awareness about the importance of sustainability in their jobs. These early-career professionals failed to prioritize or address the environmental impact of their machine learning techniques, revealing a gap in their approach to incorporating sustainability into their workflows. Additionally, none of them claimed that sustainability was highlighted during their ML education at university, indicating a lack of fundamental knowledge on the subject.

Beyond that, reasons for not considering sustainability were the perceived small impact of their work (3/13) or priorities of result quality over sustainability impact (1/13). Common arguments among those answers were that “the models that I am building are nothing in comparison to the big models” (P2) in industry or commercial products and hence academic ML model dimensions are too small to have a real impact (P1, P3). One participant highlighted the importance of balancing the quality of the outcome with the sustainability impacts, arguing that “this model is working, maybe I’ll try to do something more frugal” (P2) relies on a satisfying model in the first place.

*Challenging to evaluate the impact of a model itself:* Evaluating the impact of machine learning models on sustainability is challenging due to the complexity and variability of the factors involved. Some participants agreed that the main impact of models considered in ML is the amount of energy used and the associated carbon footprint (5/13). However, one Participant highlighted that simply measuring computation and energy consumption is insufficient, as these metrics do not fully capture the carbon emissions and environmental costs associated with machine learning applications (P7). It should be associated with the training time (P6), making deep learning approaches especially costly (P3). Three participants (P3, P4, P7) reported tracking their impact using e.g. existing toolboxes and carbon tracker. However, P4 points out that the results of existing carbon trackers can differ significantly, reducing the comparability of approaches.

Measuring carbon emissions is currently the only real indicator of potential impact on the environment. However, an open question remains “what aspects of machine learning model computation to consider? Like do you consider network requests between different nodes in a data cluster that’s running your model?” (P7). Answering this question can be challenging as data are not always available. Especially when working on or with non-public commercial models e.g. from OpenAI, “both the electricity cost and the general cost of using those models” (P5) stay hidden and make it hard to relate to a single application. Only one participant (P4) mentioned that he considers the overall impact of his work, including “all the emails, the platform we use, the video conferences[...] if I had to travel to meet with colleagues” to create a more complete overview of the environmental impact of a project.

Developers choose large ML models with a correspondingly negative effect on sustainability. Further considerations are related to the infrastructure mainly hardware (3/13) but also water consumption in the process (2/6). Participants said that using PCs (ca. 800 Watt) often exceeds mobile chips by a 10-fold (e.g. Apple M2 Pro chip 50 Watt), “even in this case, even if the MacBook runs a little slower, but still, it saves a lot of energy” (P6). There was a general consensus that working on larger clusters where “maybe 200 people use one cluster” (P6) should be preferred compared to running a dedicated PC (P4). Another factor is the production costs of CPU and GPU chips, which are often used to run large ML models, such as GPT. While there is little data available about the mining and production costs of CPUs (P4, P7) there is hardly any about GPU production. P4 explained that from a sustainability standpoint it “is completely unbelievable that in 2023 we don’t even know how much it costs in terms of environmental impact to produce a GPU”, even though foundation models increasingly rely on them. A further concern was the amount of water and cooling costs of data centers, which are often hidden when computation is outsourced to larger infrastructures (P7).

## 4.6 Large ML models negatively effect sustainability

One recurring topic throughout the interviews was the complexity of ML models and their impact on sustainability. The trend toward using large machine learning models, particularly deep learning, has raised concerns about its negative impact on sustainability. Our experts highlighted that the size and complexity have a large impact on the computation time and infrastructure required, so if an algorithm just gains a few tens of percent of accuracy the carbon footprint increase is very large. P4 emphasized that, while these models may provide small improvements in accuracy, the environmental cost can be disproportionately significant, making it critical to consider these trade-offs.

Experts further worried about the increasing trend to apply larger complex models, like deep learning, which have an especially large carbon footprint (P1, P3). Such models take “more like one month or hundreds of hours of training on hundreds of GPUs” (P2), which has a large cost attached to it.

Some have called to reconsider the impact of this trend in education, academia, and industry, reporting that students “forget about all the other simple approaches that can be better, sometimes orders of magnitude better, when you think about inference time” and similar factors (P1). This requires a rethinking about the actually needs of a model, for example “if you have 90% accuracy compared to 95% accuracy, but the 90% accuracy uses only 10% of the footprint, then is the 90% already enough for the actual use case?” (P6). So “sometimes we should use a smaller model even though it has less accuracy” (P8). This could also result in refraining from ML models and instead turning to “simpler algorithms or simpler decision processes” (P4).

## 4.7 Perspectives on ML and Sustainability beyond environmental impacts are limited

Only two participants reported being aware of ML impacts beyond the environment. One highlighted “social factors like algorithmic
bias [or] economic factors such as how do we democratize AI and increase access to compute infrastructure to people who may not be able to afford it". P4 recognized the broader context of sustainability by adding measures such as travel reduction into their overall sustainability budget. However, broader views on sustainability, particularly those involving social and economic issues, were less frequently emphasized.

Supporting economic sustainability requires “opening both the code of training, how it is trained, on which data, what is the result you have so that you can have people that test fairness, privacy” (P5). Work on sustainability hence means also to provide models and code along with ML models (P5). One highlighted the need for transparency, reproducibility, and openness in machine learning, arguing that making code and models accessible is critical for analyzing fairness, privacy, and other ethical problems. P5 also emphasized the need for models that are physically aware and less of a "black box," suggesting that sustainability should take into account the transparency and accountability of decision-making processes. Only one participant reported working on sustainability concerns by using ML models to generate guidelines for pesticide spreading and safe housing locations (P5). Further, P6 described a larger perspective, where sustainability can be also understood across the whole ML life-cycle, balancing “the training, the footprint cost during the training, and the longer time deployment”, including the used hardware in the later stages or if a model could be fine-tuned or need to be retrained after some time (P6).

# 5 DISCUSSION

## 5.1 Should I choose a smaller model?: Trends vs. needs when considering model selection

Our interviewees frequently struggle to identify project needs before selecting a model especially when no clear starting point is provided. This is understandable, given that approach differs according to the project's requirements. They reported several approaches on how to explore and define goals and metrics for the final model. Some based this exploration on their familiarity with the task at hand, and guided selection on their intuition and experience. Others consulted and followed literature recommendations, Large Language Models (LLMs), or automated algorithms to investigate different models for their project. These, however, rarely include details about underlying project goals, model characteristics, or recent improvements. Instead, they compared model characteristics based on benchmarks was mentioned as a suitable solution, with P5 creating his own by running different models on the same dataset. This can be limited by the availability of data, available model overviews, and computing power. This can lead to a lack of recommended practices, such as using the smallest model that is appropriate for the task.

Several more experienced ML practitioners reported that younger developers tend to choose large models due to their accessibility and familiarity from their educational background. At the same time, those also reported being less aware of their impact on sustainability. The trend of choosing the highest possible levels of accuracy or overall performance, even when it is unneeded, is not limited to younger developers and can lead to unnecessary complex models.

Overall, the model selection process was driven by the developer's familiarity, accessible information, and experience rather than a structured evaluation of the project's requirements. As a result, ML developers tend to choose models based on availability and accessibility, resulting in a green adoption of (pre-trained) complex models. We agree that the importance of increasing awareness and information regarding available smaller alternatives and the long-term effects of these decisions, especially for younger engineers. It further requires an increased effort to measure these impacts and provide sustainability information alongside models, e.g. in benchmarks, to allow developers to make more informed decisions.

One of our interviewees mentioned also the public trend and organizational/client-based pressure regarding model selection. Not only do employers or clients often prioritize profitability, efficiency, or short-term results over sustainability considerations, but the current public trend of ‘using AI’ adds another significant barrier to considering smaller, potentially more sustainable models. While new technological advance hold great potential for companies, the unreflected technological competition can lead to unnecessary complex model applications that come with a cost for the whole society. Overcoming this public trend requires changes not only from the individual but also from a rethinking as a society. This includes more public engagement to align organizational goals with sustainable practices, motivating environmentally conscious choices, encouraging open communication with customers and managers, and providing professionals with the tools and power to advocate for change.

## 5.2 Balancing performance, interpretability, and sustainability for Model Selection

The majority of our participants mentioned that identifying trade-offs in ML projects can be difficult due to the diversity of interacting factors. For example, some said that selecting an ML “model that is interpretable by design” (P5) is not only beneficial for the developer to better understand the model performance (P9, P10, P13) but also improves user interaction.

However, this could also strongly affect the model performance and lead to less clear outcomes (P4). Some developers therefore opted for mixed approaches, combining interpretable models with deep learning models (P1). The evaluation of trade-offs between performance and interpretability is also one of the most reported in ML literature [47] which resulted in discussions about the benefits of white-box and black-box algorithms. However, other aspects such as sustainability fall short in such discussion. Only one participant highlighted the importance of sustainability when selecting ML models. Others focused instead on accuracy more but would consider it if the model's performance stayed unchanged.

Making such decisions becomes even more difficult when the impact of the added accuracy on sustainability is unclear. Some developers said that the lack of data for guiding this process efficiently. While benchmark information, such as performance, is sometimes published in the academic literature, it may not reflect real-world settings or apply to a very limited set of tasks. Beyond benchmarks and interpretability, ML developers mentioned a lack of guidance in navigating trade-offs between different model features in the literature. Addressing these gaps requires systematic investigation
of model features such as interpretability or sustainability impacts
and the trade-offs between them and with more traditional metrics
like performance. A major obstacle to that objective is the lack of
existing standards on how to measure and report the sustainability
impact of ML models.

More discussion is needed in related fields and the general public
is necessary to encourage change. We see the public discourse on
explainability as a potential role model here, where the increased
awareness of opaque ML decisions resulted in several tools to sup-
port explainability, increased focus on means of explainability to
political guidelines in this regard, e.g. the European AI Act [14].
This led to a larger awareness and request for interpretable and
explainable models reflected in seven out of thirteen interviewees
considering interpretability when selecting a model.

## 5.3 Sustainability impact of ML models often unknown and unreported

During the study, participants said they lack of knowledge about
the impact of models and infrastructure on sustainability, which
supports the existing literature [44]. We saw that the environmen-
tal implications of machine learning (ML) models are frequently
overlooked and inadequately documented, presenting a significant
barrier to the development of more environmentally responsible
systems. Especially with the increase in training and deploying
large-scale ML models, we need more information to evaluate the
impact of such models, including the water and energy usage re-
quired to run data centers, or the resources required to produce
the hardware. This limits the understanding of the scale at which
these systems work, making it difficult for developers to measure
the environmental costs of running huge models and infrastructure.
This issue emphasizes the importance of more transparency and
education about the environmental impact of machine learning
applications. Recent work [46] made a first attempt at calculating
the environmental, water, and hardware costs of LLMs for a single
task. Such investigations help to quantify the sustainable impact
and would collectively allow for a more informed discussion.

Regarding documentation of individual algorithms, only one
participant mentioned he measures and reports what his models
consume. While recent tools such as CodeCarbon start allowing
developers to easier assess the CO2 emissions consumed by their
algorithms [22], measuring and documenting the larger impact of
ML models remains a challenge. Nevertheless, a larger effort in
publicly reporting the CO2 emissions of ML models would support
model trade-off evaluations for individual models and application
cases as well as provide arguments for a larger discussion of the
gain vs. impact of ML in the long term. This reporting could further
be included in platforms that compare machine learning models
such as Hugging Face in addition to typical performance criteria
such as accuracy and loss functions. Reporting energy consumption
should become a priority, encouraging responsibility and innova-
tion toward greener, more sustainable AI systems.

## 5.4 Limitations and Future Work

This study represents a preliminary exploration into the decision-
making processes involved in selecting ML models for various
applications. While the number of interviews conducted is limited

(13 professionals), this work serves as an important first step in
understanding the key considerations and challenges practitioners
face in this domain. By identifying initial trends and insights, our
study lays the groundwork for future research with larger and more
diverse samples to build on these findings.

We note that we sought an equal gender balance among the
participants in the study. We successfully recruited five women but
unfortunately, three candidates could not find the time or canceled
at the last minute. Nevertheless, the gender balance in our study
(15% female) is still representative (and notably higher) than the
current representation of women in machine learning roles, which
is only 12% [25].

We conducted the study with ML developers with an average of
eight years of experience to provide deeper insights into experts'
tasks and strategies when selecting models. We recruited them
primarily from research facilities and personal networks, which
potentially introduced a selection bias. However, the diversity of
professional experiences, backgrounds, and application domains,
does allow for to generalize of valuable insights from the ML model
selection process. However, we believe that future research should
also involve early-career practitioners or ML model users, such as
HCI researchers, to investigate how their experiences differ.

Finally, there is an imbalance in our work regarding the three
sustainability dimensions: social, economic, and environmental.
Most of our participants and reviewed publications focus on ML's
environmental impact, with few addressing economic and social
sustainability. We see significant potential for further research into
AI accessibility, social and workplace effects, and related social
and political initiatives to address them. Some of these ideas are
integrated into our following implications.

# 6 IMPLICATIONS FOR HCI AND ML

Our findings reveal that data scientists lack of information and
awareness regarding sustainability when selecting models which
has implications for both the ML and HCI communities and the over-
all use of AI within interactive tools. In the following, we outline
implications for both domains and hope to encourage discussions
and change within them.

## 6.1 Implications for ML

Participants reported a low awareness of the sustainability impacts
of ML models, despite the increasing discussion in the media and
academic circles. One reason mentioned was that model sustainabil-
ity and evaluation of ML model trade-offs are not part of traditional
ML education. In order to address this issue we outline below how
*Supporting critical reflection on model choice in ML education* and
*Enabling sustainability evaluation of ML model choice* could improve
sustainable decision-making in the long term.

**Supporting critical reflection on model choice in ML educa-**
**tion:** Choosing suitable models is far from an organized process.
It depends on the project goal, available data and hardware, ap-
plication context, and user; but also importantly on the degree
to which users are willing to trade performance off for a greener
model. Developers often navigate that decision space by using their
professional experience as well as searching for alternatives, often
in state-of-the-art literature or model platforms. Unfortunately, this
process requires developers to first identify the requirements of the
ML problem and then convert this into efficient search strategies
to find suitable alternatives.

Our interviewees emphasized the importance of developing a
clear understanding of their needs and goals to effectively identify
suitable solutions that minimize energy consumption. Reflection
on the problem, context, and goal parameters were mentioned as
crucial and requires developers to divide problems into parts that
can be addressed directly, a skill that requires practice. ML education
should thus put greater emphasis on problem exploration and analysis
to improve students' ability to better identify alternative approaches.

Advanced ML developers reflected on the decrease in general model knowledge among junior developers, especially since the number of released models has exploded in recent years without a corresponding increase in time spent on ML education. This directly affects how junior developers identify potential model alternatives, with a corresponding potential bias toward choosing overly complex models. This further limits their ability to identify trade-offs across models and limits their ability to creatively address new problems. Younger developers especially reported their lack of training on sustainability considerations and the impact of ML models on the environment throughout the entire ML life cycle. ML education should thus include discussions of the impact of model selection throughout the ML life-cycle, including its impact on the environment and society.

Even though most developers treated top performance as their
only goal, some also prioritized interpretability, explainability, and
sustainability as additional benefits beyond performance. However,
accomplishing these required developers to identify and contextu-
alize these benefits. Evaluating competing model characteristics
is closely aligned with the identification of project and end-user
needs. The general goal should be a reflective choice of the most
suitable model with minimum sustainability impact. This could
mean that high-risk contexts, e.g. medicine or aeronautics, might
require high-accuracy results with sufficient interpretability but po-
tentially heavier computational load, whereas task-specific models
for end-user applications were found to often outperform multi-
purpose models on quality while also reducing the environmental
impact [39]. ML education should thus encourage more critical re-
flection and discussions with respect to model trade-offs to support
more responsible decision-making in ML practice.

**Enabling sustainability evaluation of ML model choice:** En-
suring the environmental sustainability of ML includes multiple
factors, including energy consumption during training and infer-
ence, water consumption to cool the data center, and harvesting rare
minerals for hardware components. Few participants were aware
of these issues and only one of thirteen measured and reported
the impact of their work. Some ML developers mentioned that sus-
tainability is only relevant at scale and that such considerations
are not required for individual models. Ensuring that sustainability
becomes an integral part of professional ML work practices will
require greater awareness of sustainability issues, more information
about sustainability factors, and standardized evaluation measures.

Several participants had only limited or no detailed information
about the impact of their work on sustainability. We argue that

integrating sustainability-related courses in academic programs is
essential for promoting sustainable decision-making. This requires
including information about the impact of both corporate techno-
logical footprints and end-user applications on sustainability, with
a corresponding reassessment of the trade-offs associated with ma-
chine learning model characteristics, akin to the discourse on AI
explainability. We argue that more public information and education
is necessary to create more awareness among ML developers about
how ML contributes to sustainability issues.

The ML experts we interviewed highlighted the lack of informa-
tion regarding the sustainability impact of different ML factors. For
example, the environmental impact of GPU hardware production
or the training and inference consumption of popular commercial
models, such as OpenAI's GPT, is not readily available, making it
difficult to evaluate these factors within a project. While the release
of models such as Llama 3 will help to quantify the energy consump-
tion of large models, more public information is needed to evaluate
the impact of models, data centers, and hardware on sustainability.

Finally, although a number of tools can track a computer's or
model's carbon footprint, e.g., [2]), these measures are rarely stan-
dardized or published. More systematic evaluation and reporting
would not only increase awareness but could alter the priorities of
ML developers and researchers. We thus need to increase informa-
tion about and awareness of ML sustainability, including establishing
suitable, comparable evaluation measures to evaluate with regular,
standardized reporting structures.

## 6.2 Implications for HCI

The growing capability and accessibility of ML models over the
past decade have had a large impact on the HCI community. In
particular, the introduction of large commercial models such as
ChatGPT that can be integrated into interactive tools without any
specific ML knowledge has resulted in a large increase in Human-AI
applications. This suggests that the HCI community has the poten-
tial to significantly shape such applications and aid in *supporting
greater reflection on sustainability*. At the same time, selecting more
sustainable models is inherently a human-centered problem, and
we see opportunities for the HCI community to become more in-
volved in *supporting the design of more sustainable systems*.

**Supporting reflection on sustainability in HCI:** A recent cri-
tique [10] argues for more critical reflection on the use of tech-
nology, especially ML/AI use. Reflection can help practitioners
prioritize sustainable practices and ideally find a balance where
their models both meet performance standards and still remain
energy-efficient and interpretable.

As active users of ML models for building intelligent interactive
systems, we also see an increasing need for awareness of the poten-
tial negative effects of ML models on sustainability within the HCI
community. Unfortunately, few UX practitioners are aware of the
impact of their work on sustainability, which is further aggravated
by the use of external models via APIs. Steps to increase awareness
could include making carbon tracking efforts, such as [1, 22, 36],
more accessible to users and consumers of interactive ML applica-
tions. This could also strengthen efforts to educate developers about
the costs of technology within and outside the community, such as
the Green Web Foundation green counseling [1], as well as deepen discussions about the short and long-term environmental impact of HCI work, similar to social implications of AI use. These efforts would *increase awareness of the need to consider sustainability when designing interactive systems*.

Another challenge is the general lack of ML knowledge by inter-
active system designers. Comparing models and reflecting on more
sustainable alternatives requires users and system developers to
investigate the problem space and requirements to make informed
decisions. These decisions are further affected by the recent increase
in funding and publications about “Human-AI” systems, which cre-
ates a counter-motivation to develop less sustainable interactive
systems. Identifying more suitable technological approaches, in-
cluding sustainable models, and evaluating their trade-offs, will a
shift of focus from the technology to the technology’s impact on
the end user. This however requires a *more nuanced discussion of*
*the requirements of AI models* within the community.

HCI offers an important application domain for large ML models, which means that we can and should shape future interaction between humans and AI systems. This will require a critical discussion on the negative impact of complex resource-intensive ML models. For example, HCI research venues could insist on reporting the energy consumption of systems that include ML models, which would not only increase the visibility of their energy consumption and permit comparisons among them, but *reporting their sustainability impact indicators* could highlight the trade-offs involved in developing such technology.

**Supporting the design of more sustainable systems:** HCI practitioners improve ML model systems by developing sustainable UX principles, similar to work on eco-design standards, e.g. [26]. This “sustainability-by-design” approach could improve the balance between technological viability and environmental impact. For example, over the last decade, the sustainable HCI community has developed various tools that improve a user’s decision-making processes. Our interviews showed that defining the actual problem and identifying relevant alternatives is challenging not only because these developers lack sufficient background knowledge for making effective keyword searches, but also the exploratory and uncharted nature of the overall process. HCI’s emphasis on interdisciplinary collaboration could help integrate perspectives from social science, environmental studies, and ethics, and help ensure that sustainable ML practices address complex, real-world challenges holistically. Sustainable HCI researchers could *help redesign this process and better support sustainable ML model exploration*.

Another crucial problem is the limited access to alternative ML
models, which are made available primarily by ML developers. A
key driving factor in the recent increase in the use of interactive
intelligent systems is the increased availability of models that do
not require advanced ML knowledge. Unfortunately, easy-to-use
large-scale multi-purpose models have a significantly larger impact
than simpler models. This encourages the use of unnecessarily com-
plex models for end-user-facing systems and prototypes. Beyond
identifying model requirements and model alternatives, the HCI
community could also play a leading role in *building end-user tools
that help non-ML professionals to find, compare, and apply more
sustainable models* in their work.

The above suggestions seek to help ML developers, HCI researchers, and UX designers to better identify project needs, consider more sustainable alternatives and implement them in their projects. We recognize that 1) large complex models are required for some tasks and should be used as long as they are justified; but also that 2) the tools that encourage sustainable ML model selection should not themselves generate a worse environmental impact. Providing better-informed ML model choices will require efforts from both the ML and HCI communities, as well as a more critical perspective on the technology we use on a daily basis.

7 CONCLUSION

The accessibility and ease of use of large language models, such as ChatGPT, have gained attention in the ML and HCI community resulting in a plethora of new applications. However, these advancements have a significant sustainability impact, and continuing the current trends might be devastating for our environmental goals.

We identified ML model selection as a crucial decision-making
point to address this issue. In this paper, we present insights from
our interviews with 13 ML developers on how ML models are cur-
rently chosen, what trade-offs professionals consider when select-
ing them as well as what role sustainability plays in this evaluation.
We conclude by exploring the implications of our insights for the
HCI and ML.

We emphasize the importance of critical reflection on sustain-
ability and model choice in ML education, as well as greater trans-
parency and standardized reporting of ML model consumption. As
more HCI practitioners use large ML models, we think it is time
to reflect more critically on the impact of our work on intelligent
interactive systems on sustainability. This includes not only rais-
ing awareness but also lobbying for a more systematic evaluation
and reporting of energy consumption for the systems we propose
and build. We also see an opportunity for the HCI community to
contribute to the design of tools for improving the assessment of
ML model trade-offs and increasing access to more sustainable al-
ternatives. The work presented in this paper is a first step towards
such initiatives.

ACKNOWLEDGMENTS

This paper is supported by European Union's Horizon Europe re-
search and innovation programme (HORIZON-CL4-2021-HUMAN-
01) under grant agreement No 101070408, project SustainML (Ap-
plication Aware, Life-Cycle Oriented Model-Hardware Co-Design
Framework for Sustainable, Energy Efficient ML Systems).

REFERENCES

[1] Chris Adams, Michelle Thorne, Hannah Smith, Lima Dastgeer, Katrin Fritsch, and Fershad Irani. 2024. Green web foundation. https://www.thegreenwebfoundation.org/about/.

[2] Lasse F Wolff Anthony, Benjamin Kanding, and Raghavendra Selvan. 2020. Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. *arXiv preprint arXiv:2007.03051* (2020).

[3] Juby Babu. 2024. OpenAI says Chatgpt's weekly users have grown to 200 million. https://www.reuters.com/technology/artificial-intelligence/openai-says-chatgpts-weekly-users-have-grown-200-million-2024-08-29/

[4] Maria Maddalena Barbieri and James O Berger. 2004. Optimal predictive model selection. (2004).

[5] Rafael Barbudo, Sebastián Ventura, and José Raúl Romero. 2023. Eight years of AutoML: categorisation, review and trends. *Knowledge and Information Systems* 65, 12 (2023), 5097–5149.
[6] Christoph Becker, Ruzanna Chitchyan, Leticia Duboc, Steve Easterbrook, Birgit Penzenstadler, Norbert Seyff, and Colin C Venters. 2015. Sustainability design and software: The karlskrona manifesto. In *2015 IEEE/ACM 37th IEEE International Conference on Software Engineering*, Vol. 2. IEEE, 467–476.

[7] Gissella Bejarano, Adita Kulkarni, Raushan Raushan, Anand Seetharam, and Arti Ramesh. 2019. SWaP: Probabilistic Graphical and Deep Learning Models for Water Consumption Prediction. In *Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (New York, NY, USA) (BuildSys '19)*. Association for Computing Machinery, New York, NY, USA, 233–242. https://doi.org/10.1145/3360322.3360846

[8] Laetitia Bornes, Catherine Letondal, and Rob Vingerhoeds. 2024. Systemic Sustainable HCI: Integrating Collaborative Modeling into a Design Process to Address Rebound Effects. In *Proceedings of the 2024 ACM Designing Interactive Systems Conference*. 3046–3062.

[9] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. *Qualitative research in psychology* 3, 2 (2006), 77–101.

[10] Christina Bremer, Bran Knowles, and Adrian Friday. 2022. Have We Taken On Too Much?: A Critical Review of the Sustainable HCI Landscape. In *Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems*. 1–11.

[11] Gavin C Cawley and Nicola LC Talbot. 2010. On over-fitting in model selection and subsequent selection bias in performance evaluation. *The Journal of Machine Learning Research* 11 (2010), 2079–2107.

[12] Haomin Chen, Catalina Gomez, Chien-Ming Huang, and Mathias Unberath. 2022. Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review. *NPJ digital medicine* 5, 1 (2022), 156.

[13] Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. 2018. Learning to explain: An information-theoretic perspective on model interpretation. In *International conference on machine learning*. PMLR, 883–892.

[14] Council of European Union. 2024. Council regulation (EU) no 2024/1689 – AI Act. http://data.europa.eu/eli/reg/2024/1689/oj

[15] Evropi-Sofia Dalampira and Stefanos A Nastis. 2020. Back to the future: simplifying Sustainable Development Goals based on three pillars of sustainability. *International Journal of Sustainable Agricultural Management and Informatics* 6, 3 (2020), 226–240.

[16] Jens de Hoog, Ali Anwar, Peter Hellinckx, and Siegfried Mercelis. 2024. Selection of image classifiers for noisy images through metalearning. In *Proceedings of the 2024 7th International Conference on Machine Vision and Applications*. 92–99.

[17] Jie Ding, Vahid Tarokh, and Yuhong Yang. 2018. Model selection techniques: An overview. *IEEE Signal Processing Magazine* 35, 6 (2018), 16–34.

[18] Carl DiSalvo, Phoebe Sengers, and HRönß Brynjarsdóttir. 2010. Mapping the landscape of sustainable HCI. In *Proceedings of the SIGCHI conference on human factors in computing systems*. 1975–1984.

[19] Olivia Doggett, Kelly Bronson, and Robert Soden. 2023. HCI research on agriculture: competing sociotechnical imaginaries, definitions, and opportunities. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems*. 1–24.

[20] Paul Dourish. 2010. HCI and environmental sustainability: the politics of design and the design of politics. In *Proceedings of the 8th ACM conference on designing interactive systems*. 1–10.

[21] Carsten Felix Draschner, Hajira Jabeen, and Jens Lehmann. 2022. Ethical and Sustainability Considerations for Knowledge Graph based Machine Learning. In *2022 IEEE Fifth International Conference on Artificial Intelligence and Knowledge Engineering (AIKE)*. 53–60. https://doi.org/10.1109/AIKE55402.2022.00015

[22] Wilson Friedler. 2021. Track and reduce CO2 emissions from your computing. https://codecarbon.io/.

[23] Jon Froehlich, Leah Findlater, and James Landay. 2010. The design of eco-feedback technology. In *Proceedings of the SIGCHI conference on human factors in computing systems*. 1999–2008.

[24] Maliheh Ghajargar, Jeffrey Bardzell, Alison Smith Renner, Peter Gall Krogh, Kristina Höök, David Cuartielles, Laurens Boer, and Mikael Wiberg. 2021. From "explainable ai" to "graspable ai". In *Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction*. 1–4.

[25] Sakshi Gupta. 2020. Why Women Are Making It Big in Artificial Intelligence and Machine Learning. https://www.springboard.com/blog/data-science/women-in-machine-learning/. [Accessed 13-09-2024].

[26] Sofia Guridi, Matteo Iannacchero, and Emmi Pouta. 2024. Towards More Sustainable Interactive Textiles: A Literature Review on The Use of Biomaterials for eTextiles.. In *Proceedings of the CHI Conference on Human Factors in Computing Systems*. 1–19.

[27] Lon Åke Erni Johannes Hansson, Teresa Cerratto Pargman, and Daniel Sapiens Pargman. 2021. A Decade of Sustainable HCI: Connecting SHCI to the Sustainable Development Goals. In *Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems* (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 300, 19 pages. https://doi.org/10.1145/3411764.3445069

[28] Zezhen He and Yaron Shaposhnik. 2023. Visualizing the Implicit Model Selection Tradeoff. *Journal of Artificial Intelligence Research* 76 (2023), 829–881.

[29] Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. *The Journal of Machine Learning Research* 21, 1 (2020), 10039–10081.

[30] IEA. 2024. Electricity 2024: Analysis and forecast to 2026. https://www.iea.org/reports/electricity-2024

[31] Randy H Katz, David E Culler, Seth Sanders, Sara Alspaugh, Yanpei Chen, Stephen Dawson-Haggerty, Prabal Dutta, Mike He, Xiaofan Jiang, Laura Keys, et al. 2010. Sustainable Computing: Informatics and Systems. (2010).

[32] Jörg Kienzle, Gunter Mussbacher, Benoit Combemale, Lucy Bastin, Nelly Bencomo, Jean-Michel Bruel, Christoph Becker, Stefanie Betz, Ruzanna Chitchyan, Betty HC Cheng, et al. 2020. Toward model-driven sustainability evaluation. *Commun. ACM* 63, 3 (2020), 80–91.

[33] Bran Knowles, Oliver Bates, and Maria Håkansson. 2018. This changes sustainable HCI. In *Proceedings of the 2018 CHI Conference on human factors in computing systems*. 1–12.

[34] Todd Kulesza, Simone Stumpf, Weng-Keen Wong, Margaret M Burnett, Stephen Perona, Amy J Ko, and Ian Oberst. 2011. Why-oriented end-user debugging of naive Bayes text classification. *ACM Transactions on Interactive Intelligent Systems* (TiiS) 1, 1 (2011), 1–31.

[35] Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. *arXiv preprint arXiv:1910.09700* (2019).

[36] Loïc Lannelongue, Jason Grealey, and Michael Inouye. 2021. Green algorithms: quantifying the carbon footprint of computation. *Advanced science* 8, 12 (2021), 2100707.

[37] Liangde Li, Supun Nakandala, and Arun Kumar. 2021. Intermittent human-in-the-loop model selection using cerebro: a demonstration. *Proceedings of the VLDB Endowment* 14, 12 (2021).

[38] Yuanlong Li, Yonggang Wen, Dacheng Tao, and Kyle Guan. 2019. Transforming cooling optimization for green data center via deep reinforcement learning. *IEEE transactions on cybernetics* 50, 5 (2019), 2002–2013.

[39] Sasha Luccioni, Yacine Jernite, and Emma Strubell. 2024. Power Hungry Processing: Watts Driving the Cost of AI Deployment?. In *Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency* (Rio de Janeiro, Brazil) (FACCT '24). Association for Computing Machinery, New York, NY, USA, 85–99. https://doi.org/10.1145/3630106.3658542

[40] Wendy E Mackay. 2023. DOIT: The Design of Interactive Things. Selected methods for quickly and effectively designing interactive systems from the user's perspective. In *Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems*. 1–3.

[41] Armin Moin, Moharram Challenger, Atta Badii, and Stephan Günnemann. 2022. Supporting AI Engineering on the IoT Edge through Model-Driven TinyML. In *2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMP-SAC)*. 884–893. https://doi.org/10.1109/COMPSAC54236.2022.00140

[42] Rafiullah Omar. 2024. Energy-Efficient Development of ML-Enabled Systems: A Data-Centric Approach. In *Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering-Software Engineering for AI*. 259–263.

[43] Matthew Parry, A Philip Dawid, and Steffen Lauritzen. 2012. Proper local scoring rules. (2012).

[44] Priyavanshi Pathania, Rohit Mehra, Vibhu Saujanya Sharma, Vikrant Kaulgud, Sanjay Podder, and Adam P Burden. 2023. Towards a Knowledge Base of Common Sustainability Weaknesses in Green Software Development. In *2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)*. IEEE, 1718–1720.

[45] Vasileios Perifanis, Nikolaos Pavlidis, Selim F. Yilmaz, Francesc Wilhelm, Elia Guerra, Marco Miozzo, Pavlos S. Efraimidis, Paolo Dini, and Remous-Aris Koutsiamantis. 2023. Towards Energy-Aware Federated Traffic Prediction for Cellular Networks. In *2023 Eighth International Conference on Fog and Mobile Edge Computing (FMEC)*. 93–100. https://doi.org/10.1109/FMEC59375.2023.10306017

[46] Shaolei Ren, Bill Tomlinson, Rebecca W Black, and Andrew W Torrance. 2024. Reconciling the contrasting narratives on the environmental impact of large language models. *Scientific Reports* 14, 1 (2024), 26310.

[47] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should i trust you?" Explaining the predictions of any classifier. In *Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining*. 1135–1144.

[48] David Rolnick, Priya L Donti, Lynn H Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, et al. 2022. Tackling climate change with machine learning. *ACM Computing Surveys (CSUR)* 55, 2 (2022), 1–96.

[49] Soumik Saha, Biswajit Bera, Pravat Kumar Shit, Sumana Bhattacharjee, and Nairita Sengupta. 2023. Prediction of forest fire susceptibility applying machine and deep learning algorithms for conservation priorities of forest resources. *Remote Sensing Applications: Society and Environment* 29 (2023), 100917.

[50] Albrecht Schmidt, Fosca Giannotti, Wendy Mackay, Ben Shneiderman, and Kaisa Väänänen. 2021. Artificial intelligence for humankind: a panel on how to create truly interactive and Human-Centered AI for the benefit of individuals and
Society. In IFIP Conference on Human-Computer Interaction. Springer, 335-339.

[51] Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020. Green ai. Commun. ACM 63, 12 (2020), 54-63.

[52] Tor Sporsem. 2024. Discovering Explainability Requirements in ML-Based Software. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings. 170-172.

[53] Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243 (2019).

[54] Aniththa Umamahesan and Deepak Mukunthu Iyappan Babu. 2020. From zero to AI hero with automated machine learning. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3495-3495.

[55] Valeye. 2022. Tracarbon — Track your device's carbon footprint. https://medium.com/@florian.valeye/tracarbon-track-your-devices-carbon-footprint-fb051fcc9009/.

[56] Aimee Van Wynsberghe. 2021. Sustainable AI: AI for sustainability and the sustainability of AI. AI and Ethics 1, 3 (2021), 213-218.

[57] Paul Walsh, Jhilam Bera, Vibhu Saujanya Sharma, Vikrant Kaulgud, Raghotham M Rao, and Orlaith Ross. 2021. Sustainable AI in the Cloud: Exploring Machine Learning Energy Use in the Cloud. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). 265-266. https://doi.org/10.1109/ASEW52652.2021.00058

[58] Jiayi Wang, Chengliang Chai, Nan Tang, Jiabin Liu, and Guoliang Li. 2022. Core-sets over multiple tables for feature-rich and data-efficient machine learning. Proceedings of the VLDB Endowment 16, 1 (2022), 64-76.

[59] Yair Weiss and William T Freeman. 2007. What makes a good model of natural images? In 2007 IEEE conference on computer vision and pattern recognition. IEEE, 1-8.

[60] Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. 2022. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems 4 (2022), 795-813.

# A APPENDIX

Guiding questions for semi-structured story interview:

## The ML Problem

(1) Can you tell me about the last time you chose an ML model to solve a problem and what you did?

(2) What was the initial problem and who defined the problem? Do you have any notes you keep about it and if yes, can I see them?

(3) What was your goal (with the model) and how clear was it to you? What details did you have about a satisfying solution (...) and who defined them?

(4) How do you specify selection parameters for the problem and if yes, where did they come from?

(5) How to start the process

(6) How did you start looking for an ML model?

(7) How do you express the problem before the selection process, like through diagrams or notes?

(8) Did you already decide on a type of model, like classification, before you started, or where you are open to different model approaches?

(9) Did you conduct any preliminary research or analysis before starting your search for an ML model?

## Searching for ML models

(1) How do you search online for a particular keyword/model and What aspects did you consider when you looked for a model, maybe the data format?

(2) Did you look at similar problems and how they solved them?

(3) What were the main sources or references you consulted to gather information about available ML models?

## Selecting a ML model

(1) And how do you go about actually testing different algorithms to see which one works better? (do you download

them? Do you run web versions? Do you look for other people who have run them)

(2) What makes a model better than another? How do you compare between two models?

(3) What metrics do you use? Parameters: accuracy, Running time, data use, complexity ...? Could you rank them (if there are more than one)?

(4) Do you estimate how long the running phase takes?

(5) How long does it take to decide?

(6) Were there any specific challenges or limitations you faced during your search for an ML model? How did you overcome them?

(7) When can you say that this model satisfies your needs? Did you change your opinion later on?

## Sustainability of ML models

(1) How much do you know about the impact of ML models on sustainability and from where?

(2) What factors are you aware could have an impact on sustainability?

(3) Have you ever considered the sustainability of your work when you select a model? and if yes, which measures have you taken and why?