Reviewerly/exHarmony huggingface dataset:
EvoRecipes: A Generative Approach for Evolving Context-Aware Recipes

Generative AI e.g. Large Language Models (LLMs) can be used to generate new recipes. However, LLMs struggle with more complex aspects like recipe semantics and process comprehension. Furthermore, LLMs have limited ability to account for user preferences since they are based on statistical patterns. As a result, these recipes may be invalid. Evolutionary algorithms inspired by the process of natural selection are optimization algorithms that use stochastic operators to generate new solutions. These algorithms can generate large number of solutions from the set of possible solution space. Moreover, these algorithms have the capability to incorporate user preferences in fitness function to generate novel recipes that are more aligned with the fitness objective. In this paper, we propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EvoRecipes</i> framework to generate novel recipes. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EvoRecipes</i> framework utilizes both Genetic Algorithm and generative AI in addition to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RecipeOn</i> ontology, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RecipeKG</i> knowledge graph. Genetic Algorithm explore the large solution space of encoded recipe solutions and are capable of incorporating user preferences, while LLMs are used to generate recipe text from encoded recipe solutions. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EvoRecipes</i> uses a population of context-aware recipe solutions from the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RecipeKG</i> knowledge graph. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RecipeKG</i> encodes recipes in RDF format using classes and properties as defined in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RecipeOn</i> ontology. Moreover, to evaluate the alignment of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EvoRecipe</i> generated recipes with multiple intended objectives, we propose a fitness function that incorporates novelty, simplicity, visual appeal, and feasibility. Additionally, to evaluate the quality of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EvoRecipe</i> generated recipes while considering the subjective nature of recipes, we conducted a survey using multi-dimensional metrics (i.e. contextual, procedural, and novelty). Results show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EvoRecipes</i> generated recipes are novel, valid and incorporate user preferences.
FedDeepFM: A Factorization Machine-Based Neural Network for Recommendation in Federated Learning

Recommender systems are an effective way to address the information overload problem. Due to the rapid development of deep learning, recommender systems based on deep learning can efficiently process massive training samples and integrate a variety of additional information, which can alleviate the inherent data sparsity and cold start problems of recommender systems. However, deep learning recommender systems need to collect massive amounts of user data for training models, which poses challenges for data security and user privacy protection. By using users’ local, not centrally collected, data, federated learning can collaboratively train deep learning recommendation models and improve the accuracy of recommendations without compromising user privacy. In this paper, we propose FedDeepFM, a deep recommender system model based on federated learning. First, FedDeepFM follows the standard paradigm of federated learning by uploading each client’s model parameters instead of the original data for model updates, according to the data processing inequality, which ensures that the server cannot collect users’ private information directly but at the same time is able to use the private data held by each client to jointly train a global model with generalization capabilities. Second, to prevent the server from learning users’ private information through indirect inference methods, a pseudo-interaction filling method is used to expand the client data to mask the real user data. The experimental results show that FedDeepFM, the deep recommender system model based on federated learning proposed in this paper, can provide high-quality recommendations while enhancing users’ privacy.
An Optimized Multi-Organ Cancer Cells Segmentation for Histopathological Images Based on CBAM-Residual U-Net

In digital pathology, the accurate segmentation of cell nuclei in histopathology images is essential for medical image analysis. Histopathologists visually evaluate the patterns of cellular architecture and tissue patterns in histopathology image analysis for cancer detection to determine the malignant tissue portions and assess the severity of malignancy. However, manually analyzing scans using a high-resolution microscope requires significant effort and time. A computer-assisted diagnosis system utilizing deep learning (DL) algorithms rapidly, reliably, and automatically segments cell nuclei. However, the existing research studies have limited accuracy, high computational costs, and a lack of robustness and generalizability on diverse datasets. To address these issues, this paper proposes a novel and improved DL architecture based on the U-Net, namely, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CBAM-Residual U-Net</i> for improving accuracy, robustness, and generalized segmentation algorithm that can be applied to various staining techniques and tissue structures. The proposed architecture utilizes a ResConv and convolution block attention modules (CBAM). These modules help the proposed architecture learn the image’s shallow and deep features. The CBAM module uses an attention mechanism concentrating on essential features such as cell nuclei’s shape, texture, and intensity to accurately segment the raw input patterns. The proposed CBAM-Residual U-Net involves fewer trainable parameters, reducing the computational and time costs compared to state-of-the-art techniques. Extensive experiments and comprehensive evaluations are conducted to demonstrate the performance of the proposed scheme on publicly available datasets: i) Data Science Bowl (DSB) 2018, ii) The GlaS, iii) Triple-Negative Breast Cancer (TNBC). The experimental results show that our proposed model considerably outperforms the state-of-the-art techniques and detects cellular boundaries well, providing fine-grained segmentation results.
Chatbots as Advisers: the Effects of Response Variability and Reply Suggestion Buttons

As chatbots gain popularity across a variety of applications, from investment to health, they employ an increasing number of features that can influence the perception of the system. Since chatbots often provide advice or guidance, we ask: do these aspects affect the user’s decision to follow their advice? We focus on two chatbot features that can influence user perception: 1) response variability in answers and delays and 2) reply suggestion buttons. We report on a between-subject study where participants made investment decisions on a simulated social trading platform by interacting with a chatbot providing advice. Performance-based study incentives made the consequences of following the advice tangible to participants. We measured how often and to what extent participants followed the chatbot’s advice compared to an alternative source of information. Results indicate that both response variability and reply suggestion buttons significantly increased the inclination to follow the advice of the chatbot.
“A Painless Way to Learn:” Designing an Interactive Storytelling Voice User Interface to Engage Older Adults in Informal Health Information Learning

We present “Mystery Agent,” an interactive storytelling voice user interface (VUI) equipped with self-regulated learning strategies to deliver informal health-related learning to older adults through a murder mystery story. We conducted a mixed methods user study with 10 older adults to evaluate Mystery Agent, using usability and perception-based questionnaires, followed by a semi-structured interview and co-design activity to generate design insights and identify design priorities. We found older adults had a positive experience interacting with Mystery Agent and considered storytelling to be an appropriate and engaging way to learn health information. However, older adults identified credibility, compassion, and control as crucial factors influencing long-term use. To address this, we present design guidelines using Mystery Agent as an example to help practitioners and researchers devise novel solutions to address the informal health information learning needs of older adults.
Trends, Challenges and Processes in Conversational Agent Design: Exploring Practitioners’ Views through Semi-Structured Interviews

The aim of this study is to explore the challenges and experiences of conversational agent (CA) practitioners in order to highlight their practical needs and bring them into consideration within the scholarly sphere. A range of data scientists, conversational designers, executive managers and researchers shared their opinions and experiences through semi-structured interviews. They were asked about emerging trends, the challenges they face, and the design processes they follow when creating CAs. In terms of trends, findings included mixed feelings regarding no-code solutions and a desire for a separation of roles. The challenges mentioned included a lack of socio-technical tools and conversational archetypes. Finally, practitioners followed different design processes and did not use the design processes described in the academic literature. These findings were analyzed to establish links between practitioners' insights and discussions in related literature. The goal of this analysis is to highlight research-practice gaps by synthesising five practitioner needs that are not currently being met. By highlighting these research-practice gaps and foregrounding the challenges and experiences of CA practitioners, we can begin to understand the extent to which emerging literature is influencing industrial settings and where more research is needed to better support CA practitioners in their work.
Facilitating Mindful Eating with a Voice Assistant

In our busy lives, meals are being squeezed into shorter times, or fit into multitasking routines of watching videos and scrolling through social media. This can lead to eating quickly without considering the food choices and overeating. Mindful eating has been promoted as an effective method to combat mindless eating with various tools available to facilitate mindfulness practices. However, the use of voice assistants to facilitate mindful eating is rather limited. We developed a voice assistant to facilitate mindful eating activities and conducted a field study with four participants over six days. We examined how the social role of the voice assistant, acting as a friend or a counselor, affects mindful eating experiences. The results suggest that voice assistants can assist users in mindful eating, however, participants preferred the friend version of the voice assistant. Implications for the design of voice assistants for mindfulness activities are provided.
Voicing Suggestions and Enabling Reflection: Results of an Expert Discussion on Proactive Assistants for Time Management

While voice-controllable Intelligent Personal Assistants (IPAs) have become widespread in recent years, they remain primarily reactive with rather constrained calendaring capabilities. Anticipating more adaptive and complex assistants in the future, we organised a multidisciplinary expert discussion investigating potential use cases, interaction principles, and user modelling challenges within proactive IPAs for time management. This paper presents the identified themes and deliberations on enticing self-reflection, longitudinal task assistance, interaction modality, dialogue design, perception of the system, usage willingness, onboarding, and explainability. These findings outline a framework of advanced IPAs for time management.
Understanding and Answering Incomplete Questions

Voice assistants interrupt people when they pause mid-question, a frustrating interaction that requires the full repetition of the entire question again. This impacts all users, but particularly people with cognitive impairments. In human-human conversation, these situations are recovered naturally as people understand the words that were uttered. In this paper we build answer pipelines which parse incomplete questions and repair them following human recovery strategies. We evaluated these pipelines on our new corpus, SLUICE. It contains 21,000 interrupted questions, from LC-QuAD 2.0 and QALD-9-plus, paired with their underspecified SPARQL queries. Compared to a system that is given the full question, our best partial understanding pipeline answered only 0.77% fewer questions. Results show that our pipeline correctly identifies what information is required to provide an answer but is not yet provided by the incomplete question. It also accurately identifies where that missing information belongs in the semantic structure of the question.
Moral Transparency as a Mitigator of Moral Bias in Conversational User Interfaces

From straightforward interactions to full-fledged open-ended dialogues, Conversational User Interfaces (CUIs) are designed to support end-user goals and follow their requests. As CUIs become more capable, investigating how to restrict or limit their ability to carry out user requests becomes increasingly critical. Currently, such intentionally constrained user interactions are accompanied by a generic explanation (e.g., "I'm sorry, but as an AI language model, I cannot say..."). We describe the role of moral bias in such user restrictions as a potential source of conflict between CUI users' autonomy and system characterisation as generated by CUI designers. Just as the users of CUIs have diverging moral viewpoints, so do CUI designers—which either intentionally or unintentionally affects how CUIs communicate. Mitigating user moral biases and making the moral viewpoints of CUI designers apparent is a critical path forward in CUI design. We describe how moral transparency in CUIs can support this goal, as exemplified through intelligent disobedience. Finally, we discuss the risks and rewards of moral transparency in CUIs and outline research opportunities to inform the design of future CUIs.
Conversational User Interfaces in Smart Homecare Interactions: A Conversation Analytic Case Study

Policymakers are increasingly interested in using virtual assistants to augment social care services in the context of a demographic ageing crisis. At the same time, technology companies are marketing conversational user interfaces (CUIs) and smart home systems as assistive technologies for elderly and disabled people. However, we know relatively little about how today’s commercially available CUIs are used to assist in everyday homecare activities, or how care service users and human care assistants interpret and adapt these technologies in practice. Here we report on a longitudinal conversation analytic case study to identify, describe, and share how CUIs can be used as assistive conversational agents in practice. The analysis reveals that, while CUIs can augment and support new capabilities in a homecare environment, they cannot replace the delicate interactional work of human care assistants. We argue that CUI design is best inspired and underpinned by a better understanding of the joint coordination of homecare activities
From Writing Dialogue to Designing Conversation: Considering the potential of Conversation Analysis for Voice User Interfaces

Conversation design at least partly aspires to create Voice User Interfaces which emulate human speech production. And yet, there is no established approach for the development of naturalistic conversational infrastructure for VUIs; conversation designers are advised to work from their common sense understanding of conversation, producing written scripts, based on memory and imagination, which are later converted into speech. This is a shortcoming in conversation design which needs to be addressed. In this provocation paper, we argue that the starting point in the development of any VUI should be the examination of natural spoken conversation, preferably from the same interactional context in which the VUI will be deployed. We provide a short example to illustrate how the current process of conversation scriptwriting can be a barrier to this, and demonstrate how this can be overcome using the social scientific approach of Conversation Analysis (CA).
The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents

Conversational agents mimic natural conversation to interact with users. Since the effectiveness of interactions strongly depends on users’ perception of agents, it is crucial to design agents’ behaviors to provide the intended user perceptions. Research on human-agent and human-human communication suggests that speech specifics are associated with perceptions of communicating parties, but there is a lack of systematic understanding of how speech specifics of agents affect users’ perceptions. To address this gap, we present a framework outlining the relationships between elements of agents’ conversation architecture (dialog strategy, content affectiveness, content style and speech format) and aspects of users’ perception (interaction, ability, sociability and humanness). Synthesized based on literature reviewed from the domains of HCI, NLP and linguistics (n=57), this framework demonstrates both the identified relationships and the areas lacking empirical evidence. We discuss the implications of the framework for conversation design and highlight the inconsistencies with terminology and measurements.
Are you sure you want to order that?

Conversational agents (CAs) that deliver proactive interventions can benefit users by reducing their cognitive workload and improving performance. However, little is known regarding how such interventions would impact perception of CA’s appropriateness in voice-only, decision-making tasks. We conducted a within-subjects experiment (N=30) to evaluate the effect of CA’s feedback delivery strategy at three levels (no feedback, unsolicited, and solicited feedback) in an interactive food ordering scenario. We discovered that unsolicited feedback was perceived to be more appropriate than solicited feedback. Our results provide preliminary insights regarding the impact of proactive feedback on CA perception in decision-making tasks.
Towards Cross-Content Conversational Agents for Behaviour Change: Investigating Domain Independence and the Role of Lexical Features in Written Language Around Change

Valuable insights into an individual's current thoughts and stance regarding behaviour change can be obtained by analysing the language they use, which can be conceptualized using Motivational Interviewing concepts. Training conversational agents (CAs) to detect and employ these concepts could help them provide more personalized and effective assistance. This study investigates the similarity of written language around behaviour change spanning diverse conversational and social contexts and change objectives. Drawing on previous research that applied MI concepts to texts about health behaviour change, we evaluate the performance of existing classifiers on six newly constructed datasets from diverse contexts. To gain insights in determining factors when identifying change language, we explore the impact of lexical features on classification. The results suggest that patterns of change language remain stable across contexts and domains, leading us to conclude that peer-to-peer online data may be sufficient to train CAs to understand user utterances related to behaviour change.
RoboClean: Contextual Language Grounding for Human-Robot Interactions in Specialised Low-Resource Environments

Building effective voice interfaces for the instruction of service robots in specialised environments is difficult due to the local knowledge of workers, such as specific terminology for objects and space, leading to limited data to train language models (known as 'low-resource' domains) and challenges in language grounding. We present a language grounding study in which we a) elicit spoken natural language of context experts in situ through a Wizard of Oz study and compile a dataset, b) qualitatively examine linguistic properties of the resulting instructions to reveal referential categories and parameters employed to construct instructions in context. We discuss how our language grounding protocol may be applied to bootstrap a language model in its targeted use context. Our work contributes a linguistic understanding of robot instructions that can be applied by designers and researchers to develop spoken language understanding for human-robot interactions in specialised, low-resource environments.
Voice Assistant Use in Long-Term Care

Research on voice assistants has primarily studied how people use them for informational needs, music requests, and to control electronic devices (e.g., IoT). Recent research suggests people such as older adults want to use them to address social and relational needs, but lacks empirical evidence to show how older adults are currently engaging in these behaviors. In this paper, we use a machine learning approach to analyze more than 600,000 queries that 456 older adults in assisted living communities made to Amazon Alexa devices over two years, classifying how older adults use voice assistants for social well-being purposes. We present empirical evidence showing how older adults engage in three primary relational behaviors with Alexa – 1) asking personal questions to "get to know" the assistant, 2) asking for advice and 3) engaging with the voice assistant to alleviate stress. We use these findings to discuss ethical implications of voice assistant use in long-term care settings.