Miracle-dz/UniFAR-FTUs huggingface dataset:
Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications through Visual Programming

In recent years, there has been a proliferation of multimedia applications that leverage machine learning (ML) for interactive experiences. Prototyping ML-based applications is, however, still challenging, given complex workflows that are not ideal for design and experimentation. To better understand these challenges, we conducted a formative study with seven ML practitioners to gather insights about common ML evaluation workflows. The study helped us derive six design goals, which informed Rapsai1, a visual programming platform for rapid and iterative development of end-to-end ML-based multimedia applications. Rapsai features a node-graph editor to facilitate interactive characterization and visualization of ML model performance. Rapsai streamlines end-to-end prototyping with interactive data augmentation and model comparison capabilities in its no-coding environment. Our evaluation of Rapsai in four real-world case studies (N=15) suggests that practitioners can accelerate their workflow, make more informed decisions, analyze strengths and weaknesses, and holistically evaluate model behavior with real-world input.
Olfactory Wearables for Mobile Targeted Memory Reactivation

This paper investigates how a smartphone-controlled olfactory wearable might improve memory recall. We conducted a within-subjects experiment with 32 participants using the device and without (control). In the experimental condition, bursts of odor were released during visuo-spatial memory navigation tasks, and replayed during sleep the following night in the subjects’ home. We found that compared to control, there was an improvement in memory performance when using the scent wearable in memory tasks that involved walking in a physical space. Furthermore, participants recalled more objects and translations when re-exposed to the same scent during the recall test, in addition to during sleep. These effects were statistically significant, and, in the object recall task, they also persisted for more than one week. This experiment demonstrates a potential practical application of olfactory interfaces that can interact with a user during wake as well as sleep to support memory.
An Evaluation of Other-Avatar Facial Animation Methods for Social VR

We report a mixed-design study on the effect of facial animation method (static, synthesized, or tracked expressions) and its synchronization to speaker audio (in sync or delayed by the method’s inherent latency) on an avatar’s perceived naturalness and plausibility. We created a virtual human for an actor and recorded his spontaneous half-minute responses to conversation prompts. As a simulated immersive interaction, 44 participants unfamiliar with the actor observed and rated performances rendered with the avatar, each with the different facial animation methods. Half of them observed performances in sync and the others with the animation method’s latency. Results show audio synchronization did not influence ratings and static faces were rated less natural and less plausible than animated faces. Notably, synthesized expressions were rated as more natural and more plausible than tracked expressions. Moreover, ratings of verbal behavior naturalness differed in the same way. We discuss implications of these results for avatar-mediated communication.
Face verification for mobile personal devices

In this thesis, we presented a detailed study of the face verification problem on the mobile device, covering every component of the system. The study includes face detection, registration, normalization, and verification. Furthermore, the information fusion problem is studied to verify face sequences, and to fuse different modalities. Although the work is application-specific, the thesis is not limited to the application, but more extensive. In every step, we have justified the methods we choose both from the theoretical and the practical point of view. In the review part of each chapter, we discussed principles and methodologies on a higher level, for a better understanding of the problems in general. In our solutions, on the other hand, we have taken care of the application requirements, and put much emphasis on the efficiency and simplicity of the methods. The system has dealt with the security, convenience, and complexity requirements, which are put forward in Chapter 1. As discussed in Chapters 3 and 4, we verify the input pattern in a high-dimensional space, which is in theory sufficiently discriminative and guarantees security. For the user convenience, a low false rejection rate is essential, and this has been taken care of by the illumination normalization and the decision fusion as discussed in Chapter 5, 6, and 7. The time-sequence verification, furthermore, not only improves security, preventing the scenario that the device is taken away after being logged-on, but also increases the user-convenience by lowering the false rejection rate with the fusion rule. Finally, the algorithmic complexity has always been an important concern of our work, and the system has been successfully implemented on the mobile device with limited computational resources.
Seeking Love and Companionship through Streaming: Unpacking Livestreamer-moderated Senior Matchmaking in China

Livestreamer-moderated matchmaking has gained wide popularity among the elderly population in China. Compared to algorithm-mediated online dating, it is characterized by (1) the mediation of matchmakers in a synchronous virtual environment and (2) the natural development of livestreaming-based matchmaking communities. Nonetheless, how these new features influence single seniors’ match-seeking remains unknown. To fill this research gap, we conduct a qualitative study consisting of observations and semi-structured interviews with 6 livestreaming matchmakers and 12 senior match-seekers (age: 50-70). We uncover matchmakers’ mediation roles during and beyond livestreaming to facilitate seniors’ match-seeking, and their additional duties to enhance seniors’ safety in this process. Livestreaming-based matchmaking communities afford multiple important values for single seniors to acquire companionship and help in seeking late-life love. We unpack the perceived benefits and challenges of livestreamer-moderated matchmaking, and discuss how to support single seniors’ match-seeking in an accessible, safe, and convenient manner.
Investigating the Synonyms of Conversational Agents to Aid Cross-Disciplinary CA Research

With the advent of artificial intelligence, the CHI community has regained a large interest in Conversational Agents (CAs). Designing CAs involves cross-disciplinary efforts, such as that of computer science (e.g., language models), psychology (e.g., cognition and emotions), linguistics (e.g., conversation design), or communication (e.g., trust, communication theory). However, CAs are named differently depending on the discipline and purpose (e.g., chatbot, embodied avatar, virtual butler). This divergence of vocabulary on CA brings challenges to researchers and designers in researching the full landscape of CA literature and sharing cross-disciplinary knowledge. We performed bibliometric and qualitative analyses to systematically assess divergent terms used for CA nomenclatures. We present 54 CA-terms and how these terms are used differently depending on the discipline and design characteristics (e.g., voice vs. non-verbal vs. text-based). Our work contributes to helping CA researchers effectively review the CA literature and build on each other’s work for novel cross-disciplinary CA research.
Playing with Power Tools: Design Toolkits and the Framing of Equity

Design toolkits that aim to to promote equity offer designers simplified approaches to creating more equitable technology. However, it is important to understand how equity is conceptualized in practice. As a curated collection of methods, toolkits signal how equity is imagined in design. In this paper, we perform a qualitative analysis of 17 design toolkits related to equity. We explore alternative design approaches that address inequity in design. We evaluate whether equity toolkits align with calls for changes to design practice, as well as Nancy Fraser’s dimensions of justice. Finally, we find that design toolkits focus on the ‘digital divide’ rather than redistributing world-building power, and thus continue to keep design power with professional designers. We also find that ‘design thinking’ continues to influence design toolkits. Furthermore, the simplicity of toolkits does not engage with the complexities that shape equity in practice. We conclude with suggestions to help researchers and designers rethink design toolkits.
Improving Environmental Knowledge with a Serious Game: An experimental study

Climate change mitigation is the century’s obligation to humanity. People all throughout the world have too little knowledge and comprehension of the planet’s ecosystem. Increasing people’s awareness and knowledge about the environment might help reduce environmental-hostile behavior. We propose a serious game that helps players to learn more about the environment in a fun and engaging manner. The game’s effectiveness in teaching players about the environment is evaluated, and its role as a mediation tool is estimated by measuring environmental knowledge and consciousness using literature-based questionnaires before and after the game is played. Our results show significant differences in participants’ environmental knowledge. Differences in environmental consciousness are not significant. Strengths and weaknesses of the developed serious game based on user feedback are identified to improve future iterations.
AutomataStage: an AR-mediated Creativity Support Tool for Hands-on Multidisciplinary Learning

The creativity support tools can enhance the hands-on multidisciplinary learning experience by drawing interest in the process of creating the outcome. We present AutomataStage, an AR-mediated creativity support tool for hands-on multidisciplinary learning. AutomataStage utilizes a video see-through interface to support the creation of Interactive Automata. The combination of building blocks and low-cost materials increases the expressiveness. The generative design method and one-to-one guide support the idea development process. It also provides a hardware see-through feature with which inside parts and circuits can be seen and an operational see-through feature that shows the operation in real-time. The visual programming method with a state transition diagram supports the iterative process during the creation process. A user study shows that AutomataStage enabled the students to create diverse Interactive Automata within 40-minute sessions. By creating Interactive Automata, the participants could learn the basic concepts of components. See-through features allowed active exploration with interest while integrating the components. We discuss the implications of hands-on tools with interactive and kinetic content beyond multidisciplinary learning.
VisaudiBlow: Fine-grained Blowing Interaction Based on Visual and Auditory Detection

More and more studies regard blowing as a means of reflecting the user’s physiological conditions. Its potential as an independent input channel has not been fully explored. In this paper, we analyze the design space of blowing as an independent interactive channel and propose a novel blowing recognition approach, VisaudiBlow, based on visual and audio detection. VisaudiBlow uses Facial Tracker to capture the mouth expression when doing specific blowing actions, and audio information generated by blowing is obtained by a microphone beside the mouth, enabling identification and detection of the input parameters such as blowing type, direction, and intensity. In order to verify the validity of blowing as an independent input channel, a prototype system BlowUI is implemented. The experimental results demonstrate that blow interaction can effectively complete the menu control tasks and achieve the same interaction efficiency as controllers in specific scenarios.
4DCurve: A Shape-Changing Fabrication Method Based on Curved Paths with a 3D Printing Pen

In recent years, 4D printing is of continuous interest and attention in the field of Human-Computer Interaction. However, the advanced settings and complex g-code controls are still raising barriers for newcomers. 3D printing pen, in contrast, provides maximum versatility and accessibility for novices and even children. In this paper, we propose 4DCurve as a low-cost and easy-accessible 4D printing method to create curves with 3D printing pens. Compared with the strictly flat and layer-by-layer printing strategy of machines, the manual use of 3D printing pens enables freely curved and non-planar printing paths for spatial creativity and constant combination with daily objects. In addition, a guidance system has been built up for proposing curve path guidance based on the users' deformation needs, we also demonstrate various application cases on explosive design space for fashion designers, craft enthusiasts and designers in the field of personal fabrication.
How Older Adults Use Online Videos for Learning

Online videos are a promising medium for older adults to learn. Yet, few studies have investigated what, how, and why they learn through online videos. In this study, we investigated older adults’ motivation, watching patterns, and difficulties in using online videos for learning by (1) running interviews with 13 older adults and (2) analyzing large-scale video event logs (N=41.8M) from a Korean Massive Online Open Course (MOOC) platform. Our results show that older adults (1) are motivated to learn practical topics, leading to less consumption of STEM domains than non-older adults, (2) watch videos with less interaction and watch a larger portion of a single video compared to non-older adults, and (3) face various difficulties (e.g., inconvenience arisen due to their unfamiliarity with technologies) that limit their learning through online videos. Based on the findings, we propose design guidelines for online videos and platforms targeted to support older adults’ learning.
TitleIX: Step Up & Step In! A Mobile Augmented Reality Game Featuring Interactive Embodied Conversational Agents for Sexual Assault Bystander Intervention Training on US College Campuses

Despite the efforts of existing Title IX training programs in the US, current intervention and prevention programs fail to address the problems caused by sexual violence on US college campuses. To address this issue, we designed a mobile augmented reality (AR) game – TitleIX: Step Up & Step In! – that encourages students to become more active and supportive of bystanders through innovative game play, while aiming to improve current sexual assault bystander intervention training. Utilizing AR technology and embodied conversational agents (ECAs), this game provides highly immersive scenario based training for sexual assault bystander intervention while connecting the users to realistic campus experiences. This inventive game design leverages innovative technology to increase awareness of real-world problems; specifically, sexual harassment targeting women and LGBTQ+ students on college campuses. The design implemented in this paper can inform the future construction of AR serious games for social justice.
Motivations for Collecting Digital NFT Fashion: A Uses and Gratifications Theory Approach for Understanding Digital NFT Fashion Collectors’ Behavior

Non-fungible tokens (NFTs) allow individuals to demonstrate ownership of digital and physical assets. NFTs are scarce, unique, and authentic; three properties known to be key for determining perceived value. Whilst previous research has primarily focused on NFTs as a source of economic value, here we assess the psychological motivations of collectors of digital fashion NFTs. Specifically, NTFs related to digital fashion are particularly relevant to HCI researchers as they sit at the intersection between business, culture, and self-expression. Here, we survey 19 users of a digital avatar fashion company, Genies, to understand the gratifications users derive from collecting digital NFT fashion. Results demonstrate that the primary motivations for collecting fashion NFTs are self-expression and utility and that motivations associated with value are secondary. We make design recommendations based on these results, indicating that developers should distinguish between expression-based motivations and value-based motivations.
Be Our Guest: Intercultural Heritage Exchange through Augmented Reality (AR)

This paper explores how interactive applications can help mitigate the adversity of facing cultural differences between migrants and the host community, and between migrants of diverse backgrounds to foster intercultural exchange. Based on literature about situated cognition, immersive theater, and affordance, we designed and built Be Our Guest: an augmented reality application where a user is invited to the houses of people from different cultures and is asked to help with one of their cultural rituals around simple everyday objects. We detail the various phases we took to collect the cultural stories and construct the application. We then report the results of a user study with the developed application. Our findings show that participants were easily immersed in the augmented space due to the app’s narrative, visuals, and interactive nature. Moreover, they enjoyed exploring cultural rituals, including their own, and felt more confident connecting with people from other cultures.
Fine-grained classification network for fish based on the attention mechanism and EfficientNet

Many countries have been interested in the sustainable development of marine resources. Countries can protect the sustainable development of fishery resources by joining international fisheries organizations such as CCSBT, IATTC, IOTC, ICCAT, and others, and by adhering to the Code of Conduct for Responsible Fisheries. Traditional fishing supervision methods, on the other hand, frequently have the issue that the amount of original data is cumbersome and expensive, which cannot be handled manually and wastes a lot of manpower. This paper investigates fine-grained fish classification methods based on deep learning and attention mechanisms and proposes one that can target complex background and actual working scenarios, allowing countries to reallocate human capital to management and law enforcement activities.
The Application of Audio Control in Social Robotics

Along with the development of the voice recognition technology, voice control plays an increasingly important role in daily life as it offers an intuitive interface that enables hands-free control of various devices, as a result blurring the sense of boundaries between human and robot. The rising demand for voice control in daily life has motivated this paper to develop a voice control system using machine learning models for the robot model LoCoBot WX250. In conjunction with robot hardware, this trained audio control system achieves the goal of improving the experience of interaction between human and robot. Based on this system, the LoCoBot WX250 can serve as a tour guide in museums by communicating with tourists via speaker and microphone. What's more, while talking to people, the robot can respond to certain instructions, such as navigation, change position and so on. This audio control system improves upon all previous similar projects in terms of the accuracy of intent classification and keyword recognition by implementing the state-of-the-art BERT (Bidirectional Encoder Representations from Transformers) model, so that it can have a smoother conversation with human. Furthermore, to accommodate the foreign visitors in the museum and tourist spot, the audio control system can be instructed to change the language, making it even more accommodating than a human tourist guide.
Investigating Deceptive Design in GDPR’s Legitimate Interest

Legitimate interest is one of the six grounds for processing data under the European Union’s General Data Protection Regulation (GDPR). The flexibility and ambiguity of the term "legitimate interests" can be problematic; coupled with the lack of enforcement from legal authorities and different interpretations from the various data protection authorities, legitimate interests can be taken advantage of as a loophole to collect more user data. Drawing insights from multiple disciplines, we ran two studies to empirically investigate the deceptive designs being used when legitimate interests are applied in privacy notices, and how user perceptions line up with these practices. We identified six deceptive designs, and found that the ways legitimate interest is applied in practice does not match user expectations.
Pencil-E: Crafting Functional Electronics Using Pencils and Paper

We present Pencil-E, a low-cost, ubiquitous, and accessible sensing and actuating method using pencils and paper for personal fabrication. The basic architectures of this method are the principal electrical features of the pencil as a conductive material and the properties of the paper as a carrier of pencil traces. By combining paper craft methods, including single-layer, multi-layer, and constructive structures, we developed resistive and capacitive sensors with haptic feedback and allowed customization. This paper describes the design guidelines, sensing mechanics, primitive constructions, and the fabrication process. Multiple pre-designed elements and application scenarios are demonstrated, including tangible user interfaces, wearable devices, and interactive art crafts.
Research on Intelligent Guidance Method for Vehicles in High-Speed Railway Station Based on GNSS Indoor Positioning

With the increasing complexity of the environment in high-speed railway stations and the growing demand for in-station navigation and location services, it is critical to investigate an accurate and dependable intelligent guidance system for cars in the in-station network. This paper intends to use GNSS indoor satellite base station positioning system and an indoor navigation path planning method based on improved A* algorithm to realize mutual intelligent guidance between passengers in the station and the network car by analyzing the technical status and existing problems of indoor positioning and path planning, combined with the actual situation in high-speed railway station. To begin, the indoor satellite positioning system is deployed, and satellite analog signals are broadcast to provide positioning services for general navigation and positioning terminals, resulting in accurate station positioning. The search efficiency of the A* algorithm is then improved by optimizing the search strategy and heuristic function of the traditional A* algorithm. The path length is reduced by optimizing redundant nodes. The actual verification shows that the improved A* algorithm has a 5% shorter path length than the traditional A* algorithm. When compared to the traditional A* algorithm, the improved A* algorithm can save more than 60% of the planning time. Finally, network car driver and passenger services will be provided to guide passengers and drivers to the best possible position to board the bus quickly and intelligently.