Reviewerly/exHarmony huggingface dataset:
SUPREYES: SUPer Resolutin for EYES Using Implicit Neural Representation Learning

We introduce SUPREYES – a novel self-supervised method to increase the spatio-temporal resolution of gaze data recorded using low(er)-resolution eye trackers. Despite continuing advances in eye tracking technology, the vast majority of current eye trackers – particularly mobile ones and those integrated into mobile devices – suffer from low-resolution gaze data, thus fundamentally limiting their practical usefulness. SUPREYES learns a continuous implicit neural representation from low-resolution gaze data to up-sample the gaze data to arbitrary resolutions. We compare our method with commonly used interpolation methods on arbitrary scale super-resolution and demonstrate that SUPREYES outperforms these baselines by a significant margin. We also test on the sample downstream task of gaze-based user identification and show that our method improves the performance of original low-resolution gaze data and outperforms other baselines. These results are promising as they open up a new direction for increasing eye tracking fidelity as well as enabling new gaze-based applications without the need for new eye tracking equipment.
Storyfier: Exploring Vocabulary Learning Support with Text Generation Models

Vocabulary learning support tools have widely exploited existing materials, e.g., stories or video clips, as contexts to help users memorize each target word. However, these tools could not provide a coherent context for any target words of learners' interests, and they seldom help practice word usage. In this paper, we work with teachers and students to iteratively develop Storyfier, which leverages text generation models to enable learners to read a generated story that covers any target words, conduct a story cloze test, and use these words to write a new story with adaptive AI assistance. Our within-subjects study (N=28) shows that learners generally favor the generated stories for connecting target words and writing assistance for easing their learning workload. However, in the read-cloze-write learning sessions, participants using Storyfier perform worse in recalling and using target words than learning with a baseline tool without our AI features. We discuss insights into supporting learning tasks with generative models.
Hypothesizer: A Hypothesis-Based Debugger to Find and Test Debugging Hypotheses

When software defects occur, developers begin the debugging process by formulating hypotheses to explain the cause. These hypotheses guide the investigation process, determining which evidence developers gather to accept or reject the hypothesis, such as parts of the code and program state developers examine. However, existing debugging techniques do not offer support in finding relevant hypotheses, leading to wasted time testing hypotheses and examining code that ultimately does not lead to a fix. To address this issue, we introduce a new type of debugging tool, the hypothesis-based debugger, and an implementation of this tool in Hypothesizer. Hypothesis-based debuggers support developers from the beginning of the debugging process by finding relevant hypotheses until the defect is fixed. To debug using Hypothesizer, developers first demonstrate the defect, generating a recording of the program behavior with code execution, user interface events, network communications, and user interface changes. Based on this information and the developer's descriptions of the symptoms, Hypothesizer finds relevant hypotheses, analyzes the code to identify relevant evidence to test the hypothesis, and generates an investigation plan through a timeline view. This summarizes all evidence items related to the hypothesis, indicates whether the hypothesis is likely to be true by showing which evidence items were confirmed in the recording, and enables the developer to quickly check evidence in the recording by viewing code snippets for each evidence item. A randomized controlled experiment with 16 professional developers found that, compared to traditional debugging tools and techniques such as breakpoint debuggers and Stack Overflow, Hypothesizer dramatically improved the success rate of fixing defects by a factor of five and decreased the time to debug by a factor of three.
Color Field: Developing Professional Vision by Visualizing the Effects of Color Filters

Color filters are ubiquitous across visual digital media due to their transformative effect. However, it can be difficult to understand how a color filter will affect an image, especially for novices. In order to become experts, we argue that novices need to develop Goodwin's notion of Professional Vision [29]. Then, they can "see" and interpret their work in terms of their domain knowledge like experts. Using the theory of Professional Vision, we present two design objectives for systems that aim to help users develop expertise. These goals were used to develop Color Field, an interactive visualization of color filters as a vector field over the Hue-Saturation-Lightness color space. We conducted an exploratory user study in which five color grading novices and four experts were asked to analyze color filters. We found that Color Field enabled multiple strategies to make sense of filters (e.g. reviewing the overall shape of the vector field) and discuss them (e.g. using spatial language). We conclude with other applications of Color Field and future work to leverages Professional Vision in HCI.
KnitScript: A Domain-Specific Scripting Language for Advanced Machine Knitting

Knitting machines can fabricate complex fabric structures using robust industrial fabrication machines. However, machine knitting's full capabilities are only available through low-level programming languages that operate on individual machine operations. We present KnitScript, a domain-specific machine knitting scripting language that supports computationally driven knitting designs. KnitScript provides a comprehensive virtual model of knitting machines, giving access to machine-level capabilities as they are needed while automating a variety of tedious and error-prone details. Programmers can extend KnitScript with Python programs to create more complex programs and user interfaces. We evaluate the expressivity of KnitScript through a user study where nine machine knitters used KnitScript code to modify knitting patterns. We demonstrate the capabilities of KnitScript through three demonstrations where we create: a program for generating knitted figures of randomized trees, a parameterized hat template that can be modified with accessibility features, and a pattern for a parametric mixed-material lampshade. KnitScript advances the state of machine-knitting research by providing a platform to develop and share complex knitting algorithms, design tools, and patterns. 1
Data2MV - A user behaviour dataset for multi-view scenarios

The Data2MV dataset contains gaze fixation data obtained through experimental procedures from a total of 45 participants using an Intel RealSense F200 camera module and seven different video playlists. Each of the playlists had an approximate duration of 20 minutes and was viewed at least 17 times, with raw tracking data being recorded with a 0.05 second interval. The Data2MV dataset encompasses a total of 1.000.845 gaze fixations, gathered across a total of 128 experiments. It is also composed of 68.393 image frames, extracted from each of the 6 videos selected for these experiments, and an equal quantity of saliency maps, generated from aggregate fixation data. Software tools to obtain saliency maps and generate complementary plots are also provided as an open-source software package. The Data2MV dataset was publicly released to the research community on Mendeley Data and constitutes an important contribution to reduce the current scarcity of such data, particularly in immersive, multi-view streaming scenarios.
AR-Enhanced Workouts: Exploring Visual Cues for At-Home Workout Videos in AR Environment

In recent years, with growing health consciousness, at-home workout has become increasingly popular for its convenience and safety. Most people choose to follow video guidance during exercising. However, our preliminary study revealed that fitness-minded people face challenges when watching exercise videos on handheld devices or fixed monitors, such as limited movement comprehension due to static camera angles and insufficient feedback. To address these issues, we reviewed popular workout videos, identified user requirements, and came up with an augmented reality (AR) solution. Following a user-centered iterative design process, we proposed a design space of AR visual cues for workouts and implemented an AR-based application. Specifically, we captured users' exercise performance with pose-tracking technology and provided feedback via AR visual cues. Two user experiments showed that incorporating AR visual cues could improve movement comprehension and enable users to adjust their movements based on real-time feedback. Finally, we presented several suggestions to inspire future design and apply AR visual cues to sports training.
Ubi-TOUCH: Ubiquitous Tangible Object Utilization through Consistent Hand-object interaction in Augmented Reality

Utilizing everyday objects as tangible proxies for Augmented Reality (AR) provides users with haptic feedback while interacting with virtual objects. Yet, existing methods focus on the attributes of the objects, constraining the possible proxies and yielding inconsistency in user experience. Therefore, we propose Ubi-TOUCH, an AR system that assists users in seeking a wider range of tangible proxies for AR applications based on the hand-object interaction (HOI) they desire. Given the target interaction with a virtual object, the system scans the users' vicinity and recommends object proxies with similar interactions. Upon user selection, the system simultaneously tracks and maps users' physical HOI to the virtual HOI, adaptively optimizing object 6 DoF and the hand gesture to provide consistency between the interactions. We showcase promising use cases of Ubi-TOUCH, such as remote tutorials, AR gaming, and Smart Home control. Finally, we evaluate the performance and usability of Ubi-TOUCH with a user study.
Learning Custom Experience Ontologies via Embedding-based Feedback Loops

Organizations increasingly rely on behavioral analytics tools like Google Analytics to monitor their digital experiences. Making sense of the data these tools capture, however, requires manual event tagging and filtering — often a tedious process. Prior approaches have trained machine learning models to automatically tag interaction data, but draw from fixed digital experience vocabularies which cannot be easily augmented or customized. This paper introduces a novel machine learning interaction pattern that generates customized tag predictions for organizations. The approach employs a general user experience word embedding to bootstrap an initial set of predictions, which can then be refined and customized by users to adapt the underlying vector space, iteratively improving the quality of future predictions. The paper presents a needfinding study that grounds the design choices of the system, and describes a real-world deployment as part of UserTesting.com that demonstrates the efficacy of the approach.
Theta-mechanism based cluster search algorithm for global constrained optimization

This study concerns constructing an evolutionary search system to solve the global constrained optimization problems. Firstly, we proposed a hybrid constraint-handling method, called theta-mechanism, which blends two types of constraint-handling functions and alternates use of them in the searching process to balance two competing objectives: seeking as much as possible feasible regions and quickly converging to the optimum point in the found feasible regions. Secondly, to enable the search system to cooperate well with the theta-mechanism, we designed the cluster search algorithm (CSA) and developed the search reachability analysis (SRA) method. Based on SRA, we evaluated the characteristics of several typical search operators in order to assemble them into different operator combinations in CSA to maximize its performance, which enables CSA with theta-mechanism to accomplish the two inconsistent search objectives effectively. We tested the proposed method on 18 benchmark functions from IEEE CEC2010 and 32 real-world constrained optimization problems collected in IEEE CEC2020. Our results show the CSA with theta-mechanism is more competitive than the existing state-of-the-art approaches.
Towards Flexible and Robust User Interface Adaptations With Multiple Objectives

This paper proposes a new approach for online UI adaptation that aims to overcome the limitations of the most commonly used UI optimization method involving multiple objectives: weighted sum optimization. Weighted sums are highly sensitive to objective formulation, limiting the effectiveness of UI adaptations. We propose ParetoAdapt, an adaptation approach that uses online multi-objective optimization with a posteriori articulated preferences—that is, articulation of preferences after the optimization has concluded—to make UI adaptation robust to incomplete and inaccurate objective formulations. It offers users a flexible way to control adaptations by selecting from a set of Pareto optimal adaptation proposals and adjusting them to fit their needs. We showcase the feasibility and flexibility of ParetoAdapt by implementing an online layout adaptation system in a state-of-the-art 3D UI adaptation framework. We further evaluate its robustness and run-time in simulation-based experiments that allow us to systematically change the accuracy of the estimated user preferences. We conclude by discussing how our approach may impact the usability and practicality of online UI adaptations.
Front Row: Automatically Generating Immersive Audio Representations of Tennis Broadcasts for Blind Viewers

Blind and low-vision (BLV) people face challenges watching sports due to the lack of accessibility of sports broadcasts. Currently, BLV people rely on descriptions from TV commentators, radio announcers, or their friends to understand the game. These descriptions, however, do not allow BLV viewers to visualize the action by themselves. We present Front Row, a system that automatically generates an immersive audio representation of sports broadcasts, specifically tennis, allowing BLV viewers to more directly perceive what is happening in the game. Front Row first recognizes gameplay from the video feed using computer vision, then renders players' positions and shots via spatialized (3D) audio cues. User evaluations with 12 BLV participants show that Front Row gives BLV viewers a more accurate understanding of the game compared to TV and radio, enabling viewers to form their own opinions on players' moods and strategies. We discuss future implications of Front Row and illustrate several applications, including a Front Row plug-in for video streaming platforms to enable BLV people to visualize the action in sports videos across the Web.
Usable and Fast Interactive Mental Face Reconstruction

We introduce an end-to-end interactive system for mental face reconstruction – the challenging task of visually reconstructing a face image a person only has in their mind. In contrast to existing methods that suffer from low usability and high mental load, our approach only requires the user to rank images over multiple iterations according to the perceived similarity with their mental image. Based on these rankings, our mental face reconstruction system extracts image features in each iteration, combines them into a joint feature vector, and then uses a generative model to visually reconstruct the mental image. To avoid the need for collecting large amounts of human training data, we further propose a computational user model that can simulate human ranking behaviour using data from an online crowd-sourcing study (N=215). Results from a 12-participant user study show that our method can reconstruct mental images that are visually similar to existing approaches but has significantly higher usability, lower perceived workload, and is faster. In addition, results from a third 22-participant lineup study in which we validated our reconstructions on a face ranking task show a identification rate of , which is in line with prior work. These results represent an important step towards new interactive intelligent systems that can robustly and effortlessly reconstruct a user's mental image.
Living Papers: A Language Toolkit for Augmented Scholarly Communication

Computing technology has deeply shaped how academic articles are written and produced, yet article formats and affordances have changed little over centuries. The status quo consists of digital files optimized for printed paper—ill-suited to interactive reading aids, accessibility, dynamic figures, or easy information extraction and reuse. Guided by formative discussions with scholarly communication researchers and publishing tool developers, we present Living Papers, a language toolkit for producing augmented academic articles that span print, interactive, and computational media. Living Papers articles may include formatted text, references, executable code, and interactive components. Articles are parsed into a standardized document format from which a variety of outputs are generated, including static PDFs, dynamic web pages, and extraction APIs for paper content and metadata. We describe Living Papers' architecture, document model, and reactive runtime, and detail key aspects such as citation processing and conversion of interactive components to static content. We demonstrate the use and extension of Living Papers through examples spanning traditional research papers, explorable explanations, information extraction, and reading aids such as enhanced citations, cross-references, and equations. Living Papers is available as an extensible, open source platform intended to support both article authors and researchers of augmented reading and writing experiences.
Exploring Locomotion Methods with Upright Redirected Views for VR Users in Reclining &amp; Lying Positions

Using VR in reclining & lying positions is getting common for users, but upward views caused by posture have to be redirected to be parallel to the ground as when users are standing. This affects users' locomotion performances in VR due to potential physical restrictions, and the visual-vestibular-proprioceptive conflict. This paper is among the first to investigate the suited locomotion methods and how reclining & lying positions and redirection affect them in such conditions. A user-elicitation study was carried out to construct a set of locomotion methods based on users' preferences when they were in different reclining & lying positions. A second study developed user-preferred 'tapping' and 'chair rotating' gestures, by evaluating their performances at various body reclining angles, we measured the general impacts of posture and redirection. The results showed that these methods worked effectively, but exposed some shortcomings, and users performed worst at 45° reclining angles. Finally, four upgraded methods were designed and verified to improve the locomotion performances.
EmTex: Prototyping Textile-Based Interfaces through An Embroidered Construction Kit

As electronic textiles have become more advanced in sensing, actuating, and manufacturing, incorporating smartness into fabrics has become of special interest to ubiquitous computing and interaction researchers and designers. However, innovating smart textile interfaces for numerous input and output modalities usually requires expert-level knowledge of specific materials, fabrication, and protocols. This paper presents EmTex, a construction kit based on embroidered textiles, patterned with dedicated sensing, actuating, and connecting components to facilitate the design and prototyping of smart textile interfaces. With machine embroidery, EmTex is compatible with a wide range of threads and underlay fabrics, proficient in various stitches to control the electric parameters, and capable of integrating versatile and reliable interaction functionalities with aesthetic patterns and precise designs. EmTex consists of 28 textile-based sensors, actuators, connectors, and displays, presented with standardized visual and tactile effects. Along with a visual programming tool, EmTex enables the prototyping of everyday textile interfaces for diverse life-living scenarios, that embody their touch input, and visual and haptic output properties. With EmTex, we conducted a workshop and invited 25 designers and makers to create freeform textile interfaces. Our findings revealed that EmTex helped the participants explore novel interaction opportunities with various smart textile prototypes. We also identified challenges EmTex shall face for practical use in promoting the design innovation of smart textiles.
Evaluation of Horizontal Single‐Axis Solar Tracker Algorithms in Terms of Energy Production and Operational Performance

Horizontal single‐axis solar tracking systems with Astronomical tracking algorithm are commonly used in photovoltaic (PV) installations. However, different algorithms can increase the PV installation's performance without implementing new equipment or technologies. In this article, the performance of three tracking algorithms is compared to the Astronomical one. Two algorithms aim at optimizing the received irradiance focusing on the diffuse component. The third one, called Analytical, is presented as a new development in this study, defining the optimal inclination angle depending on all radiation components. The comparison focuses on different parameters like the in‐plane irradiance, DC power output from a monofacial PV system and operational aspects like the performed number of movements. High‐time‐resolution (1 min) simulations are performed with proprietary software taking into account several effects on the effective irradiance and power output estimation, such as shading effects or PV configuration. Eight locations with different climatic conditions are analyzed. All three algorithms derive in higher in‐plane irradiance and power output values than the Astronomical algorithm, although the gain depends on the climatic conditions. At locations with high diffuse fraction, the power output from the Analytical algorithm, which outperforms the others, can be up to 3% higher than the Astronomical one.
GenAssist: Making Image Generation Accessible

Blind and low vision (BLV) creators use images to communicate with sighted audiences. However, creating or retrieving images is challenging for BLV creators as it is difficult to use authoring tools or assess image search results. Thus, creators limit the types of images they create or recruit sighted collaborators. While text-to-image generation models let creators generate high-fidelity images based on a text description (i.e. prompt), it is difficult to assess the content and quality of generated images. We present GenAssist, a system to make text-to-image generation accessible. Using our interface, creators can verify whether generated image candidates followed the prompt, access additional details in the image not specified in the prompt, and skim a summary of similarities and differences between image candidates. To power the interface, GenAssist uses a large language model to generate visual questions, vision-language models to extract answers, and a large language model to summarize the results. Our study with 12 BLV creators demonstrated that GenAssist enables and simplifies the process of image selection and generation, making visual authoring more accessible to all.