huashen218/convxai-cia-dataset huggingface dataset:
Title: Detecting and Defending Against Seizure-Inducing GIFs in Social Media
Authors: family: South
given: Laura; family: Saffo
given: David; family: Borkin
given: Michelle A.
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: human-computer interaction, GIFs, photosensitive epilepsy, accessibility

Abstract: Despite recent improvements in online accessibility, the Internet remains an inhospitable place for users with photosensitive epilepsy, a chronic condition in which certain light stimuli can trigger seizures and even lead to death in extreme cases. In this paper, we explore how current risk detection systems have allowed attackers to take advantage of design oversights and target vulnerable users with photosensitivity on popular social media platforms. Through interviews with photosensitive individuals and a critical review of existing systems, we constructed design requirements for consumer-driven protective systems and developed a prototype browser extension for actively detecting and disarming potentially seizure-inducing GIFs and videos. We validate our system with a comprehensive dataset of simulated GIFs and GIFs collected from social media. Finally, we conduct a novel quantitative analysis of the prevalence of seizure-inducing GIFs across popular social media platforms and contribute recommendations for improving online accessibility for individuals with photosensitivity. All study materials are available at https://osf.io/5a3dy/.
Title: Latte: Use-Case and Assistive-Service Driven Automated Accessibility Testing Framework for Android
Authors: family: Salehnamadi
given: Navid; family: Alshayban
given: Abdulaziz; family: Lin
given: Jun-Wei; family: Ahmed
given: Iftekhar; family: Branham
given: Stacy; family: Malek
given: Sam
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Mobile Application, Accessibility, Automated Testing

Abstract: For 15% of the world population with disabilities, accessibility is arguably the most critical software quality attribute. The ever-growing reliance of users with disability on mobile apps further underscores the need for accessible software in this domain. Existing automated accessibility assessment techniques primarily aim to detect violations of predefined guidelines, thereby produce a massive amount of accessibility warnings that often overlook the way software is actually used by users with disability. This paper presents a novel, high-fidelity form of accessibility testing for Android apps, called Latte, that automatically reuses tests written to evaluate an app’s functional correctness to assess its accessibility as well. Latte first extracts the use case corresponding to each test, and then executes each use case in the way disabled users would, i.e., using assistive services. Our empirical evaluation on real-world Android apps demonstrates Latte’s effectiveness in detecting substantially more useful defects than prior techniques.
Title: Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels
Authors: family: Zhang
given: Xiaoyi; family: de Greef
given: Lilian; family: Swearngin
given: Amanda; family: White
given: Samuel; family: Murray
given: Kyle; family: Yu
given: Lisa; family: Shan
given: Qi; family: Nichols
given: Jeffrey; family: Wu
given: Jason; family: Fleizach
given: Chris; family: Everitt
given: Aaron; family: Bigham
given: Jeffrey P
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: mobile accessibility, ui detection, accessibility enhancement

Abstract: Many accessibility features available on mobile platforms require applications (apps) to provide complete and accurate metadata describing user interface (UI) components. Unfortunately, many apps do not provide sufficient metadata for accessibility features to work as expected. In this paper, we explore inferring accessibility metadata for mobile apps from their pixels, as the visual interfaces often best reflect an app’s full functionality. We trained a robust, fast, memory-efficient, on-device model to detect UI elements using a dataset of 77,637 screens (from 4,068 iPhone apps) that we collected and annotated. To further improve UI detections and add semantic information, we introduced heuristics (e.g., UI grouping and ordering) and additional models (e.g., recognize UI content, state, interactivity). We built Screen Recognition to generate accessibility metadata to augment iOS VoiceOver. In a study with 9 screen reader users, we validated that our approach improves the accessibility of existing mobile apps, enabling even previously inaccessible apps to be used.
Title: Say It All: Feedback for Improving Non-Visual Presentation Accessibility
Authors: family: Peng
given: Yi-Hao; family: Jang
given: JiWoong; family: Bigham
given: Jeffrey P; family: Pavel
given: Amy
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Audio description, Accessibility, Slides, Video, Presentation

Abstract: Presenters commonly use slides as visual aids for informative talks. When presenters fail to verbally describe the content on their slides, blind and visually impaired audience members lose access to necessary content, making the presentation difficult to follow. Our analysis of 90 presentation videos revealed that 72% of 610 visual elements (e.g., images, text) were insufficiently described. To help presenters create accessible presentations, we introduce Presentation A11y, a system that provides real-time and post-presentation accessibility feedback. Our system analyzes visual elements on the slide and the transcript of the verbal presentation to provide element-level feedback on what visual content needs to be further described or even removed. Presenters using our system with their own slide-based presentations described more of the content on their slides, and identified  3.26 times more accessibility problems to fix after the talk than when using a traditional slide-based presentation interface. Integrating accessibility feedback into content creation tools will improve the accessibility of informational content for all.
Title: Toward Automatic Audio Description Generation for Accessible Videos
Authors: family: Wang
given: Yujia; family: Liang
given: Wei; family: Huang
given: Haikun; family: Zhang
given: Yongqi; family: Li
given: Dingzeyu; family: Yu
given: Lap-Fai
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: audio description, accessibility, video description, audio-visual consistency, sentence-level embedding, video captioning

Abstract: Video accessibility is essential for people with visual impairments. Audio descriptions describe what is happening on-screen, e.g., physical actions, facial expressions, and scene changes. Generating high-quality audio descriptions requires a lot of manual description generation [50]. To address this accessibility obstacle, we built a system that analyzes the audiovisual contents of a video and generates the audio descriptions. The system consisted of three modules: AD insertion time prediction, AD generation, and AD optimization. We evaluated the quality of our system on five types of videos by conducting qualitative studies with 20 sighted users and 12 users who were blind or visually impaired. Our findings revealed how audio description preferences varied with user types and video types. Based on our study’s analysis, we provided recommendations for the development of future audio description generation technologies.
Title: Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020
Authors: family: Butler
given: Matthew; family: Holloway
given: Leona M; family: Reinders
given: Samuel; family: Goncu
given: Cagatay; family: Marriott
given: Kim
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Blind, Low Vision, Systematic Literature Review, Tactile Graphics, Assistive Technology

Abstract: This paper presents a systematic literature review of 292 publications from 97 unique venues on touch-based graphics for people who are blind or have low vision, from 2010 to mid-2020. It is the first review of its kind on touch-based accessible graphics. It is timely because it allows us to assess the impact of new technologies such as commodity 3D printing and low-cost electronics on the production and presentation of accessible graphics. As expected our review shows an increase in publications from 2014 that we can attribute to these developments. It also reveals the need to: broaden application areas, especially to the workplace; broaden end-user participation throughout the full design process; and conduct more in situ evaluation. This work is linked to an online living resource to be shared with the wider community.
Title: Comparison of Methods for Evaluating Complexity of Simplified Texts among Deaf and Hard-of-Hearing Adults at Different Literacy Levels
Authors: family: Alonzo
given: Oliver; family: Trussell
given: Jessica; family: Dingman
given: Becca; family: Huenerfauth
given: Matt
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Deaf and Hard-of-hearing, Automatic Text Simplification, Accessibility, Methodological Research

Abstract: Research has explored using Automatic Text Simplification for reading assistance, with prior work identifying benefits and interests from Deaf and Hard-of-Hearing (DHH) adults. While the evaluation of these technologies remains a crucial aspect of research in the area, researchers lack guidance in terms of how to evaluate text complexity with DHH readers. Thus, in this work we conduct methodological research to evaluate metrics identified from prior work (including reading speed, comprehension questions, and subjective judgements of understandability and readability) in terms of their effectiveness for evaluating texts modified to be at various complexity levels with DHH adults at different literacy levels. Subjective metrics and low-linguistic-complexity comprehension questions distinguished certain text complexity levels with participants with lower literacy. Among participants with higher literacy, only subjective judgements of text readability distinguished certain text complexity levels. For all metrics, participants with higher literacy scored higher or provided more positive subjective judgements overall.
Title: TapeBlocks: A Making Toolkit for People Living with Intellectual Disabilities
Authors: family: Ellis
given: Kirsten; family: Dao
given: Emily; family: Smith
given: Osian; family: Lindsay
given: Stephen; family: Olivier
given: Patrick
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: tangibles, Making, intellectual disability, accessibility

Abstract: The accessibility and affordability of tangible electronic toolkits are significant barriers to their uptake by people with disabilities. We present the design and evaluation of TapeBlocks, a low-cost, low-fidelity toolkit intended to be accessible for people with intellectual disabilities while promoting creativity and engagement. We evaluated TapeBlocks by interviewing makers, special educational needs teachers and support coaches. Analysis of these interviews informed the design of a series of maker workshops using TapeBlocks with young adults living with intellectual disabilities, led by support coaches with support from the research team. Participants were able to engage with TapeBlocks and making, eventually building their own TapeBlocks to make personal creations. Our evaluation reveals how TapeBlocks supports accessible making and playful discovery of electronics for people living with disabilities, and addresses a gap in existing toolkits by being tinkerable, affordable and having a low threshold for engagement.
Title: Optimal Action-based or User Prediction-based Haptic Guidance: Can You Do Even Better?
Authors: family: Moon
given: Hee-Seung; family: Seo
given: Jiwon
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: User adaptation, Physical human–robot interaction (pHRI), Deep learning, Haptic guidance

Abstract: The recently advanced robotics technology enables robots to assist users in their daily lives. Haptic guidance (HG) improves users’ task performance through physical interaction between robots and users. It can be classified into optimal action-based HG (OAHG), which assists users with an optimal action, and user prediction-based HG (UPHG), which assists users with their next predicted action. This study aims to understand the difference between OAHG and UPHG and propose a combined HG (CombHG) that achieves optimal performance by complementing each HG type, which has important implications for HG design. We propose implementation methods for each HG type using deep learning-based approaches. A user study (n=20) in a haptic task environment indicated that UPHG induces better subjective evaluations, such as naturalness and comfort, than OAHG. In addition, the CombHG that we proposed further decreases the disagreement between the user intention and HG, without reducing the objective and subjective scores.
Title: TapNet: The Design, Training, Implementation, and Applications of a Multi-Task Learning CNN for Off-Screen Mobile Input
Authors: family: Huang
given: Michael Xuelin; family: Li
given: Yang; family: Nazneen
given: Nazneen; family: Chao
given: Alexander; family: Zhai
given: Shumin
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: IMU, Back-of-device, gesture recognition

Abstract: To make off-screen interaction without specialized hardware practical, we investigate using deep learning methods to process the common built-in IMU sensor (accelerometers and gyroscopes) on mobile phones into a useful set of one-handed interaction events. We present the design, training, implementation and applications of TapNet, a multi-task network that detects tapping on the smartphone. With phone form factor as auxiliary information, TapNet can jointly learn from data across devices and simultaneously recognize multiple tap properties, including tap direction and tap location. We developed two datasets consisting of over 135K training samples, 38K testing samples, and 32 participants in total. Experimental evaluation demonstrated the effectiveness of the TapNet design and its significant improvement over the state of the art. Along with the datasets, codebase1, and extensive experiments, TapNet establishes a new technical foundation for off-screen mobile input.
Title: FingerText: Exploring and Optimizing Performance for Wearable, Mobile and One-Handed Typing
Authors: family: Lee
given: DoYoung; family: Kim
given: Jiwan; family: Oakley
given: Ian
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: wearable input, optimization, head-mounted display

Abstract: Typing on wearables while situationally impaired, such as while walking, is challenging. However, while HCI research on wearable typing is diverse, existing work focuses on stationary scenarios and fine-grained input that will likely perform poorly when users are on-the-go. To address this issue we explore single-handed wearable typing using intra-hand touches between the thumb and fingers, a modality we argue will be robust to the physical disturbances inherent to input while mobile. We first examine the impact of walking on performance of these touches, noting no significant differences in accuracy or speed, then feed our study data into a multi-objective optimization process in order to design keyboard layouts (for both five and ten keys) capable of supporting rapid, accurate, comfortable, and unambiguous typing. A final study tests these layouts against QWERTY baselines and reports performance improvements of up to 10.45% WPM and 39.44% WER when users type while walking.
Title: A Computational Approach to Magnetic Force Feedback Design
Authors: family: Ogata
given: Masa; family: Koyama
given: Yuki
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Inverse design, Optimization, Haptics, Magnetic force feedback, Digital fabrication

Abstract: We present a computational approach to haptic design embedded in everyday tangible interaction with digital fabrication. To generate haptic feedback, the use of permanent magnets as the mechanism potentially contributes to simpleness and robustness; however, it is difficult to manually design how magnets should be embedded in the objects. Our approach enables the inverse design of magnetic force feedback; that is, we computationally solve an inverse problem to obtain an optimal arrangement of permanent magnets that renders the user-specified haptic sensation. To solve the inverse problem in a practical manner, we also present techniques on magnetic simulation and optimization. We demonstrate applications to explore the design possibility of augmenting digital fabrication for everyday use.
Title: A Simulation Model of Intermittently Controlled Point-and-Click Behaviour
Authors: family: Do
given: Seungwon; family: Chang
given: Minsuk; family: Lee
given: Byungjoo
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: point-and-click, Fitts’ law, user performance, simulation model

Abstract: We present a novel simulation model of point-and-click behaviour that is applicable both when a target is stationary or moving. To enable more realistic simulation than existing models, the model proposed in this study takes into account key features of the user and the external environment, such as intermittent motor control, click decision-making, visual perception, upper limb kinematics and the effect of input device. The simulated user’s point-and-click behaviour is formulated as a Markov decision process (MDP), and the user’s policy of action is optimised through deep reinforcement learning. As a result, our model successfully and accurately reproduced the trial completion time, distribution of click endpoints, and cursor trajectories of real users. Through an ablation study, we showed how the simulation results change when the model’s sub-modules are individually removed. The implemented model and dataset are publicly available.
Title: Understanding, Detecting and Mitigating the Effects of Coactivations in Ten-Finger Mid-Air Typing in Virtual Reality
Authors: family: Foy
given: Conor R.; family: Dudley
given: John J.; family: Gupta
given: Aakar; family: Benko
given: Hrvoje; family: Kristensson
given: Per Ola
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: virtual reality, text entry

Abstract: Typing with ten fingers on a virtual keyboard in virtual or augmented reality exposes a challenging input interpretation problem. There are many sources of noise in this interaction context and these exacerbate the challenge of accurately translating human actions into text. A particularly challenging input noise source arises from the physiology of the hand. Intentional finger movements can produce unintentional coactivations in other fingers. On a physical keyboard, the resistance of the keys alleviates this issue. On a virtual keyboard, coactivations are likely to introduce spurious input events under a naïve solution to input detection. In this paper we examine the features that discriminate intentional activations from coactivations. Based on this analysis, we demonstrate three alternative coactivation detection strategies with high discrimination power. Finally, we integrate coactivation detection into a probabilistic decoder and demonstrate its ability to further reduce uncorrected character error rates by approximately 10% relative and 0.9% absolute.
Title: An Adaptive Model of Gaze-based Selection
Authors: family: Chen
given: Xiuli; family: Acharya
given: Aditya; family: Oulasvirta
given: Antti; family: Howes
given: Andrew
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: adaptive model, gaze-based selection, Reinforcement learning, computational rationality

Abstract: Gaze-based selection has received significant academic attention over a number of years. While advances have been made, it is possible that further progress could be made if there were a deeper understanding of the adaptive nature of the mechanisms that guide eye movement and vision. Control of eye movement typically results in a sequence of movements (saccades) and fixations followed by a ‘dwell’ at a target and a selection. To shed light on how these sequences are planned, this paper presents a computational model of the control of eye movements in gaze-based selection. We formulate the model as an optimal sequential planning problem bounded by the limits of the human visual and motor systems and use reinforcement learning to approximate optimal solutions. The model accurately replicates earlier results on the effects of target size and distance and captures a number of other aspects of performance. The model can be used to predict number of fixations and duration required to make a gaze-based selection. The future development of the model is discussed.
Title: CapContact: Super-resolution Contact Areas from Capacitive Touchscreens
Authors: family: Streli
given: Paul; family: Holz
given: Christian
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Keywords: Super-resolution, Generative adversarial networks;, Accuracy, Touch input, Contact area, Capacitive sensing

Abstract: Touch input is dominantly detected using mutual-capacitance sensing, which measures the proximity of close-by objects that change the electric field between the sensor lines. The exponential drop-off in intensities with growing distance enables software to detect touch events, but does not reveal true contact areas. In this paper, we introduce CapContact, a novel method to precisely infer the contact area between the user’s finger and the surface from a single capacitive image. At 8 × super-resolution, our convolutional neural network generates refined touch masks from 16-bit capacitive images as input, which can even discriminate adjacent touches that are not distinguishable with existing methods. We trained and evaluated our method using supervised learning on data from 10 participants who performed touch gestures. Our capture apparatus integrates optical touch sensing to obtain ground-truth contact through high-resolution frustrated total internal reflection. We compare our method with a baseline using bicubic upsampling as well as the ground truth from FTIR images. We separately evaluate our method’s performance in discriminating adjacent touches. CapContact successfully separated closely adjacent touch contacts in 494 of 570 cases (87%) compared to the baseline’s 43 of 570 cases (8%). Importantly, we demonstrate that our method accurately performs even at half of the sensing resolution at twice the grid-line pitch across the same surface area, challenging the current industry-wide standard of a ∼ 4 mm sensing pitch. We conclude this paper with implications for capacitive touch sensing in general and for touch-input accuracy in particular.