Reviewerly/exHarmony huggingface dataset:
Unsupervised Domain Adaptation for Cross-Modality Retinal Vessel Segmentation via Disentangling Representation Style Transfer and Collaborative Consistency Learning

Various deep learning models have been developed to segment anatomical structures from medical images, but they typically have poor performance when tested on another target domain with different data distribution. Recently, unsupervised domain adaptation methods have been proposed to alleviate this so-called domain shift issue, but most of them are designed for scenarios with relatively small domain shifts and are likely to fail when encountering a large domain gap. In this paper, we propose DCDA, a novel cross-modality unsupervised domain adaptation framework for tasks with large domain shifts, e.g., segmenting retinal vessels from OCTA and OCT images. DCDA mainly consists of a disentangling representation style transfer (DRST) module and a collaborative consistency learning (CCL) module. DRST decomposes images into content components and style codes and performs style transfer and image reconstruction. CCL contains two segmentation models, one for source domain and the other for target domain. The two models use labeled data (together with the corresponding transferred images) for supervised learning and perform collaborative consistency learning on unlabeled data. Each model focuses on the corresponding single domain and aims to yield an expertized domain-specific segmentation model. Through extensive experiments on retinal vessel segmentation, our framework achieves Dice scores close to target-trained oracle both from OCTA to OCT and from OCT to OCTA, significantly outperforming other state-of-the-art methods.
Self-supervised monocular depth estimation in dynamic scenes with moving instance loss

Estimating depth from monocular images is a powerful method to perceive valuable environmental information, which is essential for applications that require three-dimensional (3D) environmental models such as autonomous driving and virtual reality. The monocular self-supervised depth estimation method based on deep learning has made rapid progress without depth ground truth information. However, the existing methods are based on the assumption of a static world during training, and depth estimation in a dynamic environment needs further development. To solve this problem, we propose a new monocular self-supervised depth estimation method in dynamic scenes to eliminate the negative impact of moving objects in the image sequence when calculating the self-supervised loss. Specifically, for the self-supervised depth estimation framework, we propose a moving object mask based on the minimum instance photometric residual and then combine it with the mask based on instance re-projection residual in the existing instance-level moving object segmentation methods. In addition, we design a moving instance loss function to process the moving object, so that the training of the model can achieve better performance. Experiments are conducted on public datasets to verify the effectiveness of the proposed method and each of its components, and the results show that our method achieves better performance for depth estimation in dynamic scenes compared to state-of-the-art methods.
Keep it Short: A Comparison of Voice Assistants’ Response Behavior

Voice assistants (VAs) are present in homes, smartphones, and cars. They allow users to perform tasks without graphical or tactile user interfaces, as they are designed for natural language interaction. However, we found that currently, VAs are emulating human behavior by responding in complete sentences, limiting the design options, and preventing VAs from meeting their full potential as a utilitarian tool. We implemented a VA that handles requests in three response styles: two differing short keyword-based response styles and a full-sentence baseline. In a user study, 72 participants interacted with our VA by issuing eight requests. Results show that the short responses were perceived similarly useful and likable while being perceived as more efficient, especially for commands, and sometimes better to comprehend than the baseline. To achieve widespread adoption, we argue that VAs should be customizable and adapt to users instead of always responding in full sentences.
Deep unsupervised endoscopic image enhancement based on multi-image fusion

A deep unsupervised endoscopic image enhancement method is proposed based on multi-image fusion to achieve high quality endoscope images from poorly illuminated, low contrast and color deviated images through an unsupervised mapping and deep learning network without the need for ground truth.Firstly, three image enhancement methods are used to process original endoscopic images to obtain three derived images, which are then transformed into HSI color space. Secondly, a deep unsupervised multi-image fusion network (DerivedFuse) is proposed to extract and fuse features of the derived images accurately by utilizing a new no-reference quality metric as loss function. I-channel images of the three derived images are inputted into the DerivedFuse network to enhance the intensity component of the original image. Finally, a saturation adjustment function is proposed to adaptive adjusting the saturation component of HSI color space to enrich the color information of the original input image.Three evaluation metrics: Entropy, Contrast Improvement Index (CII) and Average Gradient (AG) are used to evaluate the performance of the proposed method. The results are compared with that of fourteen state-of-the-art algorithms. Experiments on endoscopic image enhancement show that the Entropy value of our method is 3.27% higher than the optimal entropy value of comparison algorithms. The CII of our proposed method is 6.19% higher than that of comparison algorithms. The AG of our method is 7.83% higher than the optimal AG of comparison algorithms.The proposed deep unsupervised multi-image fusion method can obtain image information details, enhance endoscopic images with high contrast, rich and natural color information, visual and image quality. Sixteen doctors and medical students have given their assessments on the proposed method for assisting clinical diagnoses.
A Conversational Approach for Modifying Service Mashups in IoT Environments

Existing conversational approaches for Internet of Things (IoT) service mashup do not support modification because of the usability challenge, although it is common for users to modify the service mashups in IoT environments. To support the modification of IoT service mashups through conversational interfaces in a usable manner, we propose the conversational mashup modification agent (CoMMA). Users can modify IoT service mashups using CoMMA through natural language conversations. CoMMA has a two-step mashup modification interaction, an implicature-based localization step, and a modification step with a disambiguation strategy. The localization step allows users to easily search for a mashup by vocalizing their expressions in the environment. The modification step supports users to modify mashups by speaking simple modification commands. We conducted a user study and the results show that CoMMA is as effective as visual approaches in terms of task completion time and perceived task workload for modifying IoT service mashups.
A new machine learning model based on the broad learning system and wavelets

In this work, we present a new neural network named WAvelet-Based Broad LEarning System (WABBLES). WABBLES is based on the flat structure of the broad learning system. Such structure offers an alternative to deep learning models, such as convolutional neural networks. The WABBLES network uses multiresolution analysis to look for subtle, yet important features from the input data for a better classification performance. WABBLES uses wavelets to map the input signal, to obtain more relevant features from it. This is achieved by autonomously learning and adjusting the dilation and translation parameters of a wavelet, which control its shape. In this way, the resulting mapping nodes have a better representation of the most important features for the classification problem. The construction of the model is described here, along with special considerations and algorithms involved. Finally, the proposed model is tested using a database of synthetic astronomical data and a benchmark dataset called the Breast Cancer Wisconsin Dataset (Original). The conducted experiments provide a comparison between the proposed model and several machine learning algorithms with different performance metrics applied to the context of exoplanet identification and breast cancer detection. Our results confirm that the WABBLES model obtains superior accuracy and F-score percentages than the other models.
Hybridised Loss Functions for Improved Neural Network Generalisation

Loss functions play an important role in the training of artificial neural networks (ANNs), and can affect the generalisation ability of the ANN model, among other properties. Specifically, it has been shown that the cross entropy and sum squared error loss functions result in different training dynamics, and exhibit different properties that are complementary to one another. It has previously been suggested that a hybrid of the entropy and sum squared error loss functions could combine the advantages of the two functions, while limiting their disadvantages. The effectiveness of such hybrid loss functions is investigated in this study. It is shown that hybridisation of the two loss functions improves the generalisation ability of the ANNs on all problems considered. The hybrid loss function that starts training with the sum squared error loss function and later switches to the cross entropy error loss function is shown to either perform the best on average, or to not be significantly different than the best loss function tested for all problems considered. This study shows that the minima discovered by the sum squared error loss function can be further exploited by switching to cross entropy error loss function. It can thus be concluded that hybridisation of the two loss functions could lead to better performance in ANNs.
Avatar Interpreter: Improving Classroom Experiences for Deaf and Hard-of-Hearing People Based on Augmented Reality

Deaf and hard-of-hearing (DHH) people experience difficulties in accessing education. One reason is that they miss out on oral information in the classroom. Although some classrooms are equipped with sign language interpreters specifically for deaf students, DHH students have trouble switching their gaze and attention when they cannot see the interpreter and the visual information in the classroom simultaneously. To address this challenge, this paper develops Avatar Interpreter, a visual interface for synchronized sign language interpreters on an augmented reality head-mounted display. Avatar Interpreter can be displayed at different sizes and in different locations in space to fit different scenarios, helping deaf students to better receive sign language information and enhancing the classroom experience for deaf students. In the work presented in this paper, we discuss our prototype, make design recommendations, and discuss configuration and implementation. Finally, we propose questions and research methods for an upcoming user study.
The Future of Work is No Work: A Call to Action for Designers in the Abolition of Work

We endure decades of preparation for, enduring of, and recovery from work. It manifests and corrupts most of our life. It doesn't have to be like this. We can envision and design towards a future that gives life back to us: a world without work as we know it. HCI as a field should be oriented towards abolishing work for everyone rather than designing a future of work, often depicted as a metaverse of three-dimensional avatars, virtual workplaces, and automated oppression. In this paper, we detail what work is, what the abolition of work means, and a call to action for HCI to abolish work following a transformative justice framework.
Interactive Fiction Provotypes for Coping with Interpersonal Racism

Reducing uncertainty around the nature of racist interactions is one of the key motivations driving individual behaviors for coping with those incidents. However, there are few appropriate technologies to support BIPOC (Black, Indigenous, People of Color) in engaging in social uncertainty reduction around this vulnerable, sensitive topic. This paper reports on an exploratory design study investigating how social technology might facilitate uncertainty reduction through three “provotypes” - provocative prototypes of user-generated speculative design concepts. U.S.-based participants engaged with the provotypes through an interactive fiction to explore their usefulness in the context of a racist microaggression. Results showed that engaging the provotypes through interactive fiction facilitated complex and productive interactions and critiques. This work contributes a novel method for conducting exploratory design, remote user studies using interactive fiction as well as priorities, tensions, and further information what role, if any, technology might play in managing racist interactions.
Demonstrating InfraredTags: Decoding Invisible 3D Printed Tags with Convolutional Neural Networks

We demonstrate InfraredTags, which are invisible 2D markers and barcodes that are 3D printed as part of objects. We show how InfraredTags can be decoded using a convolutional neural network (CNN) after being captured by low-cost near-infrared cameras. InfraredTags are formed by printing objects from an infrared-transmitting filament, which infrared cameras can see through, and by having air gaps inside for the tag's bits, which appear at a different intensity in the infrared image. We built a user interface that facilitates the integration of common tags (QR codes, ArUco markers) with the object geometry to make them 3D printable as InfraredTags. Once printed, the tags can be robustly decoded with a U-Net model that we trained using a custom dataset for optimal binarization and detection. We show how our method enables different applications, such as object tracking and embedding metadata for tangible interactions and augmented reality.
LEMON: A Lightweight Facial Emotion Recognition System for Assistive Robotics Based on Dilated Residual Convolutional Neural Networks

The development of a Social Intelligence System based on artificial intelligence is one of the cutting edge technologies in Assistive Robotics. Such systems need to create an empathic interaction with the users; therefore, it os required to include an Emotion Recognition (ER) framework which has to run, in near real-time, together with several other intelligent services. Most of the low-cost commercial robots, however, although more accessible by users and healthcare facilities, have to balance costs and effectiveness, resulting in under-performing hardware in terms of memory and processing unit. This aspect makes the design of the systems challenging, requiring a trade-off between the accuracy and the complexity of the adopted models. This paper proposes a compact and robust service for Assistive Robotics, called Lightweight EMotion recognitiON (LEMON), which uses image processing, Computer Vision and Deep Learning (DL) algorithms to recognize facial expressions. Specifically, the proposed DL model is based on Residual Convolutional Neural Networks with the combination of Dilated and Standard Convolution Layers. The first remarkable result is the few numbers (i.e., 1.6 Million) of parameters characterizing our model. In addition, Dilated Convolutions expand receptive fields exponentially with preserving resolution, less computation and memory cost to recognize the distinction among facial expressions by capturing the displacement of the pixels. Finally, to reduce the dying ReLU problem and improve the stability of the model, we apply an Exponential Linear Unit (ELU) activation function in the initial layers of the model. We have performed training and evaluation (via one- and five-fold cross validation) of the model with five datasets available in the community and one mixed dataset created by taking samples from all of them. With respect to the other approaches, our model achieves comparable results with a significant reduction in terms of the number of parameters.
Design and Evaluation of Hybrid Search for American Sign Language to English Dictionaries: Making the Most of Imperfect Sign Recognition

Searching for the meaning of an unfamiliar sign-language word in a dictionary is difficult for learners, but emerging sign-recognition technology will soon enable users to search by submitting a video of themselves performing the word they recall. However, sign-recognition technology is imperfect, and users may need to search through a long list of possible results when seeking a desired result. To speed this search, we present a hybrid-search approach, in which users begin with a video-based query and then filter the search results by linguistic properties, e.g., handshape. We interviewed 32 ASL learners about their preferences for the content and appearance of the search-results page and filtering criteria. A between-subjects experiment with 20 ASL learners revealed that our hybrid search system outperformed a video-based search system along multiple satisfaction and performance metrics. Our findings provide guidance for designers of video-based sign-language dictionary search systems, with implications for other search scenarios.
Psychometric Properties of the User Experience Questionnaire (UEQ)

User experience (UX) summarizes user perceptions and responses resulting from the interaction with a product, system, or service. The User Experience Questionnaire (UEQ) is one standardized instrument for measuring UX. With six scales, it identifies areas in which product improvements will have the highest impact. In this paper, we evaluate the reliability and validity of this questionnaire. The data of N = 1, 121 participants who interacted with one of 23 products indicated an acceptable to good reliability of all scales. The results show, however, that the scales were not independent of each other. Combining perspicuity, efficiency, and dependability to pragmatic aspects as well as novelty and stimulation to hedonic aspects of UX improved the model fit significantly. The systematic variations of product properties and correlations with the System Usability Scale (SUS) in a second experiment with N=499 participants supported the validity of these two factors. Practical implications of the results are discussed.
Towards Digital Environmental Stewardship: the Work of Caring for the Environment in Waste Management

This paper discusses Digital Environmental Stewardship as an analytical framework that can help HCI scholarship to understand, design, and assess sociotechnical interventions concerned with sustainable waste management practices. Drawing on environmental studies, we outline key concepts of environmental stewardship – namely actors, capacity, and motivations – to unpack how different initiatives for handling waste are organised, both through grassroots and top-down interventions, and through varying sociotechnical configurations. We use these dimensions to analyse three different cases of waste management that illustrate how actions of care for the environment are ecologically organised, and what challenges might hinder them beyond –or besides– behavioural motivations. We conclude with a discussion on the orientation to action that the suggested framework provides, and its role in understanding, designing and assessing digital technologies in this domain. We argue that examining how stewardship actions fold into each other helps design sociotechnical interventions for managing waste from within a relational perspective.
Object Detection Algorithm Based on Context Information and Self-Attention Mechanism

Pursuing an object detector with good detection accuracy while ensuring detection speed has always been a challenging problem in object detection. This paper proposes a multi-scale context information fusion model combined with a self-attention block (CSA-Net). First, an improved backbone network ResNet-SA is designed with self-attention to reduce the interference of the image background area and focus on the object region. Second, this work introduces a receptive field feature enhancement module (RFFE) to combine local and global features while increasing the receptive field. Then this work adopts a spatial feature fusion pyramid with a symmetrical structure, which fuses and transfers semantic information and feature information. Finally, a sibling detection head using an anchor-free detection mechanism is applied to increase the accuracy and speed of detection at the end of the model. A large number of experiments support the above analysis and conclusions. Our model achieves an average accuracy of 46.8% on the COCO 2017 test set.
An integrated curriculum of virtual/augmented reality for multiple design students

Abstract With the rapid digital developments, the practice and study of VR/AR technologies reached almost all the design disciplines by considering different applications. Therefore, there is continuous pressure on design education, where course programs in graphic design, media design, fashion design, and product design must promptly incorporate these new technologies and create complete unity between the various design professions. This paper provides a careful examination of case studies in literature and consideration of utility for the profession through a curriculum of virtual and augmented reality technologies with specifications that suit multiple areas of design (product and graphic design). This approach demystifies these new technologies from the design process to product marketing. Interviews and discussions were conducted with a sample of approximately 100 participants (60% students, 30% design educators, 10% designers, and employers). The study included different universities from UAE, Egypt, and Italy. During the academic year 2020/2021. Furthermore, the results of this research were applied to design the Virtual Reality and its Derivatives course within the new study plan of the Graphic Design Department, College of Mass Communication at Ajman University in the United Arab Emirates.
Effects, capabilities, and boxes: from scope-based reasoning to type-based reasoning and back

Reasoning about the use of external resources is an important aspect of many practical applications. Effect systems enable tracking such information in types, but at the cost of complicating signatures of common functions. Capabilities coupled with escape analysis offer safety and natural signatures, but are often overly coarse grained and restrictive. We present System C, which builds on and generalizes ideas from type-based escape analysis and demonstrates that capabilities and effects can be reconciled harmoniously. By assuming that all functions are second class, we can admit natural signatures for many common programs. By introducing a notion of boxed values, we can lift the restrictions of second-class values at the cost of needing to track degree-of-impurity information in types. The system we present is expressive enough to support effect handlers in full capacity. We practically evaluate System C in an implementation and prove its soundness.