Reviewerly/exHarmony huggingface dataset:
Hierarchical Bias Mitigation for Semi-Supervised Medical Image Classification

Semi-supervised learning (SSL) has demonstrated remarkable advances on medical image classification, by harvesting beneficial knowledge from abundant unlabeled samples. The pseudo labeling dominates current SSL approaches, however, it suffers from intrinsic biases within the process. In this paper, we retrospect the pseudo labeling and identify three hierarchical biases: perception bias, selection bias and confirmation bias, at feature extraction, pseudo label selection and momentum optimization stages, respectively. In this regard, we propose a HierArchical BIas miTigation (HABIT) framework to amend these biases, which consists of three customized modules including Mutual Reconciliation Network (MRNet), Recalibrated Feature Compensation (RFC) and Consistency-aware Momentum Heredity (CMH). Firstly, in the feature extraction, MRNet is devised to jointly utilize convolution and permutator-based paths with a mutual information transfer module to exchanges features and reconcile spatial perception bias for better representations. To address pseudo label selection bias, RFC adaptively recalibrates the strong and weak augmented distributions to be a rational discrepancy and augments features for minority categories to achieve the balanced training. Finally, in the momentum optimization stage, in order to reduce the confirmation bias, CMH models the consistency among different sample augmentations into network updating process to improve the dependability of the model. Extensive experiments on three semi-supervised medical image classification datasets demonstrate that HABIT mitigates three biases and achieves state-of-the-art performance. Our codes are available at https://github.com/CityU-AIM-Group/HABIT.
Give Me a Hand: Improving the Effectiveness of Near-field Augmented Reality Interactions By Avatarizing Users' End Effectors

Inspired by previous works showing promise for AR self-avatarization - providing users with an augmented self avatar, we investigated whether avatarizing users' end-effectors (hands) improved their interaction performance on a near-field, obstacle avoidance, object retrieval task wherein users were tasked with retrieving a target object from a field of non-target obstacles for a number of trials. We employed a 3 (Augmented hand representation) X 2 (density of obstacles) X 2 (size of obstacles) X 2 (virtual light intensity) multi-factorial design, manipulating the presence/absence and anthropomorphic fidelity of augmented self-avatars overlaid on the user's real hands, as a between subjects factor across three experimental conditions: (1) No-Augmented Avatar (using only real hands); (2) Iconic-Augmented Avatar; (3) Realistic Augmented Avatar. Results indicated that self-avatarization improved interaction performance and was perceived as more usable regardless of the anthropomorphic fidelity of avatar. We also found that the virtual light intensity used in illuminating holograms affects how visible one's real hands are. Overall, our findings seem to indicate that interaction performance may improve when users are provided with a visual representation of the AR system's interacting layer in the form of an augmented self-avatar.
Evaluation of AR visualization approaches for catheter insertion into the ventricle cavity

Augmented reality (AR) has shown potential in computer-aided surgery. It allows for the visualization of hidden anatomical structures as well as assists in navigating and locating surgical instruments at the surgical site. Various modalities (devices and/or visualizations) have been used in the literature, but few studies investigated the adequacy/superiority of one modality over the other. For instance, the use of optical see-through (OST) HMDs has not always been scientifically justified. Our goal is to compare various visualization modalities for catheter insertion in external ventricular drain and ventricular shunt procedures. We investigate two AR approaches: (1) 2D approaches consisting of a smartphone and a 2D window visualized through an OST (Microsoft HoloLens 2), and (2) 3D approaches consisting of a fully aligned patient model and a model that is adjacent to the patient and is rotationally aligned using an OST. 32 participants joined this study. For each visualization approach, participants were asked to perform five insertions after which they filled NASA-TLX and SUS forms. Moreover, the position and orientation of the needle with respect to the planning during the insertion task were collected. The results show that participants achieved a better insertion performance significantly under 3D visualizations, and the NASA-TLX and SUS forms reflected the preference of participants for these approaches compared to 2D approaches.
Latent Class-Conditional Noise Model

Learning with noisy labels has become imperative in the Big Data era, which saves expensive human labors on accurate annotations. Previous noise-transition-based methods have achieved theoretically-grounded performance under the Class-Conditional Noise model (CCN). However, these approaches builds upon an ideal but impractical anchor set available to pre-estimate the noise transition. Even though subsequent works adapt the estimation as a neural layer, the ill-posed stochastic learning of its parameters in back-propagation easily falls into undesired local minimums. We solve this problem by introducing a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. By projecting the noise transition into the Dirichlet space, the learning is constrained on a simplex characterized by the complete dataset, instead of some ad-hoc parametric space wrapped by the neural layer. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels to train the classifier and to model the noise. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples. We further generalize LCCN to different counterparts compatible with open-set noisy labels, semi-supervised learning as well as cross-model training. A range of experiments demonstrate the advantages of LCCN and its variants over the current state-of-the-art methods. The code is available at here.
ConeSpeech: Exploring Directional Speech Interaction for Multi-Person Remote Communication in Virtual Reality

Remote communication is essential for efficient collaboration among people at different locations. We present ConeSpeech, a virtual reality (VR) based multi-user remote communication technique, which enables users to selectively speak to target listeners without distracting bystanders. With ConeSpeech, the user looks at the target listener and only in a cone-shaped area in the direction can the listeners hear the speech. This manner alleviates the disturbance to and avoids overhearing from surrounding irrelevant people. Three featured functions are supported, directional speech delivery, size-adjustable delivery range, and multiple delivery areas, to facilitate speaking to more than one listener and to listeners spatially mixed up with bystanders. We conducted a user study to determine the modality to control the cone-shaped delivery area. Then we implemented the technique and evaluated its performance in three typical multi-user communication tasks by comparing it to two baseline methods. Results show that ConeSpeech balanced the convenience and flexibility of voice communication.
A Novel Two-Stage Destriping Algorithm Based on MWIR Energy Separation and Image Guidance (MES-IG)

Long-wave infrared (LWIR) bands in multispectral datasets are extremely useful in many applications. However, the LWIR bands usually suffer from undesirable stripe noise, which impedes their further application. Compared with emission-dominated LWIR, the mid-wave infrared (MWIR) bands containing both emitted and reflected radiation usually exhibit higher image quality. In this article, we propose a novel two-stage MWIR energy separation and image guidance (MES-IG) algorithm to destripe the LWIR images with the assistance of the MWIR bands. In the first stage, we decompose the MWIR image into the emitted and reflected components by solving a constrained optimization problem. Specifically, we impose the low-rank penalty to enforce the similarities between MWIR and LWIR, and we use the total variation (TV) regularization to exploit the similarities between MWIR and visible and near-infrared (VNIR) images. In the second stage, the obtained emitted component of MWIR is considered as the guidance image to remove the stripes in the LWIR images by adopting the 1-D guided filter algorithm. Numerical experiments on the Chinese Gaofen-5 satellite and the Moderate Resolution Imaging Spectroradiometer (MODIS) data demonstrate the utility of the proposed method in providing improved LWIR image destriping performance over the state-of-the-art algorithms.
Modified Egocentric Viewpoint for Softer Seated Experience in Virtual Reality

Users in a prolonged experience of virtual reality adopt a sitting position according to their task, as they do in the real world. However, inconsistencies in the haptic feedback from a chair they sit on in the real world and that which is expected in the virtual world decrease the feeling of presence. We aimed to change the perceived haptic features of a chair by shifting the position and angle of the users' viewpoints in the virtual reality environment. The targeted features in this study were seat softness and backrest flexibility. To enhance the seat softness, we shifted the virtual viewpoint using an exponential formula soon after a user's bottom contacted the seat surface. The flexibility of the backrest was manipulated by moving the viewpoint, which followed the tilt of the virtual backrest. These shifts make users feel as if their body moves along with the viewpoint; as a result, they would perceive pseudo-softness or flexibility consistently with the body movement. Based on subjective evaluations, we confirmed that the participants perceived the seat as being softer and the backrest as being more flexible than the actual ones. These results demonstrated that only shifting the viewpoint could change the participants' perceptions of the haptic features of their seats, although significant changes created strong discomfort.
Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration, and Lower Bounds

We consider distributed optimization methods for problems where forming the Hessian is computationally challenging and communication is a significant bottleneck. We leverage randomized sketches for reducing the problem dimensions as well as preserving privacy and improving straggler resilience in asynchronous distributed systems. We derive novel approximation guarantees for classical sketching methods and establish tight concentration results that serve as both upper and lower bounds on the error. We then extend our analysis to the accuracy of parameter averaging for distributed sketches. Furthermore, we develop unbiased parameter averaging methods for randomized second order optimization in regularized problems that employ sketching of the Hessian. Existing works do not take the bias of the estimators into consideration, which limits their application to massively parallel computation. We provide closed-form formulas for regularization parameters and step sizes that provably minimize the bias for sketched Newton directions. Additionally, we demonstrate the implications of our theoretical findings via large scale experiments on a serverless cloud computing platform.
A Comparative Measurement Study of Point Cloud-Based Volumetric Video Codecs

Volumetric video is fully three-dimensional and immersive, which shows great potential in various application scenarios. However, the naive delivery of volumetric video over the Internet requires a huge amount of bandwidth and compute resources. A volumetric video should be compressed by a specific codec before transmission. In spite of the existence of a few pioneering efforts on prototype design, there is very limited work to investigate the efficiency and practicality of existing volumetric video codecs. In this paper, we conduct a comparative measurement study on five representative point cloud-based volumetric video codecs, including 3 conventional codecs (e.g., Draco, G-PCC, V-PCC) and 2 neural-based codecs (e.g., PU-GCN+, MPU+), on six real volumetric video datasets. We investigate the applicability of the above codecs in different application scenarios, and examine how the features of point cloud (e.g., quality, texture, geometric and scene complexity) affect the quality of compressed volumetric video from multiple perspectives. In addition, we also study the impact of users’ viewing behaviors on the coding and delivery efficiency with Quality of Experience (QoE) metrics. Our results shed a number of important insights, which provide useful guidelines for optimizing the design of future volumetric video streaming systems.
Wavelet-Based Fast Decoding of 360° Videos

In this paper, we propose a wavelet-based video codec specifically designed for VR displays that enables real-time playback of high-resolution 360° videos. Our codec exploits the fact that only a fraction of the full 360° video frame is visible on the display at any time. To load and decode the video viewport-dependently in real time, we make use of the wavelet transform for intra- as well as inter-frame coding. Thereby, the relevant content is directly streamed from the drive, without the need to hold the entire frames in memory. With an average of 193 frames per second at 8192 × 8192 -pixel full-frame resolution, the conducted evaluation demonstrates that our codec's decoding performance is up to 272% higher than that of the state-of-the-art video codecs H.265 and AV1 for typical VR displays. By means of a perceptual study, we further illustrate the necessity of high frame rates for a better VR experience. Finally, we demonstrate how our wavelet-based codec can also directly be used in conjunction with foveation for further performance increase.
Two-Domain Joint Attention Mechanism Based on Sensor Data for Group Activity Recognition

In this article, a two-domain joint attention mechanism based on sensor data (TJAMSD) for group activity recognition (GAR) is proposed. We build two networks in the semantic domain and data domain as the teacher network and student network. In the data domain, a GAR network based on graph convolutional network (GCN) with group relation graph (GRG) is proposed. In this network, in order to reflect the relationship between individuals, the individual action feature correlation and position correlation in a group are calculated to construct two relation graphs. Then, the two relation graphs are fused to obtain the final GRG. Finally, the GRG and the individual action features obtained by a hybrid convolutional neural network (CNN) and bi-directional long short-term memory (BLSTM) network are used as the input of the GCN to infer the group activity. Besides, a semantic-domain network is constructed by the known individual action semantics and the group activity semantics. A joint attention mechanism based on the data-domain network and semantic-domain network is proposed. The attention weights learned in the semantic-domain network are used to guide the learning of attention weights in the data-domain network, which allocates attention to different individuals. In this way, TJAMSD makes the networks pay more attention to the key individual actions in the group and overcome the interference caused by noncritical individual actions in GAR. Experiments are conducted on two constructed datasets, the Garsensors dataset and the UT-Data-gar dataset. Different group cases are considered in the experiments and the experimental results show that in all cases, GCN with GRG can better express the interaction features of groups and improve the recognition performance. Furthermore, the TJAMSD can effectively suppress the interference of noncritical actions to advance the model robustness.
DisKeyword: Tweet Corpora Exploration for Keyword Selection

How to accelerate the search for relevant topical keywords within a tweet corpus? Computational social scientists conducting topical studies employ large, self-collected or crowdsourced social media datasets such as tweet corpora. Comprehensive sets of relevant keywords are often necessary to sample or analyze these data sources. However, naively skimming through thousands of keywords can quickly become a daunting task. In this study, we present a web-based application to simplify the search for relevant topical hashtags in a tweet corpus. DisKeyword allows users to grasp high-level trends in their dataset, while iteratively labeling keywords recommended based on their links to prior labeled hashtags. We open-source our code under the MIT license.
Friendly Conditional Text Generator

Our goal is to control text generation with more fine-grained conditions at lower computational cost than is possible with current alternatives; these conditions are attributes (i.e., multiple codes and free-text). As large-scale pre-trained language models (PLMs) offer excellent performance in free-form text generation, we explore efficient architectures and training schemes that can best leverage PLMs. Our framework, Friendly Conditional Text Generator (FCTG), introduces a multi-view attention (MVA) mechanism and two training tasks, Masked Attribute Modeling (MAM) and Attribute Linguistic Matching (ALM), to direct various PLMs via modalities between the text and its attributes. The motivation of FCTG is to map texts and attributes into a shared space, and bridge their modality gaps, as the texts and attributes reside in different regions of semantic space. To avoid catastrophic forgetting, modality-free embedded representations are learnt, and used to direct PLMs in this space, FCTG applies MAM to learn attribute representations, maps them in the same space as text through MVA, and optimizes their alignment in this space via ALM. Experiments on publicly available datasets show that FCTG outperforms baselines over higher level conditions at lower computation cost.
Applying of Websocket and WebRTC for Video Calling in Telemedicine during COVID-19 Pandemic

During the COVID-19 pandemic around the world, it is difficult to visit a doctor at the hospital. Many hospitals are trying to adapt to develop telemedicine systems to serve their patients. Hence the origin of this research as well is to develop a Telemedicine system to use Websocket and WebRTC have been applied to support video calling functionality in hopes of creating sessions to serve clients efficiently. This paper shows the results of the tests obtained when used in real-world applications on the internet which found that the results have been used network utility very efficiently. It averaged 1.86 Mbps for down link and averaged 1.66 Mbps for uplink bitrate, but still had high memory consumption. When compared to video calling active in the video conferencing system is still considered a waste of resources at 79% more than other systems.
CLNode: Curriculum Learning for Node Classification

Node classification is a fundamental graph-based task that aims to predict the classes of unlabeled nodes, for which Graph Neural Networks (GNNs) are the state-of-the-art methods. Current GNNs assume that nodes in the training set contribute equally during training. However, the quality of training nodes varies greatly, and the performance of GNNs could be harmed by two types of low-quality training nodes: (1) inter-class nodes situated near class boundaries that lack the typical characteristics of their corresponding classes. Because GNNs are data-driven approaches, training on these nodes could degrade the accuracy. (2) mislabeled nodes. In real-world graphs, nodes are often mislabeled, which can significantly degrade the robustness of GNNs. To mitigate the detrimental effect of the low-quality training nodes, we present CLNode, which employs a selective training strategy to train GNN based on the quality of nodes. Specifically, we first design a multi-perspective difficulty measurer to accurately measure the quality of training nodes. Then, based on the measured qualities, we employ a training scheduler that selects appropriate training nodes to train GNN in each epoch. To evaluate the effectiveness of CLNode, we conduct extensive experiments by incorporating it in six representative backbone GNNs. Experimental results on real-world networks demonstrate that CLNode is a general framework that can be combined with various GNNs to improve their accuracy and robustness.
Never Too Late to Learn: Regularizing Gender Bias in Coreference Resolution

Leveraging pre-trained language models (PLMs) as initializers for efficient transfer learning has become a universal approach for text-related tasks. However, the models not only learn the language understanding abilities but also reproduce prejudices for certain groups in the datasets used for pre-training. Recent studies show that the biased knowledge acquired from the datasets affects the model predictions on downstream tasks. In this paper, we mitigate and analyze the gender biases in PLMs with coreference resolution, which is one of the natural language understanding (NLU) tasks. PLMs exhibit two types of gender biases: stereotype and skew. The primary causes for the biases are the imbalanced datasets with more male examples and the stereotypical examples on gender roles. While previous studies mainly focused on the skew problem, we aim to mitigate both gender biases in PLMs while maintaining the model's original linguistic capabilities. Our method employs two regularization terms, Stereotype Neutralization (SN) and Elastic Weight Consolidation (EWC). The models trained with the methods show to be neutralized and reduce the biases significantly on the WinoBias dataset compared to the public BERT. We also invented a new gender bias quantification metric called the Stereotype Quantification (SQ) score. In addition to the metrics, embedding visualizations were used to interpret how our methods have successfully debiased the models.
Towards Decentralized Autonomous Digital Signatures Using Smart Contracts

Owing to the spread of COVID-19, the digitalization of various services is rapidly being promoted. In particular, online services such as obtaining a digital certificate (e.g., digital signature) from an authority are becoming increasingly important. Therefore, systems that can autonomously generate digital signatures are urgently required. However, the autonomous generation of signatures is difficult because the secret key for signatures must be strictly managed. Moreover, a decentralized autonomous systems should be publicly verifiable. Thus, schemes that preclude strict control of the secret key are desirable. In this study, we propose a new decentralized scheme that autonomously generates a digital signature without a secret key, using blockchain-based smart contracts. The fundamental concept behind our scheme is to eliminate secret keys by leveraging the closed nature of the processing operations of smart contracts within the blockchain; thus, the process of generating signatures and their output values satisfies the condition of immutability. Finally, we perform a security evaluation and feasibility study of our proposed scheme and show that it works securely on the Ethereum blockchain.