Reviewerly/exHarmony huggingface dataset:
Eye-Tracking Assistive Technologies for Individuals With Amyotrophic Lateral Sclerosis

Amyotrophic lateral sclerosis, also known as ALS, is a progressive nervous system disorder that affects nerve cells in the brain and spinal cord, resulting in the loss of muscle control.For individuals with ALS, where mobility is limited to the movement of the eyes, the use of eye-tracking-based applications can be applied to achieve some basic tasks with certain digital interfaces.This paper presents a review of existing eye-tracking software and hardware through which eye-tracking their application is sketched as an assistive technology to cope with ALS.Eye-tracking also provides a suitable alternative as control of game elements.Furthermore, artificial intelligence has been utilized to improve eye-tracking technology with significant improvement in calibration and accuracy.Gaps in literature are highlighted in the study to offer a direction for future research.
RB-Net: Training Highly Accurate and Efficient Binary Neural Networks With Reshaped Point-Wise Convolution and Balanced Activation

In this paper, we find that the conventional convolution operation becomes the bottleneck for extremely efficient binary neural networks (BNNs). To address this issue, we open up a new direction by introducing a reshaped point-wise convolution (RPC) to replace the conventional one to build BNNs. Specifically, we conduct a point-wise convolution after rearranging the spatial information into depth, with which at least <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.25\times $ </tex-math></inline-formula> computation reduction can be achieved. Such an efficient RPC allows us to explore more powerful representational capacity of BNNs under a given computation complexity budget. Moreover, we propose to use a balanced activation (BA) to adjust the distribution of the scaled activations after binarization, which enables significant performance improvement of BNNs. After integrating RPC and BA, the proposed network, dubbed as RB-Net, strikes a good trade-off between accuracy and efficiency, achieving superior performance with lower computational cost against the state-of-the-art BNN methods. Specifically, our RB-Net achieves 66.8% Top-1 accuracy with ResNet-18 backbone on ImageNet, exceeding the state-of-the-art Real-to-Binary Net (65.4%) by 1.4% while achieving more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3\times $ </tex-math></inline-formula> reduction (52M vs. 165M) in computational complexity.
A multi-stream deep neural network with late fuzzy fusion for real-world anomaly detection

Abnormal event detection in video is alternatively known as outlier detection, where machine learning can be highly effective. While testing an unknown video, the objective of such methods is to verify the video’s category, e.g. normal or abnormal. This paper exploits visual information from normal as well as abnormal videos to train a deep multiple instance learning classifier for classification of videos. Existing multiple instance learning classifiers presume that the training videos contain only short-duration anomalous events. This assumption may not be valid for all real-world anomalies. Also, multiple occurrences of anomalies in training videos cannot be ruled out. This paper shows that by injecting temporal information in feature extraction, anomaly detection performance can be improved. To accomplish this, two spatio-temporal deep feature extractors have been applied in parallel on the training videos. These streams are then used to train a modified multiple instance learning-based classifier. Finally, a fuzzy aggregation is employed to fuse the anomaly scores. Additionally, two lightweight deep-learning classifiers have been used to substantiate the model’s efficacy for classifying fire and accident events. To understand the reliability and performance of the proposed method, extensive experiments have been carried out using UCF-Crime video dataset containing 13 anomaly categories. The dataset has been restructured into five broad categories based on the severity of actions to study the robustness of the proposed method. The paper provides sufficient empirical evidence which proves that by incorporating temporal features in the pipeline, anomaly detection accuracy can be significantly improved. Moreover, the model helps to detect long-duration anomalies in videos, which was not possible using existing methods. The proposed end-to-end multi-stream architecture performs abnormal event detection with accuracy as high as 84.48%, which is better than the performance of existing video anomaly detection methods. Moreover, the class-wise detection accuracy has improved by 6%–14% across various broad categories.
Vehicle-Road Environment Perception Under Low-Visibility Condition Based on Polarization Features via Deep Learning

Visual perception system is the key component of safety driving assistance and unmanned driving systems, and the perceptive performance of which will directly affect the running safety. There are various perception algorithms for the visual system, especially in recent years, deep learning algorithms are adopted with increasing popularity among them. However, the majority of existing works mainly focuses on daytime scenes with favorable illumination and weather conditions, and relies on visible light with intensity imaging equipment. This paper is based on polarization features acquired by deep learning methods, attempting to address the problems about vehicle-road environment perception in Low-Visibility Condition. Polarization is one of the essential properties of matter, here we propose a new method by studying and analyzing the transmission characteristics of the target’s polarization information. In the proposed method, the polarization features of vehicle-road environment are first acquired by a kind of three-channel polarization imaging device, then, fused together with an intensity image, and finally followed by a semantic segmentation operation by deep networks. The experimental results clearly demonstrate that the effect of perception based on polarization features via deep learning are greatly improved compared with intensity imaging, especially in low illumination and severe weather conditions. It is of great value for environmental robustness of visual perception system and further traffic safety.
Seeing Like a Human: Asynchronous Learning With Dynamic Progressive Refinement for Person Re-Identification

Learning discriminative and rich features is an important research task for person re-identification. Previous studies have attempted to capture global and local features at the same time and layer of the model in a non-interactive manner, which are called synchronous learning. However, synchronous learning leads to high similarity, and further defects in model performance. To this end, we propose asynchronous learning based on the human visual perception mechanism. Asynchronous learning emphasizes the time asynchrony and space asynchrony of feature learning and achieves mutual promotion and cyclical interaction for feature learning. Furthermore, we design a dynamic progressive refinement module to improve local features with the guidance of global features. The dynamic property allows this module to adaptively adjust the network parameters according to the input image, in both the training and testing stage. The progressive property narrows the semantic gap between the global and local features, which is due to the guidance of global features. Finally, we have conducted several experiments on four datasets, including Market1501, CUHK03, DukeMTMC-ReID, and MSMT17. The experimental results show that asynchronous learning can effectively improve feature discrimination and achieve strong performance.
ADFL: A Poisoning Attack Defense Framework for Horizontal Federated Learning

Recently, federated learning has received widespread attention, which will promote the implementation of artificial intelligence technology in various fields. Privacy-preserving technologies are applied to users’ local models to protect users’ privacy. Such operations make the server not see the true model parameters of each user, which opens wider door for a malicious user to upload malicious parameters and make the training result converge to an ineffective model. To solve this problem, in this article, we propose a poisoning attack defense framework for horizontal federated learning systems called ADFL. Specifically, we design a proof generation method for users to generate proofs to verify whether it is malicious or not. An aggregation rule is also proposed to make sure the global model has a high accuracy. Several verification experiments were conducted and the results show that our method can detect malicious user effectively and ensure the global model has a high accuracy.
Deep Perceptual Enhancement for Medical Image Analysis

Due to numerous hardware shortcomings, medical image acquisition devices are susceptible to producing low-quality (i.e., low contrast, inappropriate brightness, noisy, etc.) images. Regrettably, perceptually degraded images directly impact the diagnosis process and make the decision-making manoeuvre of medical practitioners notably complicated. This study proposes to enhance such low-quality images by incorporating end-to-end learning strategies for accelerating medical image analysis tasks. To the best concern, this is the first work in medical imaging which comprehensively tackles perceptual enhancement, including contrast correction, luminance correction, denoising, etc., with a fully convolutional deep network. The proposed network leverages residual blocks and a residual gating mechanism for diminishing visual artefacts and is guided by a multi-term objective function to perceive the perceptually plausible enhanced images. The practicability of the deep medical image enhancement method has been extensively investigated with sophisticated experiments. The experimental outcomes illustrate that the proposed method could outperform the existing enhancement methods for different medical image modalities by 5.00 to 7.00 dB in peak signal-to-noise ratio (PSNR) metrics and 4.00 to 6.00 in DeltaE metrics. Additionally, the proposed method can drastically improve the medical image analysis tasks' performance and reveal the potentiality of such an enhancement method in real-world applications.
eFL-Boost: Efficient Federated Learning for Gradient Boosting Decision Trees

Privacy protection has attracted increasing attention, and privacy concerns often prevent flexible data utilization.In most industries, data are distributed across multiple organizations due to privacy concerns.Federated learning (FL), which enables cross-organizational machine learning by communicating statistical information, is a state-of-the-art technology that is used to solve this problem.However, for gradient boosting decision tree (GBDT) in FL, balancing communication efficiency and security while maintaining sufficient accuracy remains an unresolved problem.In this paper, we propose an FL scheme for GBDT, i.e., efficient FL for GBDT (eFL-Boost), which minimizes accuracy loss, communication costs, and information leakage.The proposed scheme focuses on appropriate allocation of local computation (performed individually by each organization) and global computation (performed cooperatively by all organizations) when updating a model.It is known that tree structures incur high communication costs for global computation, whereas leaf weights do not require such costs and are expected to contribute relatively more to accuracy.Thus, in the proposed eFL-Boost, a tree structure is determined locally at one of the organizations, and leaf weights are calculated globally by aggregating the local gradients of all organizations.Specifically, eFL-Boost requires only three communications per update, and only statistical information that has low privacy risk is leaked to other organizations.Through performance evaluation on public data sets (ROC AUC, Log loss, and F1-score are used as metrics), the proposed eFL-Boost outperforms existing schemes that incur low communication costs and was comparable to a scheme that offers no privacy protection.
Integration of fuzzy logic and a convolutional neural network in three-way decision-making

With the increase in web usage, opinion mining has become a new trend in analysing opinions. Nevertheless, opinion mining still faces huge challenges, such as uncertainty in opinions which can make opinions difficult to interpret using existing opinion mining models. These models have been developed to automate the opinion mining process of the user; however, extant models which use machine learning algorithms have limitations dealing with uncertainties in opinions such as online customer reviews. Fuzzy models were introduced to solve this problem. However, the fuzzy models have issues in regard to large uncertain boundaries. To address this serious issue, this research introduces a decision framework via which positive, negative and boundary regions are classified using fuzzy concepts. Then, a convolutional neural network (CNN) is used to further classify fuzzy concepts originally allocated to the boundary region. The framework uses formal concepts to represent uncertainties and the CNN classifies the boundary region concepts into either positive or negative opinions. A series of experiments was conducted, the outcomes of which suggest the uncertainties in opinions can be effectively handled using our three-way decision-making framework. Our work contributes to boosting the performance of opinion classification models.
Weakly Supervised RGB-D Salient Object Detection With Prediction Consistency Training and Active Scribble Boosting

RGB-D salient object detection (SOD) has attracted increasingly more attention as it shows more robust results in complex scenes compared with RGB SOD. However, state-of-the-art RGB-D SOD approaches heavily rely on a large amount of pixel-wise annotated data for training. Such densely labeled annotations are often labor-intensive and costly. To reduce the annotation burden, we investigate RGB-D SOD from a weakly supervised perspective. More specifically, we use annotator-friendly scribble annotations as supervision signals for model training. Since scribble annotations are much sparser compared to ground-truth masks, some critical object structure information might be neglected. To preserve such structure information, we explicitly exploit the complementary edge information from two modalities (i.e., RGB and depth). Specifically, we leverage the dual-modal edge guidance and introduce a new network architecture with a dual-edge detection module and a modality-aware feature fusion module. In order to use the useful information of unlabeled pixels, we introduce a prediction consistency training scheme by comparing the predictions of two networks optimized by different strategies. Moreover, we develop an active scribble boosting strategy to provide extra supervision signals with negligible annotation cost, leading to significant SOD performance improvement. Extensive experiments on seven benchmarks validate the superiority of our proposed method. Remarkably, the proposed method with scribble annotations achieves competitive performance in comparison to fully supervised state-of-the-art methods.
Attitudes and Folk Theories of Data Subjects on Transparency and Accuracy in Emotion Recognition

The growth of technologies promising to infer emotions raises political and ethical concerns, including concerns regarding their accuracy and transparency. A marginalized perspective in these conversations is that of data subjects potentially affected by emotion recognition. Taking social media as one emotion recognition deployment context, we conducted interviews with data subjects (i.e., social media users) to investigate their notions about accuracy and transparency in emotion recognition and interrogate stated attitudes towards these notions and related folk theories. We find that data subjects see accurate inferences as uncomfortable and as threatening their agency, pointing to privacy and ambiguity as desired design principles for social media platforms. While some participants argued that contemporary emotion recognition must be accurate, others raised concerns about possibilities for contesting the technology and called for better transparency. Furthermore, some challenged the technology altogether, highlighting that emotions are complex, relational, performative, and situated. In interpreting our findings, we identify new folk theories about accuracy and meaningful transparency in emotion recognition. Overall, our analysis shows an unsatisfactory status quo for data subjects that is shaped by power imbalances and a lack of reflexivity and democratic deliberation within platform governance.
Background Activation Suppression for Weakly Supervised Object Localization

Weakly supervised object localization (WSOL) aims to localize objects using only image-level labels. Recently a new paradigm has emerged by generating a foreground prediction map (FPM) to achieve localization task. Existing FPM-based methods use cross-entropy (CE) to evaluate the foreground prediction map and to guide the learning of generator. We argue for using activation value to achieve more efficient learning. It is based on the experimental observation that, for a trained network, CE converges to zero when the foreground mask covers only part of the object region. While activation value increases until the mask expands to the object boundary, which indicates that more object areas can be learned by using activation value. In this paper, we propose a Background Activation Suppression (BAS) method. Specifically, an Activation Map Constraint module (AMC) is designed to facilitate the learning of generator by suppressing the background activation value. Meanwhile, by using the foreground region guidance and the area constraint, BAS can learn the whole region of the object. In the inference phase, we consider the prediction maps of different categories together to obtain the final localization results. Extensive experiments show that BAS achieves significant and consistent improvement over the baseline methods on the CUB-200-2011 and ILSVRC datasets. Code and models are available at github.com/wpy1999IBAS.
Integrated Network and Security Operation Center: A Systematic Analysis

Traditionally, network and security operation center teams have worked in silos despite commonalities. The network operating center (NOC) team is to provide operationality and availability of information technology (IT) assets, while the security operation center (SOC) team is to ensure IT assets security and protect them from cyber-security attacks. The convergence in IT assets and exponential growth in cyber-security threats in the present digital-online scenario have created many challenges in maintaining network and IT assets effectively and protecting them. It is vital to break these silos and bring them under one integrated unit to effectively counter cyber-security attacks, threats, and vandalism at a reduced operational cost. Despite its necessity, the relevant literature lacks an opinion. It focuses mainly on conceptual segments instead of a holistic view of an integrated NOC and SOC architecture, limiting further innovations in the field. A systematic literature review and analysis is conducted to collate and understand current research ideas in this paper. The mapped relevant literature and our expertise have been then used to propose the implementable state-of-the-art architecture of an integrated NOC and SOC, its definition, the main building blocks and its usefulness for the organizations. Only explicit knowledge of people is considered while neglecting the tacit knowledge in automating and integrating the processes of NOC and SOC, which is the major limitation of the relevant literature. Taping people tacit knowledge is necessary for utilizing the entire caliber of the NOC and SOC integration in the future.
A Sustainable, Interactive Elderly Healthcare System for Nursing Homes: An Interdisciplinary Design

Population ageing becomes a perplexing conundrum with social and economic development. Many senior citizens are now empty nesters because the younger generation prefer to stay in metropolises for a better life. Therefore, living in a nursing home is a popular choice for the aged. This objective-oriented paper proposes a sustainable elderly healthcare system for nursing homes. The main work is the design and implementation of a new rapid and interactive assistance service. Based on cost-effective fingerprint indoor-positioning technology, the alert message that a person is at risk will be immediately sent to nearby people before professionals arrive. Warning messages are available when nearing marked areas (e.g., slippery floors). The parallel path-finding algorithm plays a significant role in finding nearby people and alerting people who approach specific areas. Furthermore, this system provides application programming interfaces to connect to health devices, such as smart bracelets, watches, and glasses. In general, the system is designed to ensure the safety of the elderly and improve management efficiency, which corresponds to present smart elderly care proposals from governments.
Communication Efficient Federated Learning With Energy Awareness Over Wireless Networks

In federated learning (FL), reducing the communication overhead is one of the most critical challenges since the parameter server and the mobile devices share the training parameters over wireless links. With such consideration, we adopt the idea of SignSGD in which only the signs of the gradients are exchanged. Moreover, most of the existing works assume Channel State Information (CSI) available at both the mobile devices and the parameter server, and thus the mobile devices can adopt fixed transmission rates dictated by the channel capacity. In this work, only the parameter server side CSI is assumed, and channel capacity with outage is considered. In this case, an essential problem for the mobile devices is to select appropriate local processing and communication parameters (including the transmission rates) to achieve a desired balance between the overall learning performance and their energy consumption. Two optimization problems are formulated and solved, which optimize the learning performance given the energy consumption requirement, and vice versa. Furthermore, considering that the data may be distributed across the mobile devices in a highly uneven fashion in FL, a stochastic sign-based algorithm is proposed. Extensive simulations are performed to demonstrate the effectiveness of the proposed methods.
KD-Box: Line-segment-based KD-tree for Interactive Exploration of Large-scale Time-Series Data

Time-series data-usually presented in the form of lines-plays an important role in many domains such as finance, meteorology, health, and urban informatics. Yet, little has been done to support interactive exploration of large-scale time-series data, which requires a clutter-free visual representation with low-latency interactions. In this paper, we contribute a novel line-segment-based KD-tree method to enable interactive analysis of many time series. Our method enables not only fast queries over time series in selected regions of interest but also a line splatting method for efficient computation of the density field and selection of representative lines. Further, we develop KD-Box, an interactive system that provides rich interactions, e.g., timebox, attribute filtering, and coordinated multiple views. We demonstrate the effectiveness of KD-Box in supporting efficient line query and density field computation through a quantitative comparison and show its usefulness for interactive visual analysis on several real-world datasets.