Reviewerly/exHarmony huggingface dataset:
Dynamically Weighted Balanced Loss: Class Imbalanced Learning and Confidence Calibration of Deep Neural Networks

Imbalanced class distribution is an inherent problem in many real-world classification tasks where the minority class is the class of interest. Many conventional statistical and machine learning classification algorithms are subject to frequency bias, and learning discriminating boundaries between the minority and majority classes could be challenging. To address the class distribution imbalance in deep learning, we propose a class rebalancing strategy based on a class-balanced dynamically weighted loss function where weights are assigned based on the class frequency and predicted probability of ground-truth class. The ability of dynamic weighting scheme to self-adapt its weights depending on the prediction scores allows the model to adjust for instances with varying levels of difficulty resulting in gradient updates driven by hard minority class samples. We further show that the proposed loss function is classification calibrated. Experiments conducted on highly imbalanced data across different applications of cyber intrusion detection (CICIDS2017 data set) and medical imaging (ISIC2019 data set) show robust generalization. Theoretical results supported by superior empirical performance provide justification for the validity of the proposed dynamically weighted balanced (DWB) loss function.
The impact of a 360° virtual tour on the reduction of psychological stress caused by COVID-19

The continuous spread of the COVID-19 pandemic is causing people to feel anxiety and stress. This study constructs a four-layer research model to examine how a 360° virtual tour can reduce people's psychological stress through two types of presence (the sense of presence and telepresence) and affective-motivational states (enjoyment and involvement) in this extraordinary period of the COVID-19 pandemic. In order to test the moderating effect of involvement, partial least squares (PLS) analysis is employed to analyse valid data collected from 235 individuals. The results of this study indicate that telepresence has a higher impact in generating affective-motivational states than the sense of presence. Among the factors, enjoyment shows the highest effect on satisfaction with the 360° virtual tour experience and stress reduction; involvement moderates the effect of telepresence on satisfaction with the 360° virtual tour experience. This study also contributes to virtual reality research by distinguishing the concepts of 'sense of presence' and 'telepresence' as well as demonstrating the mechanisms whereby virtual reality technology influences people's psychological well-being. Timely recommendations are provided for people in order to reduce psychological stress during and after COVID-19 pandemic.
A two-stage hybrid ant colony optimization for high-dimensional feature selection

Ant colony optimization (ACO) is widely used in feature selection owing to its excellent global/local search capabilities and flexible graph representation. However, the current ACO-based feature selection methods are mainly applied to low-dimensional datasets. For thousands of dimensional datasets, the search for the optimal feature subset (OFS) becomes extremely difficult due to the exponential increase of the search space. In this paper, we propose a two-stage hybrid ACO for high-dimensional feature selection (TSHFS-ACO). As an additional stage, it uses the interval strategy to determine the size of OFS for the following OFS search. Compared to the traditional one-stage methods that determine the size of OFS and search for OFS simultaneously, the stage of checking the performance of partial feature number endpoints in advance helps to reduce the complexity of the algorithm and alleviate the algorithm from getting into a local optimum. Moreover, the advanced ACO algorithm embeds the hybrid model, which uses the features’ inherent relevance attributes and the classification performance to guide OFS search. The test results on eleven high-dimensional public datasets show that TSHFS-ACO is suitable for high-dimensional feature selection. The obtained OFS has state-of-the-art performance on most datasets. And compared with other ACO-based feature selection methods, TSHFS-ACO has a shorter running time.
Blockchain for decentralized <scp>multi‐drone</scp> to combat <scp>COVID</scp>‐19 and future pandemics: Framework and proposed solutions

Abstract Currently, drones represent a promising technology for combating Coronavirus disease 2019 (COVID‐19) due to the transport of goods, medical supplies to a given target location in the quarantine areas experiencing an epidemic outbreak. Drone missions will increasingly rely on drone collaboration, which requires the drones to reduce communication complexity and be controlled in a decentralized fashion. Blockchain technology becomes a must in industrial applications because it provides decentralized data, accessibility, immutability, and irreversibility. Therefore, Blockchain makes data public for all drones and enables drones to log information concerning world states, time, location, resources, delivery data, and drone relation to all neighbors drones. This paper introduces decentralized independent multi‐drones to accomplish the task collaboratively. Improving blockchain with a consensus algorithm can improve network partitioning and scalability in order to combat COVID‐19. The multi‐drones task is to combat COVID‐19 via monitoring and detecting, social distancing, sanitization, data analysis, delivering goods and medical supplies, and announcement while avoiding collisions with one another. We discuss End to End (E2E) delivery application of combination blockchain and multi‐drone in combating COVID‐19 and beyond future pandemics. Furthermore, the challenges and opportunities of our proposed framework are highlighted.
An Effective Cloud Workflow Scheduling Approach Combining PSO and Idle Time Slot-Aware Rules

Workflow scheduling is a key issue and remains a challenging problem in cloud computing. Faced with the large number of virtual machine (VM) types offered by cloud providers, cloud users need to choose the most appropriate VM type for each task. Multiple task scheduling sequences exist in a workflow application. Different task scheduling sequences have a significant impact on the scheduling performance. It is not easy to determine the most appropriate set of VM types for tasks and the best task scheduling sequence. Besides, the idle time slots on VM instances should be used fully to increase resources' utilization and save the execution cost of a workflow. This paper considers these three aspects simultaneously and proposes a cloud workflow scheduling approach which combines particle swarm optimization (PSO) and idle time slot-aware rules, to minimize the execution cost of a workflow application under a deadline constraint. A new particle encoding is devised to represent the VM type required by each task and the scheduling sequence of tasks. An idle time slot-aware decoding procedure is proposed to decode a particle into a scheduling solution. To handle tasks' invalid priorities caused by the randomness of PSO, a repair method is used to repair those priorities to produce valid task scheduling sequences. The proposed approach is compared with state-of-the-art cloud workflow scheduling algorithms. Experiments show that the proposed approach outperforms the comparative algorithms in terms of both of the execution cost and the success rate in meeting the deadline.
A data augmentation approach to XGboost-based mineral potential mapping: An example of carbonate-hosted Zn Pb mineral systems of Western Iran

This study intends to showcase the application of Extreme Gradient boosting (XGboost), a state-of-the-art ensemble-learning technique, for district-scale mineral potential mapping (MPM) of carbonate-hosted ZnPb mineral systems in the Emarat district, W Iran. Notwithstanding the demonstrated effectiveness of XGboost in addressing a range of classification and regression problems, the inadequate number of deposit locations employed as labeled samples, which is intrinsic to district-scale MPM, impedes the practical application of this method. Given this notion, an iterative window-based data augmentation technique was proposed and applied to generate additional labeled samples. The predictive models derived by XGboost were compared to those generated by random forests (RF). Both methods were applied to a set of mineral systems-derived exploration targeting criteria representing critical ore-forming processes. Setting aside the similarities of predictive models derived by both methods, it was recognized that RF is much less sensitive to insufficient labeled samples. It was also recognized that, unlike RF, many parameters require tuning for generating an optimum XGboost-based predictive model. Nevertheless, given the achieved results, XGboost slightly outperformed RF in generating an optimum predictive model. Therefore, XGboost coupled to data augmentation can be deemed a viable alternative for district-scale, data-driven MPM.
Integration of blockchain and federated learning for Internet of Things: Recent advances and future challenges

The role of the Internet of Things (IoT) in the revolutionized society cannot be overlooked. The IoT can leverage advanced machine learning (ML) algorithms for its applications. However, given the fact of massive data, which is stored at a central cloud server, adopting centralized machine learning algorithms is not a viable option due to immense computation cost and privacy leakage issues. Given such conditions, blockchain can be leveraged to enhance the privacy of IoT networks by making them decentralized without any central authority. Nevertheless, the sensitive and massive data that is stored in distributive fashion, leveraged it for application purpose, is still a challenging task. To overcome this challenging task, federated learning (FL), which is a new breed of ML is the most promising solution that brings learning to the end devices without sharing the private data to the central server. In the FL mechanism, the central server act as an orchestrator to start the FL learning process, and only model parameters' updates are shared between end devices and the central orchestrator. Although FL can provide better privacy and data management, it is still in the development phase and has not been adopted by various communities due to its unknown privacy issues. In this paper first, we present the notion of blockchain and its application in IoT systems. Then we describe the privacy issues related to the implementation of blockchain in IoT and present privacy preservation techniques to cope with the privacy issues. Second, we introduce the FL application in IoT systems, devise a taxonomy, and present privacy threats in FL. Afterward, we present IoT-based use cases on envisioned dispersed federated learning and introduce blockchain-based traceability functions to improve privacy. Finally, open research gaps are addressed for future work.
Scale-Aware Graph Neural Network for Few-Shot Semantic Segmentation

Few-shot semantic segmentation (FSS) aims to segment unseen class objects given very few densely-annotated support images from the same class. Existing FSS methods find the query object by using support prototypes or by directly relying on heuristic multi-scale feature fusion. However, they fail to fully leverage the high-order appearance relationships between multi-scale features among the support-query image pairs, thus leading to an inaccurate localization of the query objects. To tackle the above challenge, we propose an end-to-end scale-aware graph neural network (SAGNN) by reasoning the cross-scale relations among the support-query images for FSS. Specifically, a scale-aware graph is first built by taking support-induced multi-scale query features as nodes and, meanwhile, each edge is modeled as the pairwise interaction of its connected nodes. By progressive message passing over this graph, SAGNN is capable of capturing cross-scale relations and overcoming object variations (e.g., appearance, scale and location), and can thus learn more precise node embeddings. This in turn enables it to predict more accurate foreground objects. Moreover, to make full use of the location relations across scales for the query image, a novel self-node collaboration mechanism is proposed to enrich the current node, which endows SAGNN the ability of perceiving different resolutions of the same objects. Extensive experiments on PASCAL-5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> and COCO-20 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> show that SAGNN achieves state-of-the-art results.
PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

Existing 3D human pose estimators suffer poor generalization performance to new datasets, largely due to the limited diversity of 2D-3D pose pairs in the training data. To address this problem, we present PoseAug, a new auto-augmentation framework that learns to augment the available training poses towards a greater diversity and thus improve generalization of the trained 2D-to-3D pose estimator. Specifically, PoseAug introduces a novel pose augmentor that learns to adjust various geometry factors (e.g., posture, body size, view point and position) of a pose through differentiable operations. With such differentiable capacity, the augmentor can be jointly optimized with the 3D pose estimator and take the estimation error as feedback to generate more diverse and harder poses in an online manner. Moreover, PoseAug introduces a novel part-aware Kinematic Chain Space for evaluating local joint-angle plausibility and develops a discriminative module accordingly to ensure the plausibility of the augmented poses. These elaborate designs enable PoseAug to generate more diverse yet plausible poses than existing offline augmentation methods, and thus yield better generalization of the pose estimator. PoseAug is generic and easy to be applied to various 3D pose estimators. Extensive experiments demonstrate that PoseAug brings clear improvements on both intra-scenario and cross-scenario datasets. Notably, it achieves 88.6% 3D PCK on MPI-INF-3DHP under cross-dataset evaluation setup, improving upon the previous best data augmentation based method [22] by 9.1%. Code can be found at: https://github.com/jfzhang95/PoseAug.
Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model

Recently, intelligent video surveillance applications have become essential in public security by the use of computer vision technologies to investigate and understand long video streams. Anomaly detection and classification are considered a major element of intelligent video surveillance. The aim of anomaly detection is to automatically determine the existence of abnormalities in a short time period. Deep reinforcement learning (DRL) techniques can be employed for anomaly detection, which integrates the concepts of reinforcement learning and deep learning enabling the artificial agents in learning the knowledge and experience from actual data directly. With this motivation, this paper presents an Intelligent Video Anomaly Detection and Classification using Faster RCNN with Deep Reinforcement Learning Model, called IVADC-FDRL model. The presented IVADC-FDRL model operates on two major stages namely anomaly detection and classification. Firstly, Faster RCNN model is applied as an object detector with Residual Network as a baseline model, which detects the anomalies as objects. Besides, deep Q-learning (DQL) based DRL model is employed for the classification of detected anomalies. In order to validate the effective anomaly detection and classification performance of the IVADC-FDRL model, an extensive set of experimentations were carried out on the benchmark UCSD anomaly dataset. The experimental results showcased the better performance of the IVADC-FDRL model over the other compared methods with the maximum accuracy of 98.50% and 94.80% on the applied Test004 and Test007 dataset respectively.
How Well Do Self-Supervised Models Transfer?

Self-supervised visual representation learning has seen huge progress recently, but no large scale evaluation has compared the many models now available. We evaluate the transfer performance of 13 top self-supervised models on 40 downstream tasks, including many-shot and few-shot recognition, object detection, and dense prediction. We compare their performance to a supervised baseline and show that on most tasks the best self-supervised models outperform supervision, confirming the recently observed trend in the literature. We find ImageNet Top-1 accuracy to be highly correlated with transfer to many-shot recognition, but increasingly less so for few-shot, object detection and dense prediction. No single self-supervised method dominates overall, suggesting that universal pre-training is still unsolved. Our analysis of features suggests that top self-supervised learners fail to preserve colour information as well as supervised alternatives, but tend to induce better classifier calibration, and less attentive overfitting than supervised learners.
Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation

Scene graph generation is an important visual understanding task with a broad range of vision applications. Despite recent tremendous progress, it remains challenging due to the intrinsic long-tailed class distribution and large intra-class variation. To address these issues, we introduce a novel confidence-aware bipartite graph neural network with adaptive message propagation mechanism for unbiased scene graph generation. In addition, we propose an efficient bi-level data resampling strategy to alleviate the imbalanced data distribution problem in training our graph network. Our approach achieves superior or competitive performance over previous methods on several challenging datasets, including Visual Genome, Open Images V4/V6, demonstrating its effectiveness and generality.
TF-Blender: Temporal Feature Blender for Video Object Detection

Video objection detection is a challenging task because isolated video frames may encounter appearance deterioration, which introduces great confusion for detection. One of the popular solutions is to exploit the temporal information and enhance per-frame representation through aggregating features from neighboring frames. Despite achieving improvements in detection, existing methods focus on the selection of higher-level video frames for aggregation rather than modeling lower-level temporal relations to increase the feature representation. To address this limitation, we propose a novel solution named TF-Blender, which includes three modules: 1) Temporal relation models the relations between the current frame and its neigh-boring frames to preserve spatial information. 2). Feature adjustment enriches the representation of every neigh-boring feature map; 3) Feature blender combines outputs from the first two modules and produces stronger features for the later detection tasks. For its simplicity, TF-Blender can be effortlessly plugged into any detection network to improve detection behavior. Extensive evaluations on ImageNet VID and YouTube-VIS benchmarks indicate the performance guarantees of using TF-Blender on recent state-of-the-art methods. Code is available at https://github.com/goodproj13/TF-Blender.
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting

We propose ResRep, a novel method for lossless channel pruning (a.k.a. filter pruning), which slims down a CNN by reducing the width (number of output channels) of convolutional layers. Inspired by the neurobiology research about the independence of remembering and forgetting, we propose to re-parameterize a CNN into the remembering parts and forgetting parts, where the former learn to maintain the performance and the latter learn to prune. Via training with regular SGD on the former but a novel update rule with penalty gradients on the latter, we realize structured sparsity. Then we equivalently merge the remembering and forgetting parts into the original architecture with narrower layers. In this sense, ResRep can be viewed as a successful application of Structural Re-parameterization. Such a methodology distinguishes ResRep from the traditional learning-based pruning paradigm that applies a penalty on parameters to produce sparsity, which may suppress the parameters essential for the remembering. ResRep slims down a standard ResNet-50 with 76.15% accuracy on ImageNet to a narrower one with only 45% FLOPs and no accuracy drop, which is the first to achieve lossless pruning with such a high compression ratio. The code and models are at https://github.com/DingXiaoH/ResRep.
A survey of visual analytics for Explainable Artificial Intelligence methods

Deep learning (DL) models have achieved impressive performance in various domains such as medicine, finance, and autonomous vehicle systems with advances in computing power and technologies. However, due to the black-box structure of DL models, the decisions of these learning models often need to be explained to end-users. Explainable Artificial Intelligence (XAI) provides explanations of black-box models to reveal the behavior and underlying decision-making mechanisms of the models through tools, techniques, and algorithms. Visualization techniques help to present model and prediction explanations in a more understandable, explainable, and interpretable way. This survey paper aims to review current trends and challenges of visual analytics in interpreting DL models by adopting XAI methods and present future research directions in this area. We reviewed literature based on two different aspects, model usage and visual approaches. We addressed several research questions based on our findings and then discussed missing points, research gaps, and potential future research directions. This survey provides guidelines to develop a better interpretation of neural networks through XAI methods in the field of visual analytics.
Bitcoin's growing e-waste problem

Bitcoin's increasing energy consumption has triggered a passionate debate about the sustainability of the digital currency. And yet, most studies have thus far ignored that Bitcoin miners cycle through a growing amount of short-lived hardware that could exacerbate the growth in global electronic waste. E-waste represents a growing threat to our environment, from toxic chemicals and heavy metals leaching into soils, to air and water pollutions caused by improper recycling. Here we present a methodology to estimate Bitcoin's e-waste and find that it adds up to 30.7 metric kilotons annually, per May 2021. This number is comparable to the amount of small IT and telecommunication equipment waste produced by a country like the Netherlands. At peak Bitcoin price levels seen early in 2021, the annual amount of e-waste may grow beyond 64.4 metric kilotons in the midterm, which highlights the dynamic trend if the Bitcoin price rises further. Moreover, the demand for mining hardware already today disrupts the global semiconductor supply chain. The strategies we present may help to mitigate Bitcoin's growing e-waste problem.