Reviewerly/exHarmony huggingface dataset:
The Sailfish Optimizer: A novel nature-inspired metaheuristic algorithm for solving constrained engineering optimization problems

Nature-inspired optimization algorithms, especially swarm based algorithms (SAs), solve many scientific and engineering problems due to their flexibility and simplicity. These algorithms are applicable for optimization problems without structural modifications. This work presents a novel nature-inspired metaheuristic optimization algorithm, called SailFish Optimizer (SFO), which is inspired by a group of hunting sailfish. This method consists of two tips of populations, sailfish population for intensification of the search around the best so far and sardines population for diversification of the search space. The SFO algorithm is evaluated on 20 well-known unimodal and multimodal mathematical functions to test different characteristics of the algorithm. In addition, SFO is compared with the six state-of-art metaheuristic algorithms in low and high dimensions. It also indicates competitive results for improvement of exploration and exploitation phases, avoidance of local optima, and high speed convergence especially on large-scale global optimization. The SFO algorithm outperforms the best algorithms in the literature on the majority of the test functions and it shows the statistically significant difference among other algorithms. Moreover, the SFO algorithm shows significantly great results for non-convex, non-separable and scalable test functions. Eventually, the promising results on five real world optimization problems indicate that the SFO is applicable for problem solving with constrained and unknown search spaces.
VerifyNet: Secure and Verifiable Federated Learning

As an emerging training model with neural networks, federated learning has received widespread attention due to its ability to update parameters without collecting users' raw data. However, since adversaries can track and derive participants' privacy from the shared gradients, federated learning is still exposed to various security and privacy threats. In this paper, we consider two major issues in the training process over deep neural networks (DNNs): 1) how to protect user's privacy (i.e., local gradients) in the training process and 2) how to verify the integrity (or correctness) of the aggregated results returned from the server. To solve the above problems, several approaches focusing on secure or privacy-preserving federated learning have been proposed and applied in diverse scenarios. However, it is still an open problem enabling clients to verify whether the cloud server is operating correctly, while guaranteeing user's privacy in the training process. In this paper, we propose VerifyNet, the first privacy-preserving and verifiable federated learning framework. In specific, we first propose a double-masking protocol to guarantee the confidentiality of users' local gradients during the federated learning. Then, the cloud server is required to provide the Proof about the correctness of its aggregated results to each user. We claim that it is impossible that an adversary can deceive users by forging Proof, unless it can solve the NP-hard problem adopted in our model. In addition, VerifyNet is also supportive of users dropping out during the training process. The extensive experiments conducted on real-world data also demonstrate the practical performance of our proposed scheme.
Attention-guided CNN for image denoising

Deep convolutional neural networks (CNNs) have attracted considerable interest in low-level computer vision. Researches are usually devoted to improving the performance via very deep CNNs. However, as the depth increases, influences of the shallow layers on deep layers are weakened. Inspired by the fact, we propose an attention-guided denoising convolutional neural network (ADNet), mainly including a sparse block (SB), a feature enhancement block (FEB), an attention block (AB) and a reconstruction block (RB) for image denoising. Specifically, the SB makes a tradeoff between performance and efficiency by using dilated and common convolutions to remove the noise. The FEB integrates global and local features information via a long path to enhance the expressive ability of the denoising model. The AB is used to finely extract the noise information hidden in the complex background, which is very effective for complex noisy images, especially real noisy images and bind denoising. Also, the FEB is integrated with the AB to improve the efficiency and reduce the complexity for training a denoising model. Finally, a RB aims to construct the clean image through the obtained noise mapping and the given noisy image. Additionally, comprehensive experiments show that the proposed ADNet performs very well in three tasks (i.e. synthetic and real noisy images, and blind denoising) in terms of both quantitative and qualitative evaluations. The code of ADNet is accessible at https://github.com/hellloxiaotian/ADNet.
Blockchain for the future of sustainable supply chain management in Industry 4.0

The objective of this study is to provide an overview of Blockchain technology and Industry 4.0 for advancing supply chains towards sustainability. First, extracted from the existing literature, we evaluate the capabilities of Industry 4.0 for sustainability under three main topics of (1) Internet of things (IoT)-enabled energy management in smart factories; (2) smart logistics and transportation; and (3) smart business models. We expand beyond Industry 4.0 with unfolding the capabilities that Blockchain offers for increasing sustainability, under four main areas: (1) design of incentive mechanisms and tokenization to promote consumer green behavior; (2) enhance visibility across the entire product lifecycle; (3) increase systems efficiency while decreasing development and operational costs; and (4) foster sustainability monitoring and reporting performance across supply chain networks. Furthermore, Blockchain technology capabilities for contributing to social and environmental sustainability, research gaps, adversary effects of Blockchain, and future research directions are discussed.
Probabilistic Regression for Visual Tracking

Visual tracking is fundamentally the problem of regressing the state of the target in each video frame. While significant progress has been achieved, trackers are still prone to failures and inaccuracies. It is therefore crucial to represent the uncertainty in the target estimation. Although current prominent paradigms rely on estimating a state-dependent confidence score, this value lacks a clear probabilistic interpretation, complicating its use. In this work, we therefore propose a probabilistic regression formulation and apply it to tracking. Our network predicts the conditional probability density of the target state given an input image. Crucially, our formulation is capable of modeling label noise stemming from inaccurate annotations and ambiguities in the task. The regression network is trained by minimizing the Kullback-Leibler divergence. When applied for tracking, our formulation not only allows a probabilistic representation of the output, but also substantially improves the performance. Our tracker sets a new state-of-the-art on six datasets, achieving 59.8% AUC on LaSOT and 75.8% Success on TrackingNet. The code and models are available at https://github.com/visionml/pytracking.
Artificial hummingbird algorithm: A new bio-inspired optimizer with its engineering applications

A new bio-inspired optimization algorithm called artificial hummingbird algorithm (AHA) is proposed in this work to solve optimization problems. The AHA algorithm simulates the special flight skills and intelligent foraging strategies of hummingbirds in nature. Three kinds of flight skills utilized in foraging strategies, including axial, diagonal, and omnidirectional flights, are modeled. In addition, guided foraging, territorial foraging, and migrating foraging are implemented, and a visit table is constructed to model the memory function of hummingbirds for food sources. AHA is validated using two sets of numerical test functions, and the results are compared with those obtained from various algorithms. The comparisons demonstrate that AHA is more competitive than other meta-heuristic algorithms and determine high-quality solutions with fewer control parameters. Additionally, the performance of AHA is validated on ten challenging engineering design cases studies. The results show the superior effectiveness of AHA in terms of computational burden and solution precision compared with the existing optimization techniques in literature. The study also explores the application of AHA in hydropower operation design to further demonstrate its potential in practice. The source code of AHA is publicly available at https://seyedalimirjalili.com/aha and https://www.mathworks.com/matlabcentral/fileexchange/101133-artificial-hummingbird-algorithm?s_tid=srchtitle.
Feature selection based on artificial bee colony and gradient boosting decision tree

Data from many real-world applications can be high dimensional and features of such data are usually highly redundant. Identifying informative features has become an important step for data mining to not only circumvent the curse of dimensionality but to reduce the amount of data for processing. In this paper, we propose a novel feature selection method based on bee colony and gradient boosting decision tree aiming at addressing problems such as efficiency and informative quality of the selected features. Our method achieves global optimization of the inputs of the decision tree using the bee colony algorithm to identify the informative features. The method initializes the feature space spanned by the dataset. Less relevant features are suppressed according to the information they contribute to the decision making using an artificial bee colony algorithm. Experiments are conducted with two breast cancer datasets and six datasets from the public data repository. Experimental results demonstrate that the proposed method effectively reduces the dimensions of the dataset and achieves superior classification accuracy using the selected features.
The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI

Artificial intelligence and algorithmic decision-making processes are increasingly criticized for their black-box nature. Explainable AI approaches to trace human-interpretable decision processes from algorithms have been explored. Yet, little is known about algorithmic explainability from a human factors’ perspective. From the perspective of user interpretability and understandability, this study examines the effect of explainability in AI on user trust and attitudes toward AI. It conceptualizes causability as an antecedent of explainability and as a key cue of an algorithm and examines them in relation to trust by testing how they affect user perceived performance of AI-driven services. The results show the dual roles of causability and explainability in terms of its underlying links to trust and subsequent user behaviors. Explanations of why certain news articles are recommended generate users trust whereas causability of to what extent they can understand the explanations affords users emotional confidence. Causability lends the justification for what and how should be explained as it determines the relative importance of the properties of explainability. The results have implications for the inclusion of causability and explanatory cues in AI systems, which help to increase trust and help users to assess the quality of explanations. Causable explainable AI will help people understand the decision-making process of AI algorithms by bringing transparency and accountability into AI systems.
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers

Deep learning has recently seen rapid development and received significant attention due to its state-of-the-art performance on previously-thought hard problems. However, because of the internal complexity and nonlinear structure of deep neural networks, the underlying decision making processes for why these models are achieving such performance are challenging and sometimes mystifying to interpret. As deep learning spreads across domains, it is of paramount importance that we equip users of deep learning with tools for understanding when a model works correctly, when it fails, and ultimately how to improve its performance. Standardized toolkits for building neural networks have helped democratize deep learning; visual analytics systems have now been developed to support model explanation, interpretation, debugging, and improvement. We present a survey of the role of visual analytics in deep learning research, which highlights its short yet impactful history and thoroughly summarizes the state-of-the-art using a human-centered interrogative framework, focusing on the Five W's and How (Why, Who, What, How, When, and Where). We conclude by highlighting research directions and open research problems. This survey helps researchers and practitioners in both visual analytics and deep learning to quickly learn key aspects of this young and rapidly growing body of research, whose impact spans a diverse range of domains.
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation

In this work, we connect two distinct concepts for unsupervised domain adaptation: feature distribution alignment between domains by utilizing the task-specific decision boundary and the Wasserstein metric. Our proposed sliced Wasserstein discrepancy (SWD) is designed to capture the natural notion of dissimilarity between the outputs of task-specific classifiers. It provides a geometrically meaningful guidance to detect target samples that are far from the support of the source and enables efficient distribution alignment in an end-to-end trainable fashion. In the experiments, we validate the effectiveness and genericness of our method on digit and sign recognition, image classification, semantic segmentation, and object detection.
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

Building instance segmentation models that are data-efficient and can handle rare object categories is an important challenge in computer vision. Leveraging data augmentations is a promising direction towards addressing this challenge. Here, we perform a systematic study of the Copy-Paste augmentation (e.g., [13], [12]) for instance segmentation where we randomly paste objects onto an image. Prior studies on Copy-Paste relied on modeling the surrounding visual context for pasting the objects. However, we find that the simple mechanism of pasting objects randomly is good enough and can provide solid gains on top of strong baselines. Furthermore, we show Copy-Paste is additive with semi-supervised methods that leverage extra data through pseudo labeling (e.g. self-training). On COCO instance segmentation, we achieve 49.1 mask AP and 57.3 box AP, an improvement of +0.6 mask AP and +1.5 box AP over the previous state-of-the-art. We further demonstrate that Copy-Paste can lead to significant improvements on the LVIS benchmark. Our baseline model outperforms the LVIS 2020 Challenge winning entry by +3.6 mask AP on rare categories. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Privacy preservation in blockchain based IoT systems: Integration issues, prospects, challenges, and future research directions

Modern Internet of Things (IoT) systems are paving their path for a revolutionized world in which majority of our objects of everyday use will be interconnected. These objects will be able to link and communicate with each other and their surroundings in order to automate majority of our tasks. This interconnection of IoT nodes require security, seamless authentication, robustness and easy maintenance services. In order to provide such salient features, blockchain comes out as a viable solution. The decentralized nature of blockchain has resolved many security, maintenance, and authentication issues of IoT systems. Therefore, an immense increase in applications of blockchain-based IoT systems can be seen from the past few years. However, blockchain-based IoT network is public, so transactional details and encrypted keys are open and visible to everybody in that network. Thus, any adversary can infer critical information of users from this public infrastructure. In this paper, we discuss the privacy issues caused due to integration of blockchain in IoT applications by focusing over the applications of our daily use. Furthermore, we discuss implementation of five privacy preservation strategies in blockchain-based IoT systems named as anonymization, encryption, private contract, mixing, and differential privacy. Finally, we discuss challenges, and future directions for research in privacy preservation of blockchain-based IoT systems. This paper can serve as a basis of development of future privacy preservation strategies to address several privacy problems of IoT systems operating over blockchain.
Deep face recognition: A survey

Deep learning applies multiple processing layers to learn representations of data with multiple levels of feature extraction. This emerging technique has reshaped the research landscape of face recognition (FR) since 2014, launched by the breakthroughs of DeepFace and DeepID. Since then, deep learning technique, characterized by the hierarchical architecture to stitch together pixels into invariant face representation, has dramatically improved the state-of-the-art performance and fostered successful real-world applications. In this survey, we provide a comprehensive review of the recent developments on deep FR, covering broad topics on algorithm designs, databases, protocols, and application scenes. First, we summarize different network architectures and loss functions proposed in the rapid evolution of the deep FR methods. Second, the related face processing methods are categorized into two classes: "one-to-many augmentation" and "many-to-one normalization". Then, we summarize and compare the commonly used databases for both model training and evaluation. Third, we review miscellaneous scenes in deep FR, such as cross-factor, heterogenous, multiple-media and industrial scenes. Finally, the technical challenges and several promising directions are highlighted.
PipeDream

DNN training is extremely time-consuming, necessitating efficient multi-accelerator parallelization. Current approaches to parallelizing training primarily use intra-batch parallelization, where a single iteration of training is split over the available workers, but suffer from diminishing returns at higher worker counts. We present PipeDream, a system that adds inter-batch pipelining to intra-batch parallelism to further improve parallel training throughput, helping to better overlap computation with communication and reduce the amount of communication when possible. Unlike traditional pipelining, DNN training is bi-directional, where a forward pass through the computation graph is followed by a backward pass that uses state and intermediate data computed during the forward pass. Naïve pipelining can thus result in mismatches in state versions used in the forward and backward passes, or excessive pipeline flushes and lower hardware efficiency. To address these challenges, PipeDream versions model parameters for numerically correct gradient computations, and schedules forward and backward passes of different minibatches concurrently on different workers with minimal pipeline stalls. PipeDream also automatically partitions DNN layers among workers to balance work and minimize communication. Extensive experimentation with a range of DNN tasks, models, and hardware configurations shows that PipeDream trains models to high accuracy up to 5.3X faster than commonly used intra-batch parallelism techniques.
Supply chain re-engineering using blockchain technology: A case of smart contract based tracking process

The emergence of blockchain technology has created a number of potential innovations in handling business activities across various industries. However, few studies discuss the potential influence of blockchain technology from a business process re-engineering perspective. This study focuses on the feasibility and inceptive application of supply chain processes. We proposed a blockchain-based framework along with the use of an affiliated technology, i.e., smart contracts, to derive the feasible benefits of the supply chain process design. Through the illustrative design of an integrated process, we provide an achievable use case of the disintermediation of business processes via a conceptual, shared information ledger. This ledger not only facilitates the sharing of tracking information but also promotes a network for multilateral collaboration among supply chain members. The pursuit of transparency and accountability across supply chain processes can potentially influence decentralization and automation. A comparative analysis of the current and proposed frameworks is conducted to support the core reasoning of this study. Additionally, future implications on managerial practice and academic research are explored to provide pervasive suggestions for similar attempts in different sectors. We conclude with an evaluation of the potential influence of blockchain technology on supply chain management.
Adversarial Attacks and Defenses in Deep Learning

With the rapid developments of artificial intelligence (AI) and deep learning (DL) techniques, it is critical to ensure the security and robustness of the deployed algorithms. Recently, the security vulnerability of DL algorithms to adversarial samples has been widely recognized. The fabricated samples can lead to various misbehaviors of the DL models while being perceived as benign by humans. Successful implementations of adversarial attacks in real physical-world scenarios further demonstrate their practicality. Hence, adversarial attack and defense techniques have attracted increasing attention from both machine learning and security communities and have become a hot research topic in recent years. In this paper, we first introduce the theoretical foundations, algorithms, and applications of adversarial attack techniques. We then describe a few research efforts on the defense techniques, which cover the broad frontier in the field. Several open problems and challenges are subsequently discussed, which we hope will provoke further research efforts in this critical area.
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution

In natural images, information is conveyed at different frequencies where higher frequencies are usually encoded with fine details and lower frequencies are usually encoded with global structures. Similarly, the output feature maps of a convolution layer can also be seen as a mixture of information at different frequencies. In this work, we propose to factorize the mixed feature maps by their frequencies, and design a novel Octave Convolution (OctConv) operation to store and process feature maps that vary spatially “slower” at a lower spatial resolution reducing both memory and computation cost. Unlike existing multi-scale methods, OctConv is formulated as a single, generic, plug-and-play convolutional unit that can be used as a direct replacement of (vanilla) convolutions without any adjustments in the network architecture. It is also orthogonal and complementary to methods that suggest better topologies or reduce channel-wise redundancy like group or depth-wise convolutions. We experimentally show that by simply replacing convolutions with OctConv, we can consistently boost accuracy for both image and video recognition tasks, while reducing memory and computational cost. An OctConv-equipped ResNet-152 can achieve 82.9% top-1 classification accuracy on ImageNet with merely 22.2 GFLOPs.