Reviewerly/exHarmony huggingface dataset:
Scaling Local Self-Attention for Parameter Efficient Visual Backbones

Self-attention has the promise of improving computer vision systems due to parameter-independent scaling of receptive fields and content-dependent interactions, in contrast to parameter-dependent scaling and content-independent interactions of convolutions. Self-attention models have recently been shown to have encouraging improvements on accuracy-parameter trade-offs compared to baseline convolutional models such as ResNet-50. In this work, we develop self-attention models that can outperform not just the canonical baseline models, but even the high-performing convolutional models. We propose two extensions to self-attention that, in conjunction with a more efficient implementation of self-attention, improve the speed, memory usage, and accuracy of these models. We leverage these improvements to develop a new self-attention model family, HaloNets, which reach state-of-the-art accuracies on the parameter-limited setting of the ImageNet classification benchmark. In preliminary transfer learning experiments, we find that HaloNet models outperform much larger models and have better inference performance. On harder tasks such as object detection and instance segmentation, our simple local self-attention and convolutional hybrids show improvements over very strong baselines. These results mark another step in demonstrating the efficacy of self-attention models on settings traditionally dominated by convolutions. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Blockchain, adoption, and financial inclusion in India: Research opportunities

The economic development of rural India requires connecting remote villages to local and global supply chains. Yet, high rates of financial exclusion inhibit rural Indians from participating in these supply networks. We review the literature on financial inclusion, adoption, and blockchain in India, and posit that to resolve financial exclusion, the four challenges of geographical access, high cost, inappropriate banking products, and financial illiteracy need to be overcome. Next, we argue that blockchain technologies hold the potential to overcome most of these challenges. However, for blockchain technologies to become the cornerstone of financial inclusion initiatives, an understanding of technology adoption in India is needed. To guide the development of such understanding, we develop a research agenda on the antecedents of adoption, adoption patterns, and outcomes of adoption. Answering these research questions will lead to a nuanced understanding of adoption of blockchain-based technologies in rural India. The practical contribution of this paper is the discussion of how blockchain can alleviate the issue of financial exclusion in rural India, thereby providing a basis for a solution that could connect rural Indians to global supply chain networks. The theoretical contribution lies in the identification of knowledge gaps that should be answered to achieve financial inclusion of rural Indians.
VERI-Wild: A Large Dataset and a New Method for Vehicle Re-Identification in the Wild

Vehicle Re-identification (ReID) is of great significance to the intelligent transportation and public security. However, many challenging issues of Vehicle ReID in real-world scenarios have not been fully investigated, e.g., the high viewpoint variations, extreme illumination conditions, complex backgrounds, and different camera sources. To promote the research of vehicle ReID in the wild, we collect a new dataset called VERI-Wild with the following distinct features: 1) The vehicle images are captured by a large surveillance system containing 174 cameras covering a large urban district (more than 200km^2) The camera network continuously captures vehicles for 24 hours in each day and lasts for 1 month. 3) It is the first vehicle ReID dataset that is collected from unconstrained conditionsns. It is also a large dataset containing more than 400 thousand images of 40 thousand vehicle IDs. In this paper, we also propose a new method for vehicle ReID, in which, the ReID model is coupled into a Feature Distance Adversarial Network (FDA-Net), and a novel feature distance adversary scheme is designed to generate hard negative samples in feature space to facilitate ReID model training. The comprehensive results show the effectiveness of our method on the proposed dataset and the other two existing datasets.
A novel chaos based optical image encryption using fractional Fourier transform and DNA sequence operation

In this paper, we propose a new method for optical image encryption using fractional Fourier transform, DNA sequence operation and chaos theory. Random phase masks are generated using iterative Lorenz map and the plain image is transformed to a DNA matrix. This matrix is combined with the random phase mask and then transformed three times using the fractional Fourier transform. An Optical implementation of the encryption algorithm is proposed in our work. According to the experiment results and security analysis, we find that our algorithm has good encryption effect, larger secret key space and high sensitivity to the secret key. It can resist to most known attacks, such as statistical analysis and exhaustive attacks. All these features show that our encryption algorithm is very suitable for digital image encryption.
PrivySharing: A blockchain-based framework for privacy-preserving and secure data sharing in smart cities

The ubiquitous use of Internet of Things (IoT) ranges from industrial control systems to e-Health, e-commerce, smart cities, agriculture, supply chain management, smart cars, cyber-physical systems and a lot more. However, the data collected and processed by IoT systems especially the ones with centralized control are vulnerable to availability, integrity, and privacy threats. Hence, we present “PrivySharing,” a blockchain-based innovative framework for privacy-preserving and secure IoT data sharing in a smart city environment. The proposed scheme is distinct from existing strategies on many aspects. The data privacy is preserved by dividing the blockchain network into various channels, where every channel comprises a finite number of authorized organizations and processes a specific type of data such as health, smart car, smart energy or financial details. Moreover, access to users’ data within a channel is controlled by embedding access control rules in the smart contracts. In addition, data within a channel is further isolated and secured by using private data collection and encryption respectively. Likewise, the REST API that enables clients to interact with the blockchain network has dual security in the form of an API Key and OAuth 2.0. The proposed solution conforms to some of the significant requirements outlined in the European Union General Data Protection Regulation. We also present a system of reward in the form of a digital token named “PrivyCoin” for users sharing their data with stakeholders/third parties. Lastly, the experimental outcomes advocate that a multi-channel blockchain scales well as compared to a single-channel blockchain system.
Task-Free Continual Learning

Methods proposed in the literature towards continual deep learning typically operate in a task-based sequential learning setup. A sequence of tasks is learned, one at a time, with all data of current task available but not of previous or future tasks. Task boundaries and identities are known at all times. This setup, however, is rarely encountered in practical applications. Therefore we investigate how to transform continual learning to an online setup. We develop a system that keeps on learning over time in a streaming fashion, with data distributions gradually changing and without the notion of separate tasks. To this end, we build on the work on Memory Aware Synapses, and show how this method can be made online by providing a protocol to decide i) when to update the importance weights, ii) which data to use to update them, and iii) how to accumulate the importance weights at each update step. Experimental results show the validity of the approach in the context of two applications: (self-)supervised learning of a face recognition model by watching soap series and learning a robot to avoid collisions.
An intensify Harris Hawks optimizer for numerical and engineering optimization problems

Recently developed Harris Hawks Optimization has virtuous behavior for finding optimum solution in search space. However, it easily get trapped into local search space for constrained engineering optimization problems. In order to accelerate the global search phase of existing Harris Hawks optimizer and to stuck it out of local search space, the proposed research aims to explore the exploration phase of the existing optimizer, the hybrid variant of Harris Hawks optimizer has been developed using sine–cosinealgorithm and named as Hybrid Harris Hawks-Sine Cosine Algorithm (hHHO-SCA) The effectiveness of the proposed optimizer has been tested for various nonlinear, non-convex and highly constrained engineering design problem. In order to validate the results of the proposed algorithm, 65 standard benchmark problems including CEC2017, CEC2018 and eleven multidisciplinary engineering design optimization problems has been taken into consideration. After verification it has been observed that the outcomes of the proposed hHHO-SCA optimization algorithm is much better than standard sine–cosine optimization algorithm, Harris Hawks Optimizer, Ant Lion Optimizer algorithm, Moth Flame Optimization algorithm, grey wolf optimizer algorithm, and others recently described meta-heuristics, heuristics and hybrid type optimization search algorithm and proposed algorithm endorses its effectiveness in multi-disciplinary design and engineering optimization problems.
Artificial intelligence in oncology

Artificial intelligence (AI) has contributed substantially to the resolution of a variety of biomedical problems, including cancer, over the past decade. Deep learning, a subfield of AI that is highly flexible and supports automatic feature extraction, is increasingly being applied in various areas of both basic and clinical cancer research. In this review, we describe numerous recent examples of the application of AI in oncology, including cases in which deep learning has efficiently solved problems that were previously thought to be unsolvable, and we address obstacles that must be overcome before such application can become more widespread. We also highlight resources and datasets that can help harness the power of AI for cancer research. The development of innovative approaches to and applications of AI will yield important insights in oncology in the coming decade.
ImVoteNet: Boosting 3D Object Detection in Point Clouds With Image Votes

3D object detection has seen quick progress thanks to advances in deep learning on point clouds. A few recent works have even shown state-of-the-art performance with just point clouds input (e.g. VoteNet). However, point cloud data have inherent limitations. They are sparse, lack color information and often suffer from sensor noise. Images, on the other hand, have high resolution and rich texture. Thus they can complement the 3D geometry provided by point clouds. Yet how to effectively use image information to assist point cloud based detection is still an open question. In this work, we build on top of VoteNet and propose a 3D detection architecture called ImVoteNet specialized for RGB-D scenes. ImVoteNet is based on fusing 2D votes in images and 3D votes in point clouds. Compared to prior work on multi-modal detection, we explicitly extract both geometric and semantic features from the 2D images. We leverage camera parameters to lift these features to 3D. To improve the synergy of 2D-3D feature fusion, we also propose a multi-tower training scheme. We validate our model on the challenging SUN RGB-D dataset, advancing state-of-the-art results by 5.7 mAP. We also provide rich ablation studies to analyze the contribution of each design choice.
Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather

The fusion of multimodal sensor streams, such as camera, lidar, and radar measurements, plays a critical role in object detection for autonomous vehicles, which base their decision making on these inputs. While existing methods exploit redundant information in good environmental conditions, they fail in adverse weather where the sensory streams can be asymmetrically distorted. These rare ``edge-case'' scenarios are not represented in available datasets, and existing fusion architectures are not designed to handle them. To address this challenge we present a novel multimodal dataset acquired in over 10,000~km of driving in northern Europe. Although this dataset is the first large multimodal dataset in adverse weather, with 100k labels for lidar, camera, radar, and gated NIR sensors, it does not facilitate training as extreme weather is rare. To this end, we present a deep fusion network for robust fusion without a large corpus of labeled training data covering all asymmetric distortions. Departing from proposal-level fusion, we propose a single-shot model that adaptively fuses features, driven by measurement entropy. We validate the proposed method, trained on clean data, on our extensive validation dataset. Code and data are available here https://github.com/princeton-computational-imaging/SeeingThroughFog.
Multi-behavior Recommendation with Graph Convolutional Networks

Traditional recommendation models that usually utilize only one type of user-item interaction are faced with serious data sparsity or cold start issues. Multi-behavior recommendation taking use of multiple types of user-item interactions, such as clicks and favorites, can serve as an effective solution. Early efforts towards multi-behavior recommendation fail to capture behaviors' different influence strength on target behavior. They also ignore behaviors' semantics which is implied in multi-behavior data. Both of these two limitations make the data not fully exploited for improving the recommendation performance on the target behavior.
Object Relational Graph With Teacher-Recommended Learning for Video Captioning

Taking full advantage of the information from both vision and language is critical for the video captioning task. Existing models lack adequate visual representation due to the neglect of interaction between object, and sufficient training for content-related words due to long-tailed problems. In this paper, we propose a complete video captioning system including both a novel model and an effective training strategy. Specifically, we propose an object relational graph (ORG) based encoder, which captures more detailed interaction features to enrich visual representation. Meanwhile, we design a teacher-recommended learning (TRL) method to make full use of the successful external language model (ELM) to integrate the abundant linguistic knowledge into the caption model. The ELM generates more semantically similar word proposals which extend the groundtruth words used for training to deal with the long-tailed problem. Experimental evaluations on three benchmarks: MSVD, MSR-VTT and VATEX show the proposed ORG-TRL system achieves state-of-the-art performance. Extensive ablation studies and visualizations illustrate the effectiveness of our system.
A Blockchained Federated Learning Framework for Cognitive Computing in Industry 4.0 Networks

Cognitive computing, a revolutionary AI concept emulating human brain's reasoning process, is progressively flourishing in the Industry 4.0 automation. With the advancement of various AI and machine learning technologies the evolution toward improved decision making as well as data-driven intelligent manufacturing has already been evident. However, several emerging issues, including the poisoning attacks, performance, and inadequate data resources, etc., have to be resolved. Recent research works studied the problem lightly, which often leads to unreliable performance, inefficiency, and privacy leakage. In this article, we developed a decentralized paradigm for big data-driven cognitive computing (D2C), using federated learning and blockchain jointly. Federated learning can solve the problem of “data island” with privacy protection and efficient processing while blockchain provides incentive mechanism, fully decentralized fashion, and robust against poisoning attacks. Using blockchain-enabled federated learning help quick convergence with advanced verifications and member selections. Extensive evaluation and assessment findings demonstrate D2C's effectiveness relative to existing leading designs and models.
FCOS: A Simple and Strong Anchor-free Object Detector

In computer vision, object detection is one of most important tasks, which underpins a few instance-level recognition tasks and many downstream applications. Recently one-stage methods have gained much attention over two-stage approaches due to their simpler design and competitive performance. Here we propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to other dense prediction problems such as semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the pre-defined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating the intersection over union (IoU) scores during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code is available at: git.io/AdelaiDet.
Manipulating and Measuring Model Interpretability

With machine learning models being increasingly used to aid decision making even in high-stakes domains, there has been a growing interest in developing interpretable models. Although many supposedly interpretable models have been proposed, there have been relatively few experimental studies investigating whether these models achieve their intended effects, such as making people more closely follow a model’s predictions when it is beneficial for them to do so or enabling them to detect when a model has made a mistake. We present a sequence of pre-registered experiments (N = 3, 800) in which we showed participants functionally identical models that varied only in two factors commonly thought to make machine learning models more or less interpretable: the number of features and the transparency of the model (i.e., whether the model internals are clear or black box). Predictably, participants who saw a clear model with few features could better simulate the model’s predictions. However, we did not find that participants more closely followed its predictions. Furthermore, showing participants a clear model meant that they were less able to detect and correct for the model’s sizable mistakes, seemingly due to information overload. These counterintuitive findings emphasize the importance of testing over intuition when developing interpretable models.
Quasi-Dense Similarity Learning for Multiple Object Tracking

Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions on the images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of region proposals on a pair of images for contrastive learning. We can directly combine this similarity learning with existing detection methods to build Quasi-Dense Tracking (QDTrack) without turning to displacement regression or motion priors. We also find that the resulting distinctive feature space admits a simple nearest neighbor search at the inference time. Despite its simplicity, QD-Track outperforms all existing methods on MOT, BDD100K, Waymo, and TAO tracking benchmarks. It achieves 68.7 MOTA at 20.3 FPS on MOT17 without using external training data. Compared to methods with similar detectors, it boosts almost 10 points of MOTA and significantly decreases the number of ID switches on BDD100K and Waymo datasets. Our code and trained models are available at https://github.com/SysCV/qdtrack.
A Survey on Cooperative Co-Evolutionary Algorithms

The first cooperative co-evolutionary algorithm (CCEA) was proposed by Potter and De Jong in 1994 and since then many CCEAs have been proposed and successfully applied to solving various complex optimization problems. In applying CCEAs, the complex optimization problem is decomposed into multiple subproblems, and each subproblem is solved with a separate subpopulation, evolved by an individual evolutionary algorithm (EA). Through cooperative co-evolution of multiple EA subpopulations, a complete problem solution is acquired by assembling the representative members from each subpopulation. The underlying divide-and-conquer and collaboration mechanisms enable CCEAs to tackle complex optimization problems efficiently, and hence CCEAs have been attracting wide attention in the EA community. This paper presents a comprehensive survey of these CCEAs, covering problem decomposition, collaborator selection, individual fitness evaluation, subproblem resource allocation, implementations, benchmark test problems, control parameters, theoretical analyses, and applications. The unsolved challenges and potential directions for their solutions are discussed.
The Security of Big Data in Fog-Enabled IoT Applications Including Blockchain: A Survey

The proliferation of inter-connected devices in critical industries, such as healthcare and power grid, is changing the perception of what constitutes critical infrastructure. The rising interconnectedness of new critical industries is driven by the growing demand for seamless access to information as the world becomes more mobile and connected and as the Internet of Things (IoT) grows. Critical industries are essential to the foundation of today's society, and interruption of service in any of these sectors can reverberate through other sectors and even around the globe. In today's hyper-connected world, the critical infrastructure is more vulnerable than ever to cyber threats, whether state sponsored, criminal groups or individuals. As the number of interconnected devices increases, the number of potential access points for hackers to disrupt critical infrastructure grows. This new attack surface emerges from fundamental changes in the critical infrastructure of organizations technology systems. This paper aims to improve understanding the challenges to secure future digital infrastructure while it is still evolving. After introducing the infrastructure generating big data, the functionality-based fog architecture is defined. In addition, a comprehensive review of security requirements in fog-enabled IoT systems is presented. Then, an in-depth analysis of the fog computing security challenges and big data privacy and trust concerns in relation to fog-enabled IoT are given. We also discuss blockchain as a key enabler to address many security related issues in IoT and consider closely the complementary interrelationships between blockchain and fog computing. In this context, this work formalizes the task of securing big data and its scope, provides a taxonomy to categories threats to fog-based IoT systems, presents a comprehensive comparison of state-of-the-art contributions in the field according to their security service and recommends promising research directions for future investigations.