Reviewerly/exHarmony huggingface dataset:
Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.
SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

Attention mechanisms, which enable a neural network to accurately focus on all the relevant elements of the input, have become an essential component to improve the performance of deep neural networks. There are mainly two attention mechanisms widely used in computer vision studies, spatial attention and channel attention, which aim to capture the pixel-level pairwise relationship and channel dependency, respectively. Although fusing them together may achieve better performance than their individual implementations, it will inevitably increase the computational overhead. In this paper, we propose an efficient Shuffle Attention (SA) module to address this issue, which adopts Shuffle Units to combine two types of attention mechanisms effectively. Specifically, SA first groups channel dimensions into multiple sub-features before processing them in parallel. Then, for each sub-feature, SA utilizes a Shuffle Unit to depict feature dependencies in both spatial and channel dimensions. After that, all sub-features are aggregated and a "channel shuffle" operator is adopted to enable information communication between different sub-features. The proposed SA module is efficient yet effective, e.g., the parameters and computations of SA against the backbone ResNet50 are 300 vs. 25.56M and 2.76e-3 GFLOPs vs. 4.12 GFLOPs, respectively, and the performance boost is more than 1.34% in terms of Top-1 accuracy. Extensive experimental results on common-used benchmarks, including ImageNet-1k for classification, MS COCO for object detection, and instance segmentation, demonstrate that the proposed SA outperforms the current SOTA methods significantly by achieving higher accuracy while having lower model complexity.
An Attentive Survey of Attention Models

Attention Model has now become an important concept in neural networks that has been researched within diverse application domains. This survey provides a structured and comprehensive overview of the developments in modeling attention. In particular, we propose a taxonomy that groups existing techniques into coherent categories. We review salient neural architectures in which attention has been incorporated and discuss applications in which modeling attention has shown a significant impact. We also describe how attention has been used to improve the interpretability of neural networks. Finally, we discuss some future research directions in attention. We hope this survey will provide a succinct introduction to attention models and guide practitioners while developing approaches for their applications.
Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution

Hyperspectral images (HSIs) with high spectral resolution only have the low spatial resolution. On the contrary, multispectral images (MSIs) with much lower spectral resolution can be obtained with higher spatial resolution. Therefore, fusing the high-spatial-resolution MSI (HR-MSI) with low-spatial-resolution HSI of the same scene has become the very popular HSI super-resolution scheme. In this paper, a novel low tensor-train (TT) rank (LTTR)-based HSI super-resolution method is proposed, where an LTTR prior is designed to learn the correlations among the spatial, spectral, and nonlocal modes of the nonlocal similar high-spatial-resolution HSI (HR-HSI) cubes. First, we cluster the HR-MSI cubes as many groups based on their similarities, and the HR-HSI cubes are also clustered according to the learned cluster structure in the HR-MSI cubes. The HR-HSI cubes in each group are much similar to each other and can constitute a 4-D tensor, whose four modes are highly correlated. Therefore, we impose the LTTR constraint on these 4-D tensors, which can effectively learn the correlations among the spatial, spectral, and nonlocal modes because of the well-balanced matricization scheme of TT rank. We formulate the super-resolution problem as TT rank regularized optimization problem, which is solved via the scheme of alternating direction method of multipliers. Experiments on HSI data sets indicate the effectiveness of the LTTR-based method.
The blockchain: State-of-the-art and research challenges

The blockchain revolutionizes the creation of both scalable information technology systems and diversified applications by integrating the increasingly popular artificial intelligence, cloud computing, and big data. Various industries have recently begun to implement the exploration of blockchain. It will not take long for the blockchain to spread all over the world. In order to identify and further the development of the blockchain technology, this paper reviews the extant studies on the blockchain and its key components, blockchain-based IoT, blockchain-based security, blockchain-based data management, and the main applications based on the blockchain, and it delineates potential trends and challenges. This study provides a comprehensive overview of state-of-the-art blockchain and describes a forward-looking direction.
NTIRE 2019 Challenge on Video Deblurring and Super-Resolution: Dataset and Study

This paper introduces a novel large dataset for video deblurring, video super-resolution and studies the state-of-the-art as emerged from the NTIRE 2019 video restoration challenges. The video deblurring and video super-resolution challenges are each the first challenge of its kind, with 4 competitions, hundreds of participants and tens of proposed solutions. Our newly collected REalistic and Diverse Scenes dataset (REDS) was employed by the challenges. In our study, we compare the solutions from the challenges to a set of representative methods from the literature and evaluate them on our proposed REDS dataset. We find that the NTIRE 2019 challenges push the state-of-the-art in video deblurring and super-resolution, reaching compelling performance on our newly proposed REDS dataset.
Graph Convolutional Tracking

Tracking by siamese networks has achieved favorable performance in recent years. However, most of existing siamese methods do not take full advantage of spatial-temporal target appearance modeling under different contextual situations. In fact, the spatial-temporal information can provide diverse features to enhance the target representation, and the context information is important for online adaption of target localization. To comprehensively leverage the spatial-temporal structure of historical target exemplars and get benefit from the context information, in this work, we present a novel Graph Convolutional Tracking (GCT) method for high-performance visual tracking. Specifically, the GCT jointly incorporates two types of Graph Convolutional Networks (GCNs) into a siamese framework for target appearance modeling. Here, we adopt a spatial-temporal GCN to model the structured representation of historical target exemplars. Furthermore, a context GCN is designed to utilize the context of the current frame to learn adaptive features for target localization. Extensive results on 4 challenging benchmarks show that our GCT method performs favorably against state-of-the-art trackers while running around 50 frames per second.
Learning to Reduce Dual-Level Discrepancy for Infrared-Visible Person Re-Identification

Infrared-Visible person RE-IDentification (IV-REID) is a rising task. Compared to conventional person re-identification (re-ID), IV-REID concerns the additional modality discrepancy originated from the different imaging processes of spectrum cameras, in addition to the person's appearance discrepancy caused by viewpoint changes, pose variations and deformations presented in the conventional re-ID task. The co-existed discrepancies make IV-REID more difficult to solve. Previous methods attempt to reduce the appearance and modality discrepancies simultaneously using feature-level constraints. It is however difficult to eliminate the mixed discrepancies using only feature-level constraints. To address the problem, this paper introduces a novel Dual-level Discrepancy Reduction Learning (D$^2$RL) scheme which handles the two discrepancies separately. For reducing the modality discrepancy, an image-level sub-network is trained to translate an infrared image into its visible counterpart and a visible image to its infrared version. With the image-level sub-network, we can unify the representations for images with different modalities. With the help of the unified multi-spectral images, a feature-level sub-network is trained to reduce the remaining appearance discrepancy through feature embedding. By cascading the two sub-networks and training them jointly, the dual-level reductions take their responsibilities cooperatively and attentively. Extensive experiments demonstrate the proposed approach outperforms the state-of-the-art methods.
Barnacles Mating Optimizer: A new bio-inspired algorithm for solving engineering optimization problems

This paper presents a novel bio-inspired optimization algorithm namely the Barnacles Mating Optimizer (BMO) algorithm to solve optimization problems. The proposed algorithm mimics the mating behaviour of barnacles in nature for solving optimization problems. The BMO is first benchmarked on a set of 23 mathematical functions to test the characteristics of BMO in finding the optimal solutions. It is then applied to optimal reactive power dispatch (ORPD) problem to verify the reliability and efficiency of BMO. Extensive comparative studies with other algorithms are conducted and from the simulation results, it is observed that BMO generally provides better results and exhibits huge potential of BMO in solving real optimization problems.
Social behavior for autonomous vehicles

Deployment of autonomous vehicles on public roads promises increased efficiency and safety. It requires understanding the intent of human drivers and adapting to their driving styles. Autonomous vehicles must also behave in safe and predictable ways without requiring explicit communication. We integrate tools from social psychology into autonomous-vehicle decision making to quantify and predict the social behavior of other drivers and to behave in a socially compliant way. A key component is Social Value Orientation (SVO), which quantifies the degree of an agent's selfishness or altruism, allowing us to better predict how the agent will interact and cooperate with others. We model interactions between agents as a best-response game wherein each agent negotiates to maximize their own utility. We solve the dynamic game by finding the Nash equilibrium, yielding an online method of predicting multiagent interactions given their SVOs. This approach allows autonomous vehicles to observe human drivers, estimate their SVOs, and generate an autonomous control policy in real time. We demonstrate the capabilities and performance of our algorithm in challenging traffic scenarios: merging lanes and unprotected left turns. We validate our results in simulation and on human driving data from the NGSIM dataset. Our results illustrate how the algorithm's behavior adapts to social preferences of other drivers. By incorporating SVO, we improve autonomous performance and reduce errors in human trajectory predictions by 25%.
Learning With Average Precision: Training Image Retrieval With a Listwise Loss

Image retrieval can be formulated as a ranking problem where the goal is to order database images by decreasing similarity to the query. Recent deep models for image retrieval have outperformed traditional methods by leveraging ranking-tailored loss functions, but important theoretical and practical problems remain. First, rather than directly optimizing the global ranking, they minimize an upper-bound on the essential loss, which does not necessarily result in an optimal mean average precision (mAP). Second, these methods require significant engineering efforts to work well, e.g., special pre-training and hard-negative mining. In this paper we propose instead to directly optimize the global mAP by leveraging recent advances in listwise loss formulations. Using a histogram binning approximation, the AP can be differentiated and thus employed to end-to-end learning. Compared to existing losses, the proposed method considers thousands of images simultaneously at each iteration and eliminates the need for ad hoc tricks. It also establishes a new state of the art on many standard retrieval benchmarks. Models and evaluation scripts have been made available at: https://europe.naverlabs.com/Deep-Image-Retrieval/.
An exhaustive survey on security and privacy issues in Healthcare 4.0

The healthcare industry has revolutionized from 1.0 to 4.0 where Healthcare 1.0 was more doctor centric and Healthcare 2.0 replaced manual records with electronic healthcare records (EHRs). Healthcare 3.0 was patient-centric and Healthcare 4.0 uses cloud computing (CC), fog computing (FC), Internet of things (IoT), and telehealthcare technologies to share data between various stakeholders. However, framing a secure technique for Healthcare 4.0 has always been a challenging task. An in-secure technique for Healthcare 4.0 may lead to the healthcare data breach where hackers can gain full access to patients’ email accounts, messages, and reports. On the contrary, a secured technique for Healthcare 4.0 can provide satisfaction to all stakeholders, including patients and caregivers. Motivated from these facts, this paper presents an extensive literature review and analysis of state-of-the-art proposals to maintain security and privacy in Healthcare 4.0. We also explored the blockchain-based solution to give insights to both researchers and practitioners communities. Different taxonomies used for exploring various security and privacy issues in Healthcare 4.0 are also presented in a structured manner. Then, the advantages and limitations of various security and privacy techniques are explored and discussed in the paper. Finally, existing challenges and future research directions of security and privacy in Healthcare 4.0 are presented.
What makes an AI device human-like? The role of interaction quality, empathy and perceived psychological anthropomorphic characteristics in the acceptance of artificial intelligence in the service industry

Intelligent AI devices have become a common presence in the business landscape, offering a wide range of services, from the medical sector to the hospitality industry. From an organizational perspective, AI devices have several advantages, by performing certain tasks quicker and more accurately in comparison to humans while at the same time being more cost-efficient. However, in order to maintain the high standards of a brand, they have to be accepted by consumers and deliver socially adequate performance. Therefore, it is important to determine the characteristics of AI devices which make them accepted and trusted by consumers. Based on the Computers as Social Actors (CASA) Theory, we have researched on the role of psychological anthropomorphic characteristics, perceived empathy, and interaction quality in the acceptance of AI devices in the service industry. The results show that anthropomorphic characteristics alone do not influence acceptance and trust towards AI devices. However, both perceived empathy and interaction quality mediate the relation between anthropomorphic characteristics and acceptance. A human-like AI device has higher acceptance when it has the ability to show empathy and interaction in relation to the human consumer. This result reveals the importance of developing forms of strong intelligence and empathetic behaviour in service robots and AI devices.
Closed-Form Factorization of Latent Semantics in GANs

A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In order to identify such latent dimensions for image editing, previous methods typically annotate a collection of synthesized samples and train linear classifiers in the latent space. However, they require a clear definition of the target attribute as well as the corresponding manual annotations, limiting their applications in practice. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. In particular, we take a closer look into the generation mechanism of GANs and further propose a closedform factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights. With a lightning-fast implementation, our approach is capable of not only finding semantically meaningful dimensions comparably to the state-of-the-art supervised methods, but also resulting in far more versatile concepts across multiple GAN models trained on a wide range of datasets. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features

To fight against real-life image forgery, which commonly involves different types and combined manipulations, we propose a unified deep neural architecture called ManTraNet. Unlike many existing solutions, ManTra-Net is an end-to-end network that performs both detection and localization without extra preprocessing and postprocessing. ManTra-Net is a fully convolutional network and handles images of arbitrary sizes and many known forgery types such splicing, copy-move, removal, enhancement, and even unknown types. This paper has three salient contributions. We design a simple yet effective self-supervised learning task to learn robust image manipulation traces from classifying 385 image manipulation types. Further, we formulate the forgery localization problem as a local anomaly detection problem, design a Z-score feature to capture local anomaly, and propose a novel long short-term memory solution to assess local anomalies. Finally, we carefully conduct ablation experiments to systematically optimize the proposed network design. Our extensive experimental results demonstrate the generalizability, robustness and superiority of ManTra-Net, not only in single types of manipulations/forgeries, but also in their complicated combinations.
Federated Learning With Blockchain for Autonomous Vehicles: Analysis and Design Challenges

We propose an autonomous blockchain-based federated learning (BFL) design for privacy-aware and efficient vehicular communication networking, where local on-vehicle machine learning (oVML) model updates are exchanged and verified in a distributed fashion. BFL enables oVML without any centralized training data or coordination by utilizing the consensus mechanism of the blockchain. Relying on a renewal reward approach, we develop a mathematical framework that features the controllable network and BFL parameters (e.g., the retransmission limit, block size, block arrival rate, and the frame sizes) so as to capture their impact on the system-level performance. More importantly, our rigorous analysis of oVML system dynamics quantifies the end-to-end delay with BFL, which provides important insights into deriving optimal block arrival rate by considering communication and consensus delays. We present a variety of numerical and simulation results highlighting various non-trivial findings and insights for adaptive BFL design. In particular, based on analytical results, we minimize the system delay by exploiting the channel dynamics and demonstrate that the proposed idea of tuning the block arrival rate is provably online and capable of driving the system dynamics to the desired operating point. It also identifies the improved dependency on other blockchain parameters for a given set of channel conditions, retransmission limits, and frame sizes. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> However, a number of challenges (gaps in knowledge) need to be resolved in order to realise these changes. In particular, we identify key bottleneck challenges requiring further investigations, and provide potential future research directions. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> An early version of this work has been accepted for presentation in IEEE WCNC Wksps 2020 [1].
PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation

The requirement of fine-grained perception by autonomous driving systems has resulted in recently increased research in the online semantic segmentation of single-scan LiDAR. Emerging datasets and technological advancements have enabled researchers to benchmark this problem and improve the applicable semantic segmentation algorithms. Still, online semantic segmentation of LiDAR scans in autonomous driving applications remains challenging due to three reasons: (1) the need for near-real-time latency with limited hardware, (2) points are distributed unevenly across space, and (3) an increasing number of more fine-grained semantic classes. The combination of the aforementioned challenges motivates us to propose a new LiDAR-specific, KNN-free segmentation algorithm - PolarNet. Instead of using common spherical or bird's-eye-view projection, our polar bird's-eye-view representation balances the points per grid and thus indirectly redistributes the network's attention over the long-tailed points distribution over the radial axis in polar coordination. We find that our encoding scheme greatly increases the mIoU in three drastically different real urban LiDAR single-scan segmentation datasets while retaining ultra low latency and near real-time throughput.