Reviewerly/exHarmony huggingface dataset:
Class-Aware Feature Aggregation Network for Video Object Detection

Recent progress in video object detection (VOD) has shown that aggregating features from other frames to capture long-range contextual information is very important to deal with the challenges in VOD, such as partial occlusion, motion blur, etc. To exploit more effective feature aggregation, we propose several improvements over previous works in this paper: (1) a class-aware pixel-level feature aggregation module, which characterizes a pixel by exploiting the context information lying in the instances from both the current frame and other frames. Different from the previous non-local operation, the proposed class-aware pixel-level feature aggregation filters out most of the noisy information from the large scope of background and objects in different classes, and only enhances representation of a foreground pixel with the same class instances with limited ambiguous information; (2) a class-aware instance-level feature aggregation module, which aggregates features for object proposals by learning two kinds of relations: the temporal dependencies among the same class object proposals from support frames sampled in a long time range or even the whole sequence, and spatial topology relation among proposals of different objects in the target frame. The homogeneity constraint in instance-level feature aggregation filters out many defective proposals, making the feature aggregation more accurate; and (3) a correlation-based feature alignment module embedded in the instance-level feature aggregation, which aligns the feature maps of the support and target proposals. Without bells or whistles, the proposed method achieves state-of-the-art performance on the ImageNet VID dataset without any post-processing methods. This project is publicly available <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/LiangHann/Class-aware-Feature-Aggregation-Network-for-Video-Object-Detection</uri> .
Identity-Certifying Authority-Aided Identity-Based Searchable Encryption Framework in Cloud Systems

In the era of cloud computing, massive quantities of data are encrypted and uploaded to the cloud to realize a variety of applications and services while protecting user confidentiality. Accordingly, the formulation of methods for efficiently searching encrypted data has become a critical problem. Public-key encryption with keyword search is an efficient solution that allows the data owner to generate encrypted keywords for a given document while also allowing the data user to generate the corresponding trapdoor for searching. Huang and Li proposed a public-key authenticated encryption with keyword search (PAEKS) scheme to resist keyword guessing attacks, where the data owner not only encrypts keywords but also authenticates them. However, existing PAEKS-related schemes carry a tradeoff between efficiency, storage cost, and security. In this article, we introduce a novel framework called identity-certifying authority-aided identity-based searchable encryption, which has the advantage of reducing storage space while remaining the efficiency and security. We formally define the system model and desired security requirements to represent attacks in a real scenario. In addition, we propose a provably secure scheme based on the gap bilinear Diffie–Hellman assumption and experimentally evaluate our scheme in terms of its performance and theoretical features against its state-of-the-art counterparts.
Entity Linking Meets Deep Learning: Techniques and Solutions

Entity linking (EL) is the process of linking entity mentions appearing in web text with their corresponding entities in a knowledge base. EL plays an important role in the fields of knowledge engineering and data mining, underlying a variety of downstream applications such as knowledge base population, content analysis, relation extraction, and question answering. In recent years, deep learning (DL), which has achieved tremendous success in various domains, has also been leveraged in EL methods to surpass traditional machine learning based methods and yield the state-of-the-art performance. In this survey, we present a comprehensive review and analysis of existing DL based EL methods. First of all, we propose a new taxonomy, which organizes existing DL based EL methods using three axes: embedding, feature, and algorithm. Then we systematically survey the representative EL methods along the three axes of the taxonomy. Later, we introduce ten commonly used EL data sets and give a quantitative performance analysis of DL based EL methods over these data sets. Finally, we discuss the remaining limitations of existing methods and highlight some promising future directions.
Methodology for Multi-Aspect Ontology Development: Ontology for Decision Support Based on Human-Machine Collective Intelligence

The advantage of human-machine collective intelligence for decision support systems is the ability to make better decisions due to the mitigation of human biases in the generation of potential solutions and their evaluation. So far, the potential of human-machine collective intelligence was used only in few decision support systems, however teamwork between humans and machines has not been achieved. This is partly due to the lack of interoperability in these systems. In earlier works, the authors proposed the apparatus of multi-aspect ontologies implying the integration of multiple domain ontologies to provide interoperability between humans and machines and coordinate interrelated processes going on in the systems of the considered type. Such ontologies have proved efficient for systems that require intensive information and knowledge exchange between loosely-related dynamic autonomous domains (e.g., enterprise knowledge management, product lifecycle management, or human-machine collective intelligence systems). However, existing ontology development methodologies fail to recommend a process that would support cross-domain knowledge integration during the multi-aspect ontology development. Moreover, the structure of the multi-aspect ontology imposes some restrictions on the integration approach. The paper proposes such a methodology for the multi-aspect ontology development that incorporates the aspect integration approach at multiple levels. The methodology is applied to develop a multi-aspect ontology for decision support based on human-machine collective intelligence. An example from the “e-tourism” domain demonstrates the applicability of the proposed methodology as well as the usage of the multi-aspect ontology for a human-machine environment aimed at solving real-world problems. The proposed methodology can facilitate the development of ontologies for complex knowledge-based systems that operate with knowledge from multiple loosely-connected domains.
Novel regularization method for the class imbalance problem

In neural network models, obtaining a high-quality dataset is critical because they are generally reliant on training data. A common problem that occurs is class imbalance, in which models tend to be biased to the majority class when the training data is not balanced. To overcome this problem, we propose a novel regularization method that provides a penalty to the loss function, using two facets of the distribution of the model's output p(ŷ|x): (1) skewed mean and (2) variance divergence between p(ŷ|x∈D+) and p(ŷ|x∈D−). The experimental results demonstrate that our methods consistently improve the performance on imbalanced datasets. Moreover, the combination of two regularization methods provides a substantial performance improvement on five sentence classification datasets and also an image classification dataset; notably, state-of-the-art performances are achieved on the WikiQA and SelQA datasets.
Information-theoretic regularization for Multi-source Domain Adaptation

Adversarial learning strategy has demonstrated remarkable performance in dealing with single-source Domain Adaptation (DA) problems, and it has recently been applied to Multi-source DA (MDA) problems. Although most existing MDA strategies rely on a multiple domain discriminator setting, its effect on the latent space representations has been poorly understood. Here we adopt an information-theoretic approach to identify and. resolve the potential adverse effect of the multiple domain discriminators on MDA: disintegration of domain-discriminative information, limited computational scalability, and a large variance in the gradient of the loss during training. We examine the above issues by situating adversarial DA in the context of information regularization. This also provides a theoretical justification for using a single and unified domain discriminator. Based on this idea, we implement a novel neural architecture called a Multi-source Information-regularized Adaptation Networks (MIAN). Large-scale experiments demonstrate that MIAN, despite its structural simplicity, reliably and significantly outperforms other state-of-the-art methods.
A Next Basket Recommendation Reality Check

The goal of a next basket recommendation (NBR) system is to recommend items for the next basket for a user, based on the sequence of their prior baskets. We examine whether the performance gains of the NBR methods reported in the literature hold up under a fair and comprehensive comparison. To clarify the mixed picture that emerges from our comparison, we provide a novel angle on the evaluation of next basket recommendation (NBR) methods, centered on the distinction between repetition and exploration: the next basket is typically composed of previously consumed items (i.e., repeat items) and new items (i.e., explore items). We propose a set of metrics that measure the repetition/exploration ratio and performance of NBR models. Using these new metrics, we provide a second analysis of state-of-the-art NBR models. The results help to clarify the extent of the actual progress achieved by existing NBR methods as well as the underlying reasons for any improvements that we observe. Overall, our work sheds light on the evaluation problem of NBR, provides a new evaluation protocol, and yields useful insights for the design of models for this task.
Universal Bayes consistency in metric spaces

We extend a recently proposed 1-nearest-neighbor based multiclass learning algorithm and prove that our modification is universally strongly Bayes consistent in all metric spaces admitting any such learner, making it an "optimistically universal" Bayes-consistent learner. This is the first learning algorithm known to enjoy this property; by comparison, the k-NN classifier and its variants are not generally universally Bayes consistent, except under additional structural assumptions, such as an inner product, a norm, finite dimension or a Besicovitch-type property. The metric spaces in which universal Bayes consistency is possible are the "essentially separable" ones—a notion that we define, which is more general than standard separability. The existence of metric spaces that are not essentially separable is widely believed to be independent of the ZFC axioms of set theory. We prove that essential separability exactly characterizes the existence of a universal Bayes-consistent learner for the given metric space. In particular, this yields the first impossibility result for universal Bayes consistency. Taken together, our results completely characterize strong and weak universal Bayes consistency in metric spaces.
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness

Recognizing new objects by learning from a few labeled examples in an evolving environment is crucial to obtain excellent generalization ability for real-world machine learning systems. A typical setting across current meta learning algorithms assumes a stationary task distribution during meta training. In this paper, we explore a more practical and challenging setting where task distribution changes over time with domain shift. Particularly, we consider realistic scenarios where task distribution is highly imbalanced with domain labels unavailable in nature. We propose a kernel-based method for domain change detection and a difficulty-aware memory management mechanism that jointly considers the imbalanced domain size and domain importance to learn across domains continuously. Furthermore, we introduce an efficient adaptive task sampling method during meta training, which significantly reduces task gradient variance with theoretical guarantees. Finally, we propose a challenging benchmark with imbalanced domain sequences and varied domain difficulty. We have performed extensive evaluations on the proposed benchmark, demonstrating the effectiveness of our method.
Institutional Disruptions and the Philanthropy of Multinational Firms

This paper studies philanthropy by multinational enterprises (MNEs) during institutional disruptions—the sudden and unexpected, temporary, and systemic breakdowns in market-oriented institutions. The central argument is that, under institutional disruptions, MNEs aim to restore factors that are essential for the market to function, such as infrastructure and labor markets, and the strength of the market restoration motive is positively associated with the economic importance of the affected country to the MNE. Analyses of donations from 2,000 MNEs headquartered in 63 countries in the aftermath of 265 major epidemics, natural disasters, and terrorist attacks affecting 129 countries suggest that the economic importance of the country to the firm strongly explains donations. Country market concentration, public aid, and the country’s regulatory quality moderate this effect. These associations are robust to a matching method; a vector of firm-, country-, and event-specific time-varying and -constant variables; and alternative motives, such as reputation, altruism, media salience, market standing, and poverty-gap avoidance. They offer evidence that company philanthropy in the aftermath of institutional disruptions may deviate from predicted behavior under stable conditions. Particularly, the findings contest the expectation that philanthropy rises in market competition. Monopolistic firms are comparatively large donors and may act as an economic stop-loss mechanism during large disruptions.
Semi-supervised Adversarial Learning for Stain Normalisation in Histopathology Images

Hematoxylin and Eosin (H&E) stained histopathology images provide important clues for diagnostic and prognostic assessment of diseases. However, similar tissues can be stained with inconsistent colours which significantly hinder the diagnostic process and training of deep learning models. Various Generative Adversarial Network (GAN) based stain normalisation methods have thus been proposed as a preprocessing step for the downstream classification or detection tasks. However, most of these methods are based on either unsupervised learning which suffers from large discrepancy between domains or supervised learning which requires a target domain and only utilises the target domain images. In this work, we propose to leverage Semi-supervised Learning with GAN to incorporate the source domain images in the learning of stain normalisation without requiring their corresponding ground truth data. Our approach achieves highly effective performance on two classification tasks for brain and breast cancers.
Visuomotor Reinforcement Learning for Multirobot Cooperative Navigation

This article investigates the multirobot cooperative navigation problem based on raw visual observations. A fully end-to-end learning framework is presented, which leverages graph neural networks to learn local motion coordination and utilizes deep reinforcement learning to generate visuomotor policy that enables each robot to move to its goal without the need of environment map and global positioning information. Experimental results show that, with a few tens of robots, our approach achieves comparable performance with the state-of-the-art imitation learning-based approaches with bird-view state inputs. We also illustrate our generalizability to crowded and large environments and our scalability to ten times number of the training robots. In addition, we demonstrate that our model trained for multirobot case can also improve the success rate in the single-robot navigation task in unseen environments. Note to Practitioners—With the development of intelligent industrial and logistic systems, robotic transportation systems are widely implemented. However, existing multirobot path coordination and navigation approaches are basically under some unreasonable assumptions, which are very hard to be implemented in practical scenarios. This article aims to greatly promote the real application of learning-based multirobot cooperative navigation approach, in order to achieve the following. First, we introduce an end-to-end reinforcement learning framework instead of the commonly used imitation learning strategy, as the latter one needs exhaustive training data to cover all the scenarios and does not have the required generalizability. Second, we directly use the raw sensor data instead of the commonly used bird-eye-view semantic observations, as the latter one is generally not representative of practical application scenario from the robot perspective and cannot solve the occlusion issue. Third, we interpret our learned model to illustrate which parts of the input and shared observations contribute most to the robots’ final actions. The above interpretability ensures predictability (thus safety) of our visuomotor policy in practical applications. Our learned visuomotor policy has the ability to coordinate dozens of robots by only using raw visual observations in unknown environments without map nor global localization information, this is the first time in the literature. Our future work includes solving the sim-to-real issue and conducting physical experiments.
Reinforcement Learning with Deep Deterministic Policy Gradient

This study reviews the major developments of Deep Deterministic Policy Gradient (DDPG) in the field of reinforcement learning. It is innovated by Deep Q-network ideas and can finally handle some much challenging problems that operate over continuous action space. The main idea of DDPG is to use an actor-critic architecture (shown in Figure 5) to learn much more competitive policies. It allows the model to use neural network function approximators to learn in large state and action spaces. Due to its strong capacity, DDPG has many useful applications to real world problems in the field like robotics and control systems. But like most of the model-free reinforcement learning methods, the requirement for a large number of training steps is still a major difficulty for DDPG.
Accelerating CPU-based Distributed DNN Training on Modern HPC Clusters using BlueField-2 DPUs

The Deep Learning (DL) training process consists of multiple phases — data augmentation, training, and validation of the trained model. Traditionally, these phases are executed either on the CPUs or GPUs in a serial fashion due to lack of additional computing resources to offload independent phases of DL training. Recently, Mellanox/NVIDIA has introduced the BlueField-2 DPUs which combine the advanced capabilities of traditional ASIC based network adapters with an array of ARM processors. In this paper, we characterize and explore how one can take advantage of the additional ARM cores on the BlueField-2 DPUs to intelligently accelerate different phases of DL training. We propose multiple novel designs to efficiently offload the phases of DL training to the DPUs. We evaluate our proposed designs using multiple DL models on state-of-the-art HPC clusters. Our experimental results show that the proposed designs are able to deliver up to 15% improvement in overall DL training time. To the best of our knowledge, this is the first work to explore the use of DPUs to accelerate DL training.
Multiclass Anomaly Detection in Flight Data Using Semi-Supervised Explainable Deep Learning Model

The identification of precursors to safety incidents in aviation data is a crucial task, yet extremely challenging. The main approach in practice leverages domain expertise to define expected tolerances in system behavior and flags exceedances from such safety margins. However, this approach is incapable of identifying unknown risks and vulnerabilities. Various machine-learning approaches have been investigated and deployed to identify anomalies, with the great challenge of procuring enough labeled data to achieve reliable and accurate performance. This paper presents an explainable deep semi-supervised model for anomaly detection in aviation, building upon recent advancements described in the machine-learning literature. The proposed model combines feature engineering and classification in feature space, while leveraging all available data (labeled and unlabeled). Our approach is validated with case studies of anomaly detection during the takeoff and landing phases of commercial aircraft. Our model outperforms the state-of-the-art supervised anomaly-detection model, reaching significantly higher accuracy and fewer false alarms, even if only small proportion of data in the training set is labeled.
Channel rearrangement multi-branch network for image super-resolution

In image super-resolution task, the existing convolutional neural network methods proceed to increase the number of network layers and filters to achieve better performance. Nevertheless, such approaches not only lead to a dramatic increase in network parameters, but also restrict the practical application of super-resolution methods. In order to address this problem, we propose a lightweight model called Channel Rearrangement Multi-Branch Network (CRMBN) for super-resolution of single images. Specifically, we design a channel rearrangement algorithm based on the total variation of the feature maps. It is inspired by the fact that the feature maps with high total variation contain more high-frequency details, which are valuable for super-resolution reconstruction. Therefore, for the super-resolution networks, the more feature maps with high total variation, the better performance of reconstruction. The rearranged channels are divided into two groups, one with high total variation and the other with the opposite. In order to reduce the parameters while mining for more details, we send the features with low total variation to the deep network. Finally, high total variation features and deep features are fused to reconstruct high-resolution image. The experimental results show that compared with other lightweight models, the proposed method can offer a favorable improvement in terms of PSNR and SSIM.
A blockchain index structure based on subchain query

Abstract Blockchain technology has the characteristics of decentralization and tamper resistance, which can store data safely and reduce the cost of trust effectively. However, the existing blockchain system has weak performance in data management, and only supports traversal queries with transaction hashes as keywords. The query method based on the account transaction trace chain (ATTC) improves the query efficiency of historical transactions of the account. However, the efficiency of querying accounts with longer transaction chains has not been effectively improved. Given the inefficiency and single method of the ATTC index in the query, we propose a subchain-based account transaction chain (SCATC) index structure. First, the account transaction chain is divided into subchains, and the last block of each subchain is connected by a hash pointer. The block-by-block query mode in ATTC is converted to the subchain-by-subchain query mode, which shortens the query path. Multiple transactions of the same account in the same block are merged and stored, which simplifies the construction cost of the index and saves storage resources. then, the construction algorithm and query algorithm is given for the SCATC index structure. Simulation analysis shows that the SCATC index structure significantly improves query efficiency.