Reviewerly/exHarmony huggingface dataset:
Recent Developments in Parallel and Distributed Computing for Remotely Sensed Big Data Processing

This article gives a survey of state-of-the-art methods for processing remotely sensed big data and thoroughly investigates existing parallel implementations on diverse popular high-performance computing platforms. The pros/cons of these approaches are discussed in terms of capability, scalability, reliability, and ease of use. Among existing distributed computing platforms, cloud computing is currently the most promising solution to efficient and scalable processing of remotely sensed big data due to its advanced capabilities for high-performance and service-oriented computing. We further provide an in-depth analysis of state-of-the-art cloud implementations that seek for exploiting the parallelism of distributed processing of remotely sensed big data. In particular, we study a series of scheduling algorithms (GSs) aimed at distributing the computation load across multiple cloud computing resources in an optimized manner. We conduct a thorough review of different GSs and reveal the significance of employing scheduling strategies to fully exploit parallelism during the remotely sensed big data processing flow. We present a case study on large-scale remote sensing datasets to evaluate the parallel and distributed approaches and algorithms. Evaluation results demonstrate the advanced capabilities of cloud computing in processing remotely sensed big data and the improvements in computational efficiency obtained by employing scheduling strategies.
Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics

As a class of randomized learner model, stochastic configuration networks (SCNs) have been successfully applied in a few data analytics tasks. Given the industrial big data modeling tasks, however, the original SCNs potentially lead to excessive training time. To this end, this article extends SCNs to a hybrid parallel version, termed hybrid parallel SCNs (HPSCNs). In the hybrid parallel learning algorithm, two SCNs are synchronously constructed. The difference between the two of them is that one employs a point-incremental algorithm, and another one adopts a block-incremental algorithm. Moreover, a data parallel method is established based on a dynamic block strategy to accelerate the establishment of candidate node pool for each SCN. Additionally, this article proposes an adaptive hyperparameter adjustment method, which allows the hyperparameters in the supervisory mechanism to be automatically adjusted along with the learning process. Comparative experiments are first conducted through four large-scale benchmark datasets, followed by the fully discussion on the algorithm parameters. Finally, a practical industrial application case is made, where a grinding particle size soft sensor is developed based on HPSCN, showing the effectiveness of the proposed algorithm.
Biometrics-protected optical communication enabled by deep learning–enhanced triboelectric/photonic synergistic interface

Security is a prevailing concern in communication as conventional encryption methods are challenged by progressively powerful supercomputers. Here, we show that biometrics-protected optical communication can be constructed by synergizing triboelectric and nanophotonic technology. The synergy enables the loading of biometric information into the optical domain and the multiplexing of digital and biometric information at zero power consumption. The multiplexing process seals digital signals with a biometric envelope to avoid disrupting the original high-speed digital information and enhance the complexity of transmitted information. The system can perform demultiplexing, recover high-speed digital information, and implement deep learning to identify 15 users with around 95% accuracy, irrespective of biometric information data types (electrical, optical, or demultiplexed optical). Secure communication between users and the cloud is established after user identification for document exchange and smart home control. Through integrating triboelectric and photonics technology, our system provides a low-cost, easy-to-access, and ubiquitous solution for secure communication.
SF-YOLOv5: A Lightweight Small Object Detection Algorithm Based on Improved Feature Fusion Mode

In the research of computer vision, a very challenging problem is the detection of small objects. The existing detection algorithms often focus on detecting full-scale objects, without making proprietary optimization for detecting small-size objects. For small objects dense scenes, not only the accuracy is low, but also there is a certain waste of computing resources. An improved detection algorithm was proposed for small objects based on YOLOv5. By reasonably clipping the feature map output of the large object detection layer, the computing resources required by the model were significantly reduced and the model becomes more lightweight. An improved feature fusion method (PB-FPN) for small object detection based on PANet and BiFPN was proposed, which effectively increased the detection ability for small object of the algorithm. By introducing the spatial pyramid pooling (SPP) in the backbone network into the feature fusion network and connecting with the model prediction head, the performance of the algorithm was effectively enhanced. The experiments demonstrated that the improved algorithm has very good results in detection accuracy and real-time ability. Compared with the classical YOLOv5, the mAP@0.5 and mAP@0.5:0.95 of SF-YOLOv5 were increased by 1.6% and 0.8%, respectively, the number of parameters of the network were reduced by 68.2%, computational resources (FLOPs) were reduced by 12.7%, and the inferring time of the mode was reduced by 6.9%.
Unifying rational models of categorization via the hierarchical Dirichlet process

Models of categorization make different representational assumptions, with categories being represented by prototypes, sets of exemplars, and everything in between. Rational models of categorization justify these representational assumptions in terms of different schemes for estimating probability distributions. However, they do not answer the question of which scheme should be used in representing a given category. We show that existing rational models of categorization are special cases of a statistical model called the hierarchical Dirichlet process, which can be used to automatically infer a representation of the appropriate complexity for a given category.
Nonuniformly Sampled Data Processing Using LSTM Networks

We investigate classification and regression for nonuniformly sampled variable length sequential data and introduce a novel long short-term memory (LSTM) architecture. In particular, we extend the classical LSTM network with additional time gates, which incorporate the time information as a nonlinear scaling factor on the conventional gates. We also provide forward-pass and backward-pass update equations for the proposed LSTM architecture. We show that our approach is superior to the classical LSTM architecture when there is correlation between time samples. In our experiments, we achieve significant performance gains with respect to the classical LSTM and phased-LSTM architectures. In this sense, the proposed LSTM architecture is highly appealing for the applications involving nonuniformly sampled sequential data.
Vibration and modal analysis of a rotating disc using high-speed 3D digital image correlation

Rotating structures are of interest in many engineering fields, where knowing their dynamic behaviour is important to prevent undesired operating states. However, the experimental measurement of vibrations in rotating structures is still a relatively demanding task that requires the use of special measurement technique or method. The paper deals with the application of high-speed digital image correlation in vibration analysis of rotating structures. The method introduced in this research is based on the elimination of rigid body motion components contained in the primary responses measured by cameras. The process of separation and subsequent elimination of these components is given by numerical post-processing of three-dimensional displacement fields. The basis is to determine rotation matrix and translation vector that optimally describe rigid transformation between two positions of an analysed object. The proposed method is applicable for measuring under both constant and variable speed of rotation. Practical application is presented by three experiments in which vibrations of a flat rotating disc were analysed. The first experiment is focused on determine the operating deflection shapes of the disc rotating at 4700 rpm. The second one is a run-up analysis under variable rotational speed that ranges from 0 to 4700 rpm. The aim of the third measurement is to obtain natural frequencies and modal shapes of the disc rotating at 4800 rpm. The results of the experiments show that the elimination of rigid body motions leads to a higher accuracy of measurement and provides more pronounced frequency response spectra.
Hybrid representation learning for cross-modal retrieval

The rapid development of Deep Neural Networks (DNNs) in single-modal retrieval has promoted the wide application of DNNs in cross-modal retrieval tasks. Therefore, we propose a DNN-based method to learn the shared representation for each modality. Our method, hybrid representation learning (HRL), consists of three steps. In the first learning step, stacked restricted Boltzmann machines (SRBM) are utilized to extract the modality-friendly representation for each modality, with statistical properties that are more similar than those of the original input instances of both modalities, and a multimodal deep belief net (multimodal DBN) is utilized to extract the modality-mutual representation, which contains some missing information in the original input instances. In the second learning step, a two-level network containing a joint autoencoder and a three-layer feedforward neural net are used. From these steps, the hybrid representation is obtained, which combines the image representation constructed by the image-pathway SRBM and the modality-mutual representation, which involves the latent image representation and can be used to infer the missing values of the image via the multimodal DBN or vice-versa. In the third learning step, stacked bimodal autoencoders are used to obtain the final shared representation for each modality. The experimental results show that our proposed HRL method is superior to several advanced approaches according to three widely used cross-modal datasets.
The use of virtual reality head-mounted displays within applied sport psychology

Virtual reality (VR) technology has been employed within several domains such as medicine, education, and the military. Nonetheless, there is limited research examining how VR can supplement applied sport psychology practice. This article provides the reader with an understanding of key components and concepts associated with VR head-mounted displays (HMDs). Subsequently, a range of possible applications within applied sport psychology are discussed, such as the training of perceptual-cognitive skills, relaxation strategies, and injury rehabilitation. Thereafter, the practicalities of using VR HMDs are outlined, and recommendations are provided to applied sport psychology practitioners wishing to embed this technology within their practice.
SemEval-2019 Task 9: Suggestion Mining from Online Reviews and Forums

We present the pilot SemEval task on Suggestion Mining. The task consists of subtasks A and B, where we created labeled data from feedback forum and hotel reviews respectively. Subtask A provides training and test data from the same domain, while Subtask B evaluates the system on a test dataset from a different domain than the available training data. 33 teams participated in the shared task, with a total of 50 members. We summarize the problem definition, benchmark dataset preparation, and methods used by the participating teams, providing details of the methods used by the top ranked systems. The dataset is made freely available to help advance the research in suggestion mining, and reproduce the systems submitted under this task
Learning Behavior Trees From Demonstration

Robotic Learning from Demonstration (LfD) allows anyone, not just experts, to program a robot for an arbitrary task. Many LfD methods focus on low level primitive actions such as manipulator trajectories. Complex multistep task with many primitive actions must be learned from demonstration if LfD is to encompass the full range of task a user may desire. Existing methods represent the high level task in various forms including, finite state machines, decision trees, formal logic, among others. Behavior trees are proposed as an alternative representation of high level task. Behavior trees are an execution model for the control of a robot designed for real time execution, modularity, and, consequently, transparency. Real time execution allows the robot to reactively perform the task. Modularity allows the reuse of learned primitive actions and high level task in new situations, speeding up the process of learning in new scenarios. Transparency allows users to understand and interactively modify the learned model. Behavior trees are used to represent high level tasks by building on the relationship it has with decision trees. We demonstrate a human teaching our Fetch robot a household cleaning task.
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning

The application of stochastic variance reduction to optimization has shown remarkable recent theoretical and practical success. The applicability of these techniques to the hard non-convex optimization problems encountered during training of modern deep neural networks is an open problem. We show that naive application of the SVRG technique and related approaches fail, and explore why.
Post training 4-bit quantization of convolutional networks for rapid-deployment

Convolutional neural networks require significant memory bandwidth and storage for intermediate computations, apart from substantial computing resources. Neural network quantization has significant benefits in reducing the amount of intermediate results, but it often requires the full datasets and time-consuming fine tuning to recover the accuracy lost after quantization. This paper introduces the first practical 4-bit post training quantization approach: it does not involve training the quantized model (fine-tuning), nor it requires the availability of the full dataset. We target the quantization of both activations and weights and suggest three complementary methods for minimizing quantization error at the tensor level, two of whom obtain a closed-form analytical solution. Combining these methods, our approach achieves accuracy that is just a few percents less the state-of-the-art baseline across a wide range of convolutional models. The source code to replicate all experiments is available on GitHub: \url{https://github.com/submission2019/cnn-quantization}.
Limitations of Lazy Training of Two-layers Neural Network

We study the supervised learning problem under either of the following two models: (1) Feature vectors x_i are d-dimensional Gaussian and responses are y_i = f_*(x_i) for f_* an unknown quadratic function; (2) Feature vectors x_i are distributed as a mixture of two d-dimensional centered Gaussians, and y_i's are the corresponding class labels. We use two-layers neural networks with quadratic activations, and compare three different learning regimes: the random features (RF) regime in which we only train the second-layer weights; the neural tangent (NT) regime in which we train a linearization of the neural network around its initialization; the fully trained neural network (NN) regime in which we train all the weights in the network. We prove that, even for the simple quadratic model of point (1), there is a potentially unbounded gap between the prediction risk achieved in these three training regimes, when the number of neurons is smaller than the ambient dimension. When the number of neurons is larger than the number of dimensions, the problem is significantly easier and both NT and NN learning achieve zero risk.
Multi-Agent Path Finding – An Overview

Multi-Agent Pathfinding (MAPF) is the problem of finding paths for multiple agents such that every agent reaches its goal and the agents do not collide. In recent years, there has been a growing interest in MAPF in the Artificial Intelligence (AI) research community. This interest is partially because real-world MAPF applications, such as warehouse management, multi-robot teams, and aircraft management, are becoming more prevalent. In this overview, we discuss several possible definitions of the MAPF problem. Then, we survey MAPF algorithms, starting with fast but incomplete algorithms, then fast, complete but not optimal algorithms, and finally optimal algorithms. Then, we describe approximately optimal algorithms and conclude with non-classical MAPF and pointers for future reading and future work.
Nibbler

Developers today have access to an arsenal of toolkits and libraries for rapid application prototyping. However, when an application loads a library, the entirety of that library's code is mapped into the address space, even if only a single function is actually needed. The unused portion is bloat that can negatively impact software defenses by unnecessarily inflating their overhead or increasing their attack surface. Recent work has explored debloating as a way of alleviating the above problems, when source code is available. In this paper, we investigate whether debloating is possible and practical at the binary level. To this end, we present Nibbler: a system that identifies and erases unused functions within shared libraries. Nibbler works in tandem with defenses like continuous code re-randomization and control-flow integrity, enhancing them without incurring additional run-time overhead. We developed and tested a prototype of Nibbler on x86-64 Linux; Nibbler reduces the size of shared libraries and the number of available functions, for real-world binaries and the SPEC CINT2006 suite, by up to 56% and 82%, respectively. We also demonstrate that Nibbler benefits defenses by showing that: (i) it improves the deployability of a continuous re-randomization system for binaries, namely Shuffler, by increasing its efficiency by 20%, and (ii) it improves certain fast, but coarse and context-insensitive control-flow integrity schemes by reducing the number of gadgets reachable through returns and indirect calls by 75% and 49% on average.
NAS-BENCH-1SHOT1: BENCHMARKING AND DISSECTING ONE-SHOT NEURAL ARCHITECTURE SEARCH

One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice. Nevertheless, there is still a lack of understanding on how these weight-sharing algorithms exactly work due to the many factors controlling the dynamics of the process. In order to allow a scientific study of these components, we introduce a general framework for one-shot NAS that can be instantiated to many recently-introduced variants and introduce a general benchmarking framework that draws on the recent large-scale tabular benchmark NAS-Bench-101 for cheap anytime evaluations of one-shot NAS methods. To showcase the framework, we compare several state-of-the-art one-shot NAS methods, examine how sensitive they are to their hyperparameters and how they can be improved by careful regularization, and compare their per- formance to that of blackbox optimizers for NAS-Bench-101.
Compiler-based graph representations for deep learning models of code

In natural language processing, novel methods in deep learning, like recurrent neural networks (RNNs) on sequences of words, have been very successful. In contrast to natural languages, programming languages usually have a well-defined structure. With this structure compilers can reason about programs, using graphs such as abstract syntax trees (ASTs) or control-data flow graphs (CDFGs). In this paper, we argue that we should use these graph structures instead of sequences for learning compiler optimization tasks. To this end, we use graph neural networks (GNNs) for learning predictive compiler tasks on two representations based on ASTs and CDFGs. Experiments show that this improves upon the state-of-the-art in the task of heterogeneous OpenCL mapping, while providing orders of magnitude faster inference times, crucial for compiler optimizations. When testing on benchmark suites not included for training, our AST-based model significantly outperforms the state-of-the-art by over 12 percentage points in terms of accuracy. It is the only one to perform clearly better than a random mapping. On the task of predicting thread coarsening factors, we show that all of the methods fail to produce an overall speedup.
Learning Invariant Representations for Reinforcement Learning without Reconstruction

We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Our goal is to learn representations that both provide for effective downstream control and invariance to task-irrelevant details. Bisimulation metrics quantify behavioral similarity between states in continuous MDPs, which we propose using to learn robust latent representations which encode only the task-relevant information from observations. Our method trains encoders such that distances in latent space equal bisimulation distances in state space. We demonstrate the effectiveness of our method at disregarding task-irrelevant information using modified visual MuJoCo tasks, where the background is replaced with moving distractors and natural videos, while achieving SOTA performance. We also test a first-person highway driving task where our method learns invariance to clouds, weather, and time of day. Finally, we provide generalization results drawn from properties of bisimulation metrics, and links to causal inference.
Regularized Evolutionary Multitask Optimization: Learning to Intertask Transfer in Aligned Subspace

This article proposes a novel and computationally efficient explicit intertask information transfer strategy between optimization tasks by aligning the subspaces. In evolutionary multitasking, the tasks might have biases embedded in function landscapes and decision spaces, which often causes the threat of predominantly negative transfer. However, the complementary information among different tasks can give an enhanced performance of solving complicated problems when properly harnessed. In this article, we distill this insight by introducing an intertask knowledge transfer strategy implemented in the low-dimension subspaces via a learnable alignment matrix. Specifically, to unveil the significant features of the function landscapes, the task-specific low-dimension subspaces is established based on the distribution information of subpopulations possessed by tasks, respectively. Next, the alignment matrix between pairwise subspaces is learned by minimizing the discrepancies of the subspaces. Given the aligned subspaces by applying the alignment matrix to subspaces' base vectors, the individuals from different tasks are then projected into aligned subspaces and reproduce therein. Moreover, since this method only considers the leading eigenvectors, it turns out to be intrinsically regularized and noise-insensitive. Comprehensive experiments are conducted on the synthetic and practical benchmark problems so as to assess the efficacy of the proposed method. According to the experimental results, the proposed method exhibits a superior performance compared with existing evolutionary multitask optimization algorithms.