Reviewerly/exHarmony huggingface dataset:
Empirically Measuring Concentration: Fundamental Limits on Intrinsic Robustness

Many recent works have shown that adversarial examples that fool classifiers can be found by minimally perturbing a normal input. Recent theoretical results, starting with Gilmer et al. (2018b), show that if the inputs are drawn from a concentrated metric probability space, then adversarial examples with small perturbation are inevitable. A concentrated space has the property that any subset with Ω(1) (e.g.,1/100) measure, according to the imposed distribution, has small distance to almost all (e.g., 99/100) of the points in the space. It is not clear, however, whether these theoretical results apply to actual distributions such as images. This paper presents a method for empirically measuring and bounding the concentration of a concrete dataset which is proven to converge to the actual concentration. We use it to empirically estimate the intrinsic robustness to and L_2 and L_infinity perturbations of several image classification benchmarks. Code for our experiments is available at https://github.com/xiaozhanguva/Measure-Concentration.
Initialization of ReLUs for Dynamical Isometry

Deep learning relies on good initialization schemes and hyperparameter choices prior to training a neural network. Random weight initializations induce random network ensembles, which give rise to the trainability, training speed, and sometimes also generalization ability of an instance. In addition, such ensembles provide theoretical insights into the space of candidate models of which one is selected during training. The results obtained so far rely on mean field approximations that assume infinite layer width and that study average squared signals. We derive the joint signal output distribution exactly, without mean field assumptions, for fully-connected networks with Gaussian weights and biases, and analyze deviations from the mean field results. For rectified linear units, we further discuss limitations of the standard initialization scheme, such as its lack of dynamical isometry, and propose a simple alternative that overcomes these by initial parameter sharing.
Adaptively Aligned Image Captioning via Adaptive Attention Time

Recent neural models for image captioning usually employ an encoder-decoder framework with an attention mechanism. However, the attention mechanism in such a framework aligns one single (attended) image feature vector to one caption word, assuming one-to-one mapping from source image regions and target caption words, which is never possible. In this paper, we propose a novel attention model, namely Adaptive Attention Time (AAT), to align the source and the target adaptively for image captioning. AAT allows the framework to learn how many attention steps to take to output a caption word at each decoding step. With AAT, an image region can be mapped to an arbitrary number of caption words while a caption word can also attend to an arbitrary number of image regions. AAT is deterministic and differentiable, and doesn't introduce any noise to the parameter gradients. In this paper, we empirically show that AAT improves over state-of-the-art methods on the task of image captioning. Code is available at https://github.com/husthuaan/AAT.
Learning Deterministic Weighted Automata with Queries and Counterexamples

We present an algorithm for reconstruction of a probabilistic deterministic finite automaton (PDFA) from a given black-box language model, such as a recurrent neural network (RNN). The algorithm is a variant of the exact-learning algorithm L*, adapted to work in a probabilistic setting under noise. The key insight of the adaptation is the use of conditional probabilities when making observations on the model, and the introduction of a variation tolerance when comparing observations. When applied to RNNs, our algorithm returns models with better or equal word error rate (WER) and normalised distributed cumulative gain (NDCG) than achieved by n-gram or weighted finite automata (WFA) approximations of the same networks. The PDFAs capture a richer class of languages than n-grams, and are guaranteed to be stochastic and deterministic -- unlike the WFAs.
Bayesian Learning of Sum-Product Networks

Sum-product networks (SPNs) are flexible density estimators and have received significant attention due to their attractive inference properties. While parameter learning in SPNs is well developed, structure learning leaves something to be desired: Even though there is a plethora of SPN structure learners, most of them are somewhat ad-hoc and based on intuition rather than a clear learning principle. In this paper, we introduce a well-principled Bayesian framework for SPN structure learning. First, we decompose the problem into i) laying out a computational graph, and ii) learning the so-called scope function over the graph. The first is rather unproblematic and akin to neural network architecture validation. The second represents the effective structure of the SPN and needs to respect the usual structural constraints in SPN, i.e. completeness and decomposability. While representing and learning the scope function is somewhat involved in general, in this paper, we propose a natural parametrisation for an important and widely used special case of SPNs. These structural parameters are incorporated into a Bayesian model, such that simultaneous structure and parameter learning is cast into monolithic Bayesian posterior inference. In various experiments, our Bayesian SPNs often improve test likelihoods over greedy SPN learners. Further, since the Bayesian framework protects against overfitting, we can evaluate hyper-parameters directly on the Bayesian model score, waiving the need for a separate validation set, which is especially beneficial in low data regimes. Bayesian SPNs can be applied to heterogeneous domains and can easily be extended to nonparametric formulations. Moreover, our Bayesian approach is the first, which consistently and robustly learns SPN structures under missing data.
Arbicon-Net: Arbitrary Continuous Geometric Transformation Networks for Image Registration

This paper concerns the undetermined problem of estimating geometric transformation between image pairs. Recent methods introduce deep neural networks to predict the controlling parameters of hand-crafted geometric transformation models (e.g. thin-plate spline) for image registration and matching. However, the low-dimension parametric models are incapable of estimating a highly complex geometric transform with limited flexibility to model the actual geometric deformation from image pairs. To address this issue, we present an end-to-end trainable deep neural networks, named Arbitrary Continuous Geometric Transformation Networks (Arbicon-Net), to directly predict the dense displacement field for pairwise image alignment. Arbicon-Net is generalized from training data to predict the desired arbitrary continuous geometric transformation in a data-driven manner for unseen new pair of images. Particularly, without imposing penalization terms, the predicted displacement vector function is proven to be spatially continuous and smooth. To verify the performance of Arbicon-Net, we conducted semantic alignment tests over both synthetic and real image dataset with various experimental settings. The results demonstrate that Arbicon-Net outperforms the previous image alignment techniques in identifying the image correspondences.
Adversarial Self-Defense for Cycle-Consistent GANs

The goal of unsupervised image-to-image translation is to map images from one domain to another without the ground truth correspondence between the two domains. State-of-art methods learn the correspondence using large numbers of unpaired examples from both domains and are based on generative adversarial networks. In order to preserve the semantics of the input image, the adversarial objective is usually combined with a cycle-consistency loss that penalizes incorrect reconstruction of the input image from the translated one. However, if the target mapping is many-to-one, e.g. aerial photos to maps, such a restriction forces the generator to hide information in low-amplitude structured noise that is undetectable by human eye or by the discriminator. In this paper, we show how such self-attacking behavior of unsupervised translation methods affects their performance and provide two defense techniques. We perform a quantitative evaluation of the proposed techniques and show that making the translation model more robust to the self-adversarial attack increases its generation quality and reconstruction reliability and makes the model less sensitive to low-amplitude perturbations. Our project page can be found at ai.bu.edu/selfadv.
Using a Logarithmic Mapping to Enable Lower Discount Factors in Reinforcement Learning

In an effort to better understand the different ways in which the discount factor affects the optimization process in reinforcement learning, we designed a set of experiments to study each effect in isolation. Our analysis reveals that the common perception that poor performance of low discount factors is caused by (too) small action-gaps requires revision. We propose an alternative hypothesis that identifies the size-difference of the action-gap across the state-space as the primary cause. We then introduce a new method that enables more homogeneous action-gaps by mapping value estimates to a logarithmic space. We prove convergence for this method under standard assumptions and demonstrate empirically that it indeed enables lower discount factors for approximate reinforcement-learning methods. This in turn allows tackling a class of reinforcement-learning problems that are challenging to solve with traditional methods.
Learning to Predict Without Looking Ahead: World Models Without Forward Prediction

Much of model-based reinforcement learning involves learning a model of an agent's world, and training an agent to leverage this model to perform a task more efficiently. While these models are demonstrably useful for agents, every naturally occurring model of the world of which we are aware---e.g., a brain---arose as the byproduct of competing evolutionary pressures for survival, not minimization of a supervised forward-predictive loss via gradient descent. That useful models can arise out of the messy and slow optimization process of evolution suggests that forward-predictive modeling can arise as a side-effect of optimization under the right circumstances. Crucially, this optimization process need not explicitly be a forward-predictive loss. In this work, we introduce a modification to traditional reinforcement learning which we call observational dropout, whereby we limit the agents ability to observe the real environment at each timestep. In doing so, we can coerce an agent into learning a world model to fill in the observation gaps during reinforcement learning. We show that the emerged world model, while not explicitly trained to predict the future, can help the agent learn key skills required to perform well in its environment. Videos of our results available at https://learningtopredict.github.io/
Increasing Trust in Data-Driven Model Validation

In recent years, interest in autonomous systems has increased. To observe their environment and interact with it, such systems need to process sensor data including camera images. State-of-the-art methods for object recognition and image segmentation rely on complex data-driven models such as convolutional neural networks. Although no final answer exists yet on how to perform safety evaluation of systems containing such models, such evaluation should comprise at least validation with realistic input data, including settings with suboptimal data quality. Because many test datasets still lack a sufficient number of representative quality deficits, we consider augmenting existing data with quality deficits as necessary. For this purpose, a novel tool framework is presented and illustrated using traffic sign recognition as a use case. The extendable approach distinguishes between augmentation at the object, context, and sensor levels. To provide realistic augmentation and meta-data for existing image datasets, known context information and conditional probabilities are processed. First applications on the GTSRB dataset show promising results. The augmentation of datasets facilitates a more rigorous investigation of how various quality deficits affect the accuracy of a model in its target application scope.
Source Generator Attribution via Inversion

With advances in Generative Adversarial Networks (GANs) leading to dramatically-improved synthetic images and video, there is an increased need for algorithms which extend traditional forensics to this new category of imagery. While GANs have been shown to be helpful in a number of computer vision applications, there are other problematic uses such as `deep fakes' which necessitate such forensics. Source camera attribution algorithms using various cues have addressed this need for imagery captured by a camera, but there are fewer options for synthetic imagery. We address the problem of attributing a synthetic image to a specific generator in a white box setting, by inverting the process of generation. This enables us to simultaneously determine whether the generator produced the image and recover an input which produces a close match to the synthetic image.
Excavating the Potential of GPU for Accelerating Graph Traversal

Graph traversal is an essential procedure for a growing amount of applications today. This type of algorithms typically iterate input graph datasets until convergence and the logic of each iteration is quite simple. GPUs are used extensively as graph traversal accelerators due to the capability of massive parallelism and high-bandwidth memory access. However, existing methods are inefficient in two ways. First, streaming multiprocessors (SMs) are still underutilized due to the unbalanced load allocation and uncoalesced memory access. Second, they use space-inefficient data structures or need auxiliary data to assist traversal. It is undesirable, considering the limited GPU memory capacity. Moreover, existing designs commonly focus on optimizing kernel execution time. Data-transfer time is also notable in the whole procedure. Thus, space-efficient data structure and data-transfer policy should be concerned. In this paper, we propose EtaGraph, a novel GPU graph traversal framework optimized for GPU memory system and execution parallelism. EtaGraph has several features: 1). It uses a frontier-like kernel execution model, featuring a lightweight graph transformation procedure, named Unified Degree Cut, allowing GPU threads to process skewed graph efficiently without modification of raw data or introducing extra space overhead; 2). It uses on-demand data-transfer to overlap computation so that it optimizes the total time of data-transfer and execution; 3). It adopts an explicit utilization of Shared Memory to enhance memory coalescing and to improve effective memory bandwidth. Evaluation of EtaGraph shows significant and consistent speedups over the state-of-the-art GPU-based graph processing frameworks on both real-world and synthetic graphs.
Design of Beidou Satellite System in Ocean Logistics Real-Time Tracking System

Hao, Q.; Wang, Z.-F. and Qin, L.-L., 2019. Design of Beidou satellite system in ocean logistics real-time tracking system. In: Gong, D.; Zhu, H., and Liu, R. (eds.), Selected Topics in Coastal Research: Engineering, Industry, Economy, and Sustainable Development. Journal of Coastal Research, Special Issue No. 94, pp. 204–207. Coconut Creek (Florida), ISSN 0749-0208.In order to ensure the quality and safety of pelagic fish catches in the process of transportation, administrator needs to carry out real-time monitoring and management to the transport and quality status during this period and establish a complete management system. This paper puts forward that the real-time online monitoring of transport location, catches temperature, humidity, PH value and other physical property data can be used during the application process of Beidou System and RFID temperature tags in the cold chain logistics of ocean catches. The system consists of 3 parts, i.e. vehicle or vessel tracking, parameter monitoring and data management and analysis. The system applies the technologies such as Beidou, ZigBee and Internet as well as the Java-programming client web page structure and the traditional SSH2 as its code framework. Hence, it realizes the real-time monitoring of whole catches transportation logistics process and makes administrator timely know about the status of catches so as to ensure the product quality. This system can guarantee the quality, safety and transport efficiency of pelagic fish products and provides a support for the building of cold chain logistics system.
Asymmetric Bayesian personalized ranking for one-class collaborative filtering

In this paper, we propose a novel preference assumption for modeling users' one-class feedback such as "thumb up" in an important recommendation problem called one-class collaborative filtering (OCCF). Specifically, we address a fundamental limitation of a recent symmetric pairwise preference assumption and propose a novel and first asymmetric one, which is able to make the preferences of different users more comparable. With the proposed asymmetric pairwise preference assumption, we further design a novel recommendation algorithm called asymmetric Bayesian personalized ranking (ABPR). Extensive empirical studies on two large and public datasets show that our ABPR performs significantly better than several state-of-the-art recommendation methods with either pointwise preference assumption or pairwise preference assumption.
Reconfigurable Multi-UAV Formation Using Angle-Encoded PSO

In this paper, we propose an algorithm for the formation of multiple UAVs used in vision-based inspection of infrastructure. A path planning algorithm is first developed by using a variant of the particle swarm optimisation, named θ-PSO, to generate a feasible path for the overall formation configuration taken into account the constraints for visual inspection. Here, we introduced a cost function that includes various constraints on flight safety and visual inspection. A reconfigurable topology is then added based on the use of intermediate waypoints to allow the formation to avoid collision with obstacles during operation. The planned path and formation are then combined to derive the trajectory and velocity profiles for each UAV. Experiments have been conducted for the task of inspecting a light rail bridge. The results confirmed the validity and effectiveness of the proposed algorithm.
Visualising a “good game”: analytics as a calculative engine in a digital environment

Purpose The purpose of this paper is to concern the use of analytics as a calculative engine enabling coordination and control for the development process in a creative digital business environment. Design/methodology/approach This research employs an explorative field study approach, using interview data from professionals working with free-to-play mobile game development. Drawing on the concepts of cycles of accumulation, accounting as an engine and mediating instruments, this study examines how organisational actors using the analytics in a digital business environment participate in the data generation that accumulates knowledge about and new insights into the desired outcome. Findings The real-time metrics provided the means for organisational actors to continually monitor, visualise and if necessary intervene in the creative “good game” development process. Timely quantification and visualisation of user actions, collected as digital traces, enhanced the cycle of information accumulation. This new knowledge resulted in a desire for improvement and perfection, which directed the actions towards the organisational objectives. Originality/value This study furthers our understanding of the performativity of accounting as an engine and the user behavioural data trace as its “fuel” in a digital product development. It highlights the role of analytics as a “fact-generating” device, capable of transforming the raw user behavioural data, the fuel, into powerful explanations through visualisations of ideals. The real-time metrics, understood as mediating instruments, enable the generation of new insights and accumulation of knowledge guiding the further development towards the desired outcome, the “good game”.
Interaction among Information Sharing, Supply Chain Structure and Performance

Zhang, X.; Fang, W., and Pi, Z., 2019. Interaction among information sharing, supply chain structure and performance. In: Guido-Aldana, P.A. and Mulahasan, S. (eds.), Advances in Water Resources and Exploration. Journal of Coastal Research, Special Issue No. 93, pp. 870–878. Coconut Creek (Florida), ISSN 0749-0208.The uncertainty of external environment is an important factor affecting the performance of supply chain, while information sharing and supply chain structure optimization are important means to reduce uncertainty and improve supply chain performance. From the perspective of decision-making and global optimization, a mathematical model on the interaction among environmental uncertainty, supply chain structure, information sharing and supply chain performance was established. Through theoretical analysis and numerical simulation, this paper explores the influence of environmental uncertainty, information distortion and information cost on supply chain structure design and supply chain performance, and analyzed the influence of information sharing on supply chain performance. This study provides a theoretical basis for understanding the role of information sharing in supply chain management, reasonably grasping the level of information sharing and optimizing the supply chain structure in uncertain environment. The comparison with the classical bullwhip effect research was made, which provides a new perspective and supplement for understanding and slowing down the bullwhip effect.
The Unfairness of Popularity Bias in Recommendation.

Recommender systems are known to suffer from the popularity bias problem: popular (i.e. frequently rated) items get a lot of exposure while less popular ones are under-represented in the recommendations. Research in this area has been mainly focusing on finding ways to tackle this issue by increasing the number of recommended long-tail items or otherwise the overall catalog coverage. In this paper, however, we look at this problem from the users' perspective: we want to see how popularity bias causes the recommendations to deviate from what the user expects to get from the recommender system. We define three different groups of users according to their interest in popular items (Niche, Diverse and Blockbuster-focused) and show the impact of popularity bias on the users in each group. Our experimental results on a movie dataset show that in many recommendation algorithms the recommendations the users get are extremely concentrated on popular items even if a user is interested in long-tail and non-popular items showing an extreme bias disparity.