OpenRaiser/Intern-Atlas huggingface dataset:
On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimization
Venue: arXiv cs.LG 2020
Year: 2020

In decentralized optimization, it is common algorithmic practice to have nodes interleave (local) gradient descent iterations with gossip (i.e. averaging over the network) steps. Motivated by the training of large-scale machine learning models, it is also increasingly common to require that messages be {\em lossy compressed} versions of the local parameters. In this paper, we show that, in such compressed decentralized optimization settings, there are benefits to having {\em multiple} gossip steps between subsequent gradient iterations, even when the cost of doing so is appropriately accounted for e.g. by means of reducing the precision of compressed information. In particular, we show that having $O(\log\frac{1}ε)$ gradient iterations {with constant step size} - and $O(\log\frac{1}ε)$ gossip steps between every pair of these iterations - enables convergence to within $ε$ of the optimal value for smooth non-convex objectives satisfying Polyak-Łojasiewicz condition. This result also holds for smooth strongly convex objectives. To our knowledge, this is the first work that derives convergence results for nonconvex optimization under arbitrary communication compression.
Enhancing Poaching Predictions for Under-Resourced Wildlife Conservation Parks Using Remote Sensing Imagery
Venue: arXiv cs.LG 2020
Year: 2020

Illegal wildlife poaching is driving the loss of biodiversity. To combat poaching, rangers patrol expansive protected areas for illegal poaching activity. However, rangers often cannot comprehensively search such large parks. Thus, the Protection Assistant for Wildlife Security (PAWS) was introduced as a machine learning approach to help identify the areas with highest poaching risk. As PAWS is deployed to parks around the world, we recognized that many parks have limited resources for data collection and therefore have scarce feature sets. To ensure under-resourced parks have access to meaningful poaching predictions, we introduce the use of publicly available remote sensing data to extract features for parks. By employing this data from Google Earth Engine, we also incorporate previously unavailable dynamic data to enrich predictions with seasonal trends. We automate the entire data-to-deployment pipeline and find that, with only using publicly available data, we recuperate prediction performance comparable to predictions made using features manually computed by park specialists. We conclude that the inclusion of satellite imagery creates a robust system through which parks of any resource level can benefit from poaching risks for years to come.
A Worrying Analysis of Probabilistic Time-series Models for Sales Forecasting
Venue: arXiv cs.LG 2020
Year: 2020

Probabilistic time-series models become popular in the forecasting field as they help to make optimal decisions under uncertainty. Despite the growing interest, a lack of thorough analysis hinders choosing what is worth applying for the desired task. In this paper, we analyze the performance of three prominent probabilistic time-series models for sales forecasting. To remove the role of random chance in architecture's performance, we make two experimental principles; 1) Large-scale dataset with various cross-validation sets. 2) A standardized training and hyperparameter selection. The experimental results show that a simple Multi-layer Perceptron and Linear Regression outperform the probabilistic models on RMSE without any feature engineering. Overall, the probabilistic models fail to achieve better performance on point estimation, such as RMSE and MAPE, than comparably simple baselines. We analyze and discuss the performances of probabilistic time-series models.
Computation harvesting in road traffic dynamics
Venue: arXiv cs.LG 2020
Year: 2020

Owing to recent advances in artificial intelligence and internet of things (IoT) technologies, collected big data facilitates high computational performance, while its computational resources and energy cost are large. Moreover, data are often collected but not used. To solve these problems, we propose a framework for a computational model that follows a natural computational system, such as the human brain, and does not rely heavily on electronic computers. In particular, we propose a methodology based on the concept of `computation harvesting', which uses IoT data collected from rich sensors and leaves most of the computational processes to real-world phenomena as collected data. This aspect assumes that large-scale computations can be fast and resilient. Herein, we perform prediction tasks using real-world road traffic data to show the feasibility of computation harvesting. First, we show that the substantial computation in traffic flow is resilient against sensor failure and real-time traffic changes due to several combinations of harvesting from spatiotemporal dynamics to synthesize specific patterns. Next, we show the practicality of this method as a real-time prediction because of its low computational cost. Finally, we show that, compared to conventional methods, our method requires lower resources while providing a comparable performance.
Enhanced Innovized Repair Operator for Evolutionary Multi- and Many-objective Optimization
Venue: arXiv cs.NE 2020
Year: 2020

"Innovization" is a task of learning common relationships among some or all of the Pareto-optimal (PO) solutions in multi- and many-objective optimization problems. Recent studies have shown that a chronological sequence of non-dominated solutions obtained in consecutive iterations during an optimization run also possess salient patterns that can be used to learn problem features to help create new and improved solutions. In this paper, we propose a machine-learning- (ML-) assisted modelling approach that learns the modifications in design variables needed to advance population members towards the Pareto-optimal set. We then propose to use the resulting ML model as an additional innovized repair (IR2) operator to be applied on offspring solutions created by the usual genetic operators, as a novel mean of improving their convergence properties. In this paper, the well-known random forest (RF) method is used as the ML model and is integrated with various evolutionary multi- and many-objective optimization algorithms, including NSGA-II, NSGA-III, and MOEA/D. On several test problems ranging from two to five objectives, we demonstrate improvement in convergence behaviour using the proposed IR2-RF operator. Since the operator does not demand any additional solution evaluations, instead using the history of gradual and progressive improvements in solutions over generations, the proposed ML-based optimization opens up a new direction of optimization algorithm development with advances in AI and ML approaches.
Dynamic Data Consistency Tests Using a CRUD Matrix as an Underlying Model
Venue: arXiv cs.SE 2020
Year: 2020

In testing of software and Internet of Things (IoT) systems, one of necessary type of tests has to verify the consistency of data that are processed and stored in the system. The Data Cycle Test technique can effectively do such tests. The goal of this technique is to verify that the system processes data entities in a system under test in a correct way and that they remain in a consistent state after operations such as create, read, update and delete. Create, read, update and delete (CRUD) matrices are used for this purpose. In this paper, we propose an extension of the Data Cycle Test design technique, which is described in the TMap methodology and related literature. This extension includes a more exact definition of the test coverage, a reflection of the relationships between the tested data entities, an exact algorithm to select and combine read and update operations in test cases for a particular data entity, and verification of the consistency of the produced test cases. As verified by our experiments, in comparison to the original Data Cycle Test technique, this proposed extension helps test designers to produce more consistent test cases that reduce the number of undetected potential data consistency defects.
A Neuro-Inspired Autoencoding Defense Against Adversarial Perturbations
Venue: arXiv cs.LG 2020
Year: 2020

Deep Neural Networks (DNNs) are vulnerable to adversarial attacks: carefully constructed perturbations to an image can seriously impair classification accuracy, while being imperceptible to humans. While there has been a significant amount of research on defending against such attacks, most defenses based on systematic design principles have been defeated by appropriately modified attacks. For a fixed set of data, the most effective current defense is to train the network using adversarially perturbed examples. In this paper, we investigate a radically different, neuro-inspired defense mechanism, starting from the observation that human vision is virtually unaffected by adversarial examples designed for machines. We aim to reject L^inf bounded adversarial perturbations before they reach a classifier DNN, using an encoder with characteristics commonly observed in biological vision: sparse overcomplete representations, randomness due to synaptic noise, and drastic nonlinearities. Encoder training is unsupervised, using standard dictionary learning. A CNN-based decoder restores the size of the encoder output to that of the original image, enabling the use of a standard CNN for classification. Our nominal design is to train the decoder and classifier together in standard supervised fashion, but we also consider unsupervised decoder training based on a regression objective (as in a conventional autoencoder) with separate supervised training of the classifier. Unlike adversarial training, all training is based on clean images. Our experiments on the CIFAR-10 show performance competitive with state-of-the-art defenses based on adversarial training, and point to the promise of neuro-inspired techniques for the design of robust neural networks. In addition, we provide results for a subset of the Imagenet dataset to verify that our approach scales to larger images.
Spatio-Temporal Visualization of Interdependent Battery Bus Transit and Power Distribution Systems
Venue: arXiv cs.HC 2020
Year: 2020

The high penetration of transportation electrification and its associated charging requirements magnify the interdependency of the transportation and power distribution systems. The emergent interdependency requires that system operators fully understand the status of both systems. To this end, a visualization tool is presented to illustrate the interdependency of battery bus transit and power distribution systems and the associated components. The tool aims at monitoring components from both systems, such as the locations of electric buses, the state of charge of batteries, the price of electricity, voltage, current, and active/reactive power flow. The results showcase the success of the visualization tool in monitoring the bus transit and power distribution components to determine a reliable cost-effective scheme for spatio-temporal charging of electric buses.
Applying Multi-armed Bandit Algorithms to Computational Advertising
Venue: arXiv cs.IR 2020
Year: 2020

Over the last two decades, we have seen extensive industrial research in the area of computational advertising. In this paper, our goal is to study the performance of various online learning algorithms to identify and display the best ads/offers with the highest conversion rates to web users. We formulate our ad-selection problem as a Multi-Armed Bandit problem which is a classical paradigm in Machine Learning. We have been applying machine learning, data mining, probability, and statistics to analyze big data in the ad-tech space and devise efficient ad selection strategies. This article highlights some of our findings in the area of computational advertising from 2011 to 2015.
A decentralized aggregation mechanism for training deep learning models using smart contract system for bank loan prediction
Venue: arXiv cs.LG 2020
Year: 2020

Data privacy and sharing has always been a critical issue when trying to build complex deep learning-based systems to model data. Facilitation of a decentralized approach that could take benefit from data across multiple nodes while not needing to merge their data contents physically has been an area of active research. In this paper, we present a solution to benefit from a distributed data setup in the case of training deep learning architectures by making use of a smart contract system. Specifically, we propose a mechanism that aggregates together the intermediate representations obtained from local ANN models over a blockchain. Training of local models takes place on their respective data. The intermediate representations derived from them, when combined and trained together on the host node, helps to get a more accurate system. While federated learning primarily deals with the same features of data where the number of samples being distributed on multiple nodes, here we are dealing with the same number of samples but with their features being distributed on multiple nodes. We consider the task of bank loan prediction wherein the personal details of an individual and their bank-specific details may not be available at the same place. Our aggregation mechanism helps to train a model on such existing distributed data without having to share and concatenate together the actual data values. The obtained performance, which is better than that of individual nodes, and is at par with that of a centralized data setup makes a strong case for extending our technique across other architectures and tasks. The solution finds its application in organizations that want to train deep learning models on vertically partitioned data.
Fairness-guided SMT-based Rectification of Decision Trees and Random Forests
Venue: arXiv cs.LG 2020
Year: 2020

Data-driven decision making is gaining prominence with the popularity of various machine learning models. Unfortunately, real-life data used in machine learning training may capture human biases, and as a result the learned models may lead to unfair decision making. In this paper, we provide a solution to this problem for decision trees and random forests. Our approach converts any decision tree or random forest into a fair one with respect to a specific data set, fairness criteria, and sensitive attributes. The \emph{FairRepair} tool, built based on our approach, is inspired by automated program repair techniques for traditional programs. It uses an SMT solver to decide which paths in the decision tree could have their outcomes flipped to improve the fairness of the model. Our experiments on the well-known adult dataset from UC Irvine demonstrate that FairRepair scales to realistic decision trees and random forests. Furthermore, FairRepair provides formal guarantees about soundness and completeness of finding a repair. Since our fairness-guided repair technique repairs decision trees and random forests obtained from a given (unfair) data-set, it can help to identify and rectify biases in decision-making in an organisation.
The VLSAT-1 Benchmark Suite
Venue: arXiv cs.DS 2020
Year: 2020

This report presents VLSAT-1 (an acronym for "Very Large Boolean SATisfiability problems"), the first part of a benchmark suite to be used in scientificexperiments and software competitions addressing SAT-solving issues.VLSAT-1 contains 100~benchmarks of increasing complexity, proposed in DIMACSCNF format under a permissive Creative Commons license. These benchmarks havebeen used by the 2020 International Competition on Model Counting.
Multiresolution Knowledge Distillation for Anomaly Detection
Venue: arXiv cs.CV 2020
Year: 2020

Unsupervised representation learning has proved to be a critical component of anomaly detection/localization in images. The challenges to learn such a representation are two-fold. Firstly, the sample size is not often large enough to learn a rich generalizable representation through conventional techniques. Secondly, while only normal samples are available at training, the learned features should be discriminative of normal and anomalous samples. Here, we propose to use the "distillation" of features at various layers of an expert network, pre-trained on ImageNet, into a simpler cloner network to tackle both issues. We detect and localize anomalies using the discrepancy between the expert and cloner networks' intermediate activation values given the input data. We show that considering multiple intermediate hints in distillation leads to better exploiting the expert's knowledge and more distinctive discrepancy compared to solely utilizing the last layer activation values. Notably, previous methods either fail in precise anomaly localization or need expensive region-based training. In contrast, with no need for any special or intensive training procedure, we incorporate interpretability algorithms in our novel framework for the localization of anomalous regions. Despite the striking contrast between some test datasets and ImageNet, we achieve competitive or significantly superior results compared to the SOTA methods on MNIST, F-MNIST, CIFAR-10, MVTecAD, Retinal-OCT, and two Medical datasets on both anomaly detection and localization.
Peer-to-Peer Energy Systems for Connected Communities: A Review of Recent Advances and Emerging Challenges
Venue: arXiv cs.DC 2020
Year: 2020

After a century of relative stability of the electricity industry, extensive deployment of distributed energy resources and recent advances in computation and communication technologies have changed the nature of how we consume, trade, and apply energy. The power system is facing a transition from its traditional hierarchical structure to a more deregulated model by introducing new energy distribution models such as peer-to-peer sharing for connected communities. The proven effectiveness of P2P sharing in benefiting both prosumers and the grid has been demonstrated in many studies and pilot projects. However, there is still no extensive implementation of such sharing models in today's electricity markets. This paper aims to shed some light on this gap through a comprehensive overview of recent advances in the P2P energy system and an insightful discussion of the challenges that need to be addressed in order to establish P2P sharing as a viable energy management option in today's electricity market. To this end, in this article, we provide some background on different aspects of P2P sharing. Then, we discuss advances in P2P sharing through a systematic domain-based classification. We also review different pilot projects on P2P sharing across the globe. Finally, we identify and discuss a number of challenges that need to be addressed for scaling up P2P sharing in the electricity market followed by concluding remarks at the end of the paper.
Socially Aware Crowd Navigation with Multimodal Pedestrian Trajectory Prediction for Autonomous Vehicles
Venue: arXiv cs.RO 2020
Year: 2020

Seamlessly operating an autonomous vehicle in a crowded pedestrian environment is a very challenging task. This is because human movement and interactions are very hard to predict in such environments. Recent work has demonstrated that reinforcement learning-based methods have the ability to learn to drive in crowds. However, these methods can have very poor performance due to inaccurate predictions of the pedestrians' future state as human motion prediction has a large variance. To overcome this problem, we propose a new method, SARL-SGAN-KCE, that combines a deep socially aware attentive value network with a human multimodal trajectory prediction model to help identify the optimal driving policy. We also introduce a novel technique to extend the discrete action space with minimal additional computational requirements. The kinematic constraints of the vehicle are also considered to ensure smooth and safe trajectories. We evaluate our method against the state of art methods for crowd navigation and provide an ablation study to show that our method is safer and closer to human behaviour.
Balance Regularized Neural Network Models for Causal Effect Estimation
Venue: arXiv cs.LG 2020
Year: 2020

Estimating individual and average treatment effects from observational data is an important problem in many domains such as healthcare and e-commerce. In this paper, we advocate balance regularization of multi-head neural network architectures. Our work is motivated by representation learning techniques to reduce differences between treated and untreated distributions that potentially arise due to confounding factors. We further regularize the model by encouraging it to predict control outcomes for individuals in the treatment group that are similar to control outcomes in the control group. We empirically study the bias-variance trade-off between different weightings of the regularizers, as well as between inductive and transductive inference.
A Geometrically Exact Continuum Framework for Light-Matter Interaction in Photo-Active Polymers I. Variational Setting
Venue: arXiv cs.CE 2020
Year: 2020

Molecular photo-switches as, e.g., azobenzene molecules allow, when embedded into a polymeric matrix, for photo-active polymer compounds responding mechanically when exposed to light of certain wavelength. Photo-mechanics, i.e. light-matter interaction in photo-active polymers holds great promise for, e.g., remote and contact-free activation of photo-driven actuators. In a series of earlier contributions, Oates et al. developed a successful continuum formulation for the coupled electric, electronic and mechanical problem capturing azobenzene polymer compounds, thereby mainly focussing on geometrically linearized kinematics. Building on that formulation, we here explore the variational setting of a geometrically exact continuum framework based on Dirichlet's and Hamilton's principle as well as, noteworthy, Hamilton's equations. Thereby, when treating the dissipative case, we resort to incremental versions of the various variational problems via suited incorporation of a dissipation potential. In particular, the Hamiltonian setting of geometrically exact photo-mechanics is up to now largely under-explored even for the energetic case, arguably since the corresponding Lagrangian is degenerate in Dirac's sense. Moreover, in general, the Hamiltonian setting of dissipative dynamical systems is a matter of ongoing debate per se. In this contribution, by advocating a novel incremental version of the Hamiltonian setting exemplified for the dissipative case of photo-mechanics, we aim to also unify the variational approach to dissipative dynamical systems. Taken together, the variational setting of a geometrically exact continuum framework of photo-mechanics paves the way for forthcoming theoretical and numerical analyses.
Distributed algorithms to determine eigenvectors of matrices on spatially distributed networks
Venue: arXiv cs.DC 2020
Year: 2020

Eigenvectors of matrices on a network have been used for understanding spectral clustering and influence of a vertex. For matrices with small geodesic-width, we propose a distributed iterative algorithm in this letter to find eigenvectors associated with their given eigenvalues. We also consider the implementation of the proposed algorithm at the vertex/agent level in a spatially distributed network.