OpenRaiser/Intern-Atlas huggingface dataset:
DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning
Venue: arXiv cs.AI 2020
Year: 2020

Expert problem-solving is driven by powerful languages for thinking about problems and their solutions. Acquiring expertise means learning these languages -- systems of concepts, alongside the skills to use them. We present DreamCoder, a system that learns to solve problems by writing programs. It builds expertise by creating programming languages for expressing domain concepts, together with neural networks to guide the search for programs within these languages. A ``wake-sleep'' learning algorithm alternately extends the language with new symbolic abstractions and trains the neural network on imagined and replayed problems. DreamCoder solves both classic inductive programming tasks and creative tasks such as drawing pictures and building scenes. It rediscovers the basics of modern functional programming, vector algebra and classical physics, including Newton's and Coulomb's laws. Concepts are built compositionally from those learned earlier, yielding multi-layered symbolic representations that are interpretable and transferrable to new tasks, while still growing scalably and flexibly with experience.
Generating Reliable and Efficient Predictions of Human Motion: A Promising Encounter between Physics and Neural Networks
Venue: arXiv cs.RO 2020
Year: 2020

Generating accurate and efficient predictions for the motion of the humans present in the scene is key to the development of effective motion planning algorithms for robots moving in promiscuous areas, where wrong planning decisions could generate safety hazard or simply make the presence of the robot "socially" unacceptable. Our approach to predict human motion is based on a neural network of a peculiar kind. Contrary to conventional deep neural networks, our network embeds in its structure the popular Social Force Model, a dynamic equation describing the motion in physical terms. This choice allows us to concentrate the learning phase in the aspects, which are really unknown (i.e., the model's parameters) and to keep the structure of the network simple and manageable. As a result, we are able to obtain a good prediction accuracy with a small synthetically generated training set, and the accuracy remains acceptable even when the network is applied in scenarios quite different from those for which it was trained. Finally, the choices of the network are "explainable", as they can be interpreted in physical terms. Comparative and experimental results prove the effectiveness of the proposed approach.
Flexible Dataset Distillation: Learn Labels Instead of Images
Venue: arXiv cs.LG 2020
Year: 2020

We study the problem of dataset distillation - creating a small set of synthetic examples capable of training a good model. In particular, we study the problem of label distillation - creating synthetic labels for a small set of real images, and show it to be more effective than the prior image-based approach to dataset distillation. Methodologically, we introduce a more robust and flexible meta-learning algorithm for distillation, as well as an effective first-order strategy based on convex optimization layers. Distilling labels with our new algorithm leads to improved results over prior image-based distillation. More importantly, it leads to clear improvements in flexibility of the distilled dataset in terms of compatibility with off-the-shelf optimizers and diverse neural architectures. Interestingly, label distillation can also be applied across datasets, for example enabling learning Japanese character recognition by training only on synthetically labeled English letters.
Vulnerability Coverage for Secure Configuration
Venue: arXiv cs.CR 2020
Year: 2020

We present a novel idea on adequacy testing called ``{vulnerability coverage}.'' The introduced coverage measure examines the underlying software for the presence of certain classes of vulnerabilities often found in the National Vulnerability Database (NVD) website. The thoroughness of the test input generation procedure is performed through the adaptation of evolutionary algorithms namely Genetic Algorithms (GA) and Particle Swarm Optimization (PSO). The methodology utilizes the Common Vulnerability Scoring System (CVSS), a free and open industry standard for assessing the severity of computer system security vulnerabilities, as a fitness measure for test inputs generation. The outcomes of these evolutionary algorithms are then evaluated in order to identify the vulnerabilities that match a class of vulnerability patterns for testing purposes.
Shape Matters: Understanding the Implicit Bias of the Noise Covariance
Venue: arXiv cs.LG 2020
Year: 2020

The noise in stochastic gradient descent (SGD) provides a crucial implicit regularization effect for training overparameterized models. Prior theoretical work largely focuses on spherical Gaussian noise, whereas empirical studies demonstrate the phenomenon that parameter-dependent noise -- induced by mini-batches or label perturbation -- is far more effective than Gaussian noise. This paper theoretically characterizes this phenomenon on a quadratically-parameterized model introduced by Vaskevicius et el. and Woodworth et el. We show that in an over-parameterized setting, SGD with label noise recovers the sparse ground-truth with an arbitrary initialization, whereas SGD with Gaussian noise or gradient descent overfits to dense solutions with large norms. Our analysis reveals that parameter-dependent noise introduces a bias towards local minima with smaller noise variance, whereas spherical Gaussian noise does not. Code for our project is publicly available.
Geometrically nonlinear modelling of pre-stressed viscoelastic fibre-reinforced composites with application to arteries
Venue: arXiv cs.CE 2020
Year: 2020

Modelling of mechanical behaviour of pre-stressed fibre-reinforced composites is considered in a geometrically exact setting. A general approach which includes two different reference configurations is employed: one configuration corresponds to the load-free state of the structure and another one to the stress-free state of each material particle. The applicability of the approach is demonstrated in terms of a viscoelastic material model; both the matrix and the fibre are modelled using a multiplicative split of the deformation gradient tensor; a transformation rule for initial conditions is elaborated and specified. Apart from its simplicity, an important advantage of the approach is that well-established numerical algorithms can be used for pre-stressed inelastic structures. The interrelation between the advocated approach and the widely used "opening angle" approach is clarified. A full-scale FEM simulation confirms the main predictions of the "opening angle" approach. A locking effect is discovered; the effect is that in some cases the opening angle of the composite is essentially smaller than the opening angles of its individual layers. Thus, the standard cutting test typically used to analyse pre-stresses does not carry enough information and more refined experimental techniques are needed.
Towards Understanding the Effect of Leak in Spiking Neural Networks
Venue: arXiv cs.NE 2020
Year: 2020

Spiking Neural Networks (SNNs) are being explored to emulate the astounding capabilities of human brain that can learn and compute functions robustly and efficiently with noisy spiking activities. A variety of spiking neuron models have been proposed to resemble biological neuronal functionalities. With varying levels of bio-fidelity, these models often contain a leak path in their internal states, called membrane potentials. While the leaky models have been argued as more bioplausible, a comparative analysis between models with and without leak from a purely computational point of view demands attention. In this paper, we investigate the questions regarding the justification of leak and the pros and cons of using leaky behavior. Our experimental results reveal that leaky neuron model provides improved robustness and better generalization compared to models with no leak. However, leak decreases the sparsity of computation contrary to the common notion. Through a frequency domain analysis, we demonstrate the effect of leak in eliminating the high-frequency components from the input, thus enabling SNNs to be more robust against noisy spike-inputs.
EMU: Efficient Muscle Simulation In Deformation Space
Venue: arXiv cs.GR 2020
Year: 2020

EMU is an efficient and scalable model to simulate bulk musculoskeletal motion with heterogenous materials. First, EMU requires no model reductions, or geometric coarsening, thereby producing results visually accurate when compared to an FEM simulation. Second, EMU is efficient and scales much better than state-of-the-art FEM with the number of elements in the mesh, and is more easily parallelizable. Third, EMU can handle heterogeneously stiff meshes with an arbitrary constitutive model, thus allowing it to simulate soft muscles, stiff tendons and even stiffer bones all within one unified system. These three key characteristics of EMU enable us to efficiently orchestrate muscle activated skeletal movements. We demonstrate the efficacy of our approach via a number of examples with tendons, muscles, bones and joints.
PAC-Bayesian Generalization Bounds for MultiLayer Perceptrons
Venue: arXiv cs.LG 2020
Year: 2020

We study PAC-Bayesian generalization bounds for Multilayer Perceptrons (MLPs) with the cross entropy loss. Above all, we introduce probabilistic explanations for MLPs in two aspects: (i) MLPs formulate a family of Gibbs distributions, and (ii) minimizing the cross-entropy loss for MLPs is equivalent to Bayesian variational inference, which establish a solid probabilistic foundation for studying PAC-Bayesian bounds on MLPs. Furthermore, based on the Evidence Lower Bound (ELBO), we prove that MLPs with the cross entropy loss inherently guarantee PAC- Bayesian generalization bounds, and minimizing PAC-Bayesian generalization bounds for MLPs is equivalent to maximizing the ELBO. Finally, we validate the proposed PAC-Bayesian generalization bound on benchmark datasets.
META-Learning Eligibility Traces for More Sample Efficient Temporal Difference Learning
Venue: arXiv cs.LG 2020
Year: 2020

Temporal-Difference (TD) learning is a standard and very successful reinforcement learning approach, at the core of both algorithms that learn the value of a given policy, as well as algorithms which learn how to improve policies. TD-learning with eligibility traces provides a way to do temporal credit assignment, i.e. decide which portion of a reward should be assigned to predecessor states that occurred at different previous times, controlled by a parameter $λ$. However, tuning this parameter can be time-consuming, and not tuning it can lead to inefficient learning. To improve the sample efficiency of TD-learning, we propose a meta-learning method for adjusting the eligibility trace parameter, in a state-dependent manner. The adaptation is achieved with the help of auxiliary learners that learn distributional information about the update targets online, incurring roughly the same computational complexity per step as the usual value learner. Our approach can be used both in on-policy and off-policy learning. We prove that, under some assumptions, the proposed method improves the overall quality of the update targets, by minimizing the overall target error. This method can be viewed as a plugin which can also be used to assist prediction with function approximation by meta-learning feature (observation)-based $λ$ online, or even in the control case to assist policy improvement. Our empirical evaluation demonstrates significant performance improvements, as well as improved robustness of the proposed algorithm to learning rate variation.
Manipulating emotions for ground truth emotion analysis
Venue: arXiv cs.CL 2020
Year: 2020

Text data are being used as a lens through which human cognition can be studied at a large scale. Methods like emotion analysis are now in the standard toolkit of computational social scientists but typically rely on third-person annotation with unknown validity. As an alternative, this paper introduces online emotion induction techniques from experimental behavioural research as a method for text-based emotion analysis. Text data were collected from participants who were randomly allocated to a happy, neutral or sad condition. The findings support the mood induction procedure. We then examined how well lexicon approaches can retrieve the induced emotion. All approaches resulted in statistical differences between the true emotion conditions. Overall, only up to one-third of the variance in emotion was captured by text-based measurements. Pretrained classifiers performed poorly on detecting true emotions. The paper concludes with limitations and suggestions for future research.
ZnG: Architecting GPU Multi-Processors with New Flash for Scalable Data Analysis
Venue: arXiv cs.AR 2020
Year: 2020

We propose ZnG, a new GPU-SSD integrated architecture, which can maximize the memory capacity in a GPU and address performance penalties imposed by an SSD. Specifically, ZnG replaces all GPU internal DRAMs with an ultra-low-latency SSD to maximize the GPU memory capacity. ZnG further removes performance bottleneck of the SSD by replacing its flash channels with a high-throughput flash network and integrating SSD firmware in the GPU's MMU to reap the benefits of hardware accelerations. Although flash arrays within the SSD can deliver high accumulated bandwidth, only a small fraction of such bandwidth can be utilized by GPU's memory requests due to mismatches of their access granularity. To address this, ZnG employs a large L2 cache and flash registers to buffer the memory requests. Our evaluation results indicate that ZnG can achieve 7.5x higher performance than prior work.
Spatio-Temporal Analysis of Cellular Networks with Cell-Center/Edge Users
Venue: arXiv cs.IT 2020
Year: 2020

Emergence of various types of services has brought about explosive growth of traffic as well as diversified traffic characteristics in the cellular networks. To have a comprehensive understanding of the influences caused by various traffic status is vital for the deployment of the next-generation wireless networks. In this paper, we develop a mathematical analytical model by utilizing queuing theory and stochastic geometry where the randomness of the traffic and the geographical locations of the interferers can be captured. We derive the b-th moments of the conditional success probability and the closed-form expressions of the meta distribution for the cell-center users (CCUs) and the cell-edge users (CEUs), respectively. Fixed-point equations are then formulated to obtain the exact value of the meta distribution by taking the random arrival traffic into consideration and the impact of the random arrival traffic on the queue status is revealed. In addition, the mean local delays for CCUs nad CEUs are derived and the corresponding regions for CCUs and CEUs where the mean local delays maintain finite are obtained. Finally, the impact of the critical network parameters on the meta distribution and the mean local delay is investigated with the numerical results.
Federated Survival Analysis with Discrete-Time Cox Models
Venue: arXiv cs.LG 2020
Year: 2020

Building machine learning models from decentralized datasets located in different centers with federated learning (FL) is a promising approach to circumvent local data scarcity while preserving privacy. However, the prominent Cox proportional hazards (PH) model, used for survival analysis, does not fit the FL framework, as its loss function is non-separable with respect to the samples. The naïve method to bypass this non-separability consists in calculating the losses per center, and minimizing their sum as an approximation of the true loss. We show that the resulting model may suffer from important performance loss in some adverse settings. Instead, we leverage the discrete-time extension of the Cox PH model to formulate survival analysis as a classification problem with a separable loss function. Using this approach, we train survival models using standard FL techniques on synthetic data, as well as real-world datasets from The Cancer Genome Atlas (TCGA), showing similar performance to a Cox PH model trained on aggregated data. Compared to previous works, the proposed method is more communication-efficient, more generic, and more amenable to using privacy-preserving techniques.
Morphological stability of three-dimensional cementite rods in polycrystalline system: A phase-field analysis
Venue: arXiv cs.CE 2020
Year: 2020

Transformations accompanying shape-instability govern the morphological configuration and distribution of the phases in a microstructure. Owing to the influence of the microstructure on the properties of a material, the stability of three-dimensional rods in a representative polycrystalline system is extensively analysed. A multiphase-field model, which recovers the physical laws and sharp-interface relations, and includes grain boundary diffusion, is adopted to investigate the morphological evolution of the precipitate. Moreover, the efficiency of the numerical approach is ensured by establishing the volume-preserving chemical equilibrium through the incorporation TCFe8 (CALPHAD) data and solving phase-field evolution in the Allen-Cahn framework. The morphological evolution of the rod in the multiphase system exhibits a unique transformation mechanism which is significantly different from the evolution of an isolated finite-structure. It is realised that, in a polycrystalline arrangement, irrespective of the initial rod-size , the shape-change begins with the energy-minimising events at the triple junctions. This early transformation renders a characteristic morphology at the longitudinal ends of the structure, which introduces sufficient driving-force through the curvature-difference for the subsequent morphological changes. The continued mass transfer to the terminations, ultimately, breaks-off the rod into separate entities that are entangled in the grain boundary. With increasing aspect ratio of the rod, it is identified that the source of mass transfer, which turns into the ovulation site, shifts from the centre. This increases the number of fragmentation events and introduces satellite particle. A comprehensive understanding of the transformation kinetics and mechanism governing the morphological evolution of the rods in a polycrystalline system is rendered in this work.
Deep Learning based Segmentation of Fish in Noisy Forward Looking MBES Images
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we investigate a Deep Learning (DL) approach to fish segmentation in a small dataset of noisy low-resolution images generated by a forward-looking multibeam echosounder (MBES). We build on recent advances in DL and Convolutional Neural Networks (CNNs) for semantic segmentation and demonstrate an end-to-end approach for a fish/non-fish probability prediction for all range-azimuth positions projected by an imaging sonar. We use self-collected datasets from the Danish Sound and the Faroe Islands to train and test our model and present techniques to obtain satisfying performance and generalization even with a low-volume dataset. We show that our model proves the desired performance and has learned to harness the importance of semantic context and take this into account to separate noise and non-targets from real targets. Furthermore, we present techniques to deploy models on low-cost embedded platforms to obtain higher performance fit for edge environments - where compute and power are restricted by size/cost - for testing and prototyping.
Fine-Tuning DARTS for Image Classification
Venue: arXiv cs.CV 2020
Year: 2020

Neural Architecture Search (NAS) has gained attraction due to superior classification performance. Differential Architecture Search (DARTS) is a computationally light method. To limit computational resources DARTS makes numerous approximations. These approximations result in inferior performance. We propose to fine-tune DARTS using fixed operations as they are independent of these approximations. Our method offers a good trade-off between the number of parameters and classification accuracy. Our approach improves the top-1 accuracy on Fashion-MNIST, CompCars, and MIO-TCD datasets by 0.56%, 0.50%, and 0.39%, respectively compared to the state-of-the-art approaches. Our approach performs better than DARTS, improving the accuracy by 0.28%, 1.64%, 0.34%, 4.5%, and 3.27% compared to DARTS, on CIFAR-10, CIFAR-100, Fashion-MNIST, CompCars, and MIO-TCD datasets, respectively.
Improved Deep Point Cloud Geometry Compression
Venue: arXiv cs.CV 2020
Year: 2020

Point clouds have been recognized as a crucial data structure for 3D content and are essential in a number of applications such as virtual and mixed reality, autonomous driving, cultural heritage, etc. In this paper, we propose a set of contributions to improve deep point cloud compression, i.e.: using a scale hyperprior model for entropy coding; employing deeper transforms; a different balancing weight in the focal loss; optimal thresholding for decoding; and sequential model training. In addition, we present an extensive ablation study on the impact of each of these factors, in order to provide a better understanding about why they improve RD performance. An optimal combination of the proposed improvements achieves BD-PSNR gains over G-PCC trisoup and octree of 5.50 (6.48) dB and 6.84 (5.95) dB, respectively, when using the point-to-point (point-to-plane) metric. Code is available at https://github.com/mauriceqch/pcc_geo_cnn_v2 .
Probabilistic Decoupling of Labels in Classification
Venue: arXiv cs.LG 2020
Year: 2020

In this paper we develop a principled, probabilistic, unified approach to non-standard classification tasks, such as semi-supervised, positive-unlabelled, multi-positive-unlabelled and noisy-label learning. We train a classifier on the given labels to predict the label-distribution. We then infer the underlying class-distributions by variationally optimizing a model of label-class transitions.
Response of Vulnerable Road Users to Visual Information from Autonomous Vehicles in Shared Spaces
Venue: arXiv cs.HC 2020
Year: 2020

Completely unmanned autonomous vehicles have been anticipated for a while. Initially, these are expected to drive only under certain conditions on some roads, and advanced functionality is required to cope with the ever-increasing challenges of safety. To enhance the public's perception of road safety and trust in new vehicular technologies, we investigate in this paper the effect of several interaction paradigms with vulnerable road users by developing and applying algorithms for the automatic analysis of pedestrian body language. We assess behavioral patterns and determine the impact of the coexistence of AVs and other road users on general road safety in a shared space for VRUs and vehicles. Results showed that the implementation of visual communication cues for interacting with VRUs is not necessarily required for a shared space in which informal traffic rules apply.
Algorithms with Predictions
Venue: arXiv cs.DS 2020
Year: 2020

We introduce algorithms that use predictions from machine learning applied to the input to circumvent worst-case analysis. We aim for algorithms that have near optimal performance when these predictions are good, but recover the prediction-less worst case behavior when the predictions have large errors.