pwc-archive/papers-with-abstracts huggingface dataset:
A Neurochaos Learning Architecture for Genome Classification
Authors: Harikrishnan NB, Pranay SY, Nithin Nagaraj
Date: 2020-10-12 00:00:00
Tasks: Classification, Feature Engineering, General Classification

There has been empirical evidence of presence of non-linearity and chaos at the level of single neurons in biological neural networks. The properties of chaotic neurons inspires us to employ them in artificial learning systems. Here, we propose a Neurochaos Learning (NL) architecture, where the neurons used to extract features from data are 1D chaotic maps. ChaosFEX+SVM, an instance of this NL architecture, is proposed as a hybrid combination of chaos and classical machine learning algorithm. We formally prove that a single layer of NL with a finite number of 1D chaotic neurons satisfies the Universal Approximation Theorem with an exact value for the number of chaotic neurons needed to approximate a discrete real valued function with finite support. This is made possible due to the topological transitivity property of chaos and the existence of uncountably infinite number of dense orbits for the chosen 1D chaotic map. The chaotic neurons in NL get activated under the presence of an input stimulus (data) and output a chaotic firing trajectory. From such chaotic firing trajectories of individual neurons of NL, we extract Firing Time, Firing Rate, Energy and Entropy that constitute ChaosFEX features. These ChaosFEX features are then fed to a Support Vector Machine with linear kernel for classification. The effectiveness of chaotic feature engineering performed by NL (ChaosFEX+SVM) is demonstrated for synthetic and real world datasets in the low and high training sample regimes. Specifically, we consider the problem of classification of genome sequences of SARS-CoV-2 from other coronaviruses (SARS-CoV-1, MERS-CoV and others). With just one training sample per class for 1000 random trials of training, we report an average macro F1-score > 0.99 for the classification of SARS-CoV-2 from SARS-CoV-1 genome sequences. Robustness of ChaosFEX features to additive noise is also demonstrated.
NeuSpell: A Neural Spelling Correction Toolkit
Authors: Sai Muralidhar Jayanthi, Danish Pruthi, Graham Neubig
Date: 2020-10-21 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Spelling Correction

We introduce NeuSpell, an open-source toolkit for spelling correction in English. Our toolkit comprises ten different models, and benchmarks them on naturally occurring misspellings from multiple sources. We find that many systems do not adequately leverage the context around the misspelt token. To remedy this, (i) we train neural models using spelling errors in context, synthetically constructed by reverse engineering isolated misspellings; and (ii) use contextual representations. By training on our synthetic examples, correction rates improve by 9% (absolute) compared to the case when models are trained on randomly sampled character perturbations. Using richer contextual representations boosts the correction rate by another 3%. Our toolkit enables practitioners to use our proposed and existing spelling correction systems, both via a unified command line, as well as a web interface. Among many potential applications, we demonstrate the utility of our spell-checkers in combating adversarial misspellings. The toolkit can be accessed at neuspell.github.io. Code and pretrained models are available at http://github.com/neuspell/neuspell.
Least squares fitting of circles and lines
Authors: N. Chernov, C. Lesort
Date: 2003-01-01 00:00:00

We study theoretical and computational aspects of the least squares fit (LSF) of circles and circular arcs. First we discuss the existence and uniqueness of LSF and various parametrization schemes. Then we evaluate several popular circle fitting algorithms and propose a new one that surpasses the existing methods in reliability. We also discuss and compare direct (algebraic) circle fits.
DiSCO: Differentiable Scan Context with Orientation
Authors: Xuecheng Xu, Huan Yin, Zexi Chen, Yue Wang, Rong Xiong
Date: 2020-10-21 00:00:00
Tasks: Pose Estimation, Robot Navigation

Global localization is essential for robot navigation, of which the first step is to retrieve a query from the map database. This problem is called place recognition. In recent years, LiDAR scan based place recognition has drawn attention as it is robust against the appearance change. In this paper, we propose a LiDAR-based place recognition method, named Differentiable Scan Context with Orientation (DiSCO), which simultaneously finds the scan at a similar place and estimates their relative orientation. The orientation can further be used as the initial value for the down-stream local optimal metric pose estimation, improving the pose estimation especially when a large orientation between the current scan and retrieved scan exists. Our key idea is to transform the feature into the frequency domain. We utilize the magnitude of the spectrum as the place signature, which is theoretically rotation-invariant. In addition, based on the differentiable phase correlation, we can efficiently estimate the global optimal relative orientation using the spectrum. With such structural constraints, the network can be learned in an end-to-end manner, and the backbone is fully shared by the two tasks, achieving interpretability and light weight. Finally, DiSCO is validated on three datasets with long-term outdoor conditions, showing better performance than the compared methods.
Distributed Methods for High-dimensional and Large-scale Tensor Factorization
Authors: Kijung Shin, U. Kang
Date: 2014-10-20 00:00:00
Tasks: Recommendation Systems, Vocal Bursts Intensity Prediction

Given a high-dimensional large-scale tensor, how can we decompose it into latent factors? Can we process it on commodity computers with limited memory? These questions are closely related to recommender systems, which have modeled rating data not as a matrix but as a tensor to utilize contextual information such as time and location. This increase in the dimension requires tensor factorization methods scalable with both the dimension and size of a tensor. In this paper, we propose two distributed tensor factorization methods, SALS and CDTF. Both methods are scalable with all aspects of data, and they show an interesting trade-off between convergence speed and memory requirements. SALS updates a subset of the columns of a factor matrix at a time, and CDTF, a special case of SALS, updates one column at a time. In our experiments, only our methods factorize a 5-dimensional tensor with 1 billion observable entries, 10M mode length, and 1K rank, while all other state-of-the-art methods fail. Moreover, our methods require several orders of magnitude less memory than our competitors. We implement our methods on MapReduce with two widely-applicable optimization techniques: local disk caching and greedy row assignment. They speed up our methods up to 98.2X and also the competitors up to 5.9X.
A Concept of a Compact and Inexpensive Device for Controlling Weeds with Laser Beams
Authors: Ildar Rakhmatulin, Christian Andreasen
Date: 2020-10-21 00:00:00

A prototype of a relatively cheap laser-based weeding device was developed and tested on couch grass (Elytrigia repens (L.) Desv. ex Nevski) mixed with tomatoes. Three types of laser were used (0.3 W, 1 W, and 5 W). A neural network was trained to identify the weed plants, and a laser guidance system estimated the coordinates of the weed. An algorithm was developed to estimate the energy necessary to harm the weed plants. We also developed a decision model for the weed control device. The energy required to damage a plant depended on the diameter of the plant which was related to plant length. The 1 W laser was not sufficient to eliminate all weed plants and required too long exposure time. The 5 W laser was more efficient but also harmed the crop if the laser beam became split into two during the weeding process. There were several challenges with the device, which needs to be improved upon. In particular, the time of exposure needs to be reduced significantly. Still, the research showed that it is possible to develop a concept for laser weeding using relatively cheap equipment, which can work in complicated situations where weeds and crop are mixed.
Bayesian Attention Modules
Authors: Xinjie Fan, Shujian Zhang, Bo Chen, Mingyuan Zhou
Date: 2020-10-20 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Image Captioning, Machine Translation, Node Classification, Question Answering, Translation, Visual Question Answering, Visual Question Answering (VQA)

Attention modules, as simple and effective tools, have not only enabled deep neural networks to achieve state-of-the-art results in many domains, but also enhanced their interpretability. Most current models use deterministic attention modules due to their simplicity and ease of optimization. Stochastic counterparts, on the other hand, are less popular despite their potential benefits. The main reason is that stochastic attention often introduces optimization issues or requires significant model changes. In this paper, we propose a scalable stochastic version of attention that is easy to implement and optimize. We construct simplex-constrained attention distributions by normalizing reparameterizable distributions, making the training process differentiable. We learn their parameters in a Bayesian framework where a data-dependent prior is introduced for regularization. We apply the proposed stochastic attention modules to various attention-based models, with applications to graph node classification, visual question answering, image captioning, machine translation, and language understanding. Our experiments show the proposed method brings consistent improvements over the corresponding baselines.
Semantics-Guided Representation Learning with Applications to Visual Synthesis
Authors: Jia-Wei Yan, Ci-Siang Lin, Fu-En Yang, Yu-Jhe Li, Yu-Chiang Frank Wang
Date: 2020-10-21 00:00:00
Tasks: Representation Learning, Triplet

Learning interpretable and interpolatable latent representations has been an emerging research direction, allowing researchers to understand and utilize the derived latent space for further applications such as visual synthesis or recognition. While most existing approaches derive an interpolatable latent space and induces smooth transition in image appearance, it is still not clear how to observe desirable representations which would contain semantic information of interest. In this paper, we aim to learn meaningful representations and simultaneously perform semantic-oriented and visually-smooth interpolation. To this end, we propose an angular triplet-neighbor loss (ATNL) that enables learning a latent representation whose distribution matches the semantic information of interest. With the latent space guided by ATNL, we further utilize spherical semantic interpolation for generating semantic warping of images, allowing synthesis of desirable visual data. Experiments on MNIST and CMU Multi-PIE datasets qualitatively and quantitatively verify the effectiveness of our method.
A routine to compute the energy and wave function for one-electron two-nuclei molecular systems
Authors: Fabio Sattin
Date: 1997-10-01 00:00:00

The routine presented computes the molecular energy and wave function of one-electron two-hydrogen like nuclei systems 
within the adiabatic approximation. For particular forms of the electron-nucleus potential, also one-active electron systems 
may be considered. The method used is the one developed by Killingbeck associated with Miller's algorithm.
A Machine-learning based Probabilistic Perspective on Dynamic Security Assessment
Authors: Jochen L. Cremer, Goran Strbac
Date: 2019-12-16 00:00:00
Tasks: BIG-bench Machine Learning

Probabilistic security assessment and real-time dynamic security assessments (DSA) are promising to better handle the risks of system operations. The current methodologies of security assessments may require many time-domain simulations for some stability phenomena that are unpractical in real-time. Supervised machine learning is promising to predict DSA as their predictions are immediately available. Classifiers are offline trained on operating conditions and then used in real-time to identify operating conditions that are insecure. However, the predictions of classifiers can be sometimes wrong and hazardous if an alarm is missed for instance. A probabilistic output of the classifier is explored in more detail and proposed for probabilistic security assessment. An ensemble classifier is trained and calibrated offline by using Platt scaling to provide accurate probability estimates of the output. Imbalances in the training database and a cost-skewness addressing strategy are proposed for considering that missed alarms are significantly worse than false alarms. Subsequently, risk-minimised predictions can be made in real-time operation by applying cost-sensitive learning. Through case studies on a real data-set of the French transmission grid and on the IEEE 6 bus system using static security metrics, it is showcased how the proposed approach reduces inaccurate predictions and risks. The sensitivity on the likelihood of contingency is studied as well as on expected outage costs. Finally, the scalability to several contingencies and operating conditions are showcased.
Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies
Authors: Itai Gat, Idan Schwartz, Alexander Schwing, Tamir Hazan
Date: 2020-10-21 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Many recent datasets contain a variety of different data modalities, for instance, image, question, and answer data in visual question answering (VQA). When training deep net classifiers on those multi-modal datasets, the modalities get exploited at different scales, i.e., some modalities can more easily contribute to the classification results than others. This is suboptimal because the classifier is inherently biased towards a subset of the modalities. To alleviate this shortcoming, we propose a novel regularization term based on the functional entropy. Intuitively, this term encourages to balance the contribution of each modality to the classification result. However, regularization with the functional entropy is challenging. To address this, we develop a method based on the log-Sobolev inequality, which bounds the functional entropy with the functional-Fisher-information. Intuitively, this maximizes the amount of information that the modalities contribute. On the two challenging multi-modal datasets VQA-CPv2 and SocialIQ, we obtain state-of-the-art results while more uniformly exploiting the modalities. In addition, we demonstrate the efficacy of our method on Colored MNIST.
Deep learning based registration using spatial gradients and noisy segmentation labels
Authors: Théo Estienne, Maria Vakalopoulou, Enzo Battistella, Alexandre Carré, Théophraste Henry, Marvin Lerousseau, Charlotte Robert, Nikos Paragios, Eric Deutsch
Date: 2020-10-21 00:00:00
Tasks: Image Registration, Medical Image Analysis

Image registration is one of the most challenging problems in medical image analysis. In the recent years, deep learning based approaches became quite popular, providing fast and performing registration strategies. In this short paper, we summarise our work presented on Learn2Reg challenge 2020. The main contributions of our work rely on (i) a symmetric formulation, predicting the transformations from source to target and from target to source simultaneously, enforcing the trained representations to be similar and (ii) integration of variety of publicly available datasets used both for pretraining and for augmenting segmentation labels. Our method reports a mean dice of $0.64$ for task 3 and $0.85$ for task 4 on the test sets, taking third place on the challenge. Our code and models are publicly available at https://github.com/TheoEst/abdominal_registration and \https://github.com/TheoEst/hippocampus_registration.
Event-based Signal Processing for Radioisotope Identification
Authors: Xiaoyu Huang, Edward Jones, Siru Zhang, Steve Furber, Yannis Goulermas, Edward Marsden, Ian Baistow, Srinjoy Mitra, Alister Hamilton
Date: 2020-07-11 00:00:00

This paper identifies the problem of unnecessary high power overhead of the conventional frame-based radioisotope identification process and proposes an event-based signal processing process to address the problem established. It also presents the design flow of the neuromorphic processor.
Deep Learning-Based Concurrent Brain Registration and Tumor Segmentation
Authors: Théo Estienne, Marvin Lerousseau, Maria Vakalopoulou, Emilie Alvarez Andres, Enzo Battistella, Alexandre Carré, Siddhartha Chandra, Stergios Christodoulidis, Mihir Sahasrabudhe, Roger Sun, Charlotte Robert, Hugues Talbot, Nikos Paragios, Eric Deutsch
Date: 2020-03-20 00:00:00
Tasks: Brain Tumor Segmentation, Deep Learning, Image Registration, Medical Image Analysis, Segmentation, Tumor Segmentation

Image registration and segmentation are the two most studied problems in medical image analysis. Deep learning algorithms have recently gained a lot of attention due to their success and state-of-the-art results in variety of problems and communities. In this paper, we propose a novel, efficient, and multi-task algorithm that addresses the problems of image registration and brain tumor segmentation jointly. Our method exploits the dependencies between these tasks through a natural coupling of their interdependencies during inference. In particular, the similarity constraints are relaxed within the tumor regions using an efficient and relatively simple formulation. We evaluated the performance of our formulation both quantitatively and qualitatively for registration and segmentation problems on two publicly available datasets (BraTS 2018 and OASIS 3), reporting competitive results with other recent state-of-the-art methods. Moreover, our proposed framework reports significant amelioration (p < 0.005) for the registration performance inside the tumor locations, providing a generic method that does not need any predefined conditions (e.g., absence of abnormalities) about the volumes to be registered. Our implementation is publicly available online at https://github.com/TheoEst/joint_registration_tumor_segmentation.
U-ReSNet: Ultimate coupling of Registration and Segmentation with deep Nets
Authors: Théo Estienne, Maria Vakalopoulou, Stergios Christodoulidis, Enzo Battistella, Marvin Lerousseau, Alexandre Carre, Guillaume Klausner, Roger Sun, Charlotte Robert, Stavroula Mougiakakou, Nikos Paragios, Eric Deutsch
Date: 2019-10-10 00:00:00
Tasks: Decoder

In this study, we propose a 3D deep neural network called U-ReSNet, a joint framework that can accurately register and segment medical volumes. The proposed network learns to automatically generate linear and elastic deformation models, trained by minimizing the mean square error and the local cross-correlation similarity metrics. In parallel, a coupled architecture is integrated, seeking to provide segmentation maps for anatomies or tissue patterns using an additional decoder part trained with the dice coefficient metric. U-ReSNet is trained in an end to end fashion, while due to this joint optimization the generated network features are more informative leading to promising results compared to other deep learning-based methods existing in the literature. We evaluated the proposed architecture using the publicly available OASIS 3 dataset, measuring the dice coefficient metric for both registration and segmentation tasks. Our promising results indicate the potentials of our method which is composed from a convolutional architecture that is extremely simple and light in terms of parameters. Our code is publicly available https://github.com/TheoEst/coupling_registration_segmentation.
Attention-based scaling adaptation for target speech extraction
Authors: Jiangyu Han, Wei Rao, Yanhua Long, Jiaen Liang
Date: 2020-10-19 00:00:00
Tasks: Speech Extraction

The target speech extraction has attracted widespread attention in recent years. In this work, we focus on investigating the dynamic interaction between different mixtures and the target speaker to exploit the discriminative target speaker clues. We propose a special attention mechanism without introducing any additional parameters in a scaling adaptation layer to better adapt the network towards extracting the target speech. Furthermore, by introducing a mixture embedding matrix pooling method, our proposed attention-based scaling adaptation (ASA) can exploit the target speaker clues in a more efficient way. Experimental results on the spatialized reverberant WSJ0 2-mix dataset demonstrate that the proposed method can improve the performance of the target speech extraction effectively. Furthermore, we find that under the same network configurations, the ASA in a single-channel condition can achieve competitive performance gains as that achieved from two-channel mixtures with inter-microphone phase difference (IPD) features.
Riemannian Langevin Algorithm for Solving Semidefinite Programs
Authors: Mufan Bill Li, Murat A. Erdogdu
Date: 2020-10-21 00:00:00

We propose a Langevin diffusion-based algorithm for non-convex optimization and sampling on a product manifold of spheres. Under a logarithmic Sobolev inequality, we establish a guarantee for finite iteration convergence to the Gibbs distribution in terms of Kullback--Leibler divergence. We show that with an appropriate temperature choice, the suboptimality gap to the global minimum is guaranteed to be arbitrarily small with high probability. As an application, we consider the Burer--Monteiro approach for solving a semidefinite program (SDP) with diagonal constraints, and analyze the proposed Langevin algorithm for optimizing the non-convex objective. In particular, we establish a logarithmic Sobolev inequality for the Burer--Monteiro problem when there are no spurious local minima, but under the presence saddle points. Combining the results, we then provide a global optimality guarantee for the SDP and the Max-Cut problem. More precisely, we show that the Langevin algorithm achieves $\epsilon$ accuracy with high probability in $\widetilde{\Omega}( \epsilon^{-5} )$ iterations.
How Data Augmentation affects Optimization for Linear Regression
Authors: Boris Hanin, Yi Sun
Date: 2020-10-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Data Augmentation, regression, Stochastic Optimization

Though data augmentation has rapidly emerged as a key tool for optimization in modern machine learning, a clear picture of how augmentation schedules affect optimization and interact with optimization hyperparameters such as learning rate is nascent. In the spirit of classical convex optimization and recent work on implicit bias, the present work analyzes the effect of augmentation on optimization in the simple convex setting of linear regression with MSE loss. We find joint schedules for learning rate and data augmentation scheme under which augmented gradient descent provably converges and characterize the resulting minimum. Our results apply to arbitrary augmentation schemes, revealing complex interactions between learning rates and augmentations even in the convex setting. Our approach interprets augmented (S)GD as a stochastic optimization method for a time-varying sequence of proxy losses. This gives a unified way to analyze learning rate, batch size, and augmentations ranging from additive noise to random projections. From this perspective, our results, which also give rates of convergence, can be viewed as Monro-Robbins type conditions for augmented (S)GD.