Reviewerly/exHarmony huggingface dataset:
Sparse Variational Inference: Bayesian Coresets from Scratch

The proliferation of automated inference algorithms in Bayesian statistics has provided practitioners newfound access to fast, reproducible data analysis and powerful statistical models. Designing automated methods that are also both computationally scalable and theoretically sound, however, remains a significant challenge. Recent work on Bayesian coresets takes the approach of compressing the dataset before running a standard inference algorithm, providing both scalability and guarantees on posterior approximation error. But the automation of past coreset methods is limited because they depend on the availability of a reasonable coarse posterior approximation, which is difficult to specify in practice. In the present work we remove this requirement by formulating coreset construction as sparsity-constrained variational inference within an exponential family. This perspective leads to a novel construction via greedy optimization, and also provides a unifying information-geometric view of present and past methods. The proposed Riemannian coreset construction algorithm is fully automated, requiring no problem-specific inputs aside from the probabilistic model and dataset. In addition to being significantly easier to use than past methods, experiments demonstrate that past coreset constructions are fundamentally limited by the fixed coarse posterior approximation; in contrast, the proposed algorithm is able to continually improve the coreset, providing state-of-the-art Bayesian dataset summarization with orders-of-magnitude reduction in KL divergence to the exact posterior.
Regularized Gradient Boosting

Gradient Boosting (\GB) is a popular and very successful ensemble method for binary trees. While various types of regularization of the base predictors are used with this algorithm, the theory that connects such regularizations with generalization guarantees is poorly understood. We fill this gap by deriving data-dependent learning guarantees for \GB used with \emph{regularization}, expressed in terms of the Rademacher complexities of the constrained families of base predictors. We introduce a new algorithm, called \rgb\, that directly benefits from these generalization bounds and that, at every boosting round, applies the \emph{Structural Risk Minimization} principle to search for a base predictor with the best empirical fit versus complexity trade-off. Inspired by \emph{Randomized Coordinate Descent} we provide a scalable implementation of our algorithm, able to search over large families of base predictors. Finally, we provide experimental results, demonstrating that our algorithm achieves significantly better out-of-sample performance on multiple datasets than the standard \GB algorithm used with its regularization.
Adjusting Population Size of Ant Colony System Using Fuzzy Logic Controller

The population size has a very strong impact on the efficiency, solution quality, and computational cost in a Swarm Intelligence (SI). In Ant Colony System algorithm, and as a Swarm Intelligence and population size based algorithm, the number of ants plays a very important role in directing the colony toward a high quality solution within a reasonable time. In this paper, a Fuzzy Logic strategy for adjusting the number of ants during runtime is presented. The based indicators for this adjustment are: Iteration and Convergence Rate. Some experiments are conducted using Travelling Salesman Problems, and results show that modifying the number of ants has a crucial effect on the performance of the Ant Colony System algorithm especially on the quality of solution.
Multi-robot global sonar survey in the presence of strong currents

This paper considers conducting underwater surveys using multiple sonar-equipped Autonomous Surface Vehicles (ASVs). Given a workspace, every ASV simultaneously searches for a mine towing a Side Scan Sonar (SSS) which can provide high-resolution image of underwater environments. Under strong currents, a survey may result in blurry sonar image, which is not desirable for detecting mines. In the case where the heading of the ASV is opposite to the direction of currents, we can obtain vivid sonar images suitable for detecting mines. This paper thus introduces global path planning of multiple ASVs satisfying the following three goals: 1. there is no hole in coverage after the search is done. 2. whenever an ASV performs sonar survey inside the workspace, the heading of the ASV is opposite to the direction of currents. 3. the workload of every ASV is balanced for time-efficient search. We verified the effectiveness of our multi-ASV path planning methods using MATLAB simulations.
On Distributed Averaging for Stochastic k-PCA

In the stochastic k-PCA problem, we are given i.i.d. samples from an unknown distribution over vectors, and the goal is to compute the top k eigenvalues and eigenvectors of the moment matrix. In the simplest variant, we have 'm' machines each of which receives 'n' samples. Each machine performs some computation and sends an O(k) size summary of the local dataset to a central server. The server performs an aggregation and computes the desired eigenvalues and vectors. The goal is to achieve the same effect as the server computing using m*n samples by itself. The main choices in this framework are the choice of the summary, and the method of aggregation. We consider a slight variant of the well-studied distributed averaging approach, and prove that this leads to significantly better bounds on the dependence between 'n' and the eigenvalue gaps. Our method can also be applied directly to a setting where the right value of the parameter k (i.e., one for which there is a non-trivial eigenvalue gap) is not known exactly. This is a common issue in practice which prior methods were unable to address.
Global Convergence of Least Squares EM for Demixing Two Log-Concave Densities

This work studies the location estimation problem for a mixture of two rotation invariant log-concave densities. We demonstrate that Least Squares EM, a variant of the EM algorithm, converges to the true location parameter from a randomly initialized point. Moreover, we establish the explicit convergence rates and sample complexity bounds, revealing their dependence on the signal-to-noise ratio and the tail property of the log-concave distributions. Our analysis generalizes previous techniques for proving the convergence results of Gaussian mixtures, and highlights that an angle-decreasing property is sufficient for establishing global convergence for Least Squares EM.
MEDIA PEMBELAJARAN DALAM PROSES BELAJAR MENGAJAR

With the development of technology and information learning media play an important role in assisting the teaching and learning process, so that an educator can take advantage of learning media that began to develop to be used as a tool in delivering material so that the learning process is easier to do. Learning media is very diverse so that an educator must be clever in choosing the appropriate learning media to be applied in school or in the classroom so that a learning goal can be achieved as expected. The existence of learning media in the midst of the development of the era, the teaching and learning process is easier to implement and very important role for a student in following the teaching and learning process. With learning media that are very helpful for students and teachers in the teaching and learning process, students are not easily bored in following the teaching and learning process, students are easier to understand the material, the learning process becomes more effective and efficient, and learning goals are easily achieved as expected. Thus, the role of learning media is needed by an educator and student so that a teaching and learning process can be made easier to understand the material, and foster students' interest in learning.
The Point Where Reality Meets Fantasy: Mixed Adversarial Generators for Image Splice Detection

Modern photo editing tools allow creating realistic manipulated images easily. While fake images can be quickly generated, learning models for their detection is challenging due to the high variety of tampering artifacts and the lack of large labeled datasets of manipulated images. In this paper, we propose a new framework for training of discriminative segmentation model via an adversarial process. We simultaneously train four models: a generative retouching model G_R that translates manipulated image to the real image domain, a generative annotation model G_A that estimates the pixel-wise probability of image patch being either real or fake, and two discriminators D_R and D_A that qualify the output of G_R and G_A. The aim of model G_R is to maximize the probability of model G_A making a mistake. Our method extends the generative adversarial networks framework with two main contributions: (1) training of a generative model G_R against a deep semantic segmentation network G_A that learns rich scene semantics for manipulated region detection, (2) proposing per class semantic loss that facilitates semantically consistent image retouching by the G_R. We collected large-scale manipulated image dataset to train our model. The dataset includes 16k real and fake images with pixel-level annotations of manipulated areas. The dataset also provides ground truth pixel-level object annotations. We validate our approach on several modern manipulated image datasets, where quantitative results and ablations demonstrate that our method achieves and surpasses the state-of-the-art in manipulated image detection. We made our code and dataset publicly available.
Empirical Analysis of Object-Oriented Metrics and Centrality Measures for Predicting Fault-Prone Classes in Object-Oriented Software

A large number of metrics have been proposed in the literature to measure various structural properties of object-oriented software. Furthermore, many centrality measures have been introduced to identify central nodes in large networks. To the best of our knowledge, only few empirical software engineering studies have explored these metrics in the case of software systems. This paper aims at providing further evidence on the usefulness of centrality measures as indicators of software defect proneness by: (1) investigating the relationships between object-oriented metrics and centrality measures, and (2) exploring how they can be combined to improve the prediction of fault-prone classes in object-oriented software. We collected data from five different versions of one open source Java software system. We used the Principal Component Analysis technique to eliminate metrics (measures) providing redundant information. Then, we built different models to predict fault-prone classes using four machine learning algorithms. Results indicate that: (1) some centrality measures capture information that is not captured by traditional object-oriented metrics, and (2) combining centrality measures with object-oriented metrics improves the performance of fault-prone classes prediction.
Application of Deep Learning Techniques on Document Classification

Automatic assignment of documents into some of the categories is highly required for analysing the content of the documents. Thus, document classification plays a significant role in the field of machine learning, artificial intelligence, information extraction, natural language processing and many more. This problem of assigning a document to a particular category or class has been approached in several ways till date, and with numerous new technological advancements, this class of problem has interesting solutions. Apart from the processes related to text analysis and parsing methodologies, deep learning has offered a way to solve this classification scenario. In the present work, we represent a comparative study on some of the basic building blocks used in deep learning, each of which can be applied to get simpler models trying to assign a class of the available documents. The present comparative study shows how these components can vary the impact on the task. The evaluation of the models has been performed on a standard available dataset. The non-linearity provided by these deep learning models are useful in getting state-of-the-art results for text classification problem.
Partially Encrypted Deep Learning using Functional Encryption

Machine learning on encrypted data has received a lot of attention thanks to recent breakthroughs in homomorphic encryption and secure multi-party computation. It allows outsourcing computation to untrusted servers without sacrificing privacy of sensitive data. We propose a practical framework to perform partially encrypted and privacy-preserving predictions which combines adversarial training and functional encryption. We first present a new functional encryption scheme to efficiently compute quadratic functions so that the data owner controls what can be computed but is not involved in the calculation: it provides a decryption key which allows one to learn a specific function evaluation of some encrypted data. We then show how to use it in machine learning to partially encrypt neural networks with quadratic activation functions at evaluation time and we provide a thorough analysis of the information leaks based on indistinguishability of data items of the same label. Last, since several encryption schemes cannot deal with the last thresholding operation used for classification, we propose a training method to prevent selected sensitive features from leaking which adversarially optimizes the network against an adversary trying to identify these features. This is of great interest for several existing works using partially encrypted machine learning as it comes with almost no cost on the model's accuracy and significantly improves data privacy.
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off

Reducing the precision of weights and activation functions in neural network training, with minimal impact on performance, is essential for the deployment of these models in resource-constrained environments. We apply mean field techniques to networks with quantized activations in order to evaluate the degree to which quantization degrades signal propagation at initialization. We derive initialization schemes which maximize signal propagation in such networks, and suggest why this is helpful for generalization. Building on these results, we obtain a closed form implicit equation for $L_{\max}$, the maximal trainable depth (and hence model capacity), given $N$, the number of quantization levels in the activation function. Solving this equation numerically, we obtain asymptotically: $L_{\max}\propto N^{1.82}$.
Theoretical Limits of Pipeline Parallel Optimization and Application to Distributed Deep Learning

We investigate the theoretical limits of pipeline parallel learning of deep learning architectures, a distributed setup in which the computation is distributed per layer instead of per example. For smooth convex and non-convex objective functions, we provide matching lower and upper complexity bounds and show that a naive pipeline parallelization of Nesterov's accelerated gradient descent is optimal. For non-smooth convex functions, we provide a novel algorithm coined Pipeline Parallel Random Smoothing (PPRS) that is within a $d^{1/4}$ multiplicative factor of the optimal convergence rate, where $d$ is the underlying dimension. While the convergence rate still obeys a slow $\varepsilon^{-2}$ convergence rate, the depth-dependent part is accelerated, resulting in a near-linear speed-up and convergence time that only slightly depends on the depth of the deep learning architecture. Finally, we perform an empirical analysis of the non-smooth non-convex case and show that, for difficult and highly non-smooth problems, PPRS outperforms more traditional optimization algorithms such as gradient descent and Nesterov's accelerated gradient descent for problems where the sample size is limited, such as few-shot or adversarial learning.
Assessing Adaptability of Software Architectures for Cyber Physical Production Systems

Cyber physical production systems (CPPS) focus on increasing the flexibility and adaptability of industrial production systems, systems that comprise hardware such as sensors and actuators in machines as well as software controlling and integrating these machines. The requirements of customised mass production imply that control software needs to be adaptable after deployment in a shop floor, possibly even without interrupting production. Software architecture plays a central role in achieving run-time adaptability. In this paper we describe five architectures, that define the structure and interaction of software components in CPPS. Three of them already are already well known and used in the field. The other two we contribute as possible solution to overcome limitations of the first three architectures. We analyse the architectures’ ability to support adaptability based on Taylor et al.’s BASE framework. We compare the architectures and discuss how the implications of CPPS affect the analysis with BASE. We further highlight what lessons from “traditional” software architecture research can be applied to arrive at adaptable software architectures for cyber physical production systems.
A Learning Approach for Informative-Frame Selection in US Rheumatology Images

Rheumatoid arthritis (RA) is an autoimmune disorder that causes pain, swelling and stiffness in joints. Nowadays, ultrasound (US) has undergone an increasing role in RA screening since it is a powerful tool to assess disease activity. However, obtaining a good quality US frame is a tricky operator dependent procedure. For this reason, the purpose of this paper is to present a strategy to the automatic selection of informative US rheumatology images by means of Convolutional Neural Networks (CNNs). The proposed method is based on VGG16 and Inception V3 CNNs, which are fine tuned to classify 214 balanced metacarpal head US images (75% used for training and 25% used for testing). A repeated 3 fold cross validation for each CNN was performed. The best results were achieved with VGG16 (area under the curve = 90%). These results support the possibility of applying this method in the actual clinical practice for supporting the diagnostic process and helping young residents’ training.
A Page-Based Reject Option for Writer Identification in Medieval Books

One main goal of paleographers is to identify the different writers who wrote a given manuscript. Recently, paleographers are starting to use digital tools which provide new and more objective ways to analyze ancient documents. On the other hand, in the last few years, deep learning techniques have been applied to many domains and to overcome its requirement of a large amount of labeled data, transfer learning has been used. This latter approach uses previously trained large deep networks as starting points to solve specific classification problems. In this paper, we present a novel approach based on deep transfer learning to implement a reject option for the recognition of the writers in medieval documents. The implemented option is page-based and considers the row labels provided by the trained deep network to estimate the class probabilities. The proposed approach has been tested on a set of digital images from a Bible of the XII century. The achieved results confirmed the effectiveness of the proposed approach.
Modeling Uncertainty by Learning a Hierarchy of Deep Neural Connections

Modeling uncertainty in deep neural networks, despite recent important advances, is still an open problem. Bayesian neural networks are a powerful solution, where the prior over network weights is a design choice, often a normal distribution or other distribution encouraging sparsity. However, this prior is agnostic to the generative process of the input data, which might lead to unwarranted generalization for out-of-distribution tested data. We suggest the presence of a confounder for the relation between the input data and the discriminative function given the target label. We propose an approach for modeling this confounder by sharing neural connectivity patterns between the generative and discriminative networks. This approach leads to a new deep architecture, where networks are sampled from the posterior of local causal structures, and coupled into a compact hierarchy. We demonstrate that sampling networks from this hierarchy, proportionally to their posterior, is efficient and enables estimating various types of uncertainties. Empirical evaluations of our method demonstrate significant improvement compared to state-of-the-art calibration and out-of-distribution detection methods.
Deliberative Explanations: visualizing network insecurities

A new approach to explainable AI, denoted {\it deliberative explanations,\/} is proposed. Deliberative explanations are a visualization technique that aims to go beyond the simple visualization of the image regions (or, more generally, input variables) responsible for a network prediction. Instead, they aim to expose the deliberations carried by the network to arrive at that prediction, by uncovering the insecurities of the network about the latter. The explanation consists of a list of insecurities, each composed of 1) an image region (more generally, a set of input variables), and 2) an ambiguity formed by the pair of classes responsible for the network uncertainty about the region. Since insecurity detection requires quantifying the difficulty of network predictions, deliberative explanations combine ideas from the literatures on visual explanations and assessment of classification difficulty. More specifically, the proposed implementation combines attributions with respect to both class predictions and a difficulty score. An evaluation protocol that leverages object recognition (CUB200) and scene classification (ADE20K) datasets that combine part and attribute annotations is also introduced to evaluate the accuracy of deliberative explanations. Finally, an experimental evaluation shows that the most accurate explanations are achieved by combining non self-referential difficulty scores and second-order attributions. The resulting insecurities are shown to correlate with regions of attributes that are shared by different classes. Since these regions are also ambiguous for humans, deliberative explanations are intuitive, suggesting that the deliberative process of modern networks correlates with human reasoning.
Differentially Private Bagging: Improved utility and cheaper privacy than subsample-and-aggregate

Differential Privacy is a popular and well-studied notion of privacy. In the era ofbig data that we are in, privacy concerns are becoming ever more prevalent and thusdifferential privacy is being turned to as one such solution. A popular method forensuring differential privacy of a classifier is known as subsample-and-aggregate,in which the dataset is divided into distinct chunks and a model is learned on eachchunk, after which it is aggregated. This approach allows for easy analysis of themodel on the data and thus differential privacy can be easily applied. In this paper,we extend this approach by dividing the data several times (rather than just once)and learning models on each chunk within each division. The first benefit of thisapproach is the natural improvement of utility by aggregating models trained ona more diverse range of subsets of the data (as demonstrated by the well-knownbagging technique). The second benefit is that, through analysis that we provide inthe paper, we can derive tighter differential privacy guarantees when several queriesare made to this mechanism. In order to derive these guarantees, we introducethe upwards and downwards moments accountants and derive bounds for thesemoments accountants in a data-driven fashion. We demonstrate the improvementsour model makes over standard subsample-and-aggregate in two datasets (HeartFailure (private) and UCI Adult (public)).