OpenRaiser/Intern-Atlas huggingface dataset:
Hybrid Embedded Deep Stacked Sparse Autoencoder with w_LPPD SVM Ensemble
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning is a kind of feature learning method with strong nonliear feature transformation and becomes more and more important in many fields of artificial intelligence. Deep autoencoder is one representative method of the deep learning methods, and can effectively extract abstract the information of datasets. However, it does not consider the complementarity between the deep features and original features during deep feature transformation. Besides, it suffers from small sample problem. In order to solve these problems, a novel deep autoencoder - hybrid feature embedded stacked sparse autoencoder(HESSAE) has been proposed in this paper. HFESAE is capable to learn discriminant deep features with the help of embedding original features to filter weak hidden-layer outputs during training. For the issue that class representation ability of abstract information is limited by small sample problem, a feature fusion strategy has been designed aiming to combining abstract information learned by HFESAE with original feature and obtain hybrid features for feature reduction. The strategy is hybrid feature selection strategy based on L1 regularization followed by an support vector machine(SVM) ensemble model, in which weighted local discriminant preservation projection (w_LPPD), is designed and employed on each base classifier. At the end of this paper, several representative public datasets are used to verify the effectiveness of the proposed algorithm. The experimental results demonstrated that, the proposed feature learning method yields superior performance compared to other existing and state of art feature learning algorithms including some representative deep autoencoder methods.
Evolutionary Bi-objective Optimization for the Dynamic Chance-Constrained Knapsack Problem Based on Tail Bound Objectives
Venue: arXiv cs.NE 2020
Year: 2020

Real-world combinatorial optimization problems are often stochastic and dynamic. Therefore, it is essential to make optimal and reliable decisions with a holistic approach. In this paper, we consider the dynamic chance-constrained knapsack problem where the weight of each item is stochastic, the capacity constraint changes dynamically over time, and the objective is to maximize the total profit subject to the probability that total weight exceeds the capacity. We make use of prominent tail inequalities such as Chebyshev's inequality, and Chernoff bound to approximate the probabilistic constraint. Our key contribution is to introduce an additional objective which estimates the minimal capacity bound for a given stochastic solution that still meets the chance constraint. This objective helps to cater for dynamic changes to the stochastic problem. We apply single- and multi-objective evolutionary algorithms to the problem and show how bi-objective optimization can help to deal with dynamic chance-constrained problems.
Differentiable Bandit Exploration
Venue: arXiv cs.LG 2020
Year: 2020

Exploration policies in Bayesian bandits maximize the average reward over problem instances drawn from some distribution $\mathcal{P}$. In this work, we learn such policies for an unknown distribution $\mathcal{P}$ using samples from $\mathcal{P}$. Our approach is a form of meta-learning and exploits properties of $\mathcal{P}$ without making strong assumptions about its form. To do this, we parameterize our policies in a differentiable way and optimize them by policy gradients, an approach that is general and easy to implement. We derive effective gradient estimators and introduce novel variance reduction techniques. We also analyze and experiment with various bandit policy classes, including neural networks and a novel softmax policy. The latter has regret guarantees and is a natural starting point for our optimization. Our experiments show the versatility of our approach. We also observe that neural network policies can learn implicit biases expressed only through the sampled instances.
Byzantine Lattice Agreement in Asynchronous Systems
Venue: arXiv cs.DC 2020
Year: 2020

We study the Byzantine lattice agreement (BLA) problem in asynchronous distributed message passing systems. In the BLA problem, each process proposes a value from a join semi-lattice and needs to output a value also in the lattice such that all output values of correct processes lie on a chain despite the presence of Byzantine processes. We present an algorithm for this problem with round complexity of $O(\log f)$ which tolerates $f < \frac{n}{5}$ Byzantine failures in the asynchronous setting without digital signatures, where $n$ is the number of processes. We also show how this algorithm can be modified to work in the authenticated setting (i.e., with digital signatures) to tolerate $f < \frac{n}{3}$ Byzantine failures.
Reinforcement learning for the privacy preservation and manipulation of eye tracking data
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we present an approach based on reinforcement learning for eye tracking data manipulation. It is based on two opposing agents, where one tries to classify the data correctly and the second agent looks for patterns in the data, which get manipulated to hide specific information. We show that our approach is successfully applicable to preserve the privacy of the subjects. For this purpose, we evaluate our approach iteratively to showcase the behavior of the reinforcement learning based approach. In addition, we evaluate the importance of temporal, as well as spatial, information of eye tracking data for specific classification goals. In the last part of our evaluation, we apply the procedure to further public data sets without re-training the autoencoder or the data manipulator. The results show that the learned manipulation is generalized and applicable to unseen data as well.
Discernible Image Compression
Venue: arXiv cs.CV 2020
Year: 2020

Image compression, as one of the fundamental low-level image processing tasks, is very essential for computer vision. Tremendous computing and storage resources can be preserved with a trivial amount of visual information. Conventional image compression methods tend to obtain compressed images by minimizing their appearance discrepancy with the corresponding original images, but pay little attention to their efficacy in downstream perception tasks, e.g., image recognition and object detection. Thus, some of compressed images could be recognized with bias. In contrast, this paper aims to produce compressed images by pursuing both appearance and perceptual consistency. Based on the encoder-decoder framework, we propose using a pre-trained CNN to extract features of the original and compressed images, and making them similar. Thus the compressed images are discernible to subsequent tasks, and we name our method as Discernible Image Compression (DIC). In addition, the maximum mean discrepancy (MMD) is employed to minimize the difference between feature distributions. The resulting compression network can generate images with high image quality and preserve the consistent perception in the feature domain, so that these images can be well recognized by pre-trained machine learning models. Experiments on benchmarks demonstrate that images compressed by using the proposed method can also be well recognized by subsequent visual recognition and detection models. For instance, the mAP value of compressed images by DIC is about 0.6% higher than that of using compressed images by conventional methods.
Approximate Distance Oracles Subject to Multiple Vertex Failures
Venue: arXiv cs.DS 2020
Year: 2020

Given an undirected graph $G=(V,E)$ of $n$ vertices and $m$ edges with weights in $[1,W]$, we construct vertex sensitive distance oracles (VSDO), which are data structures that preprocess the graph, and answer the following kind of queries: Given a source vertex $u$, a target vertex $v$, and a batch of $d$ failed vertices $D$, output (an approximation of) the distance between $u$ and $v$ in $G-D$ (that is, the graph $G$ with vertices in $D$ removed). An oracle has stretch $α$ if it always holds that $δ_{G-D}(u,v)\le\tildeδ(u,v)\leα\cdotδ_{G-D}(u,v)$, where $δ_{G-D}(u,v)$ is the actual distance between $u$ and $v$ in $G-D$, and $\tildeδ(u,v)$ is the distance reported by the oracle. In this paper we construct efficient VSDOs for any number $d$ of failures. For any constant $c\geq 1$, we propose two oracles: $\bullet$ The first oracle has size $n^{2+1/c}(\log n/ε)^{O(d)}\cdot \log W$, answers a query in ${\rm poly}(\log n,d^c,\log\log W,ε^{-1})$ time, and has stretch $1+ε$, for any constant $ε>0$. $\bullet$ The second oracle has size $n^{2+1/c}{\rm poly}(\log (nW),d)$, answers a query in ${\rm poly}(\log n,d^c,\log\log W)$ time, and has stretch ${\rm poly}(\log n,d)$. Both of these oracles can be preprocessed in time polynomial in their space complexity. These results are the first approximate distance oracles of poly-logarithmic query time for any constant number of vertex failures in general undirected graphs. Previously there are $(1+ε)$-approximate $d$-edge sensitive distance oracles [Chechik et al. 2017] answering distance queries when $d$ edges fail, which have size $O(n^2(\log n/ε)^d\cdot d\log W)$ and query time ${\rm poly}(\log n, d, \log\log W)$.
Implementing Dynamic Answer Set Programming
Venue: arXiv cs.AI 2020
Year: 2020

We introduce an implementation of an extension of Answer Set Programming (ASP) with language constructs from dynamic (and temporal) logic that provides an expressive computational framework for modeling dynamic applications. Starting from logical foundations, provided by dynamic and temporal equilibrium logics over finite linear traces, we develop a translation of dynamic formulas into temporal logic programs. This provides us with a normal form result establishing the strong equivalence of formulas in different logics. Our translation relies on the introduction of auxiliary atoms to guarantee polynomial space complexity and to provide an embedding that is doomed to be impossible over the same language. Finally, the reduction of dynamic formulas to temporal logic programs allows us to extend ASP with both approaches in a uniform way and to implement both extensions via temporal ASP solvers such as telingo
A New Methodology for Information Security Risk Assessment for Medical Devices and Its Evaluation
Venue: arXiv cs.CR 2020
Year: 2020

As technology advances towards more connected and digital environments, medical devices are becoming increasingly connected to hospital networks and to the Internet, which exposes them, and thus the patients using them, to new cybersecurity threats. Currently, there is a lack of a methodology dedicated to information security risk assessment for medical devices. In this study, we present the Threat identification, ontology-based Likelihood, severity Decomposition, and Risk integration (TLDR) methodology for information security risk assessment for medical devices. The TLDR methodology uses the following steps: (1) identifying the potentially vulnerable components of medical devices, in this case, four different medical imaging devices (MIDs); (2) identifying the potential attacks, in this case, 23 potential attacks on MIDs; (3) mapping the discovered attacks into a known attack ontology - in this case, the Common Attack Pattern Enumeration and Classifications (CAPECs); (4) estimating the likelihood of the mapped CAPECs in the medical domain with the assistance of a panel of senior healthcare Information Security Experts (ISEs); (5) computing the CAPEC-based likelihood estimates of each attack; (6) decomposing each attack into several severity aspects and assigning them weights; (7) assessing the magnitude of the impact of each of the severity aspects for each attack with the assistance of a panel of senior Medical Experts (MEs); (8) computing the composite severity assessments for each attack; and finally, (9) integrating the likelihood and severity of each attack into its risk, and thus prioritizing it. The details of steps six to eight are beyond the scope of the current study; in the current study, we had replaced them by a single step that included asking the panel of MEs [in this case, radiologists], to assess the overall severity for each attack and use it as its severity...
Jittering Samples using a kd-Tree Stratification
Venue: arXiv cs.GR 2020
Year: 2020

Monte Carlo sampling techniques are used to estimate high-dimensional integrals that model the physics of light transport in virtual scenes for computer graphics applications. These methods rely on the law of large numbers to estimate expectations via simulation, typically resulting in slow convergence. Their errors usually manifest as undesirable grain in the pictures generated by image synthesis algorithms. It is well known that these errors diminish when the samples are chosen appropriately. A well known technique for reducing error operates by subdividing the integration domain, estimating integrals in each \emph{stratum} and aggregating these values into a stratified sampling estimate. Naïve methods for stratification, based on a lattice (grid) are known to improve the convergence rate of Monte Carlo, but require samples that grow exponentially with the dimensionality of the domain. We propose a simple stratification scheme for $d$ dimensional hypercubes using the kd-tree data structure. Our scheme enables the generation of an arbitrary number of equal volume partitions of the rectangular domain, and $n$ samples can be generated in $O(n)$ time. Since we do not always need to explicitly build a kd-tree, we provide a simple procedure that allows the sample set to be drawn fully in parallel without any precomputation or storage, speeding up sampling to $O(\log n)$ time per sample when executed on $n$ cores. If the tree is implicitly precomputed ($O(n)$ storage) the parallelised run time reduces to $O(1)$ on $n$ cores. In addition to these benefits, we provide an upper bound on the worst case star-discrepancy for $n$ samples matching that of lattice-based sampling strategies, which occur as a special case of our proposed method. We use a number of quantitative and qualitative tests to compare our method against state of the art samplers for image synthesis.
The Probabilistic Model Checker Storm
Venue: arXiv cs.SE 2020
Year: 2020

We present the probabilistic model checker Storm. Storm supports the analysis of discrete- and continuous-time variants of both Markov chains and Markov decision processes. Storm has three major distinguishing features. It supports multiple input languages for Markov models, including the JANI and PRISM modeling languages, dynamic fault trees, generalized stochastic Petri nets, and the probabilistic guarded command language. It has a modular set-up in which solvers and symbolic engines can easily be exchanged. Its Python API allows for rapid prototyping by encapsulating Storm's fast and scalable algorithms. This paper reports on the main features of Storm and explains how to effectively use them. A description is provided of the main distinguishing functionalities of Storm. Finally, an empirical evaluation of different configurations of Storm on the QComp 2019 benchmark set is presented.
Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
Venue: arXiv cs.LG 2020
Year: 2020

In this work, we propose a new family of generative flows on an augmented data space, with an aim to improve expressivity without drastically increasing the computational cost of sampling and evaluation of a lower bound on the likelihood. Theoretically, we prove the proposed flow can approximate a Hamiltonian ODE as a universal transport map. Empirically, we demonstrate state-of-the-art performance on standard benchmarks of flow-based generative modeling.
Controlling Computation versus Quality for Neural Sequence Models
Venue: arXiv cs.LG 2020
Year: 2020

Most neural networks utilize the same amount of compute for every example independent of the inherent complexity of the input. Further, methods that adapt the amount of computation to the example focus on finding a fixed inference-time computational graph per example, ignoring any external computational budgets or varying inference time limitations. In this work, we utilize conditional computation to make neural sequence models (Transformer) more efficient and computation-aware during inference. We first modify the Transformer architecture, making each set of operations conditionally executable depending on the output of a learned control network. We then train this model in a multi-task setting, where each task corresponds to a particular computation budget. This allows us to train a single model that can be controlled to operate on different points of the computation-quality trade-off curve, depending on the available computation budget at inference time. We evaluate our approach on two tasks: (i) WMT English-French Translation and (ii) Unsupervised representation learning (BERT). Our experiments demonstrate that the proposed Conditional Computation Transformer (CCT) is competitive with vanilla Transformers when allowed to utilize its full computational budget, while improving significantly over computationally equivalent baselines when operating on smaller computational budgets.
Targeted Forgetting and False Memory Formation in Continual Learners through Adversarial Backdoor Attacks
Venue: arXiv cs.LG 2020
Year: 2020

Artificial neural networks are well-known to be susceptible to catastrophic forgetting when continually learning from sequences of tasks. Various continual (or "incremental") learning approaches have been proposed to avoid catastrophic forgetting, but they are typically adversary agnostic, i.e., they do not consider the possibility of a malicious attack. In this effort, we explore the vulnerability of Elastic Weight Consolidation (EWC), a popular continual learning algorithm for avoiding catastrophic forgetting. We show that an intelligent adversary can bypass the EWC's defenses, and instead cause gradual and deliberate forgetting by introducing small amounts of misinformation to the model during training. We demonstrate such an adversary's ability to assume control of the model via injection of "backdoor" attack samples on both permuted and split benchmark variants of the MNIST dataset. Importantly, once the model has learned the adversarial misinformation, the adversary can then control the amount of forgetting of any task. Equivalently, the malicious actor can create a "false memory" about any task by inserting carefully-designed backdoor samples to any fraction of the test instances of that task. Perhaps most damaging, we show this vulnerability to be very acute; neural network memory can be easily compromised with the addition of backdoor samples into as little as 1% of the training data of even a single task.
A Survey on Feasibility and Suitability of Blockchain Techniques for the E-Voting Systems
Venue: arXiv cs.CR 2020
Year: 2020

In the second decade of the 21st century, blockchain definitely became one of the most trending computational technologies. This research aims to question the feasibility and suitability of using blockchain technology within e-voting systems, regarding both technical and non-technical aspects. In today's world, although the course of this spreading is considerably slow, several countries already use means of e-voting due to many social and economic reasons, which we further investigated. Nevertheless, the number of countries offering various e-government solutions, apart from e-voting, is significantly high. E-voting systems, naturally, require much more attention and assurance regarding potential security and anonymity issues, since voting is one of the few extremely critical governmental processes. Nevertheless, e-voting is not purely a governmental service, but many companies and nonprofit organizations would benefit the cost-efficiency, scalability, remote accessibility, and ease of use that it provides. Blockchain technology is claimed to be able to address some, obviously not all, important security concerns, including anonymity, confidentiality, integrity, and non-repudiation. The analysis results presented in this article mostly confirm these claims.
STANNIS: Low-Power Acceleration of Deep Neural Network Training Using Computational Storage
Venue: arXiv cs.DC 2020
Year: 2020

This paper proposes a framework for distributed, in-storage training of neural networks on clusters of computational storage devices. Such devices not only contain hardware accelerators but also eliminate data movement between the host and storage, resulting in both improved performance and power savings. More importantly, this in-storage processing style of training ensures that private data never leaves the storage while fully controlling the sharing of public data. Experimental results show up to 2.7x speedup and 69% reduction in energy consumption and no significant loss in accuracy.
Evolutionary Optimization of Deep Learning Activation Functions
Venue: arXiv cs.LG 2020
Year: 2020

The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, the Rectified Linear Unit (ReLU) remains the most commonly-used in practice. This paper shows that evolutionary algorithms can discover novel activation functions that outperform ReLU. A tree-based search space of candidate activation functions is defined and explored with mutation, crossover, and exhaustive search. Experiments on training wide residual networks on the CIFAR-10 and CIFAR-100 image datasets show that this approach is effective. Replacing ReLU with evolved activation functions results in statistically significant increases in network accuracy. Optimal performance is achieved when evolution is allowed to customize activation functions to a particular task; however, these novel activation functions are shown to generalize, achieving high performance across tasks. Evolutionary optimization of activation functions is therefore a promising new dimension of metalearning in neural networks.
Federated Extra-Trees with Privacy Preserving
Venue: arXiv cs.LG 2020
Year: 2020

It is commonly observed that the data are scattered everywhere and difficult to be centralized. The data privacy and security also become a sensitive topic. The laws and regulations such as the European Union's General Data Protection Regulation (GDPR) are designed to protect the public's data privacy. However, machine learning requires a large amount of data for better performance, and the current circumstances put deploying real-life AI applications in an extremely difficult situation. To tackle these challenges, in this paper we propose a novel privacy-preserving federated machine learning model, named Federated Extra-Trees, which applies local differential privacy in the federated trees model. A secure multi-institutional machine learning system was developed to provide superior performance by processing the modeling jointly on different clients without exchanging any raw data. We have validated the accuracy of our work by conducting extensive experiments on public datasets and the efficiency and robustness were also verified by simulating the real-world scenarios. Overall, we presented an extensible, scalable and practical solution to handle the data island problem.