OpenRaiser/Intern-Atlas huggingface dataset:
Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling
Venue: arXiv cs.LG 2020
Year: 2020

Ensembling deep learning models is a shortcut to promote its implementation in new scenarios, which can avoid tuning neural networks, losses and training algorithms from scratch. However, it is difficult to collect sufficient accurate and diverse models through once training. This paper proposes Auto-Ensemble (AE) to collect checkpoints of deep learning model and ensemble them automatically by adaptive learning rate scheduling algorithm. The advantage of this method is to make the model converge to various local optima by scheduling the learning rate in once training. When the number of lo-cal optimal solutions tends to be saturated, all the collected checkpoints are used for ensemble. Our method is universal, it can be applied to various scenarios. Experiment results on multiple datasets and neural networks demonstrate it is effective and competitive, especially on few-shot learning. Besides, we proposed a method to measure the distance among models. Then we can ensure the accuracy and diversity of collected models.
Session-based Suggestion of Topics for Geographic Exploratory Search
Venue: arXiv cs.IR 2020
Year: 2020

Exploratory information search can challenge users in the formulation of efficacious search queries. Moreover, complex information spaces, such as those managed by Geographical Information Systems, can disorient people, making it difficult to find relevant data. In order to address these issues, we developed a session-based suggestion model that proposes concepts as a "you might also be interested in" function, by taking the user's previous queries into account. Our model can be applied to incrementally generate suggestions in interactive search. It can be used for query expansion, and in general to guide users in the exploration of possibly complex spaces of data categories. Our model is based on a concept co-occurrence graph that describes how frequently concepts are searched together in search sessions. Starting from an ontological domain representation, we generated the graph by analyzing the query log of a major search engine. Moreover, we identified clusters of ontology concepts which frequently co-occur in the sessions of the log via community detection on the graph. The evaluation of our model provided satisfactory accuracy results.
Artificial Intelligence for EU Decision-Making. Effects on Citizens Perceptions of Input, Throughput and Output Legitimacy
Venue: arXiv cs.CY 2020
Year: 2020

A lack of political legitimacy undermines the ability of the European Union to resolve major crises and threatens the stability of the system as a whole. By integrating digital data into political processes, the EU seeks to base decision-making increasingly on sound empirical evidence. In particular, artificial intelligence systems have the potential to increase political legitimacy by identifying pressing societal issues, forecasting potential policy outcomes, informing the policy process, and evaluating policy effectiveness. This paper investigates how citizens perceptions of EU input, throughput, and output legitimacy are influenced by three distinct decision-making arrangements. First, independent human decision-making, HDM, Second, independent algorithmic decision-making, ADM, and, third, hybrid decision-making by EU politicians and AI-based systems together. The results of a pre-registered online experiment with 572 respondents suggest that existing EU decision-making arrangements are still perceived as the most democratic - input legitimacy. However, regarding the decision-making process itself - throughput legitimacy - and its policy outcomes - output legitimacy, no difference was observed between the status quo and hybrid decision-making involving both ADM and democratically elected EU institutions. Where ADM systems are the sole decision-maker, respondents tend to perceive these as illegitimate. The paper discusses the implications of these findings for EU legitimacy and data-driven policy-making.
Fisheye Distortion Rectification from Deep Straight Lines
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a novel line-aware rectification network (LaRecNet) to address the problem of fisheye distortion rectification based on the classical observation that straight lines in 3D space should be still straight in image planes. Specifically, the proposed LaRecNet contains three sequential modules to (1) learn the distorted straight lines from fisheye images; (2) estimate the distortion parameters from the learned heatmaps and the image appearance; and (3) rectify the input images via a proposed differentiable rectification layer. To better train and evaluate the proposed model, we create a synthetic line-rich fisheye (SLF) dataset that contains the distortion parameters and well-annotated distorted straight lines of fisheye images. The proposed method enables us to simultaneously calibrate the geometric distortion parameters and rectify fisheye images. Extensive experiments demonstrate that our model achieves state-of-the-art performance in terms of both geometric accuracy and image quality on several evaluation metrics. In particular, the images rectified by LaRecNet achieve an average reprojection error of 0.33 pixels on the SLF dataset and produce the highest peak signal-to-noise ratio (PSNR) and structure similarity index (SSIM) compared with the groundtruth.
Verification and Design Methods for the BrainScaleS Neuromorphic Hardware System
Venue: arXiv cs.AR 2020
Year: 2020

This paper presents verification and implementation methods that have been developed for the design of the BrainScaleS-2 65nm ASICs. The 2nd generation BrainScaleS chips are mixed-signal devices with tight coupling between full-custom analog neuromorphic circuits and two general purpose microprocessors (PPU) with SIMD extension for on-chip learning and plasticity. Simulation methods for automated analysis and pre-tapeout calibration of the highly parameterizable analog neuron and synapse circuits and for hardware-software co-development of the digital logic and software stack are presented. Accelerated operation of neuromorphic circuits and highly-parallel digital data buses between the full-custom neuromorphic part and the PPU require custom methodologies to close the digital signal timing at the interfaces. Novel extensions to the standard digital physical implementation design flow are highlighted. We present early results from the first full-size BrainScaleS-2 ASIC containing 512 neurons and 130K synapses, demonstrating the successful application of these methods. An application example illustrates the full functionality of the BrainScaleS-2 hybrid plasticity architecture.
A multivariate water quality parameter prediction model using recurrent neural network
Venue: arXiv cs.LG 2020
Year: 2020

The global degradation of water resources is a matter of great concern, especially for the survival of humanity. The effective monitoring and management of existing water resources is necessary to achieve and maintain optimal water quality. The prediction of the quality of water resources will aid in the timely identification of possible problem areas and thus increase the efficiency of water management. The purpose of this research is to develop a water quality prediction model based on water quality parameters through the application of a specialised recurrent neural network (RNN), Long Short-Term Memory (LSTM) and the use of historical water quality data over several years. Both multivariate single and multiple step LSTM models were developed, using a Rectified Linear Unit (ReLU) activation function and a Root Mean Square Propagation (RMSprop) optimiser was developed. The single step model attained an error of 0.01 mg/L, whilst the multiple step model achieved a Root Mean Squared Error (RMSE) of 0.227 mg/L.
Improved Techniques for Training Single-Image GANs
Venue: arXiv cs.CV 2020
Year: 2020

Recently there has been an interest in the potential of learning generative models from a single image, as opposed to from a large dataset. This task is of practical significance, as it means that generative models can be used in domains where collecting a large dataset is not feasible. However, training a model capable of generating realistic images from only a single sample is a difficult problem. In this work, we conduct a number of experiments to understand the challenges of training these methods and propose some best practices that we found allowed us to generate improved results over previous work in this space. One key piece is that unlike prior single image generation methods, we concurrently train several stages in a sequential multi-stage manner, allowing us to learn models with fewer stages of increasing image resolution. Compared to a recent state of the art baseline, our model is up to six times faster to train, has fewer parameters, and can better capture the global structure of images.
Tigrinya Neural Machine Translation with Transfer Learning for Humanitarian Response
Venue: arXiv cs.CL 2020
Year: 2020

We report our experiments in building a domain-specific Tigrinya-to-English neural machine translation system. We use transfer learning from other Ge'ez script languages and report an improvement of 1.3 BLEU points over a classic neural baseline. We publish our development pipeline as an open-source library and also provide a demonstration application.
Property Graph Schema Optimization for Domain-Specific Knowledge Graphs
Venue: arXiv cs.DB 2020
Year: 2020

Enterprises are creating domain-specific knowledge graphs by curating and integrating their business data from multiple sources. The data in these knowledge graphs can be described using ontologies, which provide a semantic abstraction to define the content in terms of the entities and the relationships of the domain. The rich semantic relationships in an ontology contain a variety of opportunities to reduce edge traversals and consequently improve the graph query performance. Although there has been a lot of effort to build systems that enable efficient querying over knowledge graphs, the problem of schema optimization for query performance has been largely ignored in the graph setting. In this work, we show that graph schema design has significant impact on query performance, and then propose optimization algorithms that exploit the opportunities from the domain ontology to generate efficient property graph schemas. To the best of our knowledge, we are the first to present an ontology-driven approach for property graph schema optimization. We conduct empirical evaluations with two real-world knowledge graphs from medical and financial domains. The results show that the schemas produced by the optimization algorithms achieve up to 2 orders of magnitude speed-up compared to the baseline approach.
Solving Raven's Progressive Matrices with Multi-Layer Relation Networks
Venue: arXiv cs.LG 2020
Year: 2020

Raven's Progressive Matrices are a benchmark originally designed to test the cognitive abilities of humans. It has recently been adapted to test relational reasoning in machine learning systems. For this purpose the so-called Procedurally Generated Matrices dataset was set up, which is so far one of the most difficult relational reasoning benchmarks. Here we show that deep neural networks are capable of solving this benchmark, reaching an accuracy of 98.0 percent over the previous state-of-the-art of 62.6 percent by combining Wild Relation Networks with Multi-Layer Relation Networks and introducing Magnitude Encoding, an encoding scheme designed for late fusion architectures.
Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search
Venue: arXiv cs.NE 2020
Year: 2020

The performance of a deep neural network is heavily dependent on its architecture and various neural architecture search strategies have been developed for automated network architecture design. Recently, evolutionary neural architecture search (ENAS) has received increasing attention due to the attractive global optimization capability of evolutionary algorithms. However, ENAS suffers from extremely high computation costs because a large number of performance evaluations is usually required in evolutionary optimization and training deep neural networks is itself computationally very intensive. To address this issue, this paper proposes a new evolutionary framework for fast ENAS based on directed acyclic graph, in which parents are randomly sampled and trained on each mini-batch of training data. In addition, a node inheritance strategy is adopted to generate offspring individuals and their fitness is directly evaluated without training. To enhance the feature processing capability of the evolved neural networks, we also encode a channel attention mechanism in the search space. We evaluate the proposed algorithm on the widely used datasets, in comparison with 26 state-of-the-art peer algorithms. Our experimental results show the proposed algorithm is not only computationally much more efficiently, but also highly competitive in learning performance.
Utilizing Differential Evolution into optimizing targeted cancer treatments
Venue: arXiv cs.NE 2020
Year: 2020

Working towards the development of an evolvable cancer treatment simulator, the investigation of Differential Evolution was considered, motivated by the high efficiency of variations of this technique in real-valued problems. A basic DE algorithm, namely "DE/rand/1" was used to optimize the simulated design of a targeted drug delivery system for tumor treatment on PhysiCell simulator. The suggested approach proved to be more efficient than a standard genetic algorithm, which was not able to escape local minima after a predefined number of generations. The key attribute of DE that enables it to outperform standard EAs, is the fact that it keeps the diversity of the population high, throughout all the generations. This work will be incorporated with ongoing research in a more wide applicability platform that will design, develop and evaluate targeted drug delivery systems aiming cancer tumours.
COEBA: A Coevolutionary Bat Algorithm for Discrete Evolutionary Multitasking
Venue: arXiv cs.NE 2020
Year: 2020

Multitasking optimization is an emerging research field which has attracted lot of attention in the scientific community. The main purpose of this paradigm is how to solve multiple optimization problems or tasks simultaneously by conducting a single search process. The main catalyst for reaching this objective is to exploit possible synergies and complementarities among the tasks to be optimized, helping each other by virtue of the transfer of knowledge among them (thereby being referred to as Transfer Optimization). In this context, Evolutionary Multitasking addresses Transfer Optimization problems by resorting to concepts from Evolutionary Computation for simultaneous solving the tasks at hand. This work contributes to this trend by proposing a novel algorithmic scheme for dealing with multitasking environments. The proposed approach, coined as Coevolutionary Bat Algorithm, finds its inspiration in concepts from both co-evolutionary strategies and the metaheuristic Bat Algorithm. We compare the performance of our proposed method with that of its Multifactorial Evolutionary Algorithm counterpart over 15 different multitasking setups, composed by eight reference instances of the discrete Traveling Salesman Problem. The experimentation and results stemming therefrom support the main hypothesis of this study: the proposed Coevolutionary Bat Algorithm is a promising meta-heuristic for solving Evolutionary Multitasking scenarios.
Unfair Exposure of Artists in Music Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

Fairness in machine learning has been studied by many researchers. In particular, fairness in recommender systems has been investigated to ensure the recommendations meet certain criteria with respect to certain sensitive features such as race, gender etc. However, often recommender systems are multi-stakeholder environments in which the fairness towards all stakeholders should be taken care of. It is well-known that the recommendation algorithms suffer from popularity bias; few popular items are over-recommended which leads to the majority of other items not getting proportionate attention. This bias has been investigated from the perspective of the users and how it makes the final recommendations skewed towards popular items in general. In this paper, however, we investigate the impact of popularity bias in recommendation algorithms on the provider of the items (i.e. the entities who are behind the recommended items). Using a music dataset for our experiments, we show that, due to some biases in the algorithms, different groups of artists with varying degrees of popularity are systematically and consistently treated differently than others.
Bio-inspired Optimization: metaheuristic algorithms for optimization
Venue: arXiv cs.NE 2020
Year: 2020

In today's day and time solving real-world complex problems has become fundamentally vital and critical task. Many of these are combinatorial problems, where optimal solutions are sought rather than exact solutions. Traditional optimization methods are found to be effective for small scale problems. However, for real-world large scale problems, traditional methods either do not scale up or fail to obtain optimal solutions or they end-up giving solutions after a long running time. Even earlier artificial intelligence based techniques used to solve these problems could not give acceptable results. However, last two decades have seen many new methods in AI based on the characteristics and behaviors of the living organisms in the nature which are categorized as bio-inspired or nature inspired optimization algorithms. These methods, are also termed meta-heuristic optimization methods, have been proved theoretically and implemented using simulation as well used to create many useful applications. They have been used extensively to solve many industrial and engineering complex problems due to being easy to understand, flexible, simple to adapt to the problem at hand and most importantly their ability to come out of local optima traps. This local optima avoidance property helps in finding global optimal solutions. This paper is aimed at understanding how nature has inspired many optimization algorithms, basic categorization of them, major bio-inspired optimization algorithms invented in recent time with their applications.
Giving Up Control: Neurons as Reinforcement Learning Agents
Venue: arXiv cs.NE 2020
Year: 2020

Artificial Intelligence has historically relied on planning, heuristics, and handcrafted approaches designed by experts. All the while claiming to pursue the creation of Intelligence. This approach fails to acknowledge that intelligence emerges from the dynamics within a complex system. Neurons in the brain are governed by local rules, where no single neuron, or group of neurons, coordinates or controls the others. This local structure gives rise to the appropriate dynamics in which intelligence can emerge. Populations of neurons must compete with their neighbors for resources, inhibition, and activity representation. At the same time, they must cooperate, so the population and organism can perform high-level functions. To this end, we introduce modeling neurons as reinforcement learning agents. Where each neuron may be viewed as an independent actor, trying to maximize its own self-interest. By framing learning in this way, we open the door to an entirely new approach to building intelligent systems.
Classification of Chinese Handwritten Numbers with Labeled Projective Dictionary Pair Learning
Venue: arXiv cs.CV 2020
Year: 2020

Dictionary learning is a cornerstone of image classification. We set out to address a longstanding challenge in using dictionary learning for classification; that is to simultaneously maximise the discriminability and sparse-representability power of the learned dictionaries. Upon this premise, we designed class-specific dictionaries incorporating three factors: discriminability, sparsity and classification error. We integrated these metrics into a unified cost function and adopted a new feature space, i.e., histogram of oriented gradients (HOG), to generate the dictionary atoms. The rationale of using HOG features for designing the dictionaries is their strength in describing fine details of crowded images. The results of applying the proposed method in the classification of Chinese handwritten numbers demonstrated enhanced classification performance $(\sim98\%)$ compared to state-of-the-art deep learning techniques (i.e., SqueezeNet, GoogLeNet and MobileNetV2), but with a fraction of parameters. Furthermore, combination of the HOG features with dictionary learning enhances the accuracy by $11\%$ compared to the case where only pixel domain data are used. These results were supported when the proposed method was applied to both Arabic and English handwritten number databases.
A Critique on the Interventional Detection of Causal Relationships
Venue: arXiv cs.AI 2020
Year: 2020

Interventions are of fundamental importance in Pearl's probabilistic causality regime. In this paper, we will inspect how interventions influence the interpretation of causation in causal models in specific situation. To this end, we will introduce a priori relationships as non-causal relationships in a causal system. Then, we will proceed to discuss the cases that interventions can lead to spurious causation interpretations. This includes the interventional detection of a priori relationships, and cases where the interventional detection of causality forms structural causal models that are not valid in natural situations. We will also discuss other properties of a priori relations and SCMs that have a priori information in their structural equations.
Reduction Theorem for Secrecy over Linear Network Code for Active Attacks
Venue: arXiv cs.IT 2020
Year: 2020

We discuss the effect of sequential error injection on information leakage under a network code. We formulate a network code for the single transmission setting and the multiple transmission setting. Under this formulation, we show that the eavesdropper cannot improve the power of eavesdropping by sequential error injection when the operations in the network are linear operations. We demonstrate the usefulness of this reduction theorem by applying a concrete example of network.
T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Coding
Venue: arXiv cs.NE 2020
Year: 2020

Spiking neural networks (SNNs) have gained considerable interest due to their energy-efficient characteristics, yet lack of a scalable training algorithm has restricted their applicability in practical machine learning problems. The deep neural network-to-SNN conversion approach has been widely studied to broaden the applicability of SNNs. Most previous studies, however, have not fully utilized spatio-temporal aspects of SNNs, which has led to inefficiency in terms of number of spikes and inference latency. In this paper, we present T2FSNN, which introduces the concept of time-to-first-spike coding into deep SNNs using the kernel-based dynamic threshold and dendrite to overcome the aforementioned drawback. In addition, we propose gradient-based optimization and early firing methods to further increase the efficiency of the T2FSNN. According to our results, the proposed methods can reduce inference latency and number of spikes to 22% and less than 1%, compared to those of burst coding, which is the state-of-the-art result on the CIFAR-100.