OpenRaiser/Intern-Atlas huggingface dataset:
Learning for DC-OPF: Classifying active sets using neural nets
Venue: arXiv cs.SY 2019
Year: 2019

The optimal power flow is an optimization problem used in power systems operational planning to maximize economic efficiency while satisfying demand and maintaining safety margins. Due to uncertainty and variability in renewable energy generation and demand, the optimal solution needs to be updated in response to observed uncertainty realizations or near real-time forecast updates. To address the challenge of computing such frequent real-time updates to the optimal solution, recent literature has proposed the use of machine learning to learn the mapping between the uncertainty realization and the optimal solution. Further, learning the active set of constraints at optimality, as opposed to directly learning the optimal solution, has been shown to significantly simplify the machine learning task, and the learnt model can be used to predict optimal solutions in real-time. In this paper, we propose the use of classification algorithms to learn the mapping between the uncertainty realization and the active set of constraints at optimality, thus further enhancing the computational efficiency of the real-time prediction. We employ neural net classifiers for this task and demonstrate the excellent performance of this approach on a number of systems in the IEEE PES PGLib-OPF benchmark library.
Coupled nonlinear delay systems as deep convolutional neural networks
Venue: arXiv cs.ET 2019
Year: 2019

Neural networks are currently transforming the field of computer algorithms, yet their emulation on current computing substrates is highly inefficient. Reservoir computing was successfully implemented on a large variety of substrates and gave new insight in overcoming this implementation bottleneck. Despite its success, the approach lags behind the state of the art in deep learning. We therefore extend time-delay reservoirs to deep networks and demonstrate that these conceptually correspond to deep convolutional neural networks. Convolution is intrinsically realized on a substrate level by generic drive-response properties of dynamical systems. The resulting novelty is avoiding vector-matrix products between layers, which cause low efficiency in today's substrates. Compared to singleton time-delay reservoirs, our deep network achieves accuracy improvements by at least an order of magnitude in Mackey-Glass and Lorenz timeseries prediction.
WaveletAE: A Wavelet-enhanced Autoencoder for Wind Turbine Blade Icing Detection
Venue: arXiv cs.LG 2019
Year: 2019

Wind power, as an alternative to burning fossil fuels, is abundant and inexhaustible. To fully utilize wind power, wind farms are usually located in areas of high altitude and facing serious ice conditions, which can lead to serious consequences. Quick detection of blade ice accretion is crucial for the maintenance of wind farms. Unlike traditional methods of installing expensive physical detectors on wind blades, data-driven approaches are increasingly popular for inspecting the wind turbine failures. In this work, we propose a wavelet enhanced autoencoder model (WaveletAE) to identify wind turbine dysfunction by analyzing the multivariate time series monitored by the SCADA system. WaveletAE is enhanced with wavelet detail coefficients to enforce the autoencoder to capture information from multiple scales, and the CNN-LSTM architecture is applied to learn channel-wise and temporal-wise relations. The empirical study shows that the proposed model outperforms other state-of-the-art time series anomaly detection methods for real-world blade icing detection.
Finding Nearest Neighbors in graphs locally
Venue: arXiv cs.DS 2019
Year: 2019

Many distributed learning techniques have been motivated by the increasing size of datasets and their inability to fit into main memory on a single machine. We propose an algorithm that finds the nearest neighbor in a graph locally without the need of visiting the whole graph. Our algorithm is distributed which further encourage scalability. We prove the convergence of the algorithm
Heat-Diffusion: Pareto Optimal Dynamic Routing for Time-Varying Wireless Networks
Venue: arXiv cs.NI 2019
Year: 2019

A dynamic routing policy, referred to as Heat-Diffusion (HD), is developed for multihop uniclass wireless networks subject to random traffic, time-varying topology and inter-channel interference.The policy uses only current condition of queue occupancies and channel states, with requiring no knowledge of traffic and topology.Besides throughput optimality, HD minimizes an average quadratic routing cost defined by endowing each channel with a time-varying cost factor. Further, HD minimizes average network delay in the class of routing policies that base decisions only on current condition of traffic congestion and channel states. Further, in this class of routing policies, HD provides a Pareto optimal tradeoff between average routing cost and average network delay, meaning that no policy can improve either one without detriment to the other. Finally, HD fluid limit follows graph combinatorial heat equation, which can open a new way to study wireless networks using heat calculus, a very active area of pure mathematics.
Generating Natural Language Explanations for Visual Question Answering using Scene Graphs and Visual Attention
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we present a novel approach for the task of eXplainable Question Answering (XQA), i.e., generating natural language (NL) explanations for the Visual Question Answering (VQA) problem. We generate NL explanations comprising of the evidence to support the answer to a question asked to an image using two sources of information: (a) annotations of entities in an image (e.g., object labels, region descriptions, relation phrases) generated from the scene graph of the image, and (b) the attention map generated by a VQA model when answering the question. We show how combining the visual attention map with the NL representation of relevant scene graph entities, carefully selected using a language model, can give reasonable textual explanations without the need of any additional collected data (explanation captions, etc). We run our algorithms on the Visual Genome (VG) dataset and conduct internal user-studies to demonstrate the efficacy of our approach over a strong baseline. We have also released a live web demo showcasing our VQA and textual explanation generation using scene graphs and visual attention.
Descriptive complexity for minimal time of cellular automata
Venue: arXiv cs.LO 2019
Year: 2019

Descriptive complexity may be useful to design programs in a natural declarative way. This is important for parallel computation models such as cellular automata, because designing parallel programs is considered difficult. Our paper establishes logical characterizations of the three classical complexity classes that model minimal time, called real-time, of one-dimensional cellular automata according to their canonical variants. Our logics are natural restrictions of the existential second-order Horn logic. They correspond to the three ways of deciding a language on a square grid circuit of side n according to the three canonical placements of an input word of length n on the grid. Our key tool is a normalization method that transforms a formula into an equivalent formula that literally mimics a grid circuit.
Deeply Supervised Multimodal Attentional Translation Embeddings for Visual Relationship Detection
Venue: arXiv cs.CV 2019
Year: 2019

Detecting visual relationships, i.e. <Subject, Predicate, Object> triplets, is a challenging Scene Understanding task approached in the past via linguistic priors or spatial information in a single feature branch. We introduce a new deeply supervised two-branch architecture, the Multimodal Attentional Translation Embeddings, where the visual features of each branch are driven by a multimodal attentional mechanism that exploits spatio-linguistic similarities in a low-dimensional space. We present a variety of experiments comparing against all related approaches in the literature, as well as by re-implementing and fine-tuning several of them. Results on the commonly employed VRD dataset [1] show that the proposed method clearly outperforms all others, while we also justify our claims both quantitatively and qualitatively.
Machine Learning on Biomedical Images: Interactive Learning, Transfer Learning, Class Imbalance, and Beyond
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we highlight three issues that limit performance of machine learning on biomedical images, and tackle them through 3 case studies: 1) Interactive Machine Learning (IML): we show how IML can drastically improve exploration time and quality of direct volume rendering. 2) transfer learning: we show how transfer learning along with intelligent pre-processing can result in better Alzheimer's diagnosis using a much smaller training set 3) data imbalance: we show how our novel focal Tversky loss function can provide better segmentation results taking into account the imbalanced nature of segmentation datasets. The case studies are accompanied by in-depth analytical discussion of results with possible future directions.
Synthesizing Number Generators for Stochastic Computing using Mixed Integer Programming
Venue: arXiv cs.ET 2019
Year: 2019

Stochastic computing (SC) is a high density, low-power computation technique which encodes values as unary bitstreams instead of binary-encoded (BE) values. Practical SC implementations require deterministic or pseudo-random number sequences which are optimally correlated to generate bitstreams and achieve accurate results. Unfortunately, the size of the search space makes manually designing optimally correlated number sequences a difficult task. To automate this design burden, we propose a synthesis formulation using mixed integer programming to automatically generate optimally correlated number sequences. In particular, our synthesis formulation improves the accuracy of arithmetic operations such as multiplication and squaring circuits by up to 2.5x and 20x respectively. We also show how our technique can be extended to scale to larger circuits.
Adaptive Sequence Submodularity
Venue: arXiv cs.LG 2019
Year: 2019

In many machine learning applications, one needs to interactively select a sequence of items (e.g., recommending movies based on a user's feedback) or make sequential decisions in a certain order (e.g., guiding an agent through a series of states). Not only do sequences already pose a dauntingly large search space, but we must also take into account past observations, as well as the uncertainty of future outcomes. Without further structure, finding an optimal sequence is notoriously challenging, if not completely intractable. In this paper, we view the problem of adaptive and sequential decision making through the lens of submodularity and propose an adaptive greedy policy with strong theoretical guarantees. Additionally, to demonstrate the practical utility of our results, we run experiments on Amazon product recommendation and Wikipedia link prediction tasks.
Information Losses in Neural Classifiers from Sampling
Venue: arXiv cs.LG 2019
Year: 2019

This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated neural model with the information about the feature space contained in the hidden representation of that model. It then bounds this expected total variation as a function of the size of randomly sampled datasets in a fairly general setting, and without bringing in any additional dependence on model complexity. It ultimately obtains bounds on information losses that are less sensitive to input compression and in general much smaller than existing bounds. The paper then uses these bounds to explain some recent experimental findings of information compression in neural networks which cannot be explained by previous work. Finally, the paper shows that not only are these bounds much smaller than existing ones, but that they also correspond well with experiments.
Delay Outage Probability of Multi-relay Selection for Mobile Relay Edge Computing Systems
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we deal with the problem of relay selection in mobile edge computing networks, where a source node transmits a computation-intensive task to a destination via the aid of multiple relay nodes. It differs from the traditional relay nodes in the way that each relay node is equipped with an edge computing server, thus each relay node can execute the received task and forwards the computed result to the destination. Accordingly, we define a delay outage probability to evaluate the impact of the relay computing ability on the link communications, and then propose a latency-best relay selection (LBRS) scheme that not only consider the communication capability, but also consider the computing ability. The performance of the proposed relay selection scheme with the traditional communication-only relay selection (CORS) and computing-only selection (CPORS) schemes in terms of the delay outage probability and the diversity order is analyzed, compared with other relay selection schemes. We show that the proposed LBRS scheme reduces to the traditional CORS scheme under the high signal-to-noise ratio (SNR) region. We further reveal that the diversity orders of both the proposed LBRS and CORS schemes are dependent on the computing ability of relay nodes.
CruzAffect at AffCon 2019 Shared Task: A feature-rich approach to characterize happiness
Venue: arXiv cs.CL 2019
Year: 2019

We present our system, CruzAffect, for the CL-Aff Shared Task 2019. CruzAffect consists of several types of robust and efficient models for affective classification tasks. We utilize both traditional classifiers, such as XGBoosted Forest, as well as a deep learning Convolutional Neural Networks (CNN) classifier. We explore rich feature sets such as syntactic features, emotional features, and profile features, and utilize several sentiment lexicons, to discover essential indicators of social involvement and control that a subject might exercise in their happy moments, as described in textual snippets from the HappyDB database. The data comes with a labeled set (10K), and a larger unlabeled set (70K). We therefore use supervised methods on the 10K dataset, and a bootstrapped semi-supervised approach for the 70K. We evaluate these models for binary classification of agency and social labels (Task 1), as well as multi-class prediction for concepts labels (Task 2). We obtain promising results on the held-out data, suggesting that the proposed feature sets effectively represent the data for affective classification tasks. We also build concepts models that discover general themes recurring in happy moments. Our results indicate that generic characteristics are shared between the classes of agency, social and concepts, suggesting it should be possible to build general models for affective classification tasks.
A Timer-Augmented Cost Function for Load Balanced DSMC
Venue: arXiv cs.DC 2019
Year: 2019

Due to a hard dependency between time steps, large-scale simulations of gas using the Direct Simulation Monte Carlo (DSMC) method proceed at the pace of the slowest processor. Scalability is therefore achievable only by ensuring that the work done each time step is as evenly apportioned among the processors as possible. Furthermore, as the simulated system evolves, the load shifts, and thus this load-balancing typically needs to be performed multiple times over the course of a simulation. Common methods generally use either crude performance models or processor-level timers. We combine both to create a timer-augmented cost function which both converges quickly and yields well-balanced processor decompositions. When compared to a particle-based performance model alone, our method achieves 2x speedup at steady-state on up to 1024 processors for a test case consisting of a Mach 9 argon jet impacting a solid wall.
Towards Automated Melanoma Detection with Deep Learning: Data Purification and Augmentation
Venue: arXiv cs.CV 2019
Year: 2019

Melanoma is one of the ten most common cancers in the US. Early detection is crucial for survival, but often the cancer is diagnosed in the fatal stage. Deep learning has the potential to improve cancer detection rates, but its applicability to melanoma detection is compromised by the limitations of the available skin lesion databases, which are small, heavily imbalanced, and contain images with occlusions. We build deep-learning-based tools for data purification and augmentation to counter-act these limitations. The developed tools can be utilized in a deep learning system for lesion classification and we show how to build such a system. The system heavily relies on the processing unit for removing image occlusions and the data generation unit, based on generative adversarial networks, for populating scarce lesion classes, or equivalently creating virtual patients with pre-defined types of lesions. We empirically verify our approach and show that incorporating these two units into melanoma detection system results in the superior performance over common baselines.
Re-determinizing Information Set Monte Carlo Tree Search in Hanabi
Venue: arXiv cs.AI 2019
Year: 2019

This technical report documents the winner of the Computational Intelligence in Games(CIG) 2018 Hanabi competition. We introduce Re-determinizing IS-MCTS, a novel extension of Information Set Monte Carlo Tree Search (IS-MCTS) that prevents a leakage of hidden information into opponent models that can occur in IS-MCTS, and is particularly severe in Hanabi. Re-determinizing IS-MCTS scores higher in Hanabi for 2-4 players than previously published work at the time of the competition. Given the 40ms competition time limit per move we use a learned evaluation function to estimate leaf node values and avoid full simulations during MCTS. For the Mixed track competition, in which the identity of the other players is unknown, a simple Bayesian opponent model is used that is updated as each game proceeds.
Combinatorial and Algorithmic Properties of One Matrix Structure at Monotone Boolean Functions
Venue: arXiv cs.DM 2019
Year: 2019

One matrix structure in the area of monotone Boolean functions is defined here. Some of its combinatorial, algebraic and algorithmic properties are derived. On the base of these properties, three algorithms are built. First of them generates all monotone Boolean functions of $n$ variables in lexicographic order. The second one determines the first (resp. the last) lexicographically minimal true (resp. maximal false) vector of an unknown monotone function $f$ of $n$ variables. The algorithm uses at most $n$ membership queries and its running time is $Θ(n)$. It serves the third algorithm, which identifies an unknown monotone Boolean function $f$ of $n$ variables by using membership queries only. The experimental results show that for $1\leq n\leq 6$, the algorithm determines $f$ by using at most $m.n$ queries, where $m$ is the combined size of the sets of minimal true and maximal false vectors of $f$.
Atlas-based automated detection of swim bladder in Medaka embryo
Venue: arXiv cs.CV 2019
Year: 2019

Fish embryo models are increasingly being used both for the assessment of chemicals efficacy and potential toxicity. This article proposes a methodology to automatically detect the swim bladder on 2D images of Medaka fish embryos seen either in dorsal view or in lateral view. After embryo segmentation and for each studied orientation, the method builds an atlas of a healthy embryo. This atlas is then used to define the region of interest and to guide the swim bladder segmentation with a discrete globally optimal active contour. Descriptors are subsequently designed from this segmentation. An automated random forest clas-sifier is built from these descriptors in order to classify embryos with and without a swim bladder. The proposed method is assessed on a dataset of 261 images, containing 202 embryos with a swim bladder (where 196 are in dorsal view and 6 are in lateral view) and 59 without (where 43 are in dorsal view and 16 are in lateral view). We obtain an average precision rate of 95% in the total dataset following 5-fold cross-validation.
Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
Venue: arXiv cs.CV 2019
Year: 2019

Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the main components and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used image and video datasets and the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning.
Enumerating Unique Computational Graphs via an Iterative Graph Invariant
Venue: arXiv cs.DM 2019
Year: 2019

In this report, we describe a novel graph invariant for computational graphs (colored directed acylic graphs) and how we used it to generate all distinct computational graphs up to isomorphism for small graphs. The algorithm iteratively applies isomorphism-invariant operations, which take into account the graph structure and coloring, and outputs a fixed-length hash that is identical for all isomorphic computational graphs. While the algorithm cannot perfectly distinguish all pairs of non-isomorphic computational graphs, we suggest that it may be useful as a heuristic for comparing graphs.
Nearest neighbor decoding for Tardos fingerprinting codes
Venue: arXiv cs.CR 2019
Year: 2019

Over the past decade, various improvements have been made to Tardos' collusion-resistant fingerprinting scheme [Tardos, STOC 2003], ultimately resulting in a good understanding of what is the minimum code length required to achieve collusion-resistance. In contrast, decreasing the cost of the actual decoding algorithm for identifying the potential colluders has received less attention, even though previous results have shown that using joint decoding strategies, deemed too expensive for decoding, may lead to better code lengths. Moreover, in dynamic settings a fast decoder may be required to provide answers in real-time, further raising the question whether the decoding costs of score-based fingerprinting schemes can be decreased with a smarter decoding algorithm. In this paper we show how to model the decoding step of score-based fingerprinting as a nearest neighbor search problem, and how this relation allows us to apply techniques from the field of (approximate) nearest neighbor searching to obtain decoding times which are sublinear in the total number of users. As this does not affect the encoding and embedding steps, this decoding mechanism can easily be deployed within existing fingerprinting schemes, and this may bring a truly efficient joint decoder closer to reality. Besides the application to fingerprinting, similar techniques can be used to decrease the decoding costs of group testing methods, which may be of independent interest.