OpenRaiser/Intern-Atlas huggingface dataset:
Contrastive Learning for Lifted Networks
Venue: arXiv cs.CV 2019
Year: 2019

In this work we address supervised learning of neural networks via lifted network formulations. Lifted networks are interesting because they allow training on massively parallel hardware and assign energy models to discriminatively trained neural networks. We demonstrate that the training methods for lifted networks proposed in the literature have significant limitations and show how to use a contrastive loss to address those limitations. We demonstrate that this contrastive training approximates back-propagation in theory and in practice and that it is superior to the training objective regularly used for lifted networks.
Programming Unikernels in the Large via Functor Driven Development
Venue: arXiv cs.PL 2019
Year: 2019

Compiling applications as unikernels allows them to be tailored to diverse execution environments. Dependency on a monolithic operating system is replaced with linkage against libraries that provide specific services. Doing so in practice has revealed a major barrier: managing the configuration matrix across heterogenous execution targets. A realistic unikernel application depends on hundreds of libraries, each of which may place different demands on the different target execution platforms (e.g.,~cryptographic acceleration). We propose a modular approach to structuring large scale codebases that cleanly separates configuration, application and operating system logic. Our implementation is built on the \mirage unikernel framework, using the \ocaml language's powerful abstraction and metaprogramming facilities. Leveraging modules allows us to build many components independently, with only loose coupling through a set of standardised signatures. Components can be parameterized by other components and composed. Our approach accounts for state, dependency ordering, and error management, and our usage over the years has demonstrated significant efficiency benefits by leveraging compiler features such as global link-time optimisation during the configuration process. We describe our application architecture and experiences via some practical applications of our approach, and discuss how library development in \mirage can facilitate adoption in other unikernel frameworks and programming languages.
Correcting Codes for Asymmetric Single Magnitude Four Error
Venue: arXiv cs.IT 2019
Year: 2019

An error model with asymmetric single magnitude four error is considered. This paper is about constructions of codes correcting single error over $\mathbb{Z}_{2^{a}3^{b}r}$. Firstly, we reduce the construction of a maximal size $B_{1}[4](2^{a}3^{b}r)$ set for $a\geq4$ and $\gcd(r,6)=1$ to the construction of a maximal size $B_{1}[4](2^{a-3}3^{b}r)$ set. Further, we will show that maximal size $B_{1}[4](8\cdot3^{b}r)$ sets can be reduced to maximal size $B_{1}[4](3^{b}r)$ sets. Finally, we give a lower bounds of maximal size $B_{1}[4](2r)$ and $B_{1}[4](2\cdot3^{b}r)$ sets.
A Type Theory for Defining Logics and Proofs
Venue: arXiv cs.LO 2019
Year: 2019

We describe a Martin-Löf-style dependent type theory, called Cocon, that allows us to mix the intensional function space that is used to represent higher-order abstract syntax (HOAS) trees with the extensional function space that describes (recursive) computations. We mediate between HOAS representations and computations using contextual modal types. Our type theory also supports an infinite hierarchy of universes and hence supports type-level computation thereby providing metaprogramming and (small-scale) reflection. Our main contribution is the development of a Kripke-style model for Cocon that allows us to prove normalization. From the normalization proof, we derive subject reduction and consistency. Our work lays the foundation to incorporate the methodology of logical frameworks into systems such as Agda and bridges the longstanding gap between these two worlds.
The structure of graphs with given number of blocks and the maximum Wiener index
Venue: arXiv cs.DM 2019
Year: 2019

The Wiener index (the distance) of a connected graph is the sum of distances between all pairs of vertices. In this paper, we study the maximum possible value of this invariant among graphs on $n$ vertices with fixed number of blocks $p$. It is known that among graphs on $n$ vertices that have just one block, the $n$-cycle has the largest Wiener index. And the $n$-path, which has $n-1$ blocks, has the maximum Wiener index in the class of graphs on $n$ vertices. We show that among all graphs on $n$ vertices which have $p\ge 2$ blocks, the maximum Wiener index is attained by a graph composed of two cycles joined by a path (here we admit that one or both cycles can be replaced by a single edge, as in the case $p=n-1$ for example).
Object Exchangeability in Reinforcement Learning: Extended Abstract
Venue: arXiv cs.LG 2019
Year: 2019

Although deep reinforcement learning has advanced significantly over the past several years, sample efficiency remains a major challenge. Careful choice of input representations can help improve efficiency depending on the structure present in the problem. In this work, we present an attention-based method to project inputs into an efficient representation space that is invariant under changes to input ordering. We show that our proposed representation results in a search space that is a factor of m! smaller for inputs of m objects. Our experiments demonstrate improvements in sample efficiency for policy gradient methods on a variety of tasks. We show that our representation allows us to solve problems that are otherwise intractable when using naive approaches.
EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

Video restoration tasks, including super-resolution, deblurring, etc, are drawing increasing attention in the computer vision community. A challenging benchmark named REDS is released in the NTIRE19 Challenge. This new benchmark challenges existing methods from two aspects: (1) how to align multiple frames given large motions, and (2) how to effectively fuse different frames with diverse motion and blur. In this work, we propose a novel Video Restoration framework with Enhanced Deformable networks, termed EDVR, to address these challenges. First, to handle large motions, we devise a Pyramid, Cascading and Deformable (PCD) alignment module, in which frame alignment is done at the feature level using deformable convolutions in a coarse-to-fine manner. Second, we propose a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration. Thanks to these modules, our EDVR wins the champions and outperforms the second place by a large margin in all four tracks in the NTIRE19 video restoration and enhancement challenges. EDVR also demonstrates superior performance to state-of-the-art published methods on video super-resolution and deblurring. The code is available at https://github.com/xinntao/EDVR.
Collaborative and Privacy-Preserving Machine Teaching via Consensus Optimization
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we define a collaborative and privacy-preserving machine teaching paradigm with multiple distributed teachers. We focus on consensus super teaching. It aims at organizing distributed teachers to jointly select a compact while informative training subset from data hosted by the teachers to make a learner learn better. The challenges arise from three perspectives. First, the state-of-the-art pool-based super teaching method applies mixed-integer non-linear programming (MINLP) which does not scale well to very large data sets. Second, it is desirable to restrict data access of the teachers to only their own data during the collaboration stage to mitigate privacy leaks. Finally, the teaching collaboration should be communication-efficient since large communication overheads can cause synchronization delays between teachers. To address these challenges, we formulate collaborative teaching as a consensus and privacy-preserving optimization process to minimize teaching risk. We theoretically demonstrate the necessity of collaboration between teachers for improving the learner's learning. Furthermore, we show that the proposed method enjoys a similar property as the Oracle property of adaptive Lasso. The empirical study illustrates that our teaching method can deliver significantly more accurate teaching results with high speed, while the non-collaborative MINLP-based super teaching becomes prohibitively expensive to compute.
From GenderMag to InclusiveMag: An Inclusive Design Meta-Method
Venue: arXiv cs.HC 2019
Year: 2019

How can software practitioners assess whether their software supports diverse users? Although there are empirical processes that can be used to find "inclusivity bugs" piecemeal, what is often needed is a systematic inspection method to assess soft-ware's support for diverse populations. To help fill this gap, this paper introduces InclusiveMag, a generalization of GenderMag that can be used to generate systematic inclusiveness methods for a particular dimension of diversity. We then present a multi-case study covering eight diversity dimensions, of eight teams' experiences applying InclusiveMag to eight under-served populations and their "mainstream" counterparts.
Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Evaluation of deep reinforcement learning (RL) is inherently challenging. In particular, learned policies are largely opaque, and hypotheses about the behavior of deep RL agents are difficult to test in black-box environments. Considerable effort has gone into addressing opacity, but almost no effort has been devoted to producing high quality environments for experimental evaluation of agent behavior. We present TOYBOX, a new high-performance, open-source* subset of Atari environments re-designed for the experimental evaluation of deep RL. We show that TOYBOX enables a wide range of experiments and analyses that are impossible in other environments. *https://kdl-umass.github.io/Toybox/
A SAT-based System for Consistent Query Answering
Venue: arXiv cs.DB 2019
Year: 2019

An inconsistent database is a database that violates one or more integrity constraints, such as functional dependencies. Consistent Query Answering is a rigorous and principled approach to the semantics of queries posed against inconsistent databases. The consistent answers to a query on an inconsistent database is the intersection of the answers to the query on every repair, i.e., on every consistent database that differs from the given inconsistent one in a minimal way. Computing the consistent answers of a fixed conjunctive query on a given inconsistent database can be a coNP-hard problem, even though every fixed conjunctive query is efficiently computable on a given consistent database. We designed, implemented, and evaluated CAvSAT, a SAT-based system for consistent query answering. CAvSAT leverages a set of natural reductions from the complement of consistent query answering to SAT and to Weighted MaxSAT. The system is capable of handling unions of conjunctive queries and arbitrary denial constraints, which include functional dependencies as a special case. We report results from experiments evaluating CAvSAT on both synthetic and real-world databases. These results provide evidence that a SAT-based approach can give rise to a comprehensive and scalable system for consistent query answering.
Optimization Modulo the Theories of Signed Bit-Vectors and Floating-Point Numbers
Venue: arXiv cs.LO 2019
Year: 2019

Optimization Modulo Theories (OMT) is an important extension of SMT which allows for finding models that optimize given objective functions, typically consisting in linear-arithmetic or pseudo-Boolean terms. However, many SMT and OMT applications, in particular from SW and HW verification, require handling bit-precise representations of numbers, which in SMT are handled by means of the theory of Bit-Vectors (BV) for the integers and that of Floating-Point Numbers (FP) for the reals respectively. Whereas an approach for OMT with (unsigned) BV has been proposed by Nadel & Ryvchin, unfortunately we are not aware of any existing approach for OMT with FP. In this paper we fill this gap. We present a novel OMT approach, based on the novel concept of attractor and dynamic attractor, which extends the work of Nadel & Ryvchin to signed BV and, most importantly, to FP. We have implemented some OMT(BV) and OMT(FP) procedures on top of OptiMathSAT and tested the latter ones on modified problems from the SMT-LIB repository. The empirical results support the validity and feasibility of the novel approach.
SAWL:A Self-adaptive Wear-leveling NVM Scheme for High Performance Storage Systems
Venue: arXiv cs.AR 2019
Year: 2019

In order to meet the needs of high performance computing (HPC) in terms of large memory, high throughput and energy savings, the non-volatile memory (NVM) has been widely studied due to its salient features of high density, near-zero standby power, byte-addressable and non-volatile properties. In HPC systems, the multi-level cell (MLC) technique is used to significantly increase device density and decrease the cost, which however leads to much weaker endurance than the single-level cell (SLC) counterpart. Although wear-leveling techniques can mitigate this weakness in MLC, the improvements upon MLC-based NVM become very limited due to not achieving uniform write distribution before some cells are really worn out. To address this problem, our paper proposes a self-adaptive wear-leveling (SAWL) scheme for MLC-based NVM. The idea behind SAWL is to dynamically tune the wear-leveling granularities and balance the writes across the cells of entire memory, thus achieving suitable tradeoff between the lifetime and cache hit rate. Moreover, to reduce the size of the address-mapping table, SAWL maintains a few recently-accessed mappings in a small on-chip cache. Experimental results demonstrate that SAWL significantly improves the NVM lifetime and the performance for HPC systems, compared with state-of-the-art schemes.
Photometric Transformer Networks and Label Adjustment for Breast Density Prediction
Venue: arXiv cs.CV 2019
Year: 2019

Grading breast density is highly sensitive to normalization settings of digital mammogram as the density is tightly correlated with the distribution of pixel intensity. Also, the grade varies with readers due to uncertain grading criteria. These issues are inherent in the density assessment of digital mammography. They are problematic when designing a computer-aided prediction model for breast density and become worse if the data comes from multiple sites. In this paper, we proposed two novel deep learning techniques for breast density prediction: 1) photometric transformation which adaptively normalizes the input mammograms, and 2) label distillation which adjusts the label by using its output prediction. The photometric transformer network predicts optimal parameters for photometric transformation on the fly, learned jointly with the main prediction network. The label distillation, a type of pseudo-label techniques, is intended to mitigate the grading variation. We experimentally showed that the proposed methods are beneficial in terms of breast density prediction, resulting in significant performance improvement compared to various previous approaches.
Multi-class Twitter Data Categorization and Geocoding with a Novel Computing Framework
Venue: arXiv cs.SI 2019
Year: 2019

This study details the progress in transportation data analysis with a novel computing framework in keeping with the continuous evolution of the computing technology. The computing framework combines the Labelled Latent Dirichlet Allocation (L-LDA)-incorporated Support Vector Machine (SVM) classifier with the supporting computing strategy on publicly available Twitter data in determining transportation-related events to provide reliable information to travelers. The analytical approach includes analyzing tweets using text classification and geocoding locations based on string similarity. A case study conducted for the New York City and its surrounding areas demonstrates the feasibility of the analytical approach. Approximately 700,010 tweets are analyzed to extract relevant transportation-related information for one week. The SVM classifier achieves more than 85% accuracy in identifying transportation-related tweets from structured data. To further categorize the transportation-related tweets into sub-classes: incident, congestion, construction, special events, and other events, three supervised classifiers are used: L-LDA, SVM, and L-LDA incorporated SVM. Findings from this study demonstrate that the analytical framework, which uses the L-LDA incorporated SVM, can classify roadway transportation-related data from Twitter with over 98.3% accuracy, which is significantly higher than the accuracies achieved by standalone L-LDA and SVM.
Redesigning Telecommunication Engineering Courses with CDIO geared for Polytechnic Education
Venue: arXiv cs.CY 2019
Year: 2019

Whether in chemical, civil, mechanical, electrical, or their related engineering subdisciplines, remaining up-to-date in the subject matter is crucial. However, due to the pace of technological evolution, information and communications technology (ICT) fields of study are impacted with much higher consequences. Meanwhile, the curricula of higher educational institutes are struggling to catch up to this reality. In order to remain competitive, engineering schools ought to offer ICT related courses that are at once modern, relevant and ultimately beneficial for the employability of their graduates. In this spirit, we were recently mandated by our engineering school to develop and design telecommunication courses with great emphasis on (i) technological modernity, and (ii) experiential learning. To accomplish these objectives, we utilized the conceive, design, implement and operate (CDIO) framework, a modern engineering education initiative of which Sheridan is a member. In this article, we chronicle the steps we took to streamline and modernize the curriculum by outlining an effective methodology for course design and development with CDIO. We then provide examples of course update and design using the proposed methodology and highlight the lessons learned from this systematic curriculum development endeavor.
Applications of Social Media in Hydroinformatics: A Survey
Venue: arXiv cs.SI 2019
Year: 2019

Floods of research and practical applications employ social media data for a wide range of public applications, including environmental monitoring, water resource managing, disaster and emergency response.Hydroinformatics can benefit from the social media technologies with newly emerged data, techniques and analytical tools to handle large datasets, from which creative ideas and new values could be mined.This paper first proposes a 4W (What, Why, When, hoW) model and a methodological structure to better understand and represent the application of social media to hydroinformatics, then provides an overview of academic research of applying social media to hydroinformatics such as water environment, water resources, flood, drought and water Scarcity management. At last,some advanced topics and suggestions of water related social media applications from data collection, data quality management, fake news detection, privacy issues, algorithms and platforms was present to hydroinformatics managers and researchers based on previous discussion.
Adaptive Image-Feature Learning for Disease Classification Using Inductive Graph Networks
Venue: arXiv cs.LG 2019
Year: 2019

Recently, Geometric Deep Learning (GDL) has been introduced as a novel and versatile framework for computer-aided disease classification. GDL uses patient meta-information such as age and gender to model patient cohort relations in a graph structure. Concepts from graph signal processing are leveraged to learn the optimal mapping of multi-modal features, e.g. from images to disease classes. Related studies so far have considered image features that are extracted in a pre-processing step. We hypothesize that such an approach prevents the network from optimizing feature representations towards achieving the best performance in the graph network. We propose a new network architecture that exploits an inductive end-to-end learning approach for disease classification, where filters from both the CNN and the graph are trained jointly. We validate this architecture against state-of-the-art inductive graph networks and demonstrate significantly improved classification scores on a modified MNIST toy dataset, as well as comparable classification results with higher stability on a chest X-ray image dataset. Additionally, we explain how the structural information of the graph affects both the image filters and the feature learning.
The Guided Team-Partitioning Problem: Definition, Complexity, and Algorithm
Venue: arXiv cs.SI 2019
Year: 2019

A long line of literature has focused on the problem of selecting a team of individuals from a large pool of candidates, such that certain constraints are respected, and a given objective function is maximized. Even though extant research has successfully considered diverse families of objective functions and constraints, one of the most common limitations is the focus on the single-team paradigm. Despite its well-documented applications in multiple domains, this paradigm is not appropriate when the team-builder needs to partition the entire population into multiple teams. Team-partitioning tasks are very common in an educational setting, in which the teacher has to partition the students in her class into teams for collaborative projects. The task also emerges in the context of organizations, when managers need to partition the workforce into teams with specific properties to tackle relevant projects. In this work, we extend the team formation literature by introducing the Guided Team-Partitioning (GTP) problem, which asks for the partitioning of a population into teams such that the centroid of each team is as close as possible to a given target vector. As we describe in detail in our work, this formulation allows the team-builder to control the composition of the produced teams and has natural applications in practical settings. Algorithms for the GTP need to simultaneously consider the composition of multiple non-overlapping teams that compete for the same population of candidates. This makes the problem considerably more challenging than formulations that focus on the optimization of a single team. In fact, we prove that GTP is NP-hard to solve and even to approximate. The complexity of the problem motivates us to consider efficient algorithmic heuristics, which we evaluate via experiments on both real and synthetic datasets.
Multi-modal Graph Fusion for Inductive Disease Classification in Incomplete Datasets
Venue: arXiv cs.LG 2019
Year: 2019

Clinical diagnostic decision making and population-based studies often rely on multi-modal data which is noisy and incomplete. Recently, several works proposed geometric deep learning approaches to solve disease classification, by modeling patients as nodes in a graph, along with graph signal processing of multi-modal features. Many of these approaches are limited by assuming modality- and feature-completeness, and by transductive inference, which requires re-training of the entire model for each new test sample. In this work, we propose a novel inductive graph-based approach that can generalize to out-of-sample patients, despite missing features from entire modalities per patient. We propose multi-modal graph fusion which is trained end-to-end towards node-level classification. We demonstrate the fundamental working principle of this method on a simplified MNIST toy dataset. In experiments on medical data, our method outperforms single static graph approach in multi-modal disease classification.