OpenRaiser/Intern-Atlas huggingface dataset:
No Delay: Latency-Driven, Application Performance-Aware, Cluster Scheduling
Venue: arXiv cs.NI 2019
Year: 2019

Given the network latency variability observed in data centers, applications' performance is also determined by their placement within the data centre. We present NoMora, a cluster scheduling architecture whose core is represented by a latency-driven, application performance-aware, cluster scheduling policy. The policy places the tasks of an application taking into account the expected performance based on the measured network latency between pairs of hosts in the data center. Furthermore, if a tenant's application experiences increased network latency, and thus lower application performance, their application may be migrated to a better placement. Preliminary results show that our policy improves the overall average application performance by up to 13.4% and by up to 42% if preemption is enabled, and improves the task placement latency by a factor of 1.79x and the median algorithm runtime by 1.16x compared to a random policy on the Google cluster workload. This demonstrates that application performance can be improved by exploiting the relationship between network latency and application performance, and the current network conditions in a data center, while preserving the demands of low-latency cluster scheduling.
Stability of the Shannon-Stam inequality via the Föllmer process
Venue: arXiv cs.IT 2019
Year: 2019

We prove stability estimates for the Shannon-Stam inequality (also known as the entropy-power inequality) for log-concave random vectors in terms of entropy and transportation distance. In particular, we give the first stability estimate for general log-concave random vectors in the following form: for log-concave random vectors $X,Y \in \mathbb{R}^d$, the deficit in the Shannon-Stam inequality is bounded from below by the expression $$ C \left(\mathrm{D}\left(X||G\right) + \mathrm{D}\left(Y||G\right)\right), $$ where $\mathrm{D}\left( \cdot ~ ||G\right)$ denotes the relative entropy with respect to the standard Gaussian and the constant $C$ depends only on the covariance structures and the spectral gaps of $X$ and $Y$. In the case of uniformly log-concave vectors our analysis gives dimension-free bounds. Our proofs are based on a new approach which uses an entropy-minimizing process from stochastic control theory.
Proximal Splitting Networks for Image Restoration
Venue: arXiv cs.CV 2019
Year: 2019

Image restoration problems are typically ill-posed requiring the design of suitable priors. These priors are typically hand-designed and are fully instantiated throughout the process. In this paper, we introduce a novel framework for handling inverse problems related to image restoration based on elements from the half quadratic splitting method and proximal operators. Modeling the proximal operator as a convolutional network, we defined an implicit prior on the image space as a function class during training. This is in contrast to the common practice in literature of having the prior to be fixed and fully instantiated even during training stages. Further, we allow this proximal operator to be tuned differently for each iteration which greatly increases modeling capacity and allows us to reduce the number of iterations by an order of magnitude as compared to other approaches. Our final network is an end-to-end one whose run time matches the previous fastest algorithms while outperforming them in recovery fidelity on two image restoration tasks. Indeed, we find our approach achieves state-of-the-art results on benchmarks in image denoising and image super resolution while recovering more complex and finer details.
Modeling and Optimization of Human-machine Interaction Processes via the Maximum Entropy Principle
Venue: arXiv cs.LG 2019
Year: 2019

We propose a data-driven framework to enable the modeling and optimization of human-machine interaction processes, e.g., systems aimed at assisting humans in decision-making or learning, work-load allocation, and interactive advertising. This is a challenging problem for several reasons. First, humans' behavior is hard to model or infer, as it may reflect biases, long term memory, and sensitivity to sequencing, i.e., transience and exponential complexity in the length of the interaction. Second, due to the interactive nature of such processes, the machine policy used to engage with a human may bias possible data-driven inferences. Finally, in choosing machine policies that optimize interaction rewards, one must, on the one hand, avoid being overly sensitive to error/variability in the estimated human model, and on the other, being overly deterministic/predictable which may result in poor human 'engagement' in the interaction. To meet these challenges, we propose a robust approach, based on the maximum entropy principle, which iteratively estimates human behavior and optimizes the machine policy--Alternating Entropy-Reward Ascent (AREA) algorithm. We characterize AREA, in terms of its space and time complexity and convergence. We also provide an initial validation based on synthetic data generated by an established noisy nonlinear model for human decision-making.
Technical notes: Syntax-aware Representation Learning With Pointer Networks
Venue: arXiv cs.CL 2019
Year: 2019

This is a work-in-progress report, which aims to share preliminary results of a novel sequence-to-sequence schema for dependency parsing that relies on a combination of a BiLSTM and two Pointer Networks (Vinyals et al., 2015), in which the final softmax function has been replaced with the logistic regression. The two pointer networks co-operate to develop a latent syntactic knowledge, by learning the lexical properties of "selection" and the lexical properties of "selectability", respectively. At the moment and without fine-tuning, the parser implementation gets a UAS of 93.14% on the English Penn-treebank (Marcus et al., 1993) annotated with Stanford Dependencies: 2-3% under the SOTA but yet attractive as a baseline of the approach.
System Level Synthesis with State and Input Constraints
Venue: arXiv cs.SY 2019
Year: 2019

This paper addresses the problem of designing distributed controllers with state and input constraints in the System Level Synthesis (SLS) framework. Using robust optimization, we show how state and actuation constraints can be incorporated into the SLS structure. Moreover, we show that the dual variable associated with the constraint has the same sparsity pattern as the SLS parametrization, and therefore the computation distributes using a simple primal-dual algorithm. We provide a stability analysis for SLS design with input saturation under the Internal Model Control (IMC) framework. We show that the closed-loop system with saturation is stable if the controller has a gain less than one. In addition, a saturation compensation scheme that incorporates the saturation information is proposed which improves the naive SLS design under saturation.
On the Computation and Applications of Large Dense Partial Correlation Networks
Venue: arXiv cs.LG 2019
Year: 2019

While sparse inverse covariance matrices are very popular for modeling network connectivity, the value of the dense solution is often overlooked. In fact the L2-regularized solution has deep connections to a number of important applications to spectral graph theory, dimensionality reduction, and uncertainty quantification. We derive an approach to directly compute the partial correlations based on concepts from inverse problem theory. This approach also leads to new insights on open problems such as model selection and data preprocessing, as well as new approaches which relate the above application areas.
Expectation-Aware Planning: A Unifying Framework for Synthesizing and Executing Self-Explaining Plans for Human-Aware Planning
Venue: arXiv cs.AI 2019
Year: 2019

In this work, we present a new planning formalism called Expectation-Aware planning for decision making with humans in the loop where the human's expectations about an agent may differ from the agent's own model. We show how this formulation allows agents to not only leverage existing strategies for handling model differences but can also exhibit novel behaviors that are generated through the combination of these different strategies. Our formulation also reveals a deep connection to existing approaches in epistemic planning. Specifically, we show how we can leverage classical planning compilations for epistemic planning to solve Expectation-Aware planning problems. To the best of our knowledge, the proposed formulation is the first complete solution to decision-making in the presence of diverging user expectations that is amenable to a classical planning compilation while successfully combining previous works on explanation and explicability. We empirically show how our approach provides a computational advantage over existing approximate approaches that unnecessarily try to search in the space of models while also failing to facilitate the full gamut of behaviors enabled by our framework.
Generating Adversarial Examples With Conditional Generative Adversarial Net
Venue: arXiv cs.LG 2019
Year: 2019

Recently, deep neural networks have significant progress and successful application in various fields, but they are found vulnerable to attack instances, e.g., adversarial examples. State-of-art attack methods can generate attack images by adding small perturbation to the source image. These attack images can fool the classifier but have little impact to human. Therefore, such attack instances are difficult to generate by searching the feature space. How to design an effective and robust generating method has become a spotlight. Inspired by adversarial examples, we propose two novel generative models to produce adaptive attack instances directly, in which conditional generative adversarial network is adopted and distinctive strategy is designed for training. Compared with the common method, such as Fast Gradient Sign Method, our models can reduce the generating cost and improve robustness and has about one fifth running time for producing attack instance.
Effects of padding on LSTMs and CNNs
Venue: arXiv cs.LG 2019
Year: 2019

Long Short-Term Memory (LSTM) Networks and Convolutional Neural Networks (CNN) have become very common and are used in many fields as they were effective in solving many problems where the general neural networks were inefficient. They were applied to various problems mostly related to images and sequences. Since LSTMs and CNNs take inputs of the same length and dimension, input images and sequences are padded to maximum length while testing and training. This padding can affect the way the networks function and can make a great deal when it comes to performance and accuracies. This paper studies this and suggests the best way to pad an input sequence. This paper uses a simple sentiment analysis task for this purpose. We use the same dataset on both the networks with various padding to show the difference. This paper also discusses some preprocessing techniques done on the data to ensure effective analysis of the data.
Appearance-Based Gaze Estimation Using Dilated-Convolutions
Venue: arXiv cs.CV 2019
Year: 2019

Appearance-based gaze estimation has attracted more and more attention because of its wide range of applications. The use of deep convolutional neural networks has improved the accuracy significantly. In order to improve the estimation accuracy further, we focus on extracting better features from eye images. Relatively large changes in gaze angles may result in relatively small changes in eye appearance. We argue that current architectures for gaze estimation may not be able to capture such small changes, as they apply multiple pooling layers or other downsampling layers so that the spatial resolution of the high-level layers is reduced significantly. To evaluate whether the use of features extracted at high resolution can benefit gaze estimation, we adopt dilated-convolutions to extract high-level features without reducing spatial resolution. In cross-subject experiments on the Columbia Gaze dataset for eye contact detection and the MPIIGaze dataset for 3D gaze vector regression, the resulting Dilated-Nets achieve significant (up to 20.8%) gains when compared to similar networks without dilated-convolutions. Our proposed Dilated-Net achieves state-of-the-art results on both the Columbia Gaze and the MPIIGaze datasets.
Data-Enabled Predictive Control for Grid-Connected Power Converters
Venue: arXiv cs.SY 2019
Year: 2019

We apply a novel data-enabled predictive control (DeePC) algorithm in grid-connected power converters to perform safe and optimal control. Rather than a model, the DeePC algorithm solely needs input/output data measured from the unknown system to predict future trajectories. We show that the DeePC can eliminate undesired oscillations in a grid-connected power converter and stabilize an unstable system. However, the DeePC algorithm may suffer from poor scalability when applied in high-order systems. To this end, we present a finite-horizon output-based model predictive control (MPC) for grid-connected power converters, which uses an N-step auto-regressive-moving-average (ARMA) model for system representation. The ARMA model is identified via an N-step prediction error method (PEM) in a recursive way. We investigate the connection between the DeePC and the concatenated PEM-MPC method, and then analytically and numerically compare their closed-loop performance. Moreover, the PEM-MPC is applied in a voltage source converter based HVDC station which is connected to a two-area power system so as to eliminate low-frequency oscillations. All of our results are illustrated with high-fidelity, nonlinear, and noisy simulations.
Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Encoder-decoder models have become an effective approach for sequence learning tasks like machine translation, image captioning and speech recognition, but have yet to show competitive results for handwritten text recognition. To this end, we propose an attention-based sequence-to-sequence model. It combines a convolutional neural network as a generic feature extractor with a recurrent neural network to encode both the visual information, as well as the temporal context between characters in the input image, and uses a separate recurrent neural network to decode the actual character sequence. We make experimental comparisons between various attention mechanisms and positional encodings, in order to find an appropriate alignment between the input and output sequence. The model can be trained end-to-end and the optional integration of a hybrid loss allows the encoder to retain an interpretable and usable output, if desired. We achieve competitive results on the IAM and ICFHR2016 READ data sets compared to the state-of-the-art without the use of a language model, and we significantly improve over any recent sequence-to-sequence approaches.
Automatic Classification of Pathology Reports using TF-IDF Features
Venue: arXiv cs.CL 2019
Year: 2019

A Pathology report is arguably one of the most important documents in medicine containing interpretive information about the visual findings from the patient's biopsy sample. Each pathology report has a retention period of up to 20 years after the treatment of a patient. Cancer registries process and encode high volumes of free-text pathology reports for surveillance of cancer and tumor diseases all across the world. In spite of their extremely valuable information they hold, pathology reports are not used in any systematic way to facilitate computational pathology. Therefore, in this study, we investigate automated machine-learning techniques to identify/predict the primary diagnosis (based on ICD-O code) from pathology reports. We performed experiments by extracting the TF-IDF features from the reports and classifying them using three different methods---SVM, XGBoost, and Logistic Regression. We constructed a new dataset with 1,949 pathology reports arranged into 37 ICD-O categories, collected from four different primary sites, namely lung, kidney, thymus, and testis. The reports were manually transcribed into text format after collecting them as PDF files from NCI Genomic Data Commons public dataset. We subsequently pre-processed the reports by removing irrelevant textual artifacts produced by OCR software. The highest classification accuracy we achieved was 92\% using XGBoost classifier on TF-IDF feature vectors, the linear SVM scored 87\% accuracy. Furthermore, the study shows that TF-IDF vectors are suitable for highlighting the important keywords within a report which can be helpful for the cancer research and diagnostic workflow. The results are encouraging in demonstrating the potential of machine learning methods for classification and encoding of pathology reports.
Complexity-entropy analysis at different levels of organization in written language
Venue: arXiv cs.CL 2019
Year: 2019

Written language is complex. A written text can be considered an attempt to convey a meaningful message which ends up being constrained by language rules, context dependence and highly redundant in its use of resources. Despite all these constraints, unpredictability is an essential element of natural language. Here we present the use of entropic measures to assert the balance between predictability and surprise in written text. In short, it is possible to measure innovation and context preservation in a document. It is shown that this can also be done at the different levels of organization of a text. The type of analysis presented is reasonably general, and can also be used to analyze the same balance in other complex messages such as DNA, where a hierarchy of organizational levels are known to exist.
Path Planning in Dynamic Environments Using Time-Warped Grids and a Parallel Implementation
Venue: arXiv cs.RO 2019
Year: 2019

This paper proposes a solution to the problem of smooth path planning for mobile robots in dynamic and unknown environments. A novel concept of Time-Warped Grid is introduced to predict the pose of obstacles in the environment and avoid collisions. The algorithm is implemented using C/C++ and the CUDA programming environment, and combines stochastic estimation (Kalman filter), harmonic potential fields and a rubber band model, and it translates naturally into the parallel paradigm of GPU programming. In simple terms, time-warped grids are progressively wider orbits around the mobile robot. Those orbits represent the variable time intervals estimated by the robot to reach detected obstacles. The proposed method was tested using several simulation scenarios for the Pioneer P3-DX robot, which demonstrated the robustness of the algorithm by finding the optimum path in terms of smoothness, distance, and collision-free, in both static or dynamic environments, and with large number of obstacles.
Ternary Syndrome Decoding with Large Weight
Venue: arXiv cs.CR 2019
Year: 2019

The Syndrome Decoding problem is at the core of many code-based cryptosystems. In this paper, we study ternary Syndrome Decoding in large weight. This problem has been introduced in the Wave signature scheme but has never been thoroughly studied. We perform an algorithmic study of this problem which results in an update of the Wave parameters. On a more fundamental level, we show that ternary Syndrome Decoding with large weight is a really harder problem than the binary Syndrome Decoding problem, which could have several applications for the design of code-based cryptosystems.
Teaching DevOps in academia and industry: reflections and vision
Venue: arXiv cs.SE 2019
Year: 2019

This paper describes our experience of delivery educational programs in academia and in industry on DevOps, compare the two approaches and sum-up the lessons learnt. We also propose a vision to implement a shift in the Software Engineering Higher Education curricula.
Visual Cue Integration for Small Target Motion Detection in Natural Cluttered Backgrounds
Venue: arXiv cs.CV 2019
Year: 2019

The robust detection of small targets against cluttered background is important for future artificial visual systems in searching and tracking applications. The insects' visual systems have demonstrated excellent ability to avoid predators, find prey or identify conspecifics - which always appear as small dim speckles in the visual field. Build a computational model of the insects' visual pathways could provide effective solutions to detect small moving targets. Although a few visual system models have been proposed, they only make use of small-field visual features for motion detection and their detection results often contain a number of false positives. To address this issue, we develop a new visual system model for small target motion detection against cluttered moving backgrounds. Compared to the existing models, the small-field and wide-field visual features are separately extracted by two motion-sensitive neurons to detect small target motion and background motion. These two types of motion information are further integrated to filter out false positives. Extensive experiments showed that the proposed model can outperform the existing models in terms of detection rates.
On the Capacity of Computation Broadcast
Venue: arXiv cs.IT 2019
Year: 2019

The two-user computation broadcast problem is introduced as the setting where User $1$ wants message $W_1$ and has side-information $W_1'$, User $2$ wants message $W_2$ and has side-information $W_2'$, and $(W_1, W_1', W_2, W_2')$ may have arbitrary dependencies. The rate of a computation broadcast scheme is defined as the ratio $H(W_1,W_2)/H(S)$, where $S$ is the information broadcast to both users to simultaneously satisfy their demands. The supremum of achievable rates is called the capacity of computation broadcast $C_{CB}$. It is shown that $C_{CB}\leq H(W_1,W_2)/\left[H(W_1|W_1')+H(W_2|W_2')-\min\Big(I(W_1; W_2, W_2'|W_1'), I(W_2; W_1, W_1'|W_2')\Big)\right]$. For the linear computation broadcast problem, where $W_1, W_1', W_2, W_2'$ are comprised of arbitrary linear combinations of a basis set of independent symbols, the bound is shown to be tight. For non-linear computation broadcast, it is shown that this bound is not tight in general. Examples are provided to prove that different instances of computation broadcast that have the same entropic structure, i.e., the same entropy for all subsets of $\{W_1,W_1',W_2,W_2'\}$, can have different capacities. Thus, extra-entropic structure matters even for two-user computation broadcast. The significance of extra-entropic structure is further explored through a class of non-linear computation broadcast problems where the extremal values of capacity are shown to correspond to minimally and maximally structured problems within that class.
Multi-Differential Fairness Auditor for Black Box Classifiers
Venue: arXiv cs.LG 2019
Year: 2019

Machine learning algorithms are increasingly involved in sensitive decision-making process with adversarial implications on individuals. This paper presents mdfa, an approach that identifies the characteristics of the victims of a classifier's discrimination. We measure discrimination as a violation of multi-differential fairness. Multi-differential fairness is a guarantee that a black box classifier's outcomes do not leak information on the sensitive attributes of a small group of individuals. We reduce the problem of identifying worst-case violations to matching distributions and predicting where sensitive attributes and classifier's outcomes coincide. We apply mdfa to a recidivism risk assessment classifier and demonstrate that individuals identified as African-American with little criminal history are three-times more likely to be considered at high risk of violent recidivism than similar individuals but not African-American.