OpenRaiser/Intern-Atlas huggingface dataset:
X-Ray Image Compression Using Convolutional Recurrent Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

In the advent of a digital health revolution, vast amounts of clinical data are being generated, stored and processed on a daily basis. This has made the storage and retrieval of large volumes of health-care data, especially, high-resolution medical images, particularly challenging. Effective image compression for medical images thus plays a vital role in today's healthcare information system, particularly in teleradiology. In this work, an X-ray image compression method based on a Convolutional Recurrent Neural Networks RNN-Conv is presented. The proposed architecture can provide variable compression rates during deployment while it requires each network to be trained only once for a specific dimension of X-ray images. The model uses a multi-level pooling scheme that learns contextualized features for effective compression. We perform our image compression experiments on the National Institute of Health (NIH) ChestX-ray8 dataset and compare the performance of the proposed architecture with a state-of-the-art RNN based technique and JPEG 2000. The experimental results depict improved compression performance achieved by the proposed method in terms of Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) metrics. To the best of our knowledge, this is the first reported evaluation on using a deep convolutional RNN for medical image compression.
Periodic Bandits and Wireless Network Selection
Venue: arXiv cs.NI 2019
Year: 2019

Bandit-style algorithms have been studied extensively in stochastic and adversarial settings. Such algorithms have been shown to be useful in multiplayer settings, e.g. to solve the wireless network selection problem, which can be formulated as an adversarial bandit problem. A leading bandit algorithm for the adversarial setting is EXP3. However, network behavior is often repetitive, where user density and network behavior follow regular patterns. Bandit algorithms, like EXP3, fail to provide good guarantees for periodic behaviors. A major reason is that these algorithms compete against fixed-action policies, which is ineffective in a periodic setting. In this paper, we define a periodic bandit setting, and periodic regret as a better performance measure for this type of setting. Instead of comparing an algorithm's performance to fixed-action policies, we aim to be competitive with policies that play arms under some set of possible periodic patterns $F$ (for example, all possible periodic functions with periods $1,2,\cdots,P$). We propose Periodic EXP4, a computationally efficient variant of the EXP4 algorithm for periodic settings. With $K$ arms, $T$ time steps, and where each periodic pattern in $F$ is of length at most $P$, we show that the periodic regret obtained by Periodic EXP4 is at most $O\big(\sqrt{PKT \log K + KT \log |F|}\big)$. We also prove a lower bound of $Ω\big(\sqrt{PKT + KT \frac{\log |F|}{\log K}} \big)$ for the periodic setting, showing that this is optimal within log-factors. As an example, we focus on the wireless network selection problem. Through simulation, we show that Periodic EXP4 learns the periodic pattern over time, adapts to changes in a dynamic environment, and far outperforms EXP3.
Vector Autoregressive POMDP Model Learning and Planning for Human-Robot Collaboration
Venue: arXiv cs.RO 2019
Year: 2019

Human-robot collaboration (HRC) has emerged as a hot research area at the intersection of control, robotics, and psychology in recent years. It is of critical importance to obtain an expressive but meanwhile tractable model for human beings in HRC. In this paper, we propose a model called Vector Autoregressive POMDP (VAR-POMDP) model which is an extension of the traditional POMDP model by considering the correlation among observations. The VAR-POMDP model is more powerful in the expressiveness of features than the traditional continuous observation POMDP since the traditional one is a special case of the VAR-POMDP model. Meanwhile, the proposed VAR-POMDP model is also tractable, as we show that it can be effectively learned from data and we can extend point-based value iteration (PBVI) to VAR-POMDP planning. Particularly, in this paper, we propose to use the Bayesian non-parametric learning to decide potential human states and learn a VAR-POMDP model using data collected from human demonstrations. Then, we consider planning with respect to PCTL which is widely used as safety and reachability requirement in robotics. Finally, the advantage of using the proposed model for HRC is validated by experimental results using data collected from a driver-assistance test-bed.
Softmax Optimizations for Intel Xeon Processor-based Platforms
Venue: arXiv cs.MS 2019
Year: 2019

Softmax is popular normalization method used in machine learning. Deep learning solutions like Transformer or BERT use the softmax function intensively, so it is worthwhile to optimize its performance. This article presents our methodology of optimization and its results applied to softmax. By presenting this methodology, we hope to increase an interest in deep learning optimizations for CPUs. We believe that the optimization process presented here could be transferred to other deep learning frameworks such as TensorFlow or PyTorch.
Enhancing Prediction Models for One-Year Mortality in Patients with Acute Myocardial Infarction and Post Myocardial Infarction Syndrome
Venue: arXiv cs.LG 2019
Year: 2019

Predicting the risk of mortality for patients with acute myocardial infarction (AMI) using electronic health records (EHRs) data can help identify risky patients who might need more tailored care. In our previous work, we built computational models to predict one-year mortality of patients admitted to an intensive care unit (ICU) with AMI or post myocardial infarction syndrome. Our prior work only used the structured clinical data from MIMIC-III, a publicly available ICU clinical database. In this study, we enhanced our work by adding the word embedding features from free-text discharge summaries. Using a richer set of features resulted in significant improvement in the performance of our deep learning models. The average accuracy of our deep learning models was 92.89% and the average F-measure was 0.928. We further reported the impact of different combinations of features extracted from structured and/or unstructured data on the performance of the deep learning models.
Application of Autoencoder-Assisted Recurrent Neural Networks to Prevent Cases of Sudden Infant Death Syndrome
Venue: arXiv cs.LG 2019
Year: 2019

This project develops and trains a Recurrent Neural Network (RNN) that monitors sleeping infants from an auxiliary microphone for cases of Sudden Infant Death Syndrome (SIDS), manifested in sudden or gradual respiratory arrest. To minimize invasiveness and maximize economic viability, an electret microphone, and parabolic concentrator, paired with a specially designed and tuned amplifier circuit, was used as a very sensitive audio monitoring device, which fed data to the RNN model. This RNN was trained and operated in the frequency domain, where the respiratory activity is most unique from noise. In both training and operation, a Fourier transform and an autoencoder compression were applied to the raw audio, and this transformed audio data was fed into the model in 1/8 second time steps. In operation, this model flagged each perceived breath, and the time between breaths was analyzed through a statistical T-test for slope, which detected dangerous trends. The entire model achieved 92.5% accuracy on continuous data and had an 11.25-second response rate on data that emulated total respiratory arrest. Because of the compatibility of the trained model with many off-the-shelf devices like Android phones and Raspberry Pi's, free-standing processing hardware deployment is a very feasible future goal.
Low-complexity control algorithm for decentralised demand response using thermostatic loads
Venue: arXiv cs.SY 2019
Year: 2019

Thermostatically controlled loads such as refrigerators are exceptionally suitable as a flexible demand resource. This paper derives a decentralised load control algorithm for refrigerators. It is adapted from an existing continuous time control approach, with the aim to achieve low computational complexity and an ability to handle discrete time steps of variable length -- desirable features for embedding in appliances and high-throughput simulations. Simulation results of large populations of heterogeneous appliances illustrate the accurate aggregate control of power consumption and high computational efficiency.
A convolution recurrent autoencoder for spatio-temporal missing data imputation
Venue: arXiv cs.LG 2019
Year: 2019

When sensors collect spatio-temporal data in a large geographical area, the existence of missing data cannot be escaped. Missing data negatively impacts the performance of data analysis and machine learning algorithms. In this paper, we study deep autoencoders for missing data imputation in spatio-temporal problems. We propose a convolution bidirectional-LSTM for capturing spatial and temporal patterns. Moreover, we analyze an autoencoder's latent feature representation in spatio-temporal data and illustrate its performance for missing data imputation. Traffic flow data are used for evaluation of our models. The result shows that the proposed convolution recurrent neural network outperforms state-of-the-art methods.
A Fast Scalable Heuristic for Bin Packing
Venue: arXiv cs.DS 2019
Year: 2019

In this paper we present a fast scalable heuristic for bin packing that partitions the given problem into identical sub-problems of constant size and solves these constant size sub-problems by considering only a constant number of bin configurations with bounded unused space. We present some empirical evidence to support the scalability of our heuristic and its tighter empirical analysis of hard instances due to improved lower bound on the necessary wastage in an optimal solution.
Over-the-Air Computation via Intelligent Reflecting Surfaces
Venue: arXiv cs.IT 2019
Year: 2019

Over-the-air computation (AirComp) becomes a promising approach for fast wireless data aggregation via exploiting the superposition property in a multiple access channel. To further overcome the unfavorable signal propagation conditions for AirComp, in this paper, we propose an intelligent reflecting surface (IRS) aided AirComp system to build controllable wireless environments, thereby boosting the received signal power significantly. This is achieved by smartly tuning the phase shifts for the incoming electromagnetic waves at IRS, resulting in reconfigurable signal propagations. Unfortunately, it turns out that the joint design problem for AirComp transceivers and IRS phase shifts becomes a highly intractable nonconvex bi-quadratic programming problem, for which a novel alternating difference-of-convex (DC) programming algorithm is developed. This is achieved by providing a novel DC function representation for the rank-one constraint in the low-rank matrix optimization problem via matrix lifting. Simulation results demonstrate the algorithmic advantages and admirable performance of the proposed approaches compared with the state-of-art solutions.
Self-Attention Capsule Networks for Object Classification
Venue: arXiv cs.CV 2019
Year: 2019

We propose a novel architecture for object classification, called Self-Attention Capsule Networks (SACN). SACN is the first model that incorporates the Self-Attention mechanism as an integral layer within the Capsule Network (CapsNet). While the Self-Attention mechanism supplies a long-range dependencies, results in selecting the more dominant image regions to focus on, the CapsNet analyzes the relevant features and their spatial correlations inside these regions only. The features are extracted in the convolutional layer. Then, the Self-Attention layer learns to suppress irrelevant regions based on features analysis and highlights salient features useful for a specific task. The attention map is then fed into the CapsNet primary layer that is followed by a classification layer. The proposed SACN model was designed to solve two main limitations of the baseline CapsNet - analysis of complex data and significant computational load. In this work, we use a shallow CapsNet architecture and compensates for the absence of a deeper network by using the Self-Attention module to significantly improve the results. The proposed Self-Attention CapsNet architecture was extensively evaluated on six different datasets, mainly on three different medical sets, in addition to the natural MNIST, SVHN and CIFAR10. The model was able to classify images and their patches with diverse and complex backgrounds better than the baseline CapsNet. As a result, the proposed Self-Attention CapsNet significantly improved classification performance within and across different datasets and outperformed the baseline CapsNet, ResNet-18 and DenseNet-40 not only in classification accuracy but also in robustness.
ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classification
Venue: arXiv cs.LG 2019
Year: 2019

Training deep neural networks often requires careful hyper-parameter tuning and significant computational resources. In this paper, we propose ConvTimeNet (CTN): an off-the-shelf deep convolutional neural network (CNN) trained on diverse univariate time series classification (TSC) source tasks. Once trained, CTN can be easily adapted to new TSC target tasks via a small amount of fine-tuning using labeled instances from the target tasks. We note that the length of convolutional filters is a key aspect when building a pre-trained model that can generalize to time series of different lengths across datasets. To achieve this, we incorporate filters of multiple lengths in all convolutional layers of CTN to capture temporal features at multiple time scales. We consider all 65 datasets with time series of lengths up to 512 points from the UCR TSC Benchmark for training and testing transferability of CTN: We train CTN on a randomly chosen subset of 24 datasets using a multi-head approach with a different softmax layer for each training dataset, and study generalizability and transferability of the learned filters on the remaining 41 TSC datasets. We observe significant gains in classification accuracy as well as computational efficiency when using pre-trained CTN as a starting point for subsequent task-specific fine-tuning compared to existing state-of-the-art TSC approaches. We also provide qualitative insights into the working of CTN by: i) analyzing the activations and filters of first convolution layer suggesting the filters in CTN are generically useful, ii) analyzing the impact of the design decision to incorporate multiple length decisions, and iii) finding regions of time series that affect the final classification decision via occlusion sensitivity analysis.
Current Trends in the Use of Eye Tracking in Mathematics Education Research: A PME Survey
Venue: arXiv cs.HC 2019
Year: 2019

Eye tracking (ET) is a research method that receives growing interest in mathematics education research (MER). This paper aims to give a literature overview, specifically focusing on the evolution of interest in this technology, ET equipment, and analysis methods used in mathematics education. To capture the current state, we focus on papers published in the proceedings of PME, one of the primary conferences dedicated to MER, of the last ten years. We identify trends in interest, methodology, and methods of analysis that are used in the community, and discuss possible future developments.
Text Classification Components for Detecting Descriptions and Names of CAD models
Venue: arXiv cs.IR 2019
Year: 2019

We apply text analysis approaches for a specialized search engine for 3D CAD models and associated products. The main goals are to distinguish between actual product descriptions and other text on a website, as well as to decide whether a given text is or contains a product name. For this we use paragraph vectors for text classification, a character-level long short-term memory network (LSTM) for a single word classification and an LSTM tagger based on word embeddings for detecting product names within sentences. Despite the need to collect bigger datasets in our specific problem domain, the first results are promising and partially fit for production use.
MANA for MPI: MPI-Agnostic Network-Agnostic Transparent Checkpointing
Venue: arXiv cs.DC 2019
Year: 2019

Transparently checkpointing MPI for fault tolerance and load balancing is a long-standing problem in HPC. The problem has been complicated by the need to provide checkpoint-restart services for all combinations of an MPI implementation over all network interconnects. This work presents MANA (MPI-Agnostic Network-Agnostic transparent checkpointing), a single code base which supports all MPI implementation and interconnect combinations. The agnostic properties imply that one can checkpoint an MPI application under one MPI implementation and perhaps over TCP, and then restart under a second MPI implementation over InfiniBand on a cluster with a different number of CPU cores per node. This technique is based on a novel "split-process" approach, which enables two separate programs to co-exist within a single process with a single address space. This work overcomes the limitations of the two most widely adopted transparent checkpointing solutions, BLCR and DMTCP/InfiniBand, which require separate modifications to each MPI implementation and/or underlying network API. The runtime overhead is found to be insignificant both for checkpoint-restart within a single host, and when comparing a local MPI computation that was migrated to a remote cluster against an ordinary MPI computation running natively on that same remote cluster.
2D and 3D Computer Graphics Algorithms under MorphoSys
Venue: arXiv cs.DC 2019
Year: 2019

This paper presents new mappings of 2D and 3D geometrical transformation on the MorphoSys (M1) reconfigurable computing (RC) prototype [2]. This improves the system performance as a graphics accelerator [1-5]. Three algorithms are mapped including two for calculating 2D transformations, and one for 3D transformations. The results presented indicate an improved performance. The speedup achieved is explained as well as the advantages in the mapping of the application. The transformations on an 8x8 RC array were run, and numerical examples were simulated to validate our results, using the MorphoSys mULATE program, which simulates MorphoSys operations. Comparisons with other systems are presented, namely, with Intel processing systems and Celoxica RC-1000 FPGA.
AnonymousNet: Natural Face De-Identification with Measurable Privacy
Venue: arXiv cs.CV 2019
Year: 2019

With billions of personal images being generated from social media and cameras of all sorts on a daily basis, security and privacy are unprecedentedly challenged. Although extensive attempts have been made, existing face image de-identification techniques are either insufficient in photo-reality or incapable of balancing privacy and usability qualitatively and quantitatively, i.e., they fail to answer counterfactual questions such as "is it private now?", "how private is it?", and "can it be more private?" In this paper, we propose a novel framework called AnonymousNet, with an effort to address these issues systematically, balance usability, and enhance privacy in a natural and measurable manner. The framework encompasses four stages: facial attribute estimation, privacy-metric-oriented face obfuscation, directed natural image synthesis, and adversarial perturbation. Not only do we achieve the state-of-the-arts in terms of image quality and attribute prediction accuracy, we are also the first to show that facial privacy is measurable, can be factorized, and accordingly be manipulated in a photo-realistic fashion to fulfill different requirements and application scenarios. Experiments further demonstrate the effectiveness of the proposed framework.
Exploring the Daschle Collection using Text Mining
Venue: arXiv cs.IR 2019
Year: 2019

A U.S. Senator from South Dakota donated documents that were accumulated during his service as a house representative and senator to be housed at the Bridges library at South Dakota State University. This project investigated the utility of quantitative statistical methods to explore some portions of this vast document collection. The available scanned documents and emails from constituents are analyzed using natural language processing methods including the Latent Dirichlet Allocation (LDA) model. This model identified major topics being discussed in a given collection of documents. Important events and popular issues from the Senator Daschles career are reflected in the changing topics from the model. These quantitative statistical methods provide a summary of the massive amount of text without requiring significant human effort or time and can be applied to similar collections.
Crowd Management in Open Spaces
Venue: arXiv cs.CV 2019
Year: 2019

Crowd analysis and management is a challenging problem to ensure public safety and security. For this purpose, many techniques have been proposed to cope with various problems. However, the generalization capabilities of these techniques is limited due to ignoring the fact that the density of crowd changes from low to extreme high depending on the scene under observation. We propose robust feature based approach to deal with the problem of crowd management for people safety and security. We have evaluated our method using a benchmark dataset and have presented details analysis.
Brewers Conjecture and a characterization of the limits, and relationships between Consistency, Availability and Partition Tolerance in a distributed service
Venue: arXiv cs.DC 2019
Year: 2019

In designing a distributed service, three desirable attributes are Consistency, Availability and Partition Tolerance. In this note we explore a framework for characterizing these three in a manner that establishes definite limits and relationships between them, and explore some implications of this characterization.
Learning from Sets of Items in Recommender Systems
Venue: arXiv cs.IR 2019
Year: 2019

Most of the existing recommender systems use the ratings provided by users on individual items. An additional source of preference information is to use the ratings that users provide on sets of items. The advantages of using preferences on sets are two-fold. First, a rating provided on a set conveys some preference information about each of the set's items, which allows us to acquire a user's preferences for more items that the number of ratings that the user provided. Second, due to privacy concerns, users may not be willing to reveal their preferences on individual items explicitly but may be willing to provide a single rating to a set of items, since it provides some level of information hiding. This paper investigates two questions related to using set-level ratings in recommender systems. First, how users' item-level ratings relate to their set-level ratings. Second, how collaborative filtering-based models for item-level rating prediction can take advantage of such set-level ratings. We have collected set-level ratings from active users of Movielens on sets of movies that they have rated in the past. Our analysis of these ratings shows that though the majority of the users provide the average of the ratings on a set's constituent items as the rating on the set, there exists a significant number of users that tend to consistently either under- or over-rate the sets. We have developed collaborative filtering-based methods to explicitly model these user behaviors that can be used to recommend items to users. Experiments on real data and on synthetic data that resembles the under- or over-rating behavior in the real data, demonstrate that these models can recover the overall characteristics of the underlying data and predict the user's ratings on individual items.