pwc-archive/papers-with-abstracts huggingface dataset:
Automated Flower Classification over a Large Number of Classes
Authors: Maria-Elena Nilsback, Andrew Zisserman
Date: 2008-12-01 00:00:00
Tasks: Classification, General Classification

We investigate to what extent combinations of features can improve classification performance on a large dataset of similar classes. To this end we introduce a 103 class flower dataset. We compute four different features for the flowers, each describing different aspects, namely the local shape/texture, the shape of the boundary, the overall spatial distribution of petals, and the colour. We combine the features using a multiple kernel framework with a SVM classifier. The weights for each class are learnt using the method of Varma and Ray [16], which has achieved state of the art performance on other large dataset, such as Caltech 101/256. Our dataset has a similar challenge in the number of classes, but with the added difficulty of large between class similarity and small within class similarity. Results show that learning the optimum kernel combination of multiple features vastly improves the performance, from 55.1% for the best single feature to 72.8% for the combination of all features.
Pointwise shape-adaptive DCT for high-quality deblocking of compressed color images
Authors: Alessandro Foi, Vladimir Katkovnik, Karen Egiazarian
Date: 2006-09-04 00:00:00
Tasks: Blocking, Deblurring, Denoising, Image Deblocking

We present an high-quality image deblocking algorithm based on the shape-adaptive DCT (SA-DCT). The SA-DCT is a low-complexity transform which can be computed on a support of arbitrary shape. This transform has been adopted by the MPEG-4 standard and it is found implemented in modern video hardware. The use of this shape-adaptive transform for denoising and deblurring has been recently proposed, showing a remarkable performance. In this paper we discuss and analyze the use of this approach for the deblocking of block-DCT compressed images. Particular emphasis is given to the deblocking of highly-compressed color images. Extensive simulation experiments attest the advanced performance of the proposed filtering method. The visual quality of the estimates is high, with sharp detail preservation, clean edges. Blocking and ringing artifacts are suppressed while salient image features are preserved. The SA-DCT filtering used for the chrominance channels allows to faithfully reconstruct the missing structural information of the chrominances, thus correcting color-bleeding artifacts.
Enhancement of Distribution System State Estimation Using Pruned Physics-Aware Neural Networks
Authors: Minh-Quan Tran, Ahmed S. Zamzam, Phuong H. Nguyen
Date: 2021-02-07 00:00:00
Tasks: State Estimation

Realizing complete observability in the three-phase distribution system remains a challenge that hinders the implementation of classic state estimation algorithms. In this paper, a new method, called the pruned physics-aware neural network (P2N2), is developed to improve the voltage estimation accuracy in the distribution system. The method relies on the physical grid topology, which is used to design the connections between different hidden layers of a neural network model. To verify the proposed method, a numerical simulation based on one-year smart meter data of load consumptions for three-phase power flow is developed to generate the measurement and voltage state data. The IEEE 123-node system is selected as the test network to benchmark the proposed algorithm against the classic weighted least squares (WLS). Numerical results show that P2N2 outperforms WLS in terms of data redundancy and estimation accuracy.
Coupled Contagion: A Two-Fears Epidemic Model
Authors: Joshua M. Epstein, Erez Hatna, Jennifer Crodelle
Date: 2021-02-22 00:00:00
Tasks: model, Vocal Bursts Valence Prediction

We present a differential equations model in which contagious disease transmission is affected by contagious fear of the disease and contagious fear of the control, in this case vaccine. The three contagions are coupled. The two fears evolve and interact in ways that shape distancing behavior, vaccine uptake, and their relaxation. These behavioral dynamics in turn can amplify or suppress disease transmission, which feeds back to affect behavior. The model reveals several coupled contagion mechanisms for multiple epidemic waves. Methodologically, the paper advances infectious disease modeling by including human behavioral adaptation, drawing on the neuroscience of fear learning, extinction, and transmission.
AutoKWS: Keyword Spotting with Differentiable Architecture Search
Authors: Bo Zhang, Wenfeng Li, Qingyuan Li, Weiji Zhuang, Xiangxiang Chu, Yujun Wang
Date: 2020-09-08 00:00:00
Tasks: Keyword Spotting, Neural Architecture Search

Smart audio devices are gated by an always-on lightweight keyword spotting program to reduce power consumption. It is however challenging to design models that have both high accuracy and low latency for accurate and fast responsiveness. Many efforts have been made to develop end-to-end neural networks, in which depthwise separable convolutions, temporal convolutions, and LSTMs are adopted as building units. Nonetheless, these networks designed with human expertise may not achieve an optimal trade-off in an expansive search space. In this paper, we propose to leverage recent advances in differentiable neural architecture search to discover more efficient networks. Our searched model attains 97.2% top-1 accuracy on Google Speech Command Dataset v1 with only nearly 100K parameters.
Interpretation of the Area Under the ROC Curve for Risk Prediction Models
Authors: Ralph H. Stern
Date: 2021-02-22 00:00:00
Tasks: Diagnostic, Prediction

The area under the curve (AUC) of the receiver operating characteristics curve (ROC) evaluates the separation between patients and nonpatients or discrimination. For risk prediction models these risk distributions can be derived from the population risk distribution so are not independent as in diagnosis. A ROC curve AUC formula based on the underlying population risk distribution clarifies how discrimination is defined mathematically and that generation of the equivalent c-statistic effects a Monte Carlo integration of the formula. For a selection of continuous risk distributions, exact analytic formulas or numerical results for the ROC curve AUC and overlap measure are presented and demonstrate a linear or near-linear dependence on their standard deviation. The ROC curve AUC is also shown to be highly dependent on the mean population risk, a distinction from the independence from disease prevalence for diagnostic tests. The converse of discrimination, overlap, has been quantified by the overlap measure, which appears to provide equivalent information. As achieving wider population risk distributions is the goal of risk prediction modeling for clinical risk stratification, interpreting the ROC curve AUC as a measure of dispersion, rather than discrimination, when comparing risk prediction models may be more relevant.
Backward Simulation for Sets of Trajectories
Authors: Yuxuan Xia, Lennart Svensson, Ángel F. García-Fernández, Karl Granström, Jason L. Williams
Date: 2020-08-05 00:00:00

This paper presents a solution for recovering full trajectory information, via the calculation of the posterior of the set of trajectories, from a sequence of multitarget (unlabelled) filtering densities and the multitarget dynamic model. Importantly, the proposed solution opens an avenue of trajectory estimation possibilities for multitarget filters that do not explicitly estimate trajectories. In this paper, we first derive a general multitrajectory forward-backward smoothing equation based on sets of trajectories and the random finite set framework. Then we show how to sample sets of trajectories using backward simulation when the multitarget filtering densities are multi-Bernoulli processes. The proposed approach is demonstrated in a simulation study.
A prognostic dynamic model applicable to infectious diseases providing easily visualized guides -- A case study of COVID-19 in the UK
Authors: Yuxuan Zhang, Chen Gong, Dawei Li, Zhi-Wei Wang, Shengda D Pu, Alex W Robertson, Hong Yu, John Parrington
Date: 2020-12-14 00:00:00

A reasonable prediction of infectious diseases transmission process under different disease control strategies is an important reference point for policy makers. Here we established a dynamic transmission model via Python and realized comprehensive regulation of disease control measures. We classified government interventions into three categories and introduced three parameters as descriptions for the key points in disease control, these being intraregional growth rate, interregional communication rate, and detection rate of infectors. Our simulation predicts the infection by COVID-19 in the UK would be out of control in 73 days without any interventions; at the same time, herd immunity acquisition will begin from the epicentre. After we introduced government interventions, single intervention is effective in disease control but at huge expense while combined interventions would be more efficient, among which, enhancing detection number is crucial in control strategy of COVID-19. In addition, we calculated requirements for the most effective vaccination strategy based on infection number in real situation. Our model was programmed with iterative algorithms, and visualized via cellular automata, it can be applied to similar epidemics in other regions if the basic parameters are inputted, and is able to synthetically mimick the effect of multiple factors in infectious disease control.
A Multi-task Network to Detect Junctions in Retinal Vasculature
Date: 2018-06-06 00:00:00
Tasks: Junction Detection

Junctions in the retinal vasculature are key points to be able to extract its topology, but they vary in appearance, depending on vessel density, width and branching/crossing angles. The complexity of junction patterns is usually accompanied by a scarcity of labels, which discourages the usage of very deep networks for their detection. We propose a multi-task network, generating labels for vessel interior, centerline, edges and junction patterns, to provide additional information to facilitate junction detection. After the initial detection of potential junctions in junction-selective probability maps, candidate locations are re-examined in centerline probability maps to verify if they connect at least 3 branches. The experiments on the DRIVE and IOSTAR showed that our method outperformed a recent study in which a popular deep network was trained as a classifier to find junctions. Moreover, the proposed approach is applicable to unseen datasets with the same degree of success, after training it only once.
Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
Authors: Benjamin Hawks, Javier Duarte, Nicholas J. Fraser, Alessandro Pappalardo, Nhan Tran, Yaman Umuroglu
Date: 2021-02-22 00:00:00
Tasks: Bayesian Optimization, Computational Efficiency, Neural Architecture Search, Quantization

Efficient machine learning implementations optimized for inference in hardware have wide-ranging benefits, depending on the application, from lower inference latency to higher data throughput and reduced energy consumption. Two popular techniques for reducing computation in neural networks are pruning, removing insignificant synapses, and quantization, reducing the precision of the calculations. In this work, we explore the interplay between pruning and quantization during the training of neural networks for ultra low latency applications targeting high energy physics use cases. Techniques developed for this study have potential applications across many other domains. We study various configurations of pruning during quantization-aware training, which we term quantization-aware pruning, and the effect of techniques like regularization, batch normalization, and different pruning schemes on performance, computational complexity, and information content metrics. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task. Further, quantization-aware pruning typically performs similar to or better in terms of computational efficiency compared to other neural architecture search techniques like Bayesian optimization. Surprisingly, while networks with different training configurations can have similar performance for the benchmark application, the information content in the network can vary significantly, affecting its generalizability.
V2W-BERT: A Framework for Effective Hierarchical Multiclass Classification of Software Vulnerabilities
Authors: Siddhartha Shankar Das, Edoardo Serra, Mahantesh Halappanavar, Alex Pothen, Ehab Al-Shaer
Date: 2021-02-23 00:00:00
Tasks: General Classification, Link Prediction, Transfer Learning

Weaknesses in computer systems such as faults, bugs and errors in the architecture, design or implementation of software provide vulnerabilities that can be exploited by attackers to compromise the security of a system. Common Weakness Enumerations (CWE) are a hierarchically designed dictionary of software weaknesses that provide a means to understand software flaws, potential impact of their exploitation, and means to mitigate these flaws. Common Vulnerabilities and Exposures (CVE) are brief low-level descriptions that uniquely identify vulnerabilities in a specific product or protocol. Classifying or mapping of CVEs to CWEs provides a means to understand the impact and mitigate the vulnerabilities. Since manual mapping of CVEs is not a viable option, automated approaches are desirable but challenging. We present a novel Transformer-based learning framework (V2W-BERT) in this paper. By using ideas from natural language processing, link prediction and transfer learning, our method outperforms previous approaches not only for CWE instances with abundant data to train, but also rare CWE classes with little or no data to train. Our approach also shows significant improvements in using historical data to predict links for future instances of CVEs, and therefore, provides a viable approach for practical applications. Using data from MITRE and National Vulnerability Database, we achieve up to 97% prediction accuracy for randomly partitioned data and up to 94% prediction accuracy in temporally partitioned data. We believe that our work will influence the design of better methods and training models, as well as applications to solve increasingly harder problems in cybersecurity.
Histo-fetch -- On-the-fly processing of gigapixel whole slide images simplifies and speeds neural network training
Authors: Brendon Lutnick, Leema Krishna Murali, Brandon Ginley, Avi Z. Rosenberg, Pinaki Sarder
Date: 2021-02-23 00:00:00
Tasks: Image Generation, whole slide images

We created a custom pipeline (histo-fetch) to efficiently extract random patches and labels from pathology whole slide images (WSIs) for input to a neural network on-the-fly. We prefetch these patches as needed during network training, avoiding the need for WSI preparation such as chopping/tiling. We demonstrate the utility of this pipeline to perform artificial stain transfer and image generation using the popular networks CycleGAN and ProGAN, respectively.
Accurate Learning of Graph Representations with Graph Multiset Pooling
Authors: Jinheon Baek, Minki Kang, Sung Ju Hwang
Date: 2021-02-23 00:00:00
Proceeding: ICLR 2021 1
Tasks: Graph Classification, Graph Clustering, Graph Embedding, Graph Generation, Graph Learning, Graph Reconstruction, Node Clustering

Graph neural networks have been widely used on modeling graph data, achieving impressive results on node classification and link prediction tasks. Yet, obtaining an accurate representation for a graph further requires a pooling function that maps a set of node representations into a compact form. A simple sum or average over all node representations considers all node features equally without consideration of their task relevance, and any structural dependencies among them. Recently proposed hierarchical graph pooling methods, on the other hand, may yield the same representation for two different graphs that are distinguished by the Weisfeiler-Lehman test, as they suboptimally preserve information from the node features. To tackle these limitations of existing graph pooling methods, we first formulate the graph pooling problem as a multiset encoding problem with auxiliary information about the graph structure, and propose a Graph Multiset Transformer (GMT) which is a multi-head attention based global pooling layer that captures the interaction between nodes according to their structural dependencies. We show that GMT satisfies both injectiveness and permutation invariance, such that it is at most as powerful as the Weisfeiler-Lehman graph isomorphism test. Moreover, our methods can be easily extended to the previous node clustering approaches for hierarchical graph pooling. Our experimental results show that GMT significantly outperforms state-of-the-art graph pooling methods on graph classification benchmarks with high memory and time efficiency, and obtains even larger performance gain on graph reconstruction and generation tasks.
SliceNStitch: Continuous CP Decomposition of Sparse Tensor Streams
Authors: Taehyung Kwon, Inkyu Park, Dongjin Lee, Kijung Shin
Date: 2021-02-23 00:00:00
Tasks: Anomaly Detection, Recommendation Systems, Stock Market Prediction

Consider traffic data (i.e., triplets in the form of source-destination-timestamp) that grow over time. Tensors (i.e., multi-dimensional arrays) with a time mode are widely used for modeling and analyzing such multi-aspect data streams. In such tensors, however, new entries are added only once per period, which is often an hour, a day, or even a year. This discreteness of tensors has limited their usage for real-time applications, where new data should be analyzed instantly as it arrives. How can we analyze time-evolving multi-aspect sparse data 'continuously' using tensors where time is'discrete'? We propose SLICENSTITCH for continuous CANDECOMP/PARAFAC (CP) decomposition, which has numerous time-critical applications, including anomaly detection, recommender systems, and stock market prediction. SLICENSTITCH changes the starting point of each period adaptively, based on the current time, and updates factor matrices (i.e., outputs of CP decomposition) instantly as new data arrives. We show, theoretically and experimentally, that SLICENSTITCH is (1) 'Any time': updating factor matrices immediately without having to wait until the current time period ends, (2) Fast: with constant-time updates up to 464x faster than online methods, and (3) Accurate: with fitness comparable (specifically, 72 ~ 100%) to offline methods.
Clustering Aware Classification for Risk Prediction and Subtyping in Clinical Data
Authors: Shivin Srivastava, Siddharth Bhatia, Lingxiao Huang, Lim Jun Heng, Kenji Kawaguchi, Vaibhav Rajan
Date: 2021-02-23 00:00:00
Tasks: Classification, Clustering, General Classification, Representation Learning

In data containing heterogeneous subpopulations, classification performance benefits from incorporating the knowledge of cluster structure in the classifier. Previous methods for such combined clustering and classification either 1) are classifier-specific and not generic, or 2) independently perform clustering and classifier training, which may not form clusters that can potentially benefit classifier performance. The question of how to perform clustering to improve the performance of classifiers trained on the clusters has received scant attention in previous literature, despite its importance in several real-world applications. In this paper, first, we theoretically analyze the generalization performance of classifiers trained on clustered data and find conditions under which clustering can potentially aid classification. This motivates the design of a simple k-means-based classification algorithm called Clustering Aware Classification (CAC) and its neural variant {DeepCAC}. DeepCAC effectively leverages deep representation learning to learn latent embeddings and finds clusters in a manner that make the clustered data suitable for training classifiers for each underlying subpopulation. Our experiments on synthetic and real benchmark datasets demonstrate the efficacy of DeepCAC over previous methods for combined clustering and classification.
UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering
Authors: Mohamed El Banani, Luya Gao, Justin Johnson
Date: 2021-02-23 00:00:00
Proceeding: CVPR 2021 1
Tasks: Point Cloud Registration

Aligning partial views of a scene into a single whole is essential to understanding one's environment and is a key component of numerous robotics tasks such as SLAM and SfM. Recent approaches have proposed end-to-end systems that can outperform traditional methods by leveraging pose supervision. However, with the rising prevalence of cameras with depth sensors, we can expect a new stream of raw RGB-D data without the annotations needed for supervision. We propose UnsupervisedR&R: an end-to-end unsupervised approach to learning point cloud registration from raw RGB-D video. The key idea is to leverage differentiable alignment and rendering to enforce photometric and geometric consistency between frames. We evaluate our approach on indoor scene datasets and find that we outperform existing traditional approaches with classic and learned descriptors while being competitive with supervised geometric point cloud registration approaches.
Machine Learning Regression for Operator Dynamics
Authors: Justin Reyes, Sayandip Dhara, Eduardo R. Mucciolo
Date: 2021-02-23 00:00:00
Tasks: BIG-bench Machine Learning, regression

Determining the dynamics of the expectation values for operators acting on a quantum many-body (QMB) system is a challenging task. Matrix product states (MPS) have traditionally been the "go-to" models for these systems because calculating expectation values in this representation can be done with relative simplicity and high accuracy. However, such calculations can become computationally costly when extended to long times. Here, we present a solution for efficiently extending the computation of expectation values to long time intervals. We utilize a multi-layer perceptron (MLP) model as a tool for regression on MPS expectation values calculated within the regime of short time intervals. With this model, the computational cost of generating long-time dynamics is significantly reduced, while maintaining a high accuracy. These results are demonstrated with operators relevant to quantum spin models in one spatial dimension.
Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality
Authors: Tengyu Xu, Zhuoran Yang, Zhaoran Wang, Yingbin Liang
Date: 2021-02-23 00:00:00

Designing off-policy reinforcement learning algorithms is typically a very challenging task, because a desirable iteration update often involves an expectation over an on-policy distribution. Prior off-policy actor-critic (AC) algorithms have introduced a new critic that uses the density ratio for adjusting the distribution mismatch in order to stabilize the convergence, but at the cost of potentially introducing high biases due to the estimation errors of both the density ratio and value function. In this paper, we develop a doubly robust off-policy AC (DR-Off-PAC) for discounted MDP, which can take advantage of learned nuisance functions to reduce estimation errors. Moreover, DR-Off-PAC adopts a single timescale structure, in which both actor and critics are updated simultaneously with constant stepsize, and is thus more sample efficient than prior algorithms that adopt either two timescale or nested-loop structure. We study the finite-time convergence rate and characterize the sample complexity for DR-Off-PAC to attain an $\epsilon$-accurate optimal policy. We also show that the overall convergence of DR-Off-PAC is doubly robust to the approximation errors that depend only on the expressive power of approximation functions. To the best of our knowledge, our study establishes the first overall sample complexity analysis for a single time-scale off-policy AC algorithm.