pwc-archive/papers-with-abstracts huggingface dataset:
Robust Attitude Controller for Unmanned Aerial Vehicle Using Dynamic Inversion and Extended State Observer
Authors: Ruyi Yuan
Date: 2009-10-16 00:00:00
Proceeding: 2009 Second International Conference on Intelligent Computation Technology and Automation 2009 10

A robust feedback linearization controller is presented for attitude control of an unmanned aerial vehicle (UAV). The objective of this controller is to make the roll angle, pitch angle, and yaw angle track the given trajectories(commands) respectively. This design is developed using dynamic inversion and extended state observer (ESO). Firstly, dynamic inversion is used to linearize and decouple UAV attitude system into three single-input-single-output (SISO) systems, then three proportional-derivative (PD) controllers are designed for these linearized systems. Extended state observers are used to estimate and compensate unmodeled dynamics and extent disturbances. Simulation results show that the proposed controller is effective and robust.
Linear Regression on a Set of Selected Templates from a Pool of Randomly Generated Templates
Authors: Peter Taraba
Date: 2021-08-08 00:00:00
Proceeding: Machine Learning with Applications 2021 8
Tasks: regression

We study linear regression for two datasets. For the MNIST dataset we do so using max convolutions, whose parameters are generated directly from training images for the digit recognition problem, hence we call them max convolution templates. From a large pool of randomly generated convolutional templates, we select by iterative process the ones which improve defined linear regression minimization problem the most. With these templates, we use linear and logistic regression and achieve high accuracy, comparable with deep neural networks. We explain why, in a production environment, using this approach has advantages over the use of deep neural networks. On a second dataset ‘Adult Data Set’ of income predictions, we show a similar convolution type approach for generating a pool of random templates and show that the same template selection process and linear regression can be used as for the MNIST dataset.
Clustering and Community Detection in Directed Networks: A Survey
Authors: Fragkiskos D. Malliaros, Michalis Vazirgiannis
Date: 2013-08-05 00:00:00
Tasks: Clustering, Community Detection, Graph Clustering, Sociology, Survey

Networks (or graphs) appear as dominant structures in diverse domains, including sociology, biology, neuroscience and computer science. In most of the aforementioned cases graphs are directed - in the sense that there is directionality on the edges, making the semantics of the edges non symmetric. An interesting feature that real networks present is the clustering or community structure property, under which the graph topology is organized into modules commonly called communities or clusters. The essence here is that nodes of the same community are highly similar while on the contrary, nodes across communities present low similarity. Revealing the underlying community structure of directed complex networks has become a crucial and interdisciplinary topic with a plethora of applications. Therefore, naturally there is a recent wealth of research production in the area of mining directed graphs - with clustering being the primary method and tool for community detection and evaluation. The goal of this paper is to offer an in-depth review of the methods presented so far for clustering directed networks along with the relevant necessary methodological background and also related applications. The survey commences by offering a concise review of the fundamental concepts and methodological base on which graph clustering algorithms capitalize on. Then we present the relevant work along two orthogonal classifications. The first one is mostly concerned with the methodological principles of the clustering algorithms, while the second one approaches the methods from the viewpoint regarding the properties of a good cluster in a directed network. Further, we present methods and metrics for evaluating graph clustering results, demonstrate interesting application domains and provide promising future research directions.
A state-space mixed membership blockmodel for dynamic network tomography
Authors: Eric P. Xing, Wenjie Fu, Le Song
Date: 2008-12-31 00:00:00

In a dynamic social or biological environment, the interactions between the actors can undergo large and systematic changes. In this paper we propose a model-based approach to analyze what we will refer to as the dynamic tomography of such time-evolving networks. Our approach offers an intuitive but powerful tool to infer the semantic underpinnings of each actor, such as its social roles or biological functions, underlying the observed network topologies. Our model builds on earlier work on a mixed membership stochastic blockmodel for static networks, and the state-space model for tracking object trajectory. It overcomes a major limitation of many current network inference techniques, which assume that each actor plays a unique and invariant role that accounts for all its interactions with other actors; instead, our method models the role of each actor as a time-evolving mixed membership vector that allows actors to behave differently over time and carry out different roles/functions when interacting with different peers, which is closer to reality. We present an efficient algorithm for approximate inference and learning using our model; and we applied our model to analyze a social network between monks (i.e., the Sampson's network), a dynamic email communication network between the Enron employees, and a rewiring gene interaction network of fruit fly collected during its full life cycle. In all cases, our model reveals interesting patterns of the dynamic roles of the actors.
The maximum entropy mortality model: forecasting mortality using statistical moments Codes
Authors: Marius D. Pascariu, Adam Lenart, Vladimir Canudas-Romo
Date: 2019-03-29 00:00:00
Proceeding: Scandinavian Actuarial Journal 2019 3
Tasks: Time Series, Time Series Analysis

The age-at-deathdistribution is arepresentation ofthe mortality experience in a population. Although it proves to be highly informative, it is often neglected when it comes to the practice of past or future mortality assessment. We propose an innovative method to mortality modeling and forecasting by making use of the location and shape measures of a density function, i.e. statistical moments. Time series methods for extrapolating a limited number of moments are used and then the reconstruction of the future age-at-death distribution is performed. The predictive power of the method seems to be net superior when compared to the results obtained using classical approaches to extrapolating age-specific-death rates, and the accuracy of the point forecast (MASE) is improved on average by 33% respective to the state-of-the-art, the Lee–Carter model. The method is tested using data from the Human Mortality Database and implemented in a publicly available R package.
Super-Samples from Kernel Herding
Authors: Yutian Chen, Max Welling, Alex Smola
Date: 2012-03-15 00:00:00

We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting "kernel herding" algorithm is an infinite memory deterministic process that learns to approximate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rate O(1/T) which is much faster than the usual O(1/pT) for iid random samples. We illustrate kernel herding by approximating Bayesian predictive distributions.
Vis-TOP: Visual Transformer Overlay Processor
Authors: Wei Hu, Dian Xu, Zimeng Fan, Fang Liu, Yanxiang He
Date: 2021-10-21 00:00:00
Tasks: CPU, GPU, Quantization

In recent years, Transformer has achieved good results in Natural Language Processing (NLP) and has also started to expand into Computer Vision (CV). Excellent models such as the Vision Transformer and Swin Transformer have emerged. At the same time, the platform for Transformer models was extended to embedded devices to meet some resource-sensitive application scenarios. However, due to the large number of parameters, the complex computational flow and the many different structural variants of Transformer models, there are a number of issues that need to be addressed in its hardware design. This is both an opportunity and a challenge. We propose Vis-TOP (Visual Transformer Overlay Processor), an overlay processor for various visual Transformer models. It differs from coarse-grained overlay processors such as CPU, GPU, NPE, and from fine-grained customized designs for a specific model. Vis-TOP summarizes the characteristics of all visual Transformer models and implements a three-layer and two-level transformation structure that allows the model to be switched or changed freely without changing the hardware architecture. At the same time, the corresponding instruction bundle and hardware architecture are designed in three-layer and two-level transformation structure. After quantization of Swin Transformer tiny model using 8-bit fixed points (fix_8), we implemented an overlay processor on the ZCU102. Compared to GPU, the TOP throughput is 1.5x higher. Compared to the existing Transformer accelerators, our throughput per DSP is between 2.2x and 11.7x higher than others. In a word, the approach in this paper meets the requirements of real-time AI in terms of both resource consumption and inference speed. Vis-TOP provides a cost-effective and power-effective solution based on reconfigurable devices for computer vision at the edge.
Toward Accurate and Reliable Iris Segmentation Using Uncertainty Learning
Authors: Jianze Wei, Huaibo Huang, Muyi Sun, Yunlong Wang, Min Ren, Ran He, Zhenan Sun
Date: 2021-10-20 00:00:00
Tasks: Iris Recognition, Iris Segmentation, Segmentation

Iris segmentation is a deterministic part of the iris recognition system. Unreliable segmentation of iris regions especially the limbic area is still the bottleneck problem, which impedes more accurate recognition. To make further efforts on accurate and reliable iris segmentation, we propose a bilateral self-attention module and design Bilateral Transformer (BiTrans) with hierarchical architecture by exploring spatial and visual relationships. The bilateral self-attention module adopts a spatial branch to capture spatial contextual information without resolution reduction and a visual branch with a large receptive field to extract the visual contextual features. BiTrans actively applies convolutional projections and cross-attention to improve spatial perception and hierarchical feature fusion. Besides, Iris Segmentation Uncertainty Learning is developed to learn the uncertainty map according to prediction discrepancy. With the estimated uncertainty, a weighting scheme and a regularization term are designed to reduce predictive uncertainty. More importantly, the uncertainty estimate reflects the reliability of the segmentation predictions. Experimental results on three publicly available databases demonstrate that the proposed approach achieves better segmentation performance using 20% FLOPs of the SOTA IrisParseNet.
Few-Shot Temporal Action Localization with Query Adaptive Transformer
Authors: Sauradip Nag, Xiatian Zhu, Tao Xiang
Date: 2021-10-20 00:00:00
Tasks: Action Localization, Action Segmentation, Few Shot Temporal Action Localization, Fine-Grained Action Detection, Temporal Action Localization, Temporal Localization, Transfer Learning

Existing temporal action localization (TAL) works rely on a large number of training videos with exhaustive segment-level annotation, preventing them from scaling to new classes. As a solution to this problem, few-shot TAL (FS-TAL) aims to adapt a model to a new class represented by as few as a single video. Exiting FS-TAL methods assume trimmed training videos for new classes. However, this setting is not only unnatural actions are typically captured in untrimmed videos, but also ignores background video segments containing vital contextual cues for foreground action segmentation. In this work, we first propose a new FS-TAL setting by proposing to use untrimmed training videos. Further, a novel FS-TAL model is proposed which maximizes the knowledge transfer from training classes whilst enabling the model to be dynamically adapted to both the new class and each video of that class simultaneously. This is achieved by introducing a query adaptive Transformer in the model. Extensive experiments on two action localization benchmarks demonstrate that our method can outperform all the state of the art alternatives significantly in both single-domain and cross-domain scenarios. The source code can be found in https://github.com/sauradip/fewshotQAT
NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters
Authors: Yoichi Hirose, Nozomu Yoshinari, Shinichi Shirakawa
Date: 2021-10-19 00:00:00
Tasks: 4k, Benchmarking, Neural Architecture Search

The benchmark datasets for neural architecture search (NAS) have been developed to alleviate the computationally expensive evaluation process and ensure a fair comparison. Recent NAS benchmarks only focus on architecture optimization, although the training hyperparameters affect the obtained model performances. Building the benchmark dataset for joint optimization of architecture and training hyperparameters is essential to further NAS research. The existing NAS-HPO-Bench is a benchmark for joint optimization, but it does not consider the network connectivity design as done in modern NAS algorithms. This paper introduces the first benchmark dataset for joint optimization of network connections and training hyperparameters, which we call NAS-HPO-Bench-II. We collect the performance data of 4K cell-based convolutional neural network architectures trained on the CIFAR-10 dataset with different learning rate and batch size settings, resulting in the data of 192K configurations. The dataset includes the exact data for 12 epoch training. We further build the surrogate model predicting the accuracies after 200 epoch training to provide the performance data of longer training epoch. By analyzing NAS-HPO-Bench-II, we confirm the dependency between architecture and training hyperparameters and the necessity of joint optimization. Finally, we demonstrate the benchmarking of the baseline optimization algorithms using NAS-HPO-Bench-II.
LOA: Logical Optimal Actions for Text-based Interaction Games
Authors: Daiki Kimura, Subhajit Chaudhury, Masaki Ono, Michiaki Tatsubori, Don Joven Agravante, Asim Munawar, Akifumi Wachi, Ryosuke Kohita, Alexander Gray
Date: 2021-10-21 00:00:00
Proceeding: ACL 2021 5
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), text-based games

We present Logical Optimal Actions (LOA), an action decision architecture of reinforcement learning applications with a neuro-symbolic framework which is a combination of neural network and symbolic knowledge acquisition approach for natural language interaction games. The demonstration for LOA experiments consists of a web-based interactive platform for text-based games and visualization for acquired knowledge for improving interpretability for trained rules. This demonstration also provides a comparison module with other neuro-symbolic approaches as well as non-symbolic state-of-the-art agent models on the same text-based games. Our LOA also provides open-sourced implementation in Python for the reinforcement learning environment to facilitate an experiment for studying neuro-symbolic agents. Code: https://github.com/ibm/loa
Moiré Attack (MA): A New Potential Risk of Screen Photos
Authors: Dantong Niu, Ruohao Guo, Yisen Wang
Date: 2021-10-20 00:00:00

Images, captured by a camera, play a critical role in training Deep Neural Networks (DNNs). Usually, we assume the images acquired by cameras are consistent with the ones perceived by human eyes. However, due to the different physical mechanisms between human-vision and computer-vision systems, the final perceived images could be very different in some cases, for example shooting on digital monitors. In this paper, we find a special phenomenon in digital image processing, the moir\'e effect, that could cause unnoticed security threats to DNNs. Based on it, we propose a Moir\'e Attack (MA) that generates the physical-world moir\'e pattern adding to the images by mimicking the shooting process of digital devices. Extensive experiments demonstrate that our proposed digital Moir\'e Attack (MA) is a perfect camouflage for attackers to tamper with DNNs with a high success rate ($100.0\%$ for untargeted and $97.0\%$ for targeted attack with the noise budget $\epsilon=4$), high transferability rate across different models, and high robustness under various defenses. Furthermore, MA owns great stealthiness because the moir\'e effect is unavoidable due to the camera's inner physical structure, which therefore hardly attracts the awareness of humans. Our code is available at https://github.com/Dantong88/Moire_Attack.
Generalized Out-of-Distribution Detection: A Survey
Authors: Jingkang Yang, Kaiyang Zhou, Yixuan Li, Ziwei Liu
Date: 2021-10-21 00:00:00
Tasks: Anomaly Detection, Autonomous Driving, Novelty Detection, Open Set Learning, Outlier Detection, Out-of-Distribution Detection, Out of Distribution (OOD) Detection, Survey

Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. For instance, in autonomous driving, we would like the driving system to issue an alert and hand over the control to humans when it detects unusual scenes or objects that it has never seen during training time and cannot make a safe decision. The term, OOD detection, first emerged in 2017 and since then has received increasing attention from the research community, leading to a plethora of methods developed, ranging from classification-based to density-based to distance-based ones. Meanwhile, several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD), are closely related to OOD detection in terms of motivation and methodology. Despite common goals, these topics develop in isolation, and their subtle differences in definition and problem setting often confuse readers and practitioners. In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e., AD, ND, OSR, OOD detection, and OD. Under our framework, these five problems can be seen as special cases or sub-tasks, and are easier to distinguish. We then review each of these five areas by summarizing their recent technical developments, with a special focus on OOD detection methodologies. We conclude this survey with open challenges and potential research directions.
QuantumNAT: Quantum Noise-Aware Training with Noise Injection, Quantization and Normalization
Authors: Hanrui Wang, Jiaqi Gu, Yongshan Ding, Zirui Li, Frederic T. Chong, David Z. Pan, Song Han
Date: 2021-10-21 00:00:00
Tasks: Denoising, Quantization

Parameterized Quantum Circuits (PQC) are promising towards quantum advantage on near-term quantum hardware. However, due to the large quantum noises (errors), the performance of PQC models has a severe degradation on real quantum devices. Take Quantum Neural Network (QNN) as an example, the accuracy gap between noise-free simulation and noisy results on IBMQ-Yorktown for MNIST-4 classification is over 60%. Existing noise mitigation methods are general ones without leveraging unique characteristics of PQC; on the other hand, existing PQC work does not consider noise effect. To this end, we present QuantumNAT, a PQC-specific framework to perform noise-aware optimizations in both training and inference stages to improve robustness. We experimentally observe that the effect of quantum noise to PQC measurement outcome is a linear map from noise-free outcome with a scaling and a shift factor. Motivated by that, we propose post-measurement normalization to mitigate the feature distribution differences between noise-free and noisy scenarios. Furthermore, to improve the robustness against noise, we propose noise injection to the training process by inserting quantum error gates to PQC according to realistic noise models of quantum hardware. Finally, post-measurement quantization is introduced to quantize the measurement outcomes to discrete values, achieving the denoising effect. Extensive experiments on 8 classification tasks using 6 quantum devices demonstrate that QuantumNAT improves accuracy by up to 43%, and achieves over 94% 2-class, 80% 4-class, and 34% 10-class classification accuracy measured on real quantum computers. The code for construction and noise-aware training of PQC is available in the TorchQuantum library.
A Fine-Grained Analysis on Distribution Shift
Authors: Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebuffi, Ira Ktena, Krishnamurthy Dvijotham, Taylan Cemgil
Date: 2021-10-21 00:00:00
Proceeding: ICLR 2022 4

Robustness to distribution shifts is critical for deploying machine learning models in the real world. Despite this necessity, there has been little work in defining the underlying mechanisms that cause these shifts and evaluating the robustness of algorithms across multiple, different distribution shifts. To this end, we introduce a framework that enables fine-grained analysis of various distribution shifts. We provide a holistic analysis of current state-of-the-art methods by evaluating 19 distinct methods grouped into five categories across both synthetic and real-world datasets. Overall, we train more than 85K models. Our experimental framework can be easily extended to include new methods, shifts, and datasets. We find, unlike previous work~\citep{Gulrajani20}, that progress has been made over a standard ERM baseline; in particular, pretraining and augmentations (learned or heuristic) offer large gains in many cases. However, the best methods are not consistent over different datasets and shifts.
StyleAlign: Analysis and Applications of Aligned StyleGAN Models
Authors: Zongze Wu, Yotam Nitzan, Eli Shechtman, Dani Lischinski
Date: 2021-10-21 00:00:00
Proceeding: ICLR 2022 4
Tasks: Image Morphing, Image-to-Image Translation, Transfer Learning, Translation

In this paper, we perform an in-depth study of the properties and applications of aligned generative models. We refer to two models as aligned if they share the same architecture, and one of them (the child) is obtained from the other (the parent) via fine-tuning to another domain, a common practice in transfer learning. Several works already utilize some basic properties of aligned StyleGAN models to perform image-to-image translation. Here, we perform the first detailed exploration of model alignment, also focusing on StyleGAN. First, we empirically analyze aligned models and provide answers to important questions regarding their nature. In particular, we find that the child model's latent spaces are semantically aligned with those of the parent, inheriting incredibly rich semantics, even for distant data domains such as human faces and churches. Second, equipped with this better understanding, we leverage aligned models to solve a diverse set of tasks. In addition to image translation, we demonstrate fully automatic cross-domain image morphing. We further show that zero-shot vision tasks may be performed in the child domain, while relying exclusively on supervision in the parent domain. We demonstrate qualitatively and quantitatively that our approach yields state-of-the-art results, while requiring only simple fine-tuning and inversion.