pwc-archive/papers-with-abstracts huggingface dataset:
DPCNet: Dual Path Multi-Excitation Collaborative Network for Facial Expression Representation Learning in Videos
Authors: Yan Wang, Yixuan Sun, Wei Song, Shuyong Gao, Yiwen Huang, Zhaoyu Chen, Weifeng Ge, and Wenqiang Zhang
Date: 2022-10-10 00:00:00
Proceeding: MM '22: Proceedings of the 30th ACM International Conference on Multimedia 2022 10
Tasks: Dynamic Facial Expression Recognition, Representation Learning

Current works of facial expression learning in video consume significant computational resources to learn spatial channel feature representations and temporal relationships. To mitigate this issue, we propose a Dual Path multi-excitation Collaborative Network (DPCNet) to learn the critical information for facial expression representation from fewer keyframes in videos. Specifically, the DPCNet learns the important regions and keyframes from a tuple of four view-grouped frames by multi-excitation modules and produces dual-path representations of one video with consistency under two regularization strategies. A spatial-frame excitation module and a channel-temporal aggregation module are introduced consecutively to learn spatial-frame representation and generate complementary channel-temporal aggregation, respectively. Moreover, we design a multi-frame regularization loss to enforce the representation of multiple frames in the dual view to be semantically coherent. To obtain consistent prediction probabilities from the dual path, we further propose a dual path regularization loss, aiming to minimize the divergence between the distributions of two-path embeddings. Extensive experiments and ablation studies show that the DPCNet can significantly improve the performance of video-based FER and achieve state-of-the-art results on the large-scale DFEW dataset.
Clip-aware expressive feature learning for video-based facial expression recognition
Authors: Yuanyuan Liu, Chuanxu Feng, Xiaohui Yuan, Lin Zhou, Wenbin Wang, Jie Qin, and Zhongwen Luo
Date: 2022-03-25 00:00:00
Proceeding: Information Sciences 2022 3
Tasks: Dynamic Facial Expression Recognition, Facial Expression Recognition, Facial Expression Recognition (FER)

Video-based facial expression recognition (FER) has received increased attention as a result of its widespread applications. However, a video often contains many redundant and irrelevant frames. How to reduce redundancy and complexity of the available information and extract the most relevant information to facial expression in video sequences is a challenging task. In this paper, we divide a video into several short clips for processing and propose a clip-aware emotion-rich feature learning network (CEFLNet) for robust video-based FER. Our proposed CEFLNet identifies the emotional intensity expressed in each short clip in a video and obtains clip-aware emotion-rich representations. Specifically, CEFLNet constructs a clip-based feature encoder (CFE) with two-cascaded self-attention and local–global relation learning, aiming to encode clip-based spatio-temporal features from the clips of a video. An emotional intensity activation network (EIAN) is devised to generate emotional activation maps for locating the salient emotion clips and obtaining clip-aware emotion-rich representations, which are used for expression classification. The effectiveness and robustness of the proposed CEFLNet are evaluated using four public facial expression video datasets, including BU-3DFE, MMI, AFEW, and DFEW. Extensive experiments demonstrate the improved performance of our proposed CEFLNet in comparison with the state-of-the-art methods.
Former-dfer: Dynamic facial expression recognition transformer
Authors: Zengqun Zhao and Qingshan Liu
Date: 2021-10-17 00:00:00
Proceeding: MM '21: Proceedings of the 29th ACM International Conference on Multimedia 2021 10

This paper proposes a dynamic facial expression recognition transformer (Former-DFER) for the in-the-wild scenario. Specifically, the proposed Former-DFER mainly consists of a convolutional spatial transformer (CS-Former) and a temporal transformer (T-Former). The CS-Former consists of five convolution blocks and N spatial encoders, which is designed to guide the network to learn occlusionand pose-robust facial features from the spatial perspective. And the temporal transformer consists of M temporal encoders, which is designed to allow the network to learn contextual facial features from the temporal perspective. The heatmaps of the leaned facial features demonstrate that the proposed Former-DFER is capable of handling the issues such as occlusion, non-frontal pose, and head motion. And the visualization of the feature distribution shows that the proposed method can learn more discriminative facial features. Moreover, our Former-DFER also achieves state-of-the-art results on the DFEW and AFEW benchmarks. Code is available at https://github.com/zengqunzhao/Former-DFER.
On-Device Speaker Anonymization of Acoustic Embeddings for ASR based onFlexible Location Gradient Reversal Layer
Authors: Md Asif Jalal, Pablo Peso Parada, Jisi Zhang, Karthikeyan Saravanan, Mete Ozay, Myoungji Han, Jung In Lee, Seokyeong Jung
Date: 2023-07-25 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speaker anonymization, Speaker Recognition, speech-recognition, Speech Recognition

Smart devices serviced by large-scale AI models necessitates user data transfer to the cloud for inference. For speech applications, this means transferring private user information, e.g., speaker identity. Our paper proposes a privacy-enhancing framework that targets speaker identity anonymization while preserving speech recognition accuracy for our downstream task~-~Automatic Speech Recognition (ASR). The proposed framework attaches flexible gradient reversal based speaker adversarial layers to target layers within an ASR model, where speaker adversarial training anonymizes acoustic embeddings generated by the targeted layers to remove speaker identity. We propose on-device deployment by execution of initial layers of the ASR model, and transmitting anonymized embeddings to the cloud, where the rest of the model is executed while preserving privacy. Experimental results show that our method efficiently reduces speaker recognition relative accuracy by 33%, and improves ASR performance by achieving 6.2% relative Word Error Rate (WER) reduction.
Pathway: a fast and flexible unified stream data processing framework for analytical and Machine Learning applications
Authors: Michal Bartoszkiewicz, Jan Chorowski, Adrian Kosowski, Jakub Kowalski, Sergey Kulik, Mateusz Lewandowski, Krzysztof Nowicki, Kamil Piechowiak, Olivier Ruas, Zuzanna Stamirowska, Przemyslaw Uznanski
Date: 2023-07-12 00:00:00
Tasks: Benchmarking

We present Pathway, a new unified data processing framework that can run workloads on both bounded and unbounded data streams. The framework was created with the original motivation of resolving challenges faced when analyzing and processing data from the physical economy, including streams of data generated by IoT and enterprise systems. These required rapid reaction while calling for the application of advanced computation paradigms (machinelearning-powered analytics, contextual analysis, and other elements of complex event processing). Pathway is equipped with a Table API tailored for Python and Python/SQL workflows, and is powered by a distributed incremental dataflow in Rust. We describe the system and present benchmarking results which demonstrate its capabilities in both batch and streaming contexts, where it is able to surpass state-of-the-art industry frameworks in both scenarios. We also discuss streaming use cases handled by Pathway which cannot be easily resolved with state-of-the-art industry frameworks, such as streaming iterative graph algorithms (PageRank, etc.).
FPGA Implementation of Robust Residual Generator
Authors: Y. M. Kim
Date: 2023-07-25 00:00:00

In this paper, one can explicitly see the process of implementing the robust residual generator on digital domain, especially on FPGA. Firstly, the baseline model is developed in double precision floating point format. To develop the baseline model, key parameters such as SNR and detection window length are selected in the identification stage. (Please refer to the uploaded paper because this box doesn't accept more typing beyond this point)
Critical-time metric for risk analysis against sharp input anomalies: computation and application case study
Authors: Arthur Perodou, Christophe Combastel, Ali Zolghadri
Date: 2023-07-25 00:00:00

This paper investigates the critical-time criteria as a security metric for controlled systems subject to sharp input anomalies (attack, fault), characterized by having high impact in a reduced amount of time (e.g. denial-of-service, attack by upper saturation). The critical-time is the maximal time-horizon for which a system can be considered to be safe after the occurrence of an anomaly. This metric is expected to be useful for risk analysis and treatment (prevention, detection, mitigation). In this work, the computational problem of the critical-time for uncertain linear systems and several classes of sharp input anomalies, depending on the input channel and the set of abnormal signal values, is formulated based on the quadratic constraints (QC) framework, representing sets by the intersection of QC inequalities and equalities. An iterative LMI-based algorithm is then proposed to provide an under-estimate of the critical-time. Finally, the potential of the critical-time as a metric for defense design is illustrated and discussed on the quadruple-tank case study through different relevant scenarios.
BIM-to-BRICK: Using graph modeling for IoT/BMS and spatial semantic data interoperability within digital data models of buildings
Authors: Filippo Vittori, Chuan Fu Tan, Anna Laura Pisello, Adrian Chong, Clayton Miller
Date: 2023-07-25 00:00:00
Tasks: Data Integration, Management

The holistic management of a building requires data from heterogeneous sources such as building management systems (BMS), Internet-of-Things (IoT) sensor networks, and building information models. Data interoperability is a key component to eliminate silos of information, and using semantic web technologies like the BRICK schema, an effort to standardize semantic descriptions of the physical, logical, and virtual assets in buildings and the relationships between them, is a suitable approach. However, current data integration processes can involve significant manual interventions. This paper presents a methodology to automatically collect, assemble, and integrate information from a building information model to a knowledge graph. The resulting application, called BIM-to-BRICK, is run on the SDE4 building located in Singapore. BIM-to-BRICK generated a bidirectional link between a BIM model of 932 instances and experimental data collected for 17 subjects into 458 BRICK objects and 1219 relationships in 17 seconds. The automation of this approach can be compared to traditional manual mapping of data types. This scientific innovation incentivizes the convergence of disparate data types and structures in built-environment applications.
Digital Twins for Moving Target Defense Validation in AC Microgrids
Authors: Suman Rath, Subham Sahoo, Shamik Sengupta
Date: 2023-07-24 00:00:00

Cyber-physical microgrids are vulnerable to stealth attacks that can degrade their stability and operability by performing low-magnitude manipulations in a coordinated manner. This paper formulates the interactions between CSAs and microgrid defenders as a non-cooperative, zero-sum game. Additionally, it presents a hybrid Moving Target Defense (MTD) strategy for distributed microgrids that can dynamically alter local control gains to achieve resiliency against Coordinated Stealth Attacks (CSAs). The proposed strategy reduces the success probability of attack(s) by making system dynamics less predictable. The framework also identifies and removes malicious injections by modifying secondary control weights assigned to them. The manipulated signals are reconstructed using an Artificial Neural Network (ANN)-based Digital Twin (DT) to preserve stability. To guarantee additional immunity against instability arising from gain alterations, MTD decisions are also validated (via utility and best response computations) using the DT before actual implementation. The DT is also used to find the minimum perturbation that defenders must achieve to invalidate an attacker's knowledge effectively.
Pilot Performance modeling via observer-based inverse reinforcement learning
Authors: Jared Town, Zachary Morrison, Rushikesh Kamalapurkar
Date: 2023-07-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning

The focus of this paper is behavior modeling for pilots of unmanned aerial vehicles. The pilot is assumed to make decisions that optimize an unknown cost functional, which is estimated from observed trajectories using a novel inverse reinforcement learning (IRL) framework. The resulting IRL problem often admits multiple solutions. In this paper, a recently developed novel IRL observer is adapted to the pilot modeling problem. The observer is shown to converge to one of the equivalent solutions of the IRL problem. The developed technique is implemented on a quadcopter where the pilot is a linear quadratic supervisory controller that generates velocity commands for the quadcopter to travel to and hover over a desired location. Experimental results demonstrate the robustness of the method and its ability to learn equivalent cost functionals.
Characteristics and Predictive Modeling of Short-term Impacts of Hurricanes on the US Employment
Authors: Gan Zhang, Wenjun Zhu
Date: 2023-07-25 00:00:00

The physical and economic damages of hurricanes can acutely affect employment and the well-being of employees. However, a comprehensive understanding of these impacts remains elusive as many studies focused on narrow subsets of regions or hurricanes. Here we present an open-source dataset that serves interdisciplinary research on hurricane impacts on US employment. Compared to past domain-specific efforts, this dataset has greater spatial-temporal granularity and variable coverage. To demonstrate potential applications of this dataset, we focus on the short-term employment disruptions related to hurricanes during 1990-2020. The observed county-level employment changes in the initial month are small on average, though large employment losses (>30%) can occur after extreme storms. The overall small changes partly result from compensation among different employment sectors, which may obscure large, concentrated employment losses after hurricanes. Additional econometric analyses concur on the post-storm employment losses in hospitality and leisure but disagree on employment changes in the other industries. The dataset also enables data-driven analyses that highlight vulnerabilities such as pronounced employment losses related to Puerto Rico and rainy hurricanes. Furthermore, predictive modeling of short-term employment changes shows promising performance for service-providing industries and high-impact storms. In the examined cases, the nonlinear Random Forests model greatly outperforms the multiple linear regression model. The nonlinear model also suggests that more severe hurricane hazards projected by physical models may cause more extreme losses in US service-providing employment. Finally, we share our dataset and analytical code to facilitate the study and modeling of hurricane impacts in a changing climate.
Large sample properties of GMM estimators under second-order identification
Authors: Hugo Kruiniger
Date: 2023-07-25 00:00:00
Tasks: Econometrics

Dovonon and Hall (Journal of Econometrics, 2018) proposed a limiting distribution theory for GMM estimators for a p - dimensional globally identified parameter vector {\phi} when local identification conditions fail at first-order but hold at second-order. They assumed that the first-order underidentification is due to the expected Jacobian having rank p-1 at the true value {\phi}_{0}, i.e., having a rank deficiency of one. After reparametrizing the model such that the last column of the Jacobian vanishes, they showed that the GMM estimator of the first p-1 parameters converges at rate T^{-1/2} and the GMM estimator of the remaining parameter, {\phi}_{p}, converges at rate T^{-1/4}. They also provided a limiting distribution of T^{1/4}({\phi}_{p}-{\phi}_{0,p}) subject to a (non-transparent) condition which they claimed to be not restrictive in general. However, as we show in this paper, their condition is in fact only satisfied when {\phi} is overidentified and the limiting distribution of T^{1/4}({\phi}_{p}-{\phi}_{0,p}), which is non-standard, depends on whether {\phi} is exactly identified or overidentified. In particular, the limiting distributions of the sign of T^{1/4}({\phi}_{p}-{\phi}_{0,p}) for the cases of exact and overidentification, respectively, are different and are obtained by using expansions of the GMM objective function of different orders. Unsurprisingly, we find that the limiting distribution theories of Dovonon and Hall (2018) for Indirect Inference (II) estimation under two different scenarios with second-order identification where the target function is a GMM estimator of the auxiliary parameter vector, are incomplete for similar reasons. We discuss how our results for GMM estimation can be used to complete both theories and how they can be used to obtain the limiting distributions of the II estimators in the case of exact identification under either scenario.
Testing for sparse idiosyncratic components in factor-augmented regression models
Authors: Jad Beyhum, Jonas Striaukas
Date: 2023-07-25 00:00:00
Tasks: regression, Time Series

We propose a novel bootstrap test of a dense model, namely factor regression, against a sparse plus dense alternative augmenting model with sparse idiosyncratic components. The asymptotic properties of the test are established under time series dependence and polynomial tails. We outline a data-driven rule to select the tuning parameter and prove its theoretical validity. In simulation experiments, our procedure exhibits high power against sparse alternatives and low power against dense deviations from the null. Moreover, we apply our test to various datasets in macroeconomics and finance and often reject the null. This suggests the presence of sparsity -- on top of a dense model -- in commonly studied economic applications. The R package FAS implements our approach.
Inference in Experiments with Matched Pairs and Imperfect Compliance
Authors: Yuehao Bai, Hongchang Guo, Azeem M. Shaikh, Max Tabord-Meehan
Date: 2023-07-24 00:00:00

This paper studies inference for the local average treatment effect in randomized controlled trials with imperfect compliance where treatment status is determined according to "matched pairs." By "matched pairs," we mean that units are sampled i.i.d. from the population of interest, paired according to observed, baseline covariates and finally, within each pair, one unit is selected at random for treatment. Under weak assumptions governing the quality of the pairings, we first derive the limit distribution of the usual Wald (i.e., two-stage least squares) estimator of the local average treatment effect. We show further that conventional heteroskedasticity-robust estimators of the Wald estimator's limiting variance are generally conservative, in that their probability limits are (typically strictly) larger than the limiting variance. We therefore provide an alternative estimator of the limiting variance that is consistent. Finally, we consider the use of additional observed, baseline covariates not used in pairing units to increase the precision with which we can estimate the local average treatment effect. To this end, we derive the limiting behavior of a two-stage least squares estimator of the local average treatment effect which includes both the additional covariates in addition to pair fixed effects, and show that its limiting variance is always less than or equal to that of the Wald estimator. To complete our analysis, we provide a consistent estimator of this limiting variance. A simulation study confirms the practical relevance of our theoretical results. Finally, we apply our results to revisit a prominent experiment studying the effect of macroinsurance on microenterprise in Egypt.
Optimal Rating Design under Moral Hazard
Authors: Maryam Saeedi, Ali Shourideh
Date: 2020-08-21 00:00:00

We examine the design of optimal rating systems in the presence of moral hazard. First, an intermediary commits to a rating scheme. Then, a decision-maker chooses an action that generates value for the buyer. The intermediary then observes a noisy signal of the decision-maker's choice and sends the buyer a signal consistent with the rating scheme. Here we fully characterize the set of allocations that can arise in equilibrium under any arbitrary rating system. We use this characterization to study various design aspects of optimal rating systems. Specifically, we study the properties of optimal ratings when the decision-maker's effort is productive and when the decision-maker can manipulate the intermediary's signal with a noise. With manipulation, rating uncertainty is a fairly robust feature of optimal rating systems.
A Visual Quality Assessment Method for Raster Images in Scanned Document
Authors: Justin Yang, Peter Bauer, Todd Harris, Changhyung Lee, Hyeon Seok Seo, Jan P Allebach, Fengqing Zhu
Date: 2023-07-25 00:00:00
Tasks: Image Quality Assessment

Image quality assessment (IQA) is an active research area in the field of image processing. Most prior works focus on visual quality of natural images captured by cameras. In this paper, we explore visual quality of scanned documents, focusing on raster image areas. Different from many existing works which aim to estimate a visual quality score, we propose a machine learning based classification method to determine whether the visual quality of a scanned raster image at a given resolution setting is acceptable. We conduct a psychophysical study to determine the acceptability at different image resolutions based on human subject ratings and use them as the ground truth to train our machine learning model. However, this dataset is unbalanced as most images were rated as visually acceptable. To address the data imbalance problem, we introduce several noise models to simulate the degradation of image quality during the scanning process. Our results show that by including augmented data in training, we can significantly improve the performance of the classifier to determine whether the visual quality of raster images in a scanned document is acceptable or not for a given resolution setting.
In-Situ Thickness Measurement of Die Silicon Using Voltage Imaging for Hardware Assurance
Authors: Olivia P. Dizon-Paradis, Nitin Varshney, M Tanjidur Rahman, Michael Strizich, Haoting Shen, Navid Asadizanjani
Date: 2023-07-24 00:00:00

Hardware assurance of electronics is a challenging task and is of great interest to the government and the electronics industry. Physical inspection-based methods such as reverse engineering (RE) and Trojan scanning (TS) play an important role in hardware assurance. Therefore, there is a growing demand for automation in RE and TS. Many state-of-the-art physical inspection methods incorporate an iterative imaging and delayering workflow. In practice, uniform delayering can be challenging if the thickness of the initial layer of material is non-uniform. Moreover, this non-uniformity can reoccur at any stage during delayering and must be corrected. Therefore, it is critical to evaluate the thickness of the layers to be removed in a real-time fashion. Our proposed method uses electron beam voltage imaging, image processing, and Monte Carlo simulation to measure the thickness of remaining silicon to guide a uniform delayering process