pwc-archive/papers-with-abstracts huggingface dataset:
Frequency-domain MLPs are More Effective Learners in Time Series Forecasting
Authors: Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Defu Lian, Ning An, Longbing Cao, Zhendong Niu
Date: 2023-11-10 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Time Series, Time Series Forecasting

Time series forecasting has played the key role in different industrial, including finance, traffic, energy, and healthcare domains. While existing literatures have designed many sophisticated architectures based on RNNs, GNNs, or Transformers, another kind of approaches based on multi-layer perceptrons (MLPs) are proposed with simple structure, low complexity, and {superior performance}. However, most MLP-based forecasting methods suffer from the point-wise mappings and information bottleneck, which largely hinders the forecasting performance. To overcome this problem, we explore a novel direction of applying MLPs in the frequency domain for time series forecasting. We investigate the learned patterns of frequency-domain MLPs and discover their two inherent characteristic benefiting forecasting, (i) global view: frequency spectrum makes MLPs own a complete view for signals and learn global dependencies more easily, and (ii) energy compaction: frequency-domain MLPs concentrate on smaller key part of frequency components with compact signal energy. Then, we propose FreTS, a simple yet effective architecture built upon Frequency-domain MLPs for Time Series forecasting. FreTS mainly involves two stages, (i) Domain Conversion, that transforms time-domain signals into complex numbers of frequency domain; (ii) Frequency Learning, that performs our redesigned MLPs for the learning of real and imaginary part of frequency components. The above stages operated on both inter-series and intra-series scales further contribute to channel-wise and time-wise dependency learning. Extensive experiments on 13 real-world benchmarks (including 7 benchmarks for short-term forecasting and 6 benchmarks for long-term forecasting) demonstrate our consistent superiority over state-of-the-art methods.
Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection
Authors: Haibao Yu, Yingjuan Tang, Enze Xie, Jilei Mao, Ping Luo, Zaiqing Nie
Date: 2023-11-03 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: 3D Object Detection, Autonomous Driving, object-detection, Object Detection

Cooperatively utilizing both ego-vehicle and infrastructure sensor data can significantly enhance autonomous driving perception abilities. However, the uncertain temporal asynchrony and limited communication conditions can lead to fusion misalignment and constrain the exploitation of infrastructure data. To address these issues in vehicle-infrastructure cooperative 3D (VIC3D) object detection, we propose the Feature Flow Net (FFNet), a novel cooperative detection framework. FFNet is a flow-based feature fusion framework that uses a feature flow prediction module to predict future features and compensate for asynchrony. Instead of transmitting feature maps extracted from still-images, FFNet transmits feature flow, leveraging the temporal coherence of sequential infrastructure frames. Furthermore, we introduce a self-supervised training approach that enables FFNet to generate feature flow with feature prediction ability from raw infrastructure sequences. Experimental results demonstrate that our proposed method outperforms existing cooperative detection methods while only requiring about 1/100 of the transmission cost of raw data and covers all latency in one model on the DAIR-V2X dataset. The code is available at \href{https://github.com/haibao-yu/FFNet-VIC3D}{https://github.com/haibao-yu/FFNet-VIC3D}.
Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models
Authors: Naman D Singh, Francesco Croce, Matthias Hein
Date: 2023-03-03 00:00:00
Proceeding: NeurIPS 2023 11

While adversarial training has been extensively studied for ResNet architectures and low resolution datasets like CIFAR, much less is known for ImageNet. Given the recent debate about whether transformers are more robust than convnets, we revisit adversarial training on ImageNet comparing ViTs and ConvNeXts. Extensive experiments show that minor changes in architecture, most notably replacing PatchStem with ConvStem, and training scheme have a significant impact on the achieved robustness. These changes not only increase robustness in the seen $\ell_\infty$-threat model, but even more so improve generalization to unseen $\ell_1/\ell_2$-attacks. Our modified ConvNeXt, ConvNeXt + ConvStem, yields the most robust $\ell_\infty$-models across different ranges of model parameters and FLOPs, while our ViT + ConvStem yields the best generalization to unseen threat models.
Beyond Normal: On the Evaluation of Mutual Information Estimators
Authors: Paweł Czyż, Frederic Grabowski, Julia E. Vogt, Niko Beerenwinkel, Alexander Marx
Date: 2023-06-19 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Benchmarking, Domain Generalization, Representation Learning

Mutual information is a general statistical dependency measure which has found applications in representation learning, causality, domain generalization and computational biology. However, mutual information estimators are typically evaluated on simple families of probability distributions, namely multivariate normal distribution and selected distributions with one-dimensional random variables. In this paper, we show how to construct a diverse family of distributions with known ground-truth mutual information and propose a language-independent benchmarking platform for mutual information estimators. We discuss the general applicability and limitations of classical and neural estimators in settings involving high dimensions, sparse interactions, long-tailed distributions, and high mutual information. Finally, we provide guidelines for practitioners on how to select appropriate estimator adapted to the difficulty of problem considered and issues one needs to consider when applying an estimator to a new data set.
TempME: Towards the Explainability of Temporal Graph Neural Networks via Motif Discovery
Authors: Jialin Chen, Rex Ying
Date: 2023-10-30 00:00:00
Proceeding: NeurIPS 2023 11

Temporal graphs are widely used to model dynamic systems with time-varying interactions. In real-world scenarios, the underlying mechanisms of generating future interactions in dynamic systems are typically governed by a set of recurring substructures within the graph, known as temporal motifs. Despite the success and prevalence of current temporal graph neural networks (TGNN), it remains uncertain which temporal motifs are recognized as the significant indications that trigger a certain prediction from the model, which is a critical challenge for advancing the explainability and trustworthiness of current TGNNs. To address this challenge, we propose a novel approach, called Temporal Motifs Explainer (TempME), which uncovers the most pivotal temporal motifs guiding the prediction of TGNNs. Derived from the information bottleneck principle, TempME extracts the most interaction-related motifs while minimizing the amount of contained information to preserve the sparsity and succinctness of the explanation. Events in the explanations generated by TempME are verified to be more spatiotemporally correlated than those of existing approaches, providing more understandable insights. Extensive experiments validate the superiority of TempME, with up to 8.21% increase in terms of explanation accuracy across six real-world datasets and up to 22.96% increase in boosting the prediction Average Precision of current TGNNs.
Towards Self-Interpretable Graph-Level Anomaly Detection
Authors: Yixin Liu, Kaize Ding, Qinghua Lu, Fuyi Li, Leo Yu Zhang, Shirui Pan
Date: 2023-10-25 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Anomaly Detection, Graph Anomaly Detection

Graph-level anomaly detection (GLAD) aims to identify graphs that exhibit notable dissimilarity compared to the majority in a collection. However, current works primarily focus on evaluating graph-level abnormality while failing to provide meaningful explanations for the predictions, which largely limits their reliability and application scope. In this paper, we investigate a new challenging problem, explainable GLAD, where the learning objective is to predict the abnormality of each graph sample with corresponding explanations, i.e., the vital subgraph that leads to the predictions. To address this challenging problem, we propose a Self-Interpretable Graph aNomaly dETection model (SIGNET for short) that detects anomalous graphs as well as generates informative explanations simultaneously. Specifically, we first introduce the multi-view subgraph information bottleneck (MSIB) framework, serving as the design basis of our self-interpretable GLAD approach. This way SIGNET is able to not only measure the abnormality of each graph based on cross-view mutual information but also provide informative graph rationales by extracting bottleneck subgraphs from the input graph and its dual hypergraph in a self-supervised way. Extensive experiments on 16 datasets demonstrate the anomaly detection capability and self-interpretability of SIGNET.
Interpretable Prototype-based Graph Information Bottleneck
Authors: Sangwoo Seo, Sungwon Kim, Chanyoung Park
Date: 2023-10-30 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Decision Making, Prediction

The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.
Facial Expression Recognition using Residual Masking Network
Authors: Luan Pham, The Huynh Vu, Tuan Anh Tran
Date: 2021-05-05 00:00:00
Proceeding: International Conference on Pattern Recognition 2021 5
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Automatic facial expression recognition (FER) has gained much attention due to its applications in human-computer interaction. Among the approaches to improve FER tasks, this paper focuses on deep architecture with the attention mechanism. We propose a novel Masking Idea to boost the performance of CNN in facial expression task. It uses a segmentation network to refine feature maps, enabling the network to focus on relevant information to make correct decisions. In experiments, we combine the ubiquitous Deep Residual Network and Unet-like architecture to produce a Residual Masking Network. The proposed method holds state-of-the-art (SOTA) accuracy on the well-known FER2013 and private VEMO datasets.
Implementation of improved RGBD 3D target detection model based on FPGA heterogeneous computing architecture
Authors: Yu Wang, Wenbin, FENG Chongchong YU, Xinyu Hu, Yuqiu ZHANG4
Date: 2022-08-15 00:00:00
Proceeding: 2022/08/15 2022 8
Tasks: CPU, Semantic Segmentation

In order to solve the problems of low model accuracy,poor computing power,poor parallel ability and excessive power consumption in the deployment of RGBD based 3 D target detection model at the embedded end,this paper first proposes an improved RGBD 3 D target detection model based on ENet semantic segmentation model,which takes ENet as the semantic segmentation network,RGB image and depth information are fused to realize 3 D target detection.Secondly,in order to apply the model at the edge,this paper constructs a lightweight network and cuts the network in the down-sampling stage of ENet model.Finally,this paper uses Xilinx ZCU104 as the hardware development kit,which takes FPGA as the auxiliary parallel operation unit and ARM as the main operation unit.It is a heterogeneous computing architecture with the ability to deal with complex operations.The architecture uses FPGA to accelerate the depth model in parallel,which improves the operation speed and reduces the power consumption.The test results of the model on ZCU104 are compared with other hardware.The results show that while ensuring the accuracy,the power consumption of the heterogeneous computing architecture used in this paper is 93% lowerthan that of Intel Xeon e5-2620 v4 CPU,the speed is 12 times higher,and the speed is more than 180 times higher than that of ARM Cortex-A53 commonly used at the edge.
Dual-domain strip attention for image restoration
Authors: Yuning Cui, Alois Knoll
Date: 2024-03-01 00:00:00
Proceeding: Neural Networks 2024 3
Tasks: Deblurring, Denoising, Image Defocus Deblurring, Image Dehazing, Image Denoising, Image Restoration, Representation Learning

Image restoration aims to reconstruct a latent high-quality image from a degraded observation. Recently, the usage of Transformer has significantly advanced the state-of-the-art performance of various image restoration tasks due to its powerful ability to model long-range dependencies. However, the quadratic complexity of self-attention hinders practical applications. Moreover, sufficiently leveraging the huge spectral disparity between clean and degraded image pairs can also be conducive to image restoration. In this paper, we develop a dual-domain strip attention mechanism for image restoration by enhancing representation learning, which consists of spatial and frequency strip attention units. Specifically, the spatial strip attention unit harvests the contextual information for each pixel from its adjacent locations in the same row or column under the guidance of the learned weights via a simple convolutional branch. In addition, the frequency strip attention unit refines features in the spectral domain via frequency separation and modulation, which is implemented by simple pooling techniques. Furthermore, we apply different strip sizes for enhancing multi-scale learning, which is beneficial for handling degradations of various sizes. By employing the dual-domain attention units in different directions, each pixel can implicitly perceive information from an expanded region. Taken together, the proposed dual-domain strip attention network (DSANet) achieves state-of-the-art performance on 12 different datasets for four image restoration tasks, including image dehazing, image desnowing, image denoising, and image defocus deblurring.
Congestion barcodes: Exploring the topology of urban congestion using persistent homology
Authors: Yu Wu    Gabriel Shindnes     Vaibhav Karve    Derrek Yager    Daniel B. Work    Arnab Chakraborty    Richard B. Sowers
Date: 2017-10-01 00:00:00
Proceeding: IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) 2017 10
Tasks: Topological Data Analysis

This work presents a new method to quantify connectivity in transportation networks. Inspired by the field of topological data analysis, we propose a novel approach to explore the robustness of road network connectivity in the presence of congestion on the roadway. The robustness of the pattern is summarized in a congestion barcode, which can be constructed directly from traffic datasets commonly used for navigation. As an initial demonstration, we illustrate the main technique on a publicly available traffic dataset in a neighborhood in New York City.
Attributed Graph Clustering with Dual Redundancy Reduction
Authors: Lei Gong, Sihang Zhou, Wenxuan Tu and Xinwang Liu∗
Date: 2022-04-18 00:00:00
Proceeding: Conference 2022 4
Tasks: Clustering, Contrastive Learning, Diversity, Graph Clustering

Attributed graph clustering is a basic yet essential
method for graph data exploration. Recent efforts
over graph contrastive learning have achieved impressive clustering performance. However, we observe that the commonly adopted InfoMax operation tends to capture redundant information, limiting the downstream clustering performance. To
this end, we develop a novel method termed attributed graph clustering with dual redundancy reduction (AGC-DRR) to reduce the information redundancy in both input space and latent feature
space. Specifcally, for the input space redundancy reduction, we introduce an adversarial learning mechanism to adaptively learn a redundant
edge-dropping matrix to ensure the diversity of the
compared sample pairs. To reduce the redundancy
in the latent space, we force the correlation matrix of the cross-augmentation sample embedding
to approximate an identity matrix. Consequently,
the learned network is forced to be robust against
perturbation while discriminative against different samples. Extensive experiments have demonstrated that AGC-DRR outperforms the state-ofthe-art clustering methods on most of our benchmarks. The corresponding code is available at
https://github.com/gongleii/AGC-DRR.
Multi-Head Spatiotemporal Attention Graph Convolutional Network for Traffic Prediction
Authors: Oluwasanmi A, Aftab MU, Qin Z, Sarfraz MS, Yu Y, Rauf HT
Date: 2023-04-09 00:00:00
Proceeding: Sensors 2023 4
Tasks: Traffic Prediction

Intelligent transportation systems (ITSs) have become an indispensable component of modern global technological development, as they play a massive role in the accurate statistical estimation of vehicles or individuals commuting to a particular transportation facility at a given time. This provides the perfect backdrop for designing and engineering an adequate infrastructural capacity for transportation analyses. However, traffic prediction remains a daunting task due to the non-Euclidean and complex distribution of road networks and the topological constraints of urbanized road networks. To solve this challenge, this paper presents a traffic forecasting model which combines a graph convolutional network, a gated recurrent unit, and a multi-head attention mechanism to simultaneously capture and incorporate the spatio-temporal dependence and dynamic variation in the topological sequence of traffic data effectively. By achieving 91.8% accuracy on the Los Angeles highway traffic (Los-loop) test data for 15-min traffic prediction and an R2 score of 85% on the Shenzhen City (SZ-taxi) test dataset for 15- and 30-min predictions, the proposed model demonstrated that it can learn the global spatial variation and the dynamic temporal sequence of traffic data over time. This has resulted in state-of-the-art traffic forecasting for the SZ-taxi and Los-loop datasets.
Scaffolding fundamentals and recent advances in sustainable scaffolding techniques for cultured meat development
Authors: AMM Nurul Alam, Chan-Jin Kim, So-Hee Kim, Swati Kumari, Eun-Yeong Lee, Young-Hwa Hwang, Seon-Tea Joo
Date: 2024-01-05 00:00:00

In cultured meat (CM) products the paramount significance lies in the fundamental attributes like texture and sensory of the processed end product. To cater to the tactile and gustatory preferences of real meat, the product needs to be designed to incorporate its texture and sensory attributes. Presently CM products are mainly grounded products like sausage, nugget, frankfurter, burger patty, surimi, and steak with less sophistication and need to mimic real meat to grapple with the traditional meat market. The existence of fibrous microstructure in connective and muscle tissues has attracted considerable interest in the realm of tissue engineering. Scaffolding plays an important role in CM production by aiding cell adhesion, growth, differentiation, and alignment. A wide array of scaffolding technologies has been developed for implementation in the realm of biomedical research. In recent years researchers also focus on edible scaffolding to ease the process of CM. However, it is imperative to implement cutting edge technologies like 3D scaffolds, 3D printing, electrospun nanofibers in order to advance the creation of sustainable and edible scaffolding methods in CM production, with the ultimate goal of replicating the sensory and nutritional attributes to mimic real meat cut. This review discusses recent advances in scaffolding techniques and biomaterials related to structured CM production and required advances to create muscle fiber structures to mimic real meat. Keywords: Cultured meat, Scaffolding, Biomaterials, Edible scaffolding, Electrospinning, 3D bioprinting, real meat.
Coming back home to start up a business? A comparison between youth from rural and urban backgrounds in China
Authors: Chih-HungnYuan, Dajiang Wang, Lihua Hong, Yehui Zou, Jiayu Wen
Date: 2022-08-09 00:00:00
Proceeding: Frontiers in  Psychology 2022 8

Youth entrepreneurship is regarded as an important part of rural revitalization. Against the backdrop of the rural revitalization strategy, the Chinese government has introduced many policies to encourage return-home entrepreneurship among young people. However, highly educated youth have a lower willingness to return home for entrepreneurship, and prefer urban entrepreneurship or getting a job in a city. Therefore, this study used a two-stage approach to explore the factors that influence young people’s contribution to the development of their homeland, the barriers they face, and the support mechanisms they need. The study found that many barriers affect young people’s intention to return home for entrepreneurship. In rural areas, young people consider lagging environmental development to be the biggest barrier. In urban areas, infrastructure, lack of entrepreneurial experience, and funding are the factors that concern young people the most. As they have limited entrepreneurial experience, young people in both rural and urban areas have a high demand for shared entrepreneurial experience, as well as entrepreneurship courses and mentoring. The government and universities should remove the barriers faced by young people, provide more assistance, improve the environment for young people engaging in return-home entrepreneurship, and form a good entrepreneurial ecology.