pwc-archive/papers-with-abstracts huggingface dataset:
Self-Supervised Learning for Point Clouds Data: A Survey
Authors: Changyu Zeng, Wei Wang, Anh Nguyen, Yutao Yue
Date: 2023-05-09 00:00:00
Tasks: Pedestrian Detection, Self-Supervised Learning, Survey

3D point clouds are a crucial type of data collected by LiDAR sensors and widely used in transportation applications due to its concise descriptions and accurate localization. Deep neural networks (DNNs) have achieved remarkable success in processing large amount of disordered and sparse 3D point clouds, especially in various computer vision tasks, such as pedestrian detection and vehicle recognition. Among all the learning paradigms, Self-Supervised Learning (SSL), an unsupervised training paradigm that mines effective information from the data itself, is considered as an essential solution to solve the time-consuming and labor-intensive data labelling problems via smart pre-training task design. This paper provides a comprehensive survey of recent advances on SSL for point clouds. We first present an innovative taxonomy, categorizing the existing SSL methods into four broad categories based on the pretexts' characteristics. Under each category, we then further categorize the methods into more fine-grained groups and summarize the strength and limitations of the representative methods. We also compare the performance of the notable SSL methods in literature on multiple downstream tasks on benchmark datasets both quantitatively and qualitatively. Finally, we propose a number of future research directions based on the identified limitations of existing SSL research on point clouds.
A Review of Benchmarks for Visual Defect Detection in the Manufacturing Industry
Authors: Philippe Carvalho, Alexandre Durupt, Yves GRANDVALET
Date: 2023-05-05 00:00:00
Tasks: Defect Detection

The field of industrial defect detection using machine learning and deep learning is a subject of active research. Datasets, also called benchmarks, are used to compare and assess research results. There is a number of datasets in industrial visual inspection, of varying quality. Thus, it is a difficult task to determine which dataset to use. Generally speaking, datasets which include a testing set, with precise labeling and made in real-world conditions should be preferred. We propose a study of existing benchmarks to compare and expose their characteristics and their use-cases. A study of industrial metrics requirements, as well as testing procedures, will be presented and applied to the studied benchmarks. We discuss our findings by examining the current state of benchmarks for industrial visual inspection, and by exposing guidelines on the usage of benchmarks.
MD-GCN: A Multi-Scale Temporal Dual Graph Convolution Network for Traffic Flow Prediction
Authors: Huang Xiaohui;Wang Junyang;Lan Yuanchun;Jiang Chaojie;Yuan Xinhua
Date: 2023-01-11 00:00:00
Proceeding: IAENG期刊 2023 1
Tasks: Graph Sampling, Management, Prediction, valid

The spatial–temporal prediction of traffic flow is very important for traffic management and planning. The most difficult challenges of traffic flow prediction are the temporal feature extraction and the spatial correlation extraction of nodes. Due to the complex spatial correlation between different roads and the dynamic trend of time patterns, traditional forecasting methods still have limitations in obtaining spatial–temporal correlation, which makes it difficult to extract more valid information. In order to improve the accuracy of the forecasting, this paper proposes a multi-scale temporal dual graph convolution network for traffic flow prediction (MD-GCN). Firstly, we propose a gated temporal convolution based on a channel attention and inception structure to extract multi-scale temporal dependence. Then, aiming at the complexity of the traffic spatial structure, we develop a dual graph convolution module including the graph sampling and aggregation submodule (GraphSAGE) and the mix-hop propagation graph convolution submodule (MGCN) to extract the local correlation and global correlation between neighbor nodes. Finally, extensive experiments are carried out on several public traffic datasets, and the experimental results show that our proposed algorithm outperforms the existing methods.
ENRICH: Multi-purposE dataset for beNchmaRking In Computer vision and pHotogrammetry
Authors: Davide Marelli, Luca Morelli, Elisa Mariarosaria Farella, Simone Bianco, Gianluigi Ciocca, Fabio Remondino
Date: 2023-04-01 00:00:00
Proceeding: ISPRS Journal of Photogrammetry and Remote Sensing 2023 4
Tasks: 3D Reconstruction, 3D Scene Reconstruction, Benchmarking, Depth Estimation, Key Point Matching, Monocular Depth Estimation

The availability of high-resolution data and accurate ground truth is essential to evaluate and compare methods and algorithms properly. Moreover, it is often difficult to acquire real data for a given application domain that is sufficiently representative and heterogeneous in terms of scene representation, acquisition conditions, point of view, etc. To overcome the limitations of available datasets, this paper presents a new synthetic, multi-purpose dataset called ENRICH for testing photogrammetric and computer vision algorithms. Compared to existing datasets, ENRICH offers higher resolution images rendered with different lighting conditions, camera orientations, scales, and fields of view. Specifically, ENRICH is composed of three sub-datasets: ENRICH-Aerial, ENRICH-Square, and ENRICH-Statue, each exhibiting different characteristics. We show the usefulness of the proposed dataset on several examples of photogrammetry and computer vision-related tasks such as: evaluation of hand-crafted and deep learning-based local features, effects of ground control points (GCPs) configuration on the 3D accuracy, and monocular depth estimation.
Single-stage uav detection and classification with yolov5: Mosaic data augmentation and panet
Authors: Fardad Dadboud, Vaibhav Patel, Varun Mehta, Miodrag Bolic
Date: 2021-11-16 00:00:00
Proceeding: 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) 2021 11
Tasks: 2D Object Detection, Data Augmentation, object-detection, Object Detection

In Drone-vs-Bird Detection Challenge in conjunction with the 4th International Workshop on Small-Drone Surveillance, Detection and Counteraction Techniques at IEEE AVSS 2021, we proposed a YOLOV5-based object detection model for small UAV detection and classification. YOLOV5 leverages PANet neck and mosaic augmentation which help in improving detection of small objects. We have combined the challenge dataset with one of the publicly available UAV air to air dataset having complex background and lighting conditions for training the model. The proposed approach achieved 0.96 Recall, 0.98 mAP0.5, and 0.71 mAP0.5:0.95 on the 10% randomly sampled dataset from the whole dataset.
A deep image retrieval network using Max-m-Min pooling and morphological feature generating residual blocks
Authors: Farzad Sabahi, M. Omair Ahmad and M.N.S. Swamy
Date: 2023-04-26 00:00:00
Proceeding: International Journal of Multimedia Information Retrieval 2023 4
Tasks: Image Retrieval, Retrieval

The textural and structural information contained in the images is very important for generating highly discriminative features for the task of image retrieval. Morphological operations are nonlinear mathematical operations that can provide such textural and structural information. In this work, a new residual block based on a module using morphological operations coupled with an edge extraction module is proposed. A novel pooling operation focusing on the edges of the images is also proposed. A deep convolutional network is then designed using the proposed residual block and the new pooling operation that significantly improves its representational capacity. Extensive experiments are carried out to show the effectiveness of the ideas used in the design of the proposed deep image retrieval network. The proposed network is shown to significantly outperform existing state-of-the-art image retrieval networks on various benchmark datasets.
LIMIT: Language Identification, Misidentification, and Translation using Hierarchical Models in 350+ Languages
Authors: Milind Agarwal, Md Mahfuz ibn Alam, Antonios Anastasopoulos
Date: 2023-05-23 00:00:00
Tasks: Language Identification, Translation

Knowing the language of an input text/audio is a necessary first step for using almost every NLP tool such as taggers, parsers, or translation systems. Language identification is a well-studied problem, sometimes even considered solved; in reality, due to lack of data and computational challenges, current systems cannot accurately identify most of the world's 7000 languages. To tackle this bottleneck, we first compile a corpus, MCS-350, of 50K multilingual and parallel children's stories in 350+ languages. MCS-350 can serve as a benchmark for language identification of short texts and for 1400+ new translation directions in low-resource Indian and African languages. Second, we propose a novel misprediction-resolution hierarchical model, LIMIt, for language identification that reduces error by 55% (from 0.71 to 0.32) on our compiled children's stories dataset and by 40% (from 0.23 to 0.14) on the FLORES-200 benchmark. Our method can expand language identification coverage into low-resource languages by relying solely on systemic misprediction patterns, bypassing the need to retrain large models from scratch.
Decoupled Kullback-Leibler Divergence Loss
Authors: Jiequan Cui, Zhuotao Tian, Zhisheng Zhong, Xiaojuan Qi, Bei Yu, Hanwang Zhang
Date: 2023-05-23 00:00:00
Tasks: Adversarial Defense, Adversarial Robustness, Knowledge Distillation

In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of 1) a weighted Mean Square Error (wMSE) loss and 2) a Cross-Entropy loss incorporating soft labels. Thanks to the decomposed formulation of DKL loss, we have identified two areas for improvement. Firstly, we address the limitation of KL/DKL in scenarios like knowledge distillation by breaking its asymmetric optimization property. This modification ensures that the $\mathbf{w}$MSE component is always effective during training, providing extra constructive cues. Secondly, we introduce class-wise global information into KL/DKL to mitigate bias from individual samples. With these two enhancements, we derive the Improved Kullback-Leibler (IKL) Divergence loss and evaluate its effectiveness by conducting experiments on CIFAR-10/100 and ImageNet datasets, focusing on adversarial training, and knowledge distillation tasks. The proposed approach achieves new state-of-the-art adversarial robustness on the public leaderboard -- RobustBench and competitive performance on knowledge distillation, demonstrating the substantial practical merits. Our code is available at https://github.com/jiequancui/DKL.
Trend Investigation of Biopotential Recording Front-End Channels for Invasive and Non-Invasive Applications
Authors: Taeju Lee, Minkyu Je
Date: 2023-05-22 00:00:00

This paper presents the trend of biopotential recording front-end channels developed from the 1970s to the 2020s while describing a basic background on the front-end channel design. Only the front-end channels that conduct electrical recording invasively and non-invasively are addressed. The front-end channels are investigated in terms of technology node, number of channels, supply voltage, noise efficiency factor, and power efficiency factor. Also, multi-faceted comparisons are made to figure out the correlation between these five categories. In each category, the design trend is presented over time, and related circuit techniques are discussed. While addressing the characteristics of circuit techniques used to improve the channel performance, what needs to be improved is also suggested.
MAILEX: Email Event and Argument Extraction
Authors: Saurabh Srivastava, Gaurav Singh, Shou Matsumoto, Ali Raz, Paulo Costa, Joshua Poore, Ziyu Yao
Date: 2023-05-22 00:00:00
Tasks: 4k, 8k, Event Extraction, In-Context Learning

In this work, we present the first dataset, MailEx, for performing event extraction from conversational email threads. To this end, we first proposed a new taxonomy covering 10 event types and 76 arguments in the email domain. Our final dataset includes 1.5K email threads and ~4K emails, which are annotated with totally ~8K event instances. To understand the task challenges, we conducted a series of experiments comparing three types of approaches, i.e., fine-tuned sequence labeling, fine-tuned generative extraction, and few-shot in-context learning. Our results showed that the task of email event extraction is far from being addressed, due to challenges lying in, e.g., extracting non-continuous, shared trigger spans, extracting non-named entity arguments, and modeling the email conversational history. Our work thus suggests more future investigations in this domain-specific event extraction task.
Segment Any Anomaly without Training via Hybrid Prompt Regularization
Authors: Yunkang Cao, Xiaohao Xu, Chen Sun, Yuqi Cheng, Zongwei Du, Liang Gao, Weiming Shen
Date: 2023-05-18 00:00:00
Tasks: Anomaly Detection, Anomaly Localization, Anomaly Segmentation, Segmentation, Zero-shot Generalization

We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for zero-shot anomaly segmentation with hybrid prompt regularization to improve the adaptability of modern foundation models. Existing anomaly segmentation models typically rely on domain-specific fine-tuning, limiting their generalization across countless anomaly patterns. In this work, inspired by the great zero-shot generalization ability of foundation models like Segment Anything, we first explore their assembly to leverage diverse multi-modal prior knowledge for anomaly localization. For non-parameter foundation model adaptation to anomaly segmentation, we further introduce hybrid prompts derived from domain expert knowledge and target image context as regularization. Our proposed SAA+ model achieves state-of-the-art performance on several anomaly segmentation benchmarks, including VisA, MVTec-AD, MTD, and KSDD2, in the zero-shot setting. We will release the code at \href{https://github.com/caoyunkang/Segment-Any-Anomaly}{https://github.com/caoyunkang/Segment-Any-Anomaly}.
SAD: Segment Any RGBD
Authors: Jun Cen, Yizheng Wu, Kewei Wang, Xingyi Li, Jingkang Yang, Yixuan Pei, Lingdong Kong, Ziwei Liu, Qifeng Chen
Date: 2023-05-23 00:00:00
Tasks: 3D Panoptic Segmentation, Open Vocabulary Semantic Segmentation, Open-Vocabulary Semantic Segmentation, Panoptic Segmentation, Segmentation, Semantic Segmentation

The Segment Anything Model (SAM) has demonstrated its effectiveness in segmenting any part of 2D RGB images. However, SAM exhibits a stronger emphasis on texture information while paying less attention to geometry information when segmenting RGB images. To address this limitation, we propose the Segment Any RGBD (SAD) model, which is specifically designed to extract geometry information directly from images. Inspired by the natural ability of humans to identify objects through the visualization of depth maps, SAD utilizes SAM to segment the rendered depth map, thus providing cues with enhanced geometry information and mitigating the issue of over-segmentation. We further include the open-vocabulary semantic segmentation in our framework, so that the 3D panoptic segmentation is fulfilled. The project is available on https://github.com/Jun-CEN/SegmentAnyRGBD.
MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra
Authors: Ye-Xin Lu, Yang Ai, Zhen-Hua Ling
Date: 2023-05-23 00:00:00
Tasks: Decoder, Denoising, Speech Enhancement

This paper proposes MP-SENet, a novel Speech Enhancement Network which directly denoises Magnitude and Phase spectra in parallel. The proposed MP-SENet adopts a codec architecture in which the encoder and decoder are bridged by convolution-augmented transformers. The encoder aims to encode time-frequency representations from the input noisy magnitude and phase spectra. The decoder is composed of parallel magnitude mask decoder and phase decoder, directly recovering clean magnitude spectra and clean-wrapped phase spectra by incorporating learnable sigmoid activation and parallel phase estimation architecture, respectively. Multi-level losses defined on magnitude spectra, phase spectra, short-time complex spectra, and time-domain waveforms are used to train the MP-SENet model jointly. Experimental results show that our proposed MP-SENet achieves a PESQ of 3.50 on the public VoiceBank+DEMAND dataset and outperforms existing advanced speech enhancement methods.
ChipGPT: How far are we from natural language hardware design
Authors: Kaiyan Chang, Ying Wang, Haimeng Ren, Mengdi Wang, Shengwen Liang, Yinhe Han, Huawei Li, Xiaowei Li
Date: 2023-05-23 00:00:00

As large language models (LLMs) like ChatGPT exhibited unprecedented machine intelligence, it also shows great performance in assisting hardware engineers to realize higher-efficiency logic design via natural language interaction. To estimate the potential of the hardware design process assisted by LLMs, this work attempts to demonstrate an automated design environment that explores LLMs to generate hardware logic designs from natural language specifications. To realize a more accessible and efficient chip development flow, we present a scalable four-stage zero-code logic design framework based on LLMs without retraining or finetuning. At first, the demo, ChipGPT, begins by generating prompts for the LLM, which then produces initial Verilog programs. Second, an output manager corrects and optimizes these programs before collecting them into the final design space. Eventually, ChipGPT will search through this space to select the optimal design under the target metrics. The evaluation sheds some light on whether LLMs can generate correct and complete hardware logic designs described by natural language for some specifications. It is shown that ChipGPT improves programmability, and controllability, and shows broader design optimization space compared to prior work and native LLMs alone.
Efficient Multi-Scale Attention Module with Cross-Spatial Learning
Authors: Daliang Ouyang, Su He, Guozhong Zhang, Mingzhu Luo, Huaiyong Guo, Jian Zhan, Zhijie Huang
Date: 2023-05-23 00:00:00
Tasks: Dimensionality Reduction, image-classification, Image Classification, object-detection, Object Detection

Remarkable effectiveness of the channel or spatial attention mechanisms for producing more discernible feature representation are illustrated in various computer vision tasks. However, modeling the cross-channel relationships with channel dimensionality reduction may bring side effect in extracting deep visual representations. In this paper, a novel efficient multi-scale attention (EMA) module is proposed. Focusing on retaining the information on per channel and decreasing the computational overhead, we reshape the partly channels into the batch dimensions and group the channel dimensions into multiple sub-features which make the spatial semantic features well-distributed inside each feature group. Specifically, apart from encoding the global information to re-calibrate the channel-wise weight in each parallel branch, the output features of the two parallel branches are further aggregated by a cross-dimension interaction for capturing pixel-level pairwise relationship. We conduct extensive ablation studies and experiments on image classification and object detection tasks with popular benchmarks (e.g., CIFAR-100, ImageNet-1k, MS COCO and VisDrone2019) for evaluating its performance.
Improving speech translation by fusing speech and text
Authors: Wenbiao Yin, Zhicheng Liu, Chengqi Zhao, Tao Wang, Jian Tong, Rong Ye
Date: 2023-05-23 00:00:00
Tasks: cross-modal alignment, Machine Translation, Translation

In speech translation, leveraging multimodal data to improve model performance and address limitations of individual modalities has shown significant effectiveness. In this paper, we harness the complementary strengths of speech and text, which are disparate modalities. We observe three levels of modality gap between them, denoted by Modal input representation, Modal semantic, and Modal hidden states. To tackle these gaps, we propose \textbf{F}use-\textbf{S}peech-\textbf{T}ext (\textbf{FST}), a cross-modal model which supports three distinct input modalities for translation: speech, text, and fused speech-text. We leverage multiple techniques for cross-modal alignment and conduct a comprehensive analysis to assess its impact on speech translation, machine translation, and fused speech-text translation. We evaluate FST on MuST-C, GigaST, and newstest benchmark. Experiments show that the proposed FST achieves an average 34.0 BLEU on MuST-C En$\rightarrow$De/Es/Fr (vs SOTA +1.1 BLEU). Further experiments demonstrate that FST does not degrade on MT task, as observed in prior works. Instead, it yields an average improvement of 3.2 BLEU over the pre-trained MT model.
Neural Image Re-Exposure
Authors: Xinyu Zhang, Hefei Huang, Xu Jia, Dong Wang, Huchuan Lu
Date: 2023-05-23 00:00:00
Tasks: Deblurring, Decoder, Joint Deblur and Frame Interpolation, Joint Deblur and Unrolling, Rolling Shutter Correction, Video Enhancement, Video Frame Interpolation, Video Restoration

The shutter strategy applied to the photo-shooting process has a significant influence on the quality of the captured photograph. An improper shutter may lead to a blurry image, video discontinuity, or rolling shutter artifact. Existing works try to provide an independent solution for each issue. In this work, we aim to re-expose the captured photo in post-processing to provide a more flexible way of addressing those issues within a unified framework. Specifically, we propose a neural network-based image re-exposure framework. It consists of an encoder for visual latent space construction, a re-exposure module for aggregating information to neural film with a desired shutter strategy, and a decoder for 'developing' neural film into a desired image. To compensate for information confusion and missing frames, event streams, which can capture almost continuous brightness changes, are leveraged in computing visual latent content. Both self-attention layers and cross-attention layers are employed in the re-exposure module to promote interaction between neural film and visual latent content and information aggregation to neural film. The proposed unified image re-exposure framework is evaluated on several shutter-related image recovery tasks and performs favorably against independent state-of-the-art methods.
Video Prediction Models as Rewards for Reinforcement Learning
Date: 2023-05-23 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Prediction, reinforcement-learning, Reinforcement Learning, Video Prediction

Specifying reward signals that allow agents to learn complex behaviors is a long-standing challenge in reinforcement learning. A promising approach is to extract preferences for behaviors from unlabeled videos, which are widely available on the internet. We present Video Prediction Rewards (VIPER), an algorithm that leverages pretrained video prediction models as action-free reward signals for reinforcement learning. Specifically, we first train an autoregressive transformer on expert videos and then use the video prediction likelihoods as reward signals for a reinforcement learning agent. VIPER enables expert-level control without programmatic task rewards across a wide range of DMC, Atari, and RLBench tasks. Moreover, generalization of the video prediction model allows us to derive rewards for an out-of-distribution environment where no expert data is available, enabling cross-embodiment generalization for tabletop manipulation. We see our work as starting point for scalable reward specification from unlabeled videos that will benefit from the rapid advances in generative modeling. Source code and datasets are available on the project website: https://escontrela.me/viper