pwc-archive/papers-with-abstracts huggingface dataset:
Blind Prediction of Natural Video Quality
Authors: Michele A. Saad, Alan C. Bovik, Christophe Charrier
Date: 2014-01-09 00:00:00
Proceeding: IEEE Transacations on Image Processing 2014 1
Tasks: Prediction, Video Quality Assessment, Visual Question Answering (VQA)

We propose a blind (no reference or NR) video quality evaluation model that is nondistortion specific. The approach relies on a spatio-temporal model of video scenes in the discrete cosine transform domain, and on a model that characterizes the type of motion occurring in the scenes, to predict video quality. We use the models to define video statistics and perceptual features that are the basis of a video quality assessment (VQA) algorithm that does not require the presence of a pristine video to compare against in order to predict a perceptual quality score. The contributions of this paper are threefold. 1) We propose a spatio-temporal natural scene statistics (NSS) model for videos. 2) We propose a motion model that quantifies motion coherency in video scenes. 3) We show that the proposed NSS and motion coherency models are appropriate for quality assessment of videos, and we utilize them to design a blind VQA algorithm that correlates highly with human judgments of quality. The proposed algorithm, called video BLIINDS, is tested on the LIVE VQA database and on the EPFL-PoliMi video database and shown to perform close to the level of top performing reduced and full reference VQA algorithms.
No-Reference Image Quality Assessment in the Spatial Domain
Authors: Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik
Date: 2012-08-17 00:00:00
Proceeding: IEEE Transacations on Image Processing 2012 8
Tasks: Blocking, Denoising, Image Denoising, Image Quality Assessment, No-Reference Image Quality Assessment

We propose a natural scene statistic-based distortion-generic blind/no-reference (NR) image quality assessment (IQA) model that operates in the spatial domain. The new model, dubbed blind/referenceless image spatial quality evaluator (BRISQUE) does not compute distortion-specific features, such as ringing, blur, or blocking, but instead uses scene statistics of locally normalized luminance coefficients to quantify possible losses of “naturalness” in the image due to the presence of distortions, thereby leading to a holistic measure of quality. The underlying features used derive from the empirical distribution of locally normalized luminances and products of locally normalized luminances under a spatial natural scene statistic model. No transformation to another coordinate frame (DCT, wavelet, etc.) is required, distinguishing it from prior NR IQA approaches. Despite its simplicity, we are able to show that BRISQUE is statistically better than the full-reference peak signal-to-noise ratio and the structural similarity index, and is highly competitive with respect to all present-day distortion-generic NR IQA algorithms. BRISQUE has very low computational complexity, making it well suited for real time applications. BRISQUE features may be used for distortion-identification as well. To illustrate a new practical application of BRISQUE, we describe how a nonblind image denoising algorithm can be augmented with BRISQUE in order to perform blind image denoising. Results show that BRISQUE augmentation leads to performance improvements over state-of-the-art methods. A software release of BRISQUE is available online: http://live.ece.utexas.edu/research/quality/BRISQUE_release.zip for public use and evaluation.
An Efficient Blood-Cell Segmentation for the Detection of Hematological Disorders
Authors: Pradeep Kumar Das, Sukadev Meher, Rutuparna Panda, and Ajith Abraham
Date: 2021-03-18 00:00:00
Proceeding: IEEE Transactions on Cybernetics 2021 3
Tasks: Cell Segmentation, Segmentation

The automatic segmentation of blood cells for detecting hematological disorders is a crucial job. It has a vital role in
diagnosis, treatment planning, and output evaluation. The existing methods suffer from the issues like noise, improper seed-point
detection, and oversegmentation problems, which are solved here
using a Laplacian-of-Gaussian (LoG)-based modified highboosting operation, bounded opening followed by fast radial symmetry
(BOFRS)-based seed-point detection, and hybrid ellipse fitting
(EF), respectively. This article proposes a novel hybrid EF-based
blood-cell segmentation approach, which may be used for detecting various hematological disorders. Our prime contributions
are: 1) more accurate seed-point detection based on BO-FRS;
2) a novel least-squares (LS)-based geometric EF approach;
and 3) an improved segmentation performance by employing
a hybridized version of geometric and algebraic EF techniques
retaining the benefits of both approaches. It is a computationally efficient approach since it hybridizes noniterative-geometric
and algebraic methods. Moreover, we propose to estimate the
minor and major axes based on the residue and residue offset
factors. The residue offset parameter, proposed here, yields more
accurate segmentation with proper EF. Our method is compared
with the state-of-the-art methods. It outperforms the existing EF
techniques in terms of dice similarity, Jaccard score, precision,
and F1 score. It may be useful for other medical and cybernetics
applications.
FTNet: Feature Transverse Network for Thermal Image Semantic Segmentation
Authors: Karen Panetta, Shreyas Kamath K.M, Srijith Rajeev and Sos S Agaian
Date: 2021-10-26 00:00:00
Proceeding: IEEE Access 2021 10
Tasks: Benchmarking, Scene Segmentation, Semantic Segmentation, Thermal Image Segmentation

Thermal imaging is a process of using infrared radiation and thermal energy to collect information about objects. It is superior to visible imaging for its ability to operate in darkness and tolerate illumination variations. In addition, it has potential to penetrate smoke, aerosol, dust, and mist, which are critical inhibitors for visible imaging applications, including semantic segmentation. Unfortunately, current state-of-the-art image semantic segmentation methods (i) mainly concentrate on visible spectrum images, which do not adequately capture the context of corresponding pixels, particularly edge details in thermal images, and (ii) accept a trade-off between higher accuracy and lower speed, or vice-versa. Here, a novel end-to-end trainable convolutional neural network architecture, feature transverse network (FTNet), has been proposed to solve the aforementioned problems. FTNet captures and optimizes feature representation at the multi-scale resolution, thereby improving the capability to process high-resolution images and producing quality output with a lower computational cost. Extensive computer experimentations were conducted on publicly available benchmarking thermal datasets, including SODA, MFNet, and SCUT-Seg, to demonstrate the effectiveness of the proposed FTNet compared to state-of-the-art methods. This comparison includes multiple aspects, including the quantitative accuracy and speed of the various approaches. The source code is available at https://github.com/shreyaskamathkm/FTNet.
Guided Attentive Feature Fusion for Multispectral Pedestrian Detection
Authors: Heng Zhang, Elisa Fromont, Sebastien Lefevre, Bruno AVIGNON3
Date: 2021-01-03 00:00:00
Proceeding: WACV 2021 1
Tasks: Multispectral Object Detection, object-detection, Object Detection, Pedestrian Detection

Multispectral image pairs can provide complementary
visual information, making pedestrian detection systems
more robust and reliable. To benefit from both RGB and
thermal IR modalities, we introduce a novel attentive multispectral feature fusion approach. Under the guidance of
the inter- and intra-modality attention modules, our deep
learning architecture learns to dynamically weigh and fuse
the multispectral features. Experiments on two public multispectral object detection datasets demonstrate that the proposed approach significantly improves the detection accuracy at a low computation cost.
HA U-Net: Improved Model for Building Extraction From High Resolution Remote Sensing Imagery
Authors: Leilei Xu; Yujun Liu; Peng Yang; Hao Chen; Hanyue Zhang; Dan Wang; Xin Zhang All Authors
Date: 2021-07-15 00:00:00
Proceeding: IEEE Access 2021 7
Tasks: Deep Learning

Deep learning
,
building extraction
,
holistically-nested neural network
,
attention mechanism
,
weight mapping
,
watershed algorithm
CN-Motifs Perceptive Graph Neural Networks
Authors: Fan Zhang, Tian-Ming Bu
Date: 2021-11-15 00:00:00
Proceeding: IEEE Access 2021 11
Tasks: Graph Representation Learning, Node Classification, Representation Learning

Graph neural networks (GNNs) have been the dominant approaches for graph representation learning. However, most GNNs are applied to homophily graphs and perform poorly on heterophily graphs. Meanwhile, these GNNs fail to directly capture long-range dependencies and complex interactions between 1-hop neighbors when generating node representations by iteratively aggregating directly connected neighbors. In addition, structural patterns, such as motifs which have been established as building blocks for graph structure, contain rich topological and semantical information and are worth studying further. In this paper, we introduce the common-neighbors based motifs, which we called CN-motifs, to generalize and enrich the definition of structural patterns. We group the 1-hop neighbors and construct a high-order graph according to CN-motifs, and propose CN-motifs Perceptive Graph Neural Networks (CNMPGNN), a novel framework which can effectively resolve problems mentioned above. Notably, by making full use of structural patterns, our model achieves the state-of-the-art results on several homophily and heterophily datasets.
Bypassing Detection of URL-based Phishing Attacks Using Generative Adversarial Deep Neural Networks
Authors: Ahmed AlEroud†, George Karabatis
Date: 2020-03-16 00:00:00
Proceeding: DOI: 10.1145/3375708.3380315 2020 3
Tasks: BIG-bench Machine Learning

The URL components of web addresses are frequently used in creating phishing detection techniques. Typically, machine learning techniques are widely used to identify anomalous patterns in URLs as signs of possible phishing. However, adversaries may have enough knowledge and motivation to bypass URL classification algorithms by creating examples that evade classification algorithms. This paper proposes an approach that generates URL-based phishing examples using Generative Adversarial Networks. The created examples can fool Blackbox phishing detectors even when those detectors are created using sophisticated approaches such as those relying on intra-URL similarities. These created instances are used to deceive Blackbox machine learning-based phishing detection models. We tested our approach using actual phishing datasets. The results show that GAN networks are very effective in creating adversarial phishing examples that can fool both simple and sophisticated machine learning phishing detection models.
DeMFI: Deep Joint Deblurring and Multi-Frame Interpolation with Flow-Guided Attentive Correlation and Recursive Boosting
Authors: Jihyong Oh, Munchurl Kim
Date: 2021-11-19 00:00:00
Tasks: Deblurring, Video Enhancement, Video Frame Interpolation, Video Restoration

In this paper, we propose a novel joint deblurring and multi-frame interpolation (DeMFI) framework, called DeMFI-Net, which accurately converts blurry videos of lower-frame-rate to sharp videos at higher-frame-rate based on flow-guided attentive-correlation-based feature bolstering (FAC-FB) module and recursive boosting (RB), in terms of multi-frame interpolation (MFI). The DeMFI-Net jointly performs deblurring and MFI where its baseline version performs feature-flow-based warping with FAC-FB module to obtain a sharp-interpolated frame as well to deblur two center-input frames. Moreover, its extended version further improves the joint task performance based on pixel-flow-based warping with GRU-based RB. Our FAC-FB module effectively gathers the distributed blurry pixel information over blurry input frames in feature-domain to improve the overall joint performances, which is computationally efficient since its attentive correlation is only focused pointwise. As a result, our DeMFI-Net achieves state-of-the-art (SOTA) performances for diverse datasets with significant margins compared to the recent SOTA methods, for both deblurring and MFI. All source codes including pretrained DeMFI-Net are publicly available at https://github.com/JihyongOh/DeMFI.
DeepQR: Neural-based Quality Ratings for Learnersourced Multiple-Choice Questions
Authors: Lin Ni, Qiming Bao, Xiaoxuan Li, Qianqian Qi, Paul Denny, Jim Warren, Michael Witbrock, Jiamou Liu
Date: 2021-11-19 00:00:00
Tasks: Contrastive Learning, Multiple-choice

Automated question quality rating (AQQR) aims to evaluate question quality through computational means, thereby addressing emerging challenges in online learnersourced question repositories. Existing methods for AQQR rely solely on explicitly-defined criteria such as readability and word count, while not fully utilising the power of state-of-the-art deep-learning techniques. We propose DeepQR, a novel neural-network model for AQQR that is trained using multiple-choice-question (MCQ) datasets collected from PeerWise, a widely-used learnersourcing platform. Along with designing DeepQR, we investigate models based on explicitly-defined features, or semantic features, or both. We also introduce a self-attention mechanism to capture semantic correlations between MCQ components, and a contrastive-learning approach to acquire question representations using quality ratings. Extensive experiments on datasets collected from eight university-level courses illustrate that DeepQR has superior performance over six comparative models.
UN-AVOIDS: Unsupervised and Nonparametric Approach for Visualizing Outliers and Invariant Detection Scoring
Authors: Waleed A. Yousef, Issa Traore, William Briguglio
Date: 2021-11-19 00:00:00
Tasks: Anomaly Detection, Data Visualization

The visualization and detection of anomalies (outliers) are of crucial importance to many fields, particularly cybersecurity. Several approaches have been proposed in these fields, yet to the best of our knowledge, none of them has fulfilled both objectives, simultaneously or cooperatively, in one coherent framework. The visualization methods of these approaches were introduced for explaining the output of a detection algorithm, not for data exploration that facilitates a standalone visual detection. This is our point of departure: UN-AVOIDS, an unsupervised and nonparametric approach for both visualization (a human process) and detection (an algorithmic process) of outliers, that assigns invariant anomalous scores (normalized to $[0,1]$), rather than hard binary-decision. The main aspect of novelty of UN-AVOIDS is that it transforms data into a new space, which is introduced in this paper as neighborhood cumulative density function (NCDF), in which both visualization and detection are carried out. In this space, outliers are remarkably visually distinguishable, and therefore the anomaly scores assigned by the detection algorithm achieved a high area under the ROC curve (AUC). We assessed UN-AVOIDS on both simulated and two recently published cybersecurity datasets, and compared it to three of the most successful anomaly detection methods: LOF, IF, and FABOD. In terms of AUC, UN-AVOIDS was almost an overall winner. The article concludes by providing a preview of new theoretical and practical avenues for UN-AVOIDS. Among them is designing a visualization aided anomaly detection (VAAD), a type of software that aids analysts by providing UN-AVOIDS' detection algorithm (running in a back engine), NCDF visualization space (rendered to plots), along with other conventional methods of visualization in the original feature space, all of which are linked in one interactive environment.
Subspace Graph Physics: Real-Time Rigid Body-Driven Granular Flow Simulation
Authors: Amin Haeri, Krzysztof Skonieczny
Date: 2021-11-18 00:00:00
Tasks: GPU

An important challenge in robotics is understanding the interactions between robots and deformable terrains that consist of granular material. Granular flows and their interactions with rigid bodies still pose several open questions. A promising direction for accurate, yet efficient, modeling is using continuum methods. Also, a new direction for real-time physics modeling is the use of deep learning. This research advances machine learning methods for modeling rigid body-driven granular flows, for application to terrestrial industrial machines as well as space robotics (where the effect of gravity is an important factor). In particular, this research considers the development of a subspace machine learning simulation approach. To generate training datasets, we utilize our high-fidelity continuum method, material point method (MPM). Principal component analysis (PCA) is used to reduce the dimensionality of data. We show that the first few principal components of our high-dimensional data keep almost the entire variance in data. A graph network simulator (GNS) is trained to learn the underlying subspace dynamics. The learned GNS is then able to predict particle positions and interaction forces with good accuracy. More importantly, PCA significantly enhances the time and memory efficiency of GNS in both training and rollout. This enables GNS to be trained using a single desktop GPU with moderate VRAM. This also makes the GNS real-time on large-scale 3D physics configurations (700x faster than our continuum method).
Rethinking Dilated Convolution for Real-time Semantic Segmentation
Authors: Roland Gao
Date: 2021-11-18 00:00:00
Tasks: Decoder, GPU, Neural Architecture Search, Real-Time Semantic Segmentation, Semantic Segmentation

The field-of-view is an important metric when designing a model for semantic segmentation. To obtain a large field-of-view, previous approaches generally choose to rapidly downsample the resolution, usually with average poolings or stride 2 convolutions. We take a different approach by using dilated convolutions with large dilation rates throughout the backbone, allowing the backbone to easily tune its field-of-view by adjusting its dilation rates, and show that it's competitive with existing approaches. To effectively use the dilated convolution, we show a simple upper bound on the dilation rate in order to not leave gaps in between the convolutional weights, and design an SE-ResNeXt inspired block structure that uses two parallel $3\times 3$ convolutions with different dilation rates to preserve the local details. Manually tuning the dilation rates for every block can be difficult, so we also introduce a differentiable neural architecture search method that uses gradient descent to optimize the dilation rates. In addition, we propose a lightweight decoder that restores local information better than common alternatives. To demonstrate the effectiveness of our approach, our model RegSeg achieves competitive results on real-time Cityscapes and CamVid datasets. Using a T4 GPU with mixed precision, RegSeg achieves 78.3 mIOU on Cityscapes test set at $37$ FPS, and 80.9 mIOU on CamVid test set at $112$ FPS, both without ImageNet pretraining.
Generalized Decision Transformer for Offline Hindsight Information Matching
Authors: Hiroki Furuta, Yutaka Matsuo, Shixiang Shane Gu
Date: 2021-11-19 00:00:00
Tasks: continuous-control, Continuous Control, Imitation Learning, MuJoCo, Reinforcement Learning (RL)

How to extract as much learning signal from each trajectory data has been a key problem in reinforcement learning (RL), where sample inefficiency has posed serious challenges for practical applications. Recent works have shown that using expressive policy function approximators and conditioning on future trajectory information -- such as future states in hindsight experience replay or returns-to-go in Decision Transformer (DT) -- enables efficient learning of multi-task policies, where at times online RL is fully replaced by offline behavioral cloning, e.g. sequence modeling. We demonstrate that all these approaches are doing hindsight information matching (HIM) -- training policies that can output the rest of trajectory that matches some statistics of future state information. We present Generalized Decision Transformer (GDT) for solving any HIM problem, and show how different choices for the feature function and the anti-causal aggregator not only recover DT as a special case, but also lead to novel Categorical DT (CDT) and Bi-directional DT (BDT) for matching different statistics of the future. For evaluating CDT and BDT, we define offline multi-task state-marginal matching (SMM) and imitation learning (IL) as two generic HIM problems, propose a Wasserstein distance loss as a metric for both, and empirically study them on MuJoCo continuous control benchmarks. CDT, which simply replaces anti-causal summation with anti-causal binning in DT, enables the first effective offline multi-task SMM algorithm that generalizes well to unseen and even synthetic multi-modal state-feature distributions. BDT, which uses an anti-causal second transformer as the aggregator, can learn to model any statistics of the future and outperforms DT variants in offline multi-task IL. Our generalized formulations from HIM and GDT greatly expand the role of powerful sequence modeling architectures in modern RL.
Solving Visual Analogies Using Neural Algorithmic Reasoning
Authors: Atharv Sonwane, Gautam Shroff, Lovekesh Vig, Ashwin Srinivasan, Tirtharaj Dash
Date: 2021-11-19 00:00:00
Tasks: Program Synthesis, Visual Analogies

We consider a class of visual analogical reasoning problems that involve discovering the sequence of transformations by which pairs of input/output images are related, so as to analogously transform future inputs. This program synthesis task can be easily solved via symbolic search. Using a variation of the `neural analogical reasoning' approach of (Velickovic and Blundell 2021), we instead search for a sequence of elementary neural network transformations that manipulate distributed representations derived from a symbolic space, to which input images are directly encoded. We evaluate the extent to which our `neural reasoning' approach generalizes for images with unseen shapes and positions.
DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion
Authors: Renrui Zhang, Ziyao Zeng, Ziyu Guo, Xinben Gao, Kexue Fu, Jianbo Shi
Date: 2021-11-19 00:00:00
Tasks: 3D Part Segmentation, 3D Point Cloud Classification, 3D Shape Classification, Scene Segmentation, Semantic Segmentation, Vocal Bursts Intensity Prediction

Point cloud processing is a challenging task due to its sparsity and irregularity. Prior works introduce delicate designs on either local feature aggregator or global geometric architecture, but few combine both advantages. We propose Dual-Scale Point Cloud Recognition with High-frequency Fusion (DSPoint) to extract local-global features by concurrently operating on voxels and points. We reverse the conventional design of applying convolution on voxels and attention to points. Specifically, we disentangle point features through channel dimension for dual-scale processing: one by point-wise convolution for fine-grained geometry parsing, the other by voxel-wise global attention for long-range structural exploration. We design a co-attention fusion module for feature alignment to blend local-global modalities, which conducts inter-scale cross-modality interaction by communicating high-frequency coordinates information. Experiments and ablations on widely-adopted ModelNet40, ShapeNet, and S3DIS demonstrate the state-of-the-art performance of our DSPoint.