pwc-archive/papers-with-abstracts huggingface dataset:
Boosting CNN-based primary quantization matrix estimation of double JPEG images via a classification-like architecture
Authors: Benedetta Tondi, Andrea Costranzo, Dequ Huang, Bin Li
Date: 2020-12-01 00:00:00
Tasks: Image Forensics, Quantization, regression

Estimating the primary quantization matrix of double JPEG compressed images is a problem of relevant importance in image forensics since it allows to infer important information about the past history of an image. In addition, the inconsistencies of the primary quantization matrices across different image regions can be used to localize splicing in double JPEG tampered images. Traditional model-based approaches work under specific assumptions on the relationship between the first and second compression qualities and on the alignment of the JPEG grid. Recently, a deep learning-based estimator capable to work under a wide variety of conditions has been proposed, that outperforms tailored existing methods in most of the cases. The method is based on a Convolutional Neural Network (CNN) that is trained to solve the estimation as a standard regression problem. By exploiting the integer nature of the quantization coefficients, in this paper, we propose a deep learning technique that performs the estimation by resorting to a simil-classification architecture. The CNN is trained with a loss function that takes into account both the accuracy and the Mean Square Error (MSE) of the estimation. Results confirm the superior performance of the proposed technique, compared to the state-of-the art methods based on statistical analysis and, in particular, deep learning regression. Moreover, the capability of the method to work under general operative conditions, regarding the alignment of the second compression grid with the one of first compression and the combinations of the JPEG qualities of former and second compression, is very relevant in practical applications, where these information are unknown a priori.
Minimal Solutions for Panoramic Stitching Given Gravity Prior
Authors: Yaqing Ding, Daniel Barath, Zuzana Kukelova
Date: 2020-12-01 00:00:00
Proceeding: ICCV 2021 10
Tasks: Image Stitching

When capturing panoramas, people tend to align their cameras with the vertical axis, i.e., the direction of gravity. Moreover, modern devices, such as smartphones and tablets, are equipped with an IMU (Inertial Measurement Unit) that can measure the gravity vector accurately. Using this prior, the y-axes of the cameras can be aligned or assumed to be already aligned, reducing their relative orientation to 1-DOF (degree of freedom). Exploiting this assumption, we propose new minimal solutions to panoramic image stitching of images taken by cameras with coinciding optical centers, i.e., undergoing pure rotation. We consider four practical camera configurations, assuming unknown fixed or varying focal length with or without radial distortion. The solvers are tested both on synthetic scenes and on more than 500k real image pairs from the Sun360 dataset and from scenes captured by us using two smartphones equipped with IMUs. It is shown, that they outperform the state-of-the-art both in terms of accuracy and processing time.
(k, l)-Medians Clustering of Trajectories Using Continuous Dynamic Time Warping
Authors: Milutin Brankovic, Kevin Buchin, Koen Klaren, André Nusser, Aleksandr Popov, Sampson Wong
Date: 2020-12-01 00:00:00
Tasks: Clustering, Dynamic Time Warping, Trajectory Clustering

Due to the massively increasing amount of available geospatial data and the need to present it in an understandable way, clustering this data is more important than ever. As clusters might contain a large number of objects, having a representative for each cluster significantly facilitates understanding a clustering. Clustering methods relying on such representatives are called center-based. In this work we consider the problem of center-based clustering of trajectories. In this setting, the representative of a cluster is again a trajectory. To obtain a compact representation of the clusters and to avoid overfitting, we restrict the complexity of the representative trajectories by a parameter l. This restriction, however, makes discrete distance measures like dynamic time warping (DTW) less suited. There is recent work on center-based clustering of trajectories with a continuous distance measure, namely, the Fr\'echet distance. While the Fr\'echet distance allows for restriction of the center complexity, it can also be sensitive to outliers, whereas averaging-type distance measures, like DTW, are less so. To obtain a trajectory clustering algorithm that allows restricting center complexity and is more robust to outliers, we propose the usage of a continuous version of DTW as distance measure, which we call continuous dynamic time warping (CDTW). Our contribution is twofold: 1. To combat the lack of practical algorithms for CDTW, we develop an approximation algorithm that computes it. 2. We develop the first clustering algorithm under this distance measure and show a practical way to compute a center from a set of trajectories and subsequently iteratively improve it. To obtain insights into the results of clustering under CDTW on practical data, we conduct extensive experiments.
Towards a Universal Features Set for IoT Botnet Attacks Detection
Authors: Faisal Hussain, Syed Ghazanfar Abbas, Ubaid U. Fayyaz, Ghalib A. Shah, Abdullah Toqeer, Ahmad Ali
Date: 2020-12-01 00:00:00
Tasks: BIG-bench Machine Learning, Diversity

The security pitfalls of IoT devices make it easy for the attackers to exploit the IoT devices and make them a part of a botnet. Once hundreds of thousands of IoT devices are compromised and become the part of a botnet, the attackers use this botnet to launch the large and complex distributed denial of service (DDoS) attacks which take down the target websites or services and make them unable to respond the legitimate users. So far, many botnet detection techniques have been proposed but their performance is limited to a specific dataset on which they are trained. This is because the features used to train a machine learning model on one botnet dataset, do not perform well on other datasets due to the diversity of attack patterns. Therefore, in this paper, we propose a universal features set to better detect the botnet attacks regardless of the underlying dataset. The proposed features set manifest preeminent results for detecting the botnet attacks when tested the trained machine learning models over three different botnet attack datasets.
Unsupervised Anomaly Detection From Semantic Similarity Scores
Authors: Nima Rafiee, Rahil Gholamipoor, Markus Kollmann
Date: 2020-12-01 00:00:00
Tasks: Anomaly Detection, Out-of-Distribution Detection, Semantic Similarity, Semantic Textual Similarity, Unsupervised Anomaly Detection

Classifying samples as in-distribution or out-of-distribution (OOD) is a challenging problem of anomaly detection and a strong test of the generalisation power for models of the in-distribution. In this paper, we present a simple and generic framework, {\it SemSAD}, that makes use of a semantic similarity score to carry out anomaly detection. The idea is to first find for any test example the semantically closest examples in the training set, where the semantic relation between examples is quantified by the cosine similarity between feature vectors that leave semantics unchanged under transformations, such as geometric transformations (images), time shifts (audio signals), and synonymous word substitutions (text). A trained discriminator is then used to classify a test example as OOD if the semantic similarity to its nearest neighbours is significantly lower than the corresponding similarity for test examples from the in-distribution. We are able to outperform previous approaches for anomaly, novelty, or out-of-distribution detection in the visual domain by a large margin. In particular, we obtain AUROC values close to one for the challenging task of detecting examples from CIFAR-10 as out-of-distribution given CIFAR-100 as in-distribution, without making use of label information.
Improved Variational Bayesian Phylogenetic Inference with Normalizing Flows
Authors: Cheng Zhang
Date: 2020-12-01 00:00:00
Proceeding: NeurIPS 2020 12

Variational Bayesian phylogenetic inference (VBPI) provides a promising general variational framework for efficient estimation of phylogenetic posteriors. However, the current diagonal Lognormal branch length approximation would significantly restrict the quality of the approximating distributions. In this paper, we propose a new type of VBPI, VBPI-NF, as a first step to empower phylogenetic posterior estimation with deep learning techniques. By handling the non-Euclidean branch length space of phylogenetic models with carefully designed permutation equivariant transformations, VBPI-NF uses normalizing flows to provide a rich family of flexible branch length distributions that generalize across different tree topologies. We show that VBPI-NF significantly improves upon the vanilla VBPI on a benchmark of challenging real data Bayesian phylogenetic inference problems. Further investigation also reveals that the structured parameterization in those permutation equivariant transformations can provide additional amortization benefit.
Globally Optimal Relative Pose Estimation with Gravity Prior
Authors: Yaqing Ding, Daniel Barath, Jian Yang, Hui Kong, Zuzana Kukelova
Date: 2020-12-01 00:00:00
Proceeding: CVPR 2021 1
Tasks: Pose Estimation

Smartphones, tablets and camera systems used, e.g., in cars and UAVs, are typically equipped with IMUs (inertial measurement units) that can measure the gravity vector accurately. Using this additional information, the $y$-axes of the cameras can be aligned, reducing their relative orientation to a single degree-of-freedom. With this assumption, we propose a novel globally optimal solver, minimizing the algebraic error in the least-squares sense, to estimate the relative pose in the over-determined case. Based on the epipolar constraint, we convert the optimization problem into solving two polynomials with only two unknowns. Also, a fast solver is proposed using the first-order approximation of the rotation. The proposed solvers are compared with the state-of-the-art ones on four real-world datasets with approx. 50000 image pairs in total. Moreover, we collected a dataset, by a smartphone, consisting of 10933 image pairs, gravity directions, and ground truth 3D reconstructions.
Counting People by Estimating People Flows
Authors: Weizhe Liu, Mathieu Salzmann, Pascal Fua
Date: 2020-12-01 00:00:00
Tasks: Active Learning, Crowd Counting, Optical Flow Estimation

Modern methods for counting people in crowded scenes rely on deep networks to estimate people densities in individual images. As such, only very few take advantage of temporal consistency in video sequences, and those that do only impose weak smoothness constraints across consecutive frames. In this paper, we advocate estimating people flows across image locations between consecutive images and inferring the people densities from these flows instead of directly regressing them. This enables us to impose much stronger constraints encoding the conservation of the number of people. As a result, it significantly boosts performance without requiring a more complex architecture. Furthermore, it allows us to exploit the correlation between people flow and optical flow to further improve the results. We also show that leveraging people conservation constraints in both a spatial and temporal manner makes it possible to train a deep crowd counting model in an active learning setting with much fewer annotations. This significantly reduces the annotation cost while still leading to similar performance to the full supervision case.
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
Authors: Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid
Date: 2020-12-01 00:00:00
Proceeding: ICCV 2021 10
Tasks: Question Answering, Question Generation, Question-Generation, Video Question Answering, Visual Question Answering, Visual Question Answering (VQA), Zero-Shot Learning

Recent methods for visual question answering rely on large-scale annotated datasets. Manual annotation of questions and answers for videos, however, is tedious, expensive and prevents scalability. In this work, we propose to avoid manual annotation and generate a large-scale training dataset for video question answering making use of automatic cross-modal supervision. We leverage a question generation transformer trained on text data and use it to generate question-answer pairs from transcribed video narrations. Given narrated videos, we then automatically generate the HowToVQA69M dataset with 69M video-question-answer triplets. To handle the open vocabulary of diverse answers in this dataset, we propose a training procedure based on a contrastive loss between a video-question multi-modal transformer and an answer transformer. We introduce the zero-shot VideoQA task and show excellent results, in particular for rare answers. Furthermore, we demonstrate our method to significantly outperform the state of the art on MSRVTT-QA, MSVD-QA, ActivityNet-QA and How2QA. Finally, for a detailed evaluation we introduce iVQA, a new VideoQA dataset with reduced language biases and high-quality redundant manual annotations. Our code, datasets and trained models are available at https://antoyang.github.io/just-ask.html.
A Unified Structure for Efficient RGB and RGB-D Salient Object Detection
Authors: Peng Peng, Yong-Jie Li
Date: 2020-12-01 00:00:00
Tasks: object-detection, Object Detection, RGB-D Salient Object Detection, Salient Object Detection

Salient object detection (SOD) has been well studied in recent years, especially using deep neural networks. However, SOD with RGB and RGB-D images is usually treated as two different tasks with different network structures that need to be designed specifically. In this paper, we proposed a unified and efficient structure with a cross-attention context extraction (CRACE) module to address both tasks of SOD efficiently. The proposed CRACE module receives and appropriately fuses two (for RGB SOD) or three (for RGB-D SOD) inputs. The simple unified feature pyramid network (FPN)-like structure with CRACE modules conveys and refines the results under the multi-level supervisions of saliency and boundaries. The proposed structure is simple yet effective; the rich context information of RGB and depth can be appropriately extracted and fused by the proposed structure efficiently. Experimental results show that our method outperforms other state-of-the-art methods in both RGB and RGB-D SOD tasks on various datasets and in terms of most metrics.
Unsupervised Segmentation for Terracotta Warrior Point Cloud (SRG-Net)
Authors: Yao Hu, Guohua Geng, Kang Li, Wei Zhou
Date: 2020-12-01 00:00:00
Tasks: Clustering, Segmentation

The repairing work of terracotta warriors in Emperor Qinshihuang Mausoleum Site Museum is handcrafted by experts, and the increasing amounts of unearthed pieces of terracotta warriors make the archaeologists too challenging to conduct the restoration of terracotta warriors efficiently. We hope to segment the 3D point cloud data of the terracotta warriors automatically and store the fragment data in the database to assist the archaeologists in matching the actual fragments with the ones in the database, which could result in higher repairing efficiency of terracotta warriors. Moreover, the existing 3D neural network research is mainly focusing on supervised classification, clustering, unsupervised representation, and reconstruction. There are few pieces of researches concentrating on unsupervised point cloud part segmentation. In this paper, we present SRG-Net for 3D point clouds of terracotta warriors to address these problems. Firstly, we adopt a customized seed-region-growing algorithm to segment the point cloud coarsely. Then we present a supervised segmentation and unsupervised reconstruction networks to learn the characteristics of 3D point clouds. Finally, we combine the SRG algorithm with our improved CNN(convolution neural network) using a refinement method. This pipeline is called SRG-Net, which aims at conducting segmentation tasks on the terracotta warriors. Our proposed SRG-Net is evaluated on the terracotta warrior data and ShapeNet dataset by measuring the accuracy and the latency. The experimental results show that our SRG-Net outperforms the state-of-the-art methods. Our code is available at https://github.com/hyoau/SRG-Net.
A Generative Model to Synthesize EEG Data for Epileptic Seizure Prediction
Authors: Khansa Rasheed, Junaid Qadir, Terence J. O'Brien, Levin Kuhlmann, Adeel Razi
Date: 2020-12-01 00:00:00
Tasks: Data Augmentation, EEG, Electroencephalogram (EEG), Generative Adversarial Network, Prediction, Seizure prediction, Transfer Learning

Prediction of seizure before they occur is vital for bringing normalcy to the lives of patients. Researchers employed machine learning methods using hand-crafted features for seizure prediction. However, ML methods are too complicated to select the best ML model or best features. Deep Learning methods are beneficial in the sense of automatic feature extraction. One of the roadblocks for accurate seizure prediction is scarcity of epileptic seizure data. This paper addresses this problem by proposing a deep convolutional generative adversarial network to generate synthetic EEG samples. We use two methods to validate synthesized data namely, one-class SVM and a new proposal which we refer to as convolutional epileptic seizure predictor (CESP). Another objective of our study is to evaluate performance of well-known deep learning models (e.g., VGG16, VGG19, ResNet50, and Inceptionv3) by training models on augmented data using transfer learning with average time of 10 min between true prediction and seizure onset. Our results show that CESP model achieves sensitivity of 78.11% and 88.21%, and FPR of 0.27/h and 0.14/h for training on synthesized and testing on real Epilepsyecosystem and CHB-MIT datasets, respectively. Effective results of CESP trained on synthesized data shows that synthetic data acquired the correlation between features and labels very well. We also show that employment of idea of transfer learning and data augmentation in patient-specific manner provides highest accuracy with sensitivity of 90.03% and 0.03 FPR/h which was achieved using Inceptionv3, and that augmenting data with samples generated from DCGAN increased prediction results of our CESP model and Inceptionv3 by 4-5% as compared to state-of-the-art traditional augmentation techniques. Finally, we note that prediction results of CESP achieved by using augmented data are better than chance level for both datasets.
Probabilistic Grammars for Equation Discovery
Authors: Jure Brence, Ljupčo Todorovski, Sašo Džeroski
Date: 2020-12-01 00:00:00
Tasks: Equation Discovery, Symbolic Regression

Equation discovery, also known as symbolic regression, is a type of automated modeling that discovers scientific laws, expressed in the form of equations, from observed data and expert knowledge. Deterministic grammars, such as context-free grammars, have been used to limit the search spaces in equation discovery by providing hard constraints that specify which equations to consider and which not. In this paper, we propose the use of probabilistic context-free grammars in equation discovery. Such grammars encode soft constraints, specifying a prior probability distribution on the space of possible equations. We show that probabilistic grammars can be used to elegantly and flexibly formulate the parsimony principle, that favors simpler equations, through probabilities attached to the rules in the grammars. We demonstrate that the use of probabilistic, rather than deterministic grammars, in the context of a Monte-Carlo algorithm for grammar-based equation discovery, leads to more efficient equation discovery. Finally, by specifying prior probability distributions over equation spaces, the foundations are laid for Bayesian approaches to equation discovery.
Edge-assisted Democratized Learning Towards Federated Analytics
Authors: Shashi Raj Pandey, Minh N. H. Nguyen, Tri Nguyen Dang, Nguyen H. Tran, Kyi Thar, Zhu Han, Choong Seon Hong
Date: 2020-12-01 00:00:00
Tasks: Distributed Computing, Edge-computing, Federated Learning

A recent take towards Federated Analytics (FA), which allows analytical insights of distributed datasets, reuses the Federated Learning (FL) infrastructure to evaluate the summary of model performances across the training devices. However, the current realization of FL adopts single server-multiple client architecture with limited scope for FA, which often results in learning models with poor generalization, i.e., an ability to handle new/unseen data, for real-world applications. Moreover, a hierarchical FL structure with distributed computing platforms demonstrates incoherent model performances at different aggregation levels. Therefore, we need to design a robust learning mechanism than the FL that (i) unleashes a viable infrastructure for FA and (ii) trains learning models with better generalization capability. In this work, we adopt the novel democratized learning (Dem-AI) principles and designs to meet these objectives. Firstly, we show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn, as a practical framework to empower generalization capability in support of FA. Secondly, we validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions by leveraging the distributed computing infrastructure. The distributed edge computing servers construct regional models, minimize the communication loads, and ensure distributed data analytic application's scalability. To that end, we adhere to a near-optimal two-sided many-to-one matching approach to handle the combinatorial constraints in Edge-DemLearn and solve it for fast knowledge acquisition with optimization of resource allocation and associations between multiple servers and devices. Extensive simulation results on real datasets demonstrate the effectiveness of the proposed methods.
Weakly-Supervised Arbitrary-Shaped Text Detection with Expectation-Maximization Algorithm
Authors: Mengbiao Zhao, Wei Feng, Fei Yin, Xu-Yao Zhang, Cheng-Lin Liu
Date: 2020-12-01 00:00:00
Tasks: Text Detection, Weakly-supervised Learning

Arbitrary-shaped text detection is an important and challenging task in computer vision. Most existing methods require heavy data labeling efforts to produce polygon-level text region labels for supervised training. In order to reduce the cost in data labeling, we study weakly-supervised arbitrary-shaped text detection for combining various weak supervision forms (e.g., image-level tags, coarse, loose and tight bounding boxes), which are far easier for annotation. We propose an Expectation-Maximization (EM) based weakly-supervised learning framework to train an accurate arbitrary-shaped text detector using only a small amount of polygon-level annotated data combined with a large amount of weakly annotated data. Meanwhile, we propose a contour-based arbitrary-shaped text detector, which is suitable for incorporating weakly-supervised learning. Extensive experiments on three arbitrary-shaped text benchmarks (CTW1500, Total-Text and ICDAR-ArT) show that (1) using only 10% strongly annotated data and 90% weakly annotated data, our method yields comparable performance to state-of-the-art methods, (2) with 100% strongly annotated data, our method outperforms existing methods on all three benchmarks. We will make the weakly annotated datasets publicly available in the future.