pwc-archive/papers-with-abstracts huggingface dataset:
Human vs. Computer in Scene and Object Recognition
Authors: Ali Borji, Laurent Itti
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Object, Object Recognition

Several decades of research in computer and primate vision have resulted in many models (some specialized for one problem, others more general) and invaluable experimental data. Here, to help focus research efforts onto the hardest unsolved problems, and bridge computer and human vision, we define a battery of 5 tests that measure the gap between human and machine performances in several dimensions (generalization across scene categories, generalization from images to edge maps and line drawings, invariance to rotation and scaling, local/global information with jumbled images, and object recognition performance). We measure model accuracy and the correlation between model and human error patterns. Experimenting over 7 datasets, where human data is available, and gauging 14 well-established models, we find that none fully resembles humans in all aspects, and we learn from each test which models and features are more promising in approaching humans in the tested dimension. Across all tests, we find that models based on local edge histograms consistently resemble humans more, while several scene statistics or "gist" models do perform well with both scenes and objects. While computer vision has long been inspired by human vision, we believe systematic efforts, such as this, will help better identify shortcomings of models and find new paths forward.
P3.5P: Pose Estimation With Unknown Focal Length
Authors: Changchang Wu
Date: 2015-06-01 00:00:00
Proceeding: CVPR 2015 6
Tasks: Camera Pose Estimation, Pose Estimation

It is well known that the problem of camera pose estimation with unknown focal length has 7 degrees of freedom. Since each image point gives 2 constraints, solving this problem requires a minimum of 3.5 image points of 4 known 3D points, where 0.5 means either x or y coordinate of an image point.  We refer to this minimal problem as P3.5P. However, the existing methods require 4 full image points to solve the camera pose and  focal length.  In this paper, we present a general solution to the true minimal P3.5P problem with up to 10 solutions.  The remaining image coordinate is then used to filter the candidate solutions, which typically results in  a single solution for good data or no solution for outliers. Experiments show the proposed method significantly improves the efficiency over the state of the art methods while maintaining a high accuracy.
Going Unconstrained With Rolling Shutter Deblurring
Authors: Mahesh Mohan M. R., A. N. Rajagopalan, Gunasekaran Seetharaman
Date: 2017-10-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: Deblurring

Most present-day imaging devices are equipped with CMOS sensors. Motion blur is a common artifact in hand-held cameras. Because CMOS sensors mostly employ a rolling shutter (RS), the motion deblurring problem takes on a new dimension. Although few works have recently addressed this problem, they suffer from many constraints including heavy computational cost, need for precise sensor information, and inability to deal with wide-angle systems (which most cell-phone and drone cameras are) and irregular camera trajectory. In this work, we propose a model for RS blind motion deblurring that mitigates these issues significantly. Comprehensive comparisons with state-of-the-art methods reveal that our approach not only exhibits significant computational gains and unconstrained functionality but also leads to improved deblurring performance.
Adversarial Robustness vs. Model Compression, or Both?
Authors: Shaokai Ye,  Kaidi Xu,  Sijia Liu,  Hao Cheng,  Jan-Henrik Lambrechts,  Huan Zhang,  Aojun Zhou,  Kaisheng Ma,  Yanzhi Wang,  Xue Lin
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Adversarial Robustness, model, Model Compression

It is well known that deep neural networks (DNNs) are vulnerable to adversarial attacks, which are implemented by adding crafted perturbations onto benign examples. Min-max robust optimization based adversarial training can provide a notion of security against adversarial attacks. However, adversarial robustness requires a significantly larger capacity of the network than that for the natural training with only benign examples. This paper proposes a framework of concurrent adversarial training and weight pruning that enables model compression while still preserving the adversarial robustness and essentially tackles the dilemma of adversarial training. Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting; training a small model from scratch even with inherited initialization from the large model cannot achieve neither adversarial robustness nor high standard accuracy. Code is available at https://github.com/yeshaokai/Robustness-Aware-Pruning-ADMM.
Talking With Hands 16.2M: A Large-Scale Dataset of Synchronized Body-Finger Motion and Audio for Conversational Motion Analysis and Synthesis
Authors: Gilwoo Lee,  Zhiwei Deng,  Shugao Ma,  Takaaki Shiratori,  Siddhartha S. Srinivasa,  Yaser Sheikh
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Motion Synthesis

We present a 16.2-million frame (50-hour) multimodal dataset of two-person face-to-face spontaneous conversations. Our dataset features synchronized body and finger motion as well as audio data. To the best of our knowledge, it represents the largest motion capture and audio dataset of natural conversations to date. The statistical analysis verifies strong intraperson and interperson covariance of arm, hand, and speech features, potentially enabling new directions on data-driven social behavior analysis, prediction, and synthesis. As an illustration, we propose a novel real-time finger motion synthesis method: a temporal neural network innovatively trained with an inverse kinematics (IK) loss, which adds skeletal structural information to the generative model. Our qualitative user study shows that the finger motion generated by our method is perceived as natural and conversation enhancing, while the quantitative ablation study demonstrates the effectiveness of IK loss.
Miss Detection vs. False Alarm: Adversarial Learning for Small Object Segmentation in Infrared Images
Authors: Huan Wang,  Luping Zhou,  Lei Wang
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Generative Adversarial Network, Segmentation, Semantic Segmentation

A key challenge of infrared small object segmentation (ISOS) is to balance miss detection (MD) and false alarm (FA). This usually needs "opposite" strategies to suppress the two terms, and has not been well resolved in the literature. In this paper, we propose a deep adversarial learning framework to improve this situation. Departing from the tradition of jointly reducing MD and FA via a single objective, we decompose this difficult task into two sub-tasks handled by two models trained adversarially, with each focusing on reducing either MD or FA. Such a new design brings forth at least three advantages. First, as each model focuses on a relatively simpler sub-task, the overall difficulty of ISOS is somehow decreased. Second, the adversarial training of the two models naturally produces a delicate balance of MD and FA, and low rates for both MD and FA could be achieved at Nash equilibrium. Third, this MD-FA detachment gives us more flexibility to develop specific models dedicated to each sub-task. To realize the above design, we propose a conditional Generative Adversarial Network comprising of two generators and one discriminator. Each generator strives for one sub-task, while the discriminator differentiates the three segmentation results from the two generators and the ground truth. Moreover, in order to better serve the sub-tasks, the two generators, based on context aggregation networks, utilzse different size of receptive fields, providing both local and global views of objects for segmentation. As verified on multiple infrared image data sets, our method consistently achieves better segmentation than many state-of-the-art ISOS methods.
Application of Monte Carlo Tree Search in Periodic Schedule Design for Networked Control Systems
Authors: Burak Demirel, Arda Aytekin
Date: 2020-06-14 00:00:00
Tasks: Combinatorial Optimization, Scheduling

We analyze the closed-loop control performance of a networked control system that consists of $N$ independent linear feedback control loops, sharing a communication network with $M$ channels ($M<N$). A centralized scheduler, employing a scheduling protocol that produces periodic communication sequences, dictates which feedback loops should utilize all these channels. Under the periodic scheduling protocol, we derive analytical expressions for quantifying the overall control performance of the networked control system in terms of a quadratic function. We also formulate the offline combinatorial optimization of communication sequences for a given collection of linear feedback control subsystems. Then, we apply Monte Carlo Tree Search to determine the period of these communication sequences that attain near-optimal control performance. Via numerical studies, we show the effectiveness of the proposed framework.
Language-based Colorization of Scene Sketches
Authors: Changqing Zou, Haoran Mo, Chengying Gao, Ruofei Du, Hongbo Fu
Date: 2019-11-17 00:00:00
Proceeding: Transactions on Graphics 2019 11
Tasks: Colorization, Image Generation, Scene Understanding, Sketch

Being natural, touchless, and fun-embracing, language-based inputs have been demonstrated effective for various tasks from image generation to literacy education for children. This paper for the first time presents a language-based system for interactive colorization of scene sketches, based on semantic comprehension. The proposed system is built upon deep neural networks trained on a large-scale repository of scene sketches and cartoonstyle color images with text descriptions. Given a scene sketch, our system allows users, via language-based instructions, to interactively localize and colorize specific foreground object instances to meet various colorization requirements in a progressive way. We demonstrate the effectiveness of our approach via comprehensive experimental results including alternative studies, comparison with the state-of-the-art methods, and generalization user studies. Given the unique characteristics of language-based inputs, we envision a combination of our interface with a traditional scribble-based interface for a practical multimodal colorization system, benefiting various applications.
An Automated Ensemble Learning Framework Using Genetic Programming for Image Classification
Authors: Ying Bi, Bing Xue, Mengjie Zhang
Date: 2019-07-01 00:00:00
Proceeding: GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference 2019 7
Tasks: Classification, Diversity, Ensemble Learning, General Classification, image-classification, Image Classification

An ensemble consists of multiple learners and can achieve a better generalisation performance than a single learner. Genetic programming (GP) has been applied to construct ensembles using different strategies such as bagging and boosting. However, no GP-based ensemble methods focus on dealing with image classification, which is a challenging task in computer vision and machine learning. This paper proposes an automated ensemble learning framework using GP (EGP) for image classification. The new method integrates feature learning, classification function selection, classifier training, and combination into a single program tree. To achieve this, a novel program structure, a new function set and a new terminal set are developed in EGP. The performance of EGP is examined on nine different image classification data sets of varying difficulty and compared with a large number of commonly used methods including recently published methods. The results demonstrate that EGP achieves better performance than most competitive methods. Further analysis reveals that EGP evolves good ensembles simultaneously balancing diversity and accuracy. To the best of our knowledge, this study is the first work using GP to automatically generate ensembles for image classification.
Representations for Stable Off-Policy Reinforcement Learning
Authors: Dibya Ghosh, Marc G. Bellemare
Date: 2020-07-10 00:00:00
Proceeding: ICML 2020 1
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning

Reinforcement learning with function approximation can be unstable and even divergent, especially when combined with off-policy learning and Bellman updates. In deep reinforcement learning, these issues have been dealt with empirically by adapting and regularizing the representation, in particular with auxiliary tasks. This suggests that representation learning may provide a means to guarantee stability. In this paper, we formally show that there are indeed nontrivial state representations under which the canonical TD algorithm is stable, even when learning off-policy. We analyze representation learning schemes that are based on the transition matrix of a policy, such as proto-value functions, along three axes: approximation error, stability, and ease of estimation. In the most general case, we show that a Schur basis provides convergence guarantees, but is difficult to estimate from samples. For a fixed reward function, we find that an orthogonal basis of the corresponding Krylov subspace is an even better choice. We conclude by empirically demonstrating that these stable representations can be learned using stochastic gradient descent, opening the door to improved techniques for representation learning with deep networks.
A Causal Linear Model to Quantify Edge Flow and Edge Unfairness for UnfairEdge Prioritization and Discrimination Removal
Authors: Pavan Ravishankar, Pranshu Malviya, Balaraman Ravindran
Date: 2020-07-10 00:00:00

Law enforcement must prioritize sources of unfairness before mitigating their underlying unfairness, considering that they have limited resources. Unlike previous works that only make cautionary claims of discrimination and de-biases data after its generation, this paper attempts to prioritize unfair sources before mitigating their unfairness in the real-world. We assume that a causal bayesian network, representative of the data generation procedure, along with the sensitive nodes, that result in unfairness, are given. We quantify Edge Flow, which is the belief flowing along an edge by attenuating the indirect path influences, and use it to quantify Edge Unfairness. We prove that cumulative unfairness is non-existent in any decision, like judicial bail, towards any sensitive groups, like race, when the edge unfairness is absent, given an error-free linear model of conditional probability. We then measure the potential to mitigate the cumulative unfairness when edge unfairness is decreased. Based on these measures, we propose an unfair edge prioritization algorithm that prioritizes the unfair edges and a discrimination removal procedure that de-biases the generated data distribution. The experimental section validates the specifications used for quantifying the above measures.
AViD Dataset: Anonymized Videos from Diverse Countries
Authors: AJ Piergiovanni, Michael S. Ryoo
Date: 2020-07-10 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Action Classification, Action Detection, Action Recognition

We introduce a new public video dataset for action recognition: Anonymized Videos from Diverse countries (AViD). Unlike existing public video datasets, AViD is a collection of action videos from many different countries. The motivation is to create a public dataset that would benefit training and pretraining of action recognition models for everybody, rather than making it useful for limited countries. Further, all the face identities in the AViD videos are properly anonymized to protect their privacy. It also is a static dataset where each video is licensed with the creative commons license. We confirm that most of the existing video datasets are statistically biased to only capture action videos from a limited number of countries. We experimentally illustrate that models trained with such biased datasets do not transfer perfectly to action videos from the other countries, and show that AViD addresses such problem. We also confirm that the new AViD dataset could serve as a good dataset for pretraining the models, performing comparably or better than prior datasets.
Neural Knowledge Extraction From Cloud Service Incidents
Authors: Manish Shetty, Chetan Bansal, Sumit Kumar, Nikitha Rao, Nachiappan Nagappan, Thomas Zimmermann
Date: 2020-07-10 00:00:00
Tasks: Cloud Computing, Entity Extraction using GAN, Management, Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In the last decade, two paradigm shifts have reshaped the software industry - the move from boxed products to services and the widespread adoption of cloud computing. This has had a huge impact on the software development life cycle and the DevOps processes. Particularly, incident management has become critical for developing and operating large-scale services. Incidents are created to ensure timely communication of service issues and, also, their resolution. Prior work on incident management has been heavily focused on the challenges with incident triaging and de-duplication. In this work, we address the fundamental problem of structured knowledge extraction from service incidents. We have built SoftNER, a framework for unsupervised knowledge extraction from service incidents. We frame the knowledge extraction problem as a Named-entity Recognition task for extracting factual information. SoftNER leverages structural patterns like key,value pairs and tables for bootstrapping the training data. Further, we build a novel multi-task learning based BiLSTM-CRF model which leverages not just the semantic context but also the data-types for named-entity extraction. We have deployed SoftNER at Microsoft, a major cloud service provider and have evaluated it on more than 2 months of cloud incidents. We show that the unsupervised machine learning based approach has a high precision of 0.96. Our multi-task learning based deep learning model also outperforms the state of the art NER models. Lastly, using the knowledge extracted by SoftNER we are able to build significantly more accurate models for important downstream tasks like incident triaging.
Scientific Discovery by Generating Counterfactuals using Image Translation
Authors: Arunachalam Narayanaswamy, Subhashini Venugopalan, Dale R. Webster, Lily Peng, Greg Corrado, Paisan Ruamviboonsuk, Pinal Bavishi, Rory Sayres, Abigail Huang, Siva Balasubramanian, Michael Brenner, Philip Nelson, Avinash V. Varadarajan
Date: 2020-07-10 00:00:00
Tasks: scientific discovery, Translation

Model explanation techniques play a critical role in understanding the source of a model's performance and making its decisions transparent. Here we investigate if explanation techniques can also be used as a mechanism for scientific discovery. We make three contributions: first, we propose a framework to convert predictions from explanation techniques to a mechanism of discovery. Second, we show how generative models in combination with black-box predictors can be used to generate hypotheses (without human priors) that can be critically examined. Third, with these techniques we study classification models for retinal images predicting Diabetic Macular Edema (DME), where recent work showed that a CNN trained on these images is likely learning novel features in the image. We demonstrate that the proposed framework is able to explain the underlying scientific mechanism, thus bridging the gap between the model's performance and human understanding.
Not Your Grandfathers Test Set: Reducing Labeling Effort for Testing
Authors: Begum Taskazan, Jiri Navratil, Matthew Arnold, Anupama Murthi, Ganesh Venkataraman, Benjamin Elder
Date: 2020-07-10 00:00:00

Building and maintaining high-quality test sets remains a laborious and expensive task. As a result, test sets in the real world are often not properly kept up to date and drift from the production traffic they are supposed to represent. The frequency and severity of this drift raises serious concerns over the value of manually labeled test sets in the QA process. This paper proposes a simple but effective technique that drastically reduces the effort needed to construct and maintain a high-quality test set (reducing labeling effort by 80-100% across a range of practical scenarios). This result encourages a fundamental rethinking of the testing process by both practitioners, who can use these techniques immediately to improve their testing, and researchers who can help address many of the open questions raised by this new approach.
STaRFlow: A SpatioTemporal Recurrent Cell for Lightweight Multi-Frame Optical Flow Estimation
Authors: Pierre Godet, Alexandre Boulch, Aurélien Plyer, Guy Le Besnerais
Date: 2020-07-10 00:00:00
Tasks: Optical Flow Estimation

We present a new lightweight CNN-based algorithm for multi-frame optical flow estimation. Our solution introduces a double recurrence over spatial scale and time through repeated use of a generic "STaR" (SpatioTemporal Recurrent) cell. It includes (i) a temporal recurrence based on conveying learned features rather than optical flow estimates; (ii) an occlusion detection process which is coupled with optical flow estimation and therefore uses a very limited number of extra parameters. The resulting STaRFlow algorithm gives state-of-the-art performances on MPI Sintel and Kitti2015 and involves significantly less parameters than all other methods with comparable results.
Impact of Legal Requirements on Explainability in Machine Learning
Authors: Adrien Bibal, Michael Lognoul, Alexandre de Streel, Benoît Frénay
Date: 2020-07-10 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

The requirements on explainability imposed by European laws and their implications for machine learning (ML) models are not always clear. In that perspective, our research analyzes explanation obligations imposed for private and public decision-making, and how they can be implemented by machine learning techniques.
Exponential Convergence of Gradient Methods in Concave Network Zero-sum Games
Authors: Amit Kadan, Hu Fu
Date: 2020-07-10 00:00:00

Motivated by Generative Adversarial Networks, we study the computation of Nash equilibrium in concave network zero-sum games (NZSGs), a multiplayer generalization of two-player zero-sum games first proposed with linear payoffs. Extending previous results, we show that various game theoretic properties of convex-concave two-player zero-sum games are preserved in this generalization. We then generalize last iterate convergence results obtained previously in two-player zero-sum games. We analyze convergence rates when players update their strategies using Gradient Ascent, and its variant, Optimistic Gradient Ascent, showing last iterate convergence in three settings -- when the payoffs of players are linear, strongly concave and Lipschitz, and strongly concave and smooth. We provide experimental results that support these theoretical findings.