pwc-archive/papers-with-abstracts huggingface dataset:
QUEST: Queue Simulation for Content Moderation at Scale
Authors: Rahul Makhijani, Parikshit Shah, Vashist Avadhanula, Caner Gocmen, Nicolás E. Stier-Moses, Julián Mestre
Date: 2021-03-31 00:00:00
Tasks: BIG-bench Machine Learning

Moderating content in social media platforms is a formidable challenge due to the unprecedented scale of such systems, which typically handle billions of posts per day. Some of the largest platforms such as Facebook blend machine learning with manual review of platform content by thousands of reviewers. Operating a large-scale human review system poses interesting and challenging methodological questions that can be addressed with operations research techniques. We investigate the problem of optimally operating such a review system at scale using ideas from queueing theory and simulation.
GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection
Authors: Abhinav Kumar, Garrick Brazil, Xiaoming Liu
Date: 2021-03-31 00:00:00
Proceeding: CVPR 2021 1
Tasks: 2D Object Detection, 3D Object Detection, 3D Object Detection From Monocular Images, Monocular 3D Object Detection, Object, object-detection, Object Detection

Modern 3D object detectors have immensely benefited from the end-to-end learning idea. However, most of them use a post-processing algorithm called Non-Maximal Suppression (NMS) only during inference. While there were attempts to include NMS in the training pipeline for tasks such as 2D object detection, they have been less widely adopted due to a non-mathematical expression of the NMS. In this paper, we present and integrate GrooMeD-NMS -- a novel Grouped Mathematically Differentiable NMS for monocular 3D object detection, such that the network is trained end-to-end with a loss on the boxes after NMS. We first formulate NMS as a matrix operation and then group and mask the boxes in an unsupervised manner to obtain a simple closed-form expression of the NMS. GrooMeD-NMS addresses the mismatch between training and inference pipelines and, therefore, forces the network to select the best 3D box in a differentiable manner. As a result, GrooMeD-NMS achieves state-of-the-art monocular 3D object detection results on the KITTI benchmark dataset performing comparably to monocular video-based methods. Code and models at https://github.com/abhi1kumar/groomed_nms
Exploiting Invariance in Training Deep Neural Networks
Authors: Chengxi Ye, Xiong Zhou, Tristan McKinney, Yanfeng Liu, Qinggang Zhou, Fedor Zhdanov
Date: 2021-03-30 00:00:00
Tasks: Image Classification, Object Detection, Semantic Segmentation

Inspired by two basic mechanisms in animal visual systems, we introduce a feature transform technique that imposes invariance properties in the training of deep neural networks. The resulting algorithm requires less parameter tuning, trains well with an initial learning rate 1.0, and easily generalizes to different tasks. We enforce scale invariance with local statistics in the data to align similar samples at diverse scales. To accelerate convergence, we enforce a GL(n)-invariance property with global statistics extracted from a batch such that the gradient descent solution should remain invariant under basis change. Profiling analysis shows our proposed modifications takes 5% of the computations of the underlying convolution layer. Tested on convolutional networks and transformer networks, our proposed technique requires fewer iterations to train, surpasses all baselines by a large margin, seamlessly works on both small and large batch size training, and applies to different computer vision and language tasks.
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
Authors: Zeke Xie, Li Yuan, Zhanxing Zhu, Masashi Sugiyama
Date: 2021-03-31 00:00:00

It is well-known that stochastic gradient noise (SGN) acts as implicit regularization for deep learning and is essentially important for both optimization and generalization of deep networks. Some works attempted to artificially simulate SGN by injecting random noise to improve deep learning. However, it turned out that the injected simple random noise cannot work as well as SGN, which is anisotropic and parameter-dependent. For simulating SGN at low computational costs and without changing the learning rate or batch size, we propose the Positive-Negative Momentum (PNM) approach that is a powerful alternative to conventional Momentum in classic optimizers. The introduced PNM method maintains two approximate independent momentum terms. Then, we can control the magnitude of SGN explicitly by adjusting the momentum difference. We theoretically prove the convergence guarantee and the generalization advantage of PNM over Stochastic Gradient Descent (SGD). By incorporating PNM into the two conventional optimizers, SGD with Momentum and Adam, our extensive experiments empirically verified the significant advantage of the PNM-based variants over the corresponding conventional Momentum-based optimizers.
Layout-Guided Novel View Synthesis from a Single Indoor Panorama
Authors: Jiale Xu, Jia Zheng, Yanyu Xu, Rui Tang, Shenghua Gao
Date: 2021-03-31 00:00:00
Proceeding: CVPR 2021 1
Tasks: Novel View Synthesis

Existing view synthesis methods mainly focus on the perspective images and have shown promising results. However, due to the limited field-of-view of the pinhole camera, the performance quickly degrades when large camera movements are adopted. In this paper, we make the first attempt to generate novel views from a single indoor panorama and take the large camera translations into consideration. To tackle this challenging problem, we first use Convolutional Neural Networks (CNNs) to extract the deep features and estimate the depth map from the source-view image. Then, we leverage the room layout prior, a strong structural constraint of the indoor scene, to guide the generation of target views. More concretely, we estimate the room layout in the source view and transform it into the target viewpoint as guidance. Meanwhile, we also constrain the room layout of the generated target-view images to enforce geometric consistency. To validate the effectiveness of our method, we further build a large-scale photo-realistic dataset containing both small and large camera translations. The experimental results on our challenging dataset demonstrate that our method achieves state-of-the-art performance. The project page is at https://github.com/bluestyle97/PNVS.
Learning Spatio-Temporal Transformer for Visual Tracking
Authors: Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, Huchuan Lu
Date: 2021-03-31 00:00:00
Proceeding: ICCV 2021 10
Tasks: Decoder, Object Tracking, Video Object Tracking, Visual Object Tracking, Visual Tracking

In this paper, we present a new tracking architecture with an encoder-decoder transformer as the key component. The encoder models the global spatio-temporal feature dependencies between target objects and search regions, while the decoder learns a query embedding to predict the spatial positions of the target objects. Our method casts object tracking as a direct bounding box prediction problem, without using any proposals or predefined anchors. With the encoder-decoder transformer, the prediction of objects just uses a simple fully-convolutional network, which estimates the corners of objects directly. The whole method is end-to-end, does not need any postprocessing steps such as cosine window and bounding box smoothing, thus largely simplifying existing tracking pipelines. The proposed tracker achieves state-of-the-art performance on five challenging short-term and long-term benchmarks, while running at real-time speed, being 6x faster than Siam R-CNN. Code and models are open-sourced at https://github.com/researchmm/Stark.
$α$-Geodesical Skew Divergence
Authors: Masanari Kimura, Hideitsu Hino
Date: 2021-03-31 00:00:00

The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter $\lambda$, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be absolutely continuous with respect to the source distribution. In this paper, an information geometric generalization of the skew divergence called the $\alpha$-geodesical skew divergence is proposed, and its properties are studied.
Facial expression and attributes recognition based on multi-task learning of lightweight neural networks
Authors: Savchenko A.V.
Date: 2021-03-31 00:00:00
Tasks: Emotion Classification, Face Identification, Facial Expression Recognition (FER), Multi-Task Learning

In this paper, we examine the multi-task training of lightweight convolutional neural networks for face identification and classification of facial attributes (age, gender, ethnicity) trained on cropped faces without margins. It is shown that it is still necessary to fine-tune these networks in order to predict facial expressions. Several models are presented based on MobileNet, EfficientNet and RexNet architectures. It was experimentally demonstrated that our models are characterized by the state-of-the-art emotion classification accuracy on AffectNet dataset and near state-of-the-art results in age, gender and race recognition for UTKFace dataset. Moreover, it is shown that the usage of our neural network as a feature extractor of facial regions in video frames and concatenation of several statistical functions (mean, max, etc.) leads to 4.5% higher accuracy than the previously known state-of-the-art single models for AFEW and VGAF datasets from the EmotiW challenges.
Parameterized Hypercomplex Graph Neural Networks for Graph Classification
Authors: Tuan Le, Marco Bertolini, Frank Noé, Djork-Arné Clevert
Date: 2021-03-30 00:00:00
Tasks: Classification, General Classification, Graph Classification, Representation Learning

Despite recent advances in representation learning in hypercomplex (HC) space, this subject is still vastly unexplored in the context of graphs. Motivated by the complex and quaternion algebras, which have been found in several contexts to enable effective representation learning that inherently incorporates a weight-sharing mechanism, we develop graph neural networks that leverage the properties of hypercomplex feature transformation. In particular, in our proposed class of models, the multiplication rule specifying the algebra itself is inferred from the data during training. Given a fixed model architecture, we present empirical evidence that our proposed model incorporates a regularization effect, alleviating the risk of overfitting. We also show that for fixed model capacity, our proposed method outperforms its corresponding real-formulated GNN, providing additional confirmation for the enhanced expressivity of HC embeddings. Finally, we test our proposed hypercomplex GNN on several open graph benchmark datasets and show that our models reach state-of-the-art performance while consuming a much lower memory footprint with 70& fewer parameters. Our implementations are available at https://github.com/bayer-science-for-a-better-life/phc-gnn.
UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language
Authors: Oleksiy Syvokon, Olena Nahorna
Date: 2021-03-31 00:00:00
Tasks: Grammatical Error Correction

We present a corpus professionally annotated for grammatical error correction (GEC) and fluency edits in the Ukrainian language. To the best of our knowledge, this is the first GEC corpus for the Ukrainian language. We collected texts with errors (20,715 sentences) from a diverse pool of contributors, including both native and non-native speakers. The data cover a wide variety of writing domains, from text chats and essays to formal writing. Professional proofreaders corrected and annotated the corpus for errors relating to fluency, grammar, punctuation, and spelling. This corpus can be used for developing and evaluating GEC systems in Ukrainian. More generally, it can be used for researching multilingual and low-resource NLP, morphologically rich languages, document-level GEC, and fluency correction. The corpus is publicly available at https://github.com/grammarly/ua-gec
Near field Acoustic Holography on arbitrary shapes using Convolutional Neural Network
Authors: Marco Olivieri, Mirco Pezzoli, Fabio Antonacci, Augusto Sarti
Date: 2021-03-31 00:00:00
Tasks: Super-Resolution

Near-field Acoustic Holography (NAH) is a well-known problem aimed at estimating the vibrational velocity field of a structure by means of acoustic measurements. In this paper, we propose a NAH technique based on Convolutional Neural Network (CNN). The devised CNN predicts the vibrational field on the surface of arbitrary shaped plates (violin plates) with orthotropic material properties from a limited number of measurements. In particular, the architecture, named Super Resolution CNN (SRCNN), is able to estimate the vibrational field with a higher spatial resolution compared to the input pressure. The pressure and velocity datasets have been generated through Finite Element Method simulations. We validate the proposed method by comparing the estimates with the synthesized ground truth and with a state-of-the-art technique. Moreover, we evaluate the robustness of the devised network against noisy input data.
Fast Certified Robust Training with Short Warmup
Authors: Zhouxing Shi, Yihan Wang, huan zhang, JinFeng Yi, Cho-Jui Hsieh
Date: 2021-03-31 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Adversarial Defense

Recently, bound propagation based certified robust training methods have been proposed for training neural networks with certifiable robustness guarantees. Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thousands epochs to reach SOTA performance and are thus still costly. In this paper, we identify two important issues in existing methods, namely exploded bounds at initialization, and the imbalance in ReLU activation states and improve IBP training. These two issues make certified training difficult and unstable, and thereby long warmup schedules were needed in prior works. To mitigate these issues and conduct faster certified training with shorter warmup, we propose three improvements based on IBP training: 1) We derive a new weight initialization method for IBP training; 2) We propose to fully add Batch Normalization (BN) to each layer in the model, since we find BN can reduce the imbalance in ReLU activation states; 3) We also design regularization to explicitly tighten certified bounds and balance ReLU activation states during wamrup. We are able to obtain 65.03% verified error on CIFAR-10 ($\epsilon=\frac{8}{255}$) and 82.36% verified error on TinyImageNet ($\epsilon=\frac{1}{255}$) using very short training schedules (160 and 80 total epochs, respectively), outperforming literature SOTA trained with hundreds or thousands epochs under the same network architecture. The code is available at https://github.com/shizhouxing/Fast-Certified-Robust-Training.
Bit-Mixer: Mixed-precision networks with runtime bit-width selection
Authors: Adrian Bulat, Georgios Tzimiropoulos
Date: 2021-03-31 00:00:00
Proceeding: ICCV 2021 10
Tasks: AutoML, Binarization, Quantization

Mixed-precision networks allow for a variable bit-width quantization for every layer in the network. A major limitation of existing work is that the bit-width for each layer must be predefined during training time. This allows little flexibility if the characteristics of the device on which the network is deployed change during runtime. In this work, we propose Bit-Mixer, the very first method to train a meta-quantized network where during test time any layer can change its bid-width without affecting at all the overall network's ability for highly accurate inference. To this end, we make 2 key contributions: (a) Transitional Batch-Norms, and (b) a 3-stage optimization process which is shown capable of training such a network. We show that our method can result in mixed precision networks that exhibit the desirable flexibility properties for on-device deployment without compromising accuracy. Code will be made available.
新冠病毒预防，为何要用肥皂洗手？肥皂不止洗涤还能杀灭病毒的原理！COVID-19 科学博士
Authors: 科学博士
Date: 2021-03-28 00:00:00

新冠病毒预防，为何要用肥皂洗手？肥皂不止洗涤还能杀灭病毒的原理！COVID-19
#科学博士​ #肥皂​ #新冠肺炎​ ＃COVID19​ #病毒​ #健康​　#科普

★推荐影片★
世上居然有三黄蛋！三黄蛋！99.9%的人一生都没有见过！
https://youtu.be/YvkXjYK0fd4​

稀奇的双黄蛋：万分之一的鸡蛋！双黄蛋密码| 科学博士
https://youtu.be/1rDJq_vUl6Q​

哇！透明的双黄蛋，怎么做的？你在家也可以的！科学博士的家庭实验室
https://youtu.be/rlRWZBZ9sf0
Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning
Authors: Hiroki Furuta, Tadashi Kozuno, Tatsuya Matsushima, Yutaka Matsuo, Shixiang Shane Gu
Date: 2021-03-31 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Recently many algorithms were devised for reinforcement learning (RL) with function approximation. While they have clear algorithmic distinctions, they also have many implementation differences that are algorithm-independent and sometimes under-emphasized. Such mixing of algorithmic novelty and implementation craftsmanship makes rigorous analyses of the sources of performance improvements across algorithms difficult. In this work, we focus on a series of off-policy inference-based actor-critic algorithms -- MPO, AWR, and SAC -- to decouple their algorithmic innovations and implementation decisions. We present unified derivations through a single control-as-inference objective, where we can categorize each algorithm as based on either Expectation-Maximization (EM) or direct Kullback-Leibler (KL) divergence minimization and treat the rest of specifications as implementation details. We performed extensive ablation studies, and identified substantial performance drops whenever implementation details are mismatched for algorithmic choices. These results show which implementation or code details are co-adapted and co-evolved with algorithms, and which are transferable across algorithms: as examples, we identified that tanh Gaussian policy and network sizes are highly adapted to algorithmic types, while layer normalization and ELU are critical for MPO's performances but also transfer to noticeable gains in SAC. We hope our work can inspire future work to further demystify sources of performance improvements across multiple algorithms and allow researchers to build on one another's both algorithmic and implementational innovations.
Digital Twin Based Disaster Management System Proposal: DT-DMS
Authors: Özgür Dogan, Oguzhan Sahin, Enis Karaarslan
Date: 2021-03-31 00:00:00
Tasks: Decision Making, Management

The damage and the impact of natural disasters are becoming more destructive with the increase of urbanization. Today's metropolitan cities are not sufficiently prepared for the pre and post-disaster situations. Digital Twin technology can provide a solution. A virtual copy of the physical city could be created by collecting data from sensors of the Internet of Things (IoT) devices and stored on the cloud infrastructure. This virtual copy is kept current and up to date with the continuous flow of the data coming from the sensors. We propose a disaster management system utilizing machine learning called DT-DMS is used to support decision-making mechanisms. This study aims to show how to educate and prepare emergency center staff by simulating potential disaster situations on the virtual copy. The event of a disaster will be simulated allowing emergency center staff to make decisions and depicting the potential outcomes of these decisions. A rescue operation after an earthquake is simulated. Test results are promising and the simulation scope is planned to be extended.
OLIVAW: Mastering Othello without Human Knowledge, nor a Fortune
Authors: Antonio Norelli, Alessandro Panconesi
Date: 2021-03-31 00:00:00

We introduce OLIVAW, an AI Othello player adopting the design principles of the famous AlphaGo programs. The main motivation behind OLIVAW was to attain exceptional competence in a non-trivial board game at a tiny fraction of the cost of its illustrious predecessors. In this paper, we show how the AlphaGo Zero's paradigm can be successfully applied to the popular game of Othello using only commodity hardware and free cloud services. While being simpler than Chess or Go, Othello maintains a considerable search space and difficulty in evaluating board positions. To achieve this result, OLIVAW implements some improvements inspired by recent works to accelerate the standard AlphaGo Zero learning process. The main modification implies doubling the positions collected per game during the training phase, by including also positions not played but largely explored by the agent. We tested the strength of OLIVAW in three different ways: by pitting it against Edax, the strongest open-source Othello engine, by playing anonymous games on the web platform OthelloQuest, and finally in two in-person matches against top-notch human players: a national champion and a former world champion.
Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability
Authors: Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova
Date: 2021-03-31 00:00:00
Proceeding: Findings (EMNLP) 2021 11
Tasks: Abuse Detection, Abusive Language, Ethics, Position

Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various platforms. The psychological effects of abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abusive language detection in the field of NLP. In this position paper, we discuss the role that modeling of users and online communities plays in abuse detection. Specifically, we review and analyze the state of the art methods that leverage user or community information to enhance the understanding and detection of abusive language. We then explore the ethical challenges of incorporating user and community information, laying out considerations to guide future research. Finally, we address the topic of explainability in abusive language detection, proposing properties that an explainable method should aim to exhibit. We describe how user and community information can facilitate the realization of these properties and discuss the effective operationalization of explainability in view of the properties.
Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes
Authors: Dmytro Kotovenko, Matthias Wright, Arthur Heimbrecht, Björn Ommer
Date: 2021-03-31 00:00:00
Proceeding: CVPR 2021 1
Tasks: Style Transfer

There have been many successful implementations of neural style transfer in recent years. In most of these works, the stylization process is confined to the pixel domain. However, we argue that this representation is unnatural because paintings usually consist of brushstrokes rather than pixels. We propose a method to stylize images by optimizing parameterized brushstrokes instead of pixels and further introduce a simple differentiable rendering mechanism. Our approach significantly improves visual quality and enables additional control over the stylization process such as controlling the flow of brushstrokes through user input. We provide qualitative and quantitative evaluations that show the efficacy of the proposed parameterized representation.