OpenRaiser/Intern-Atlas huggingface dataset:
Training Strategies and Data Augmentations in CNN-based DeepFake Video Detection
Venue: arXiv cs.CV 2020
Year: 2020

The fast and continuous growth in number and quality of deepfake videos calls for the development of reliable detection systems capable of automatically warning users on social media and on the Internet about the potential untruthfulness of such contents. While algorithms, software, and smartphone apps are getting better every day in generating manipulated videos and swapping faces, the accuracy of automated systems for face forgery detection in videos is still quite limited and generally biased toward the dataset used to design and train a specific detection system. In this paper we analyze how different training strategies and data augmentation techniques affect CNN-based deepfake detectors when training and testing on the same dataset or across different datasets.
Gradient Episodic Memory with a Soft Constraint for Continual Learning
Venue: arXiv cs.LG 2020
Year: 2020

Catastrophic forgetting in continual learning is a common destructive phenomenon in gradient-based neural networks that learn sequential tasks, and it is much different from forgetting in humans, who can learn and accumulate knowledge throughout their whole lives. Catastrophic forgetting is the fatal shortcoming of a large decrease in performance on previous tasks when the model is learning a novel task. To alleviate this problem, the model should have the capacity to learn new knowledge and preserve learned knowledge. We propose an average gradient episodic memory (A-GEM) with a soft constraint $ε\in [0, 1]$, which is a balance factor between learning new knowledge and preserving learned knowledge; our method is called gradient episodic memory with a soft constraint $ε$ ($ε$-SOFT-GEM). $ε$-SOFT-GEM outperforms A-GEM and several continual learning benchmarks in a single training epoch; additionally, it has state-of-the-art average accuracy and efficiency for computation and memory, like A-GEM, and provides a better trade-off between the stability of preserving learned knowledge and the plasticity of learning new knowledge.
The Secretary Problem with Independent Sampling
Venue: arXiv cs.GT 2020
Year: 2020

In the secretary problem we are faced with an online sequence of elements with values. Upon seeing an element we have to make an irrevocable take-it-or-leave-it decision. The goal is to maximize the probability of picking the element of maximum value. The most classic version of the problem is that in which the elements arrive in random order and their values are arbitrary. However, by varying the available information, new interesting problems arise. Also the case in which the arrival order is adversarial instead of random leads to interesting variants that have been considered in the literature. In this paper we study both the random order and adversarial order secretary problems with an additional twist. The values are arbitrary, but before starting the online sequence we independently sample each element with a fixed probability $p$. The sampled elements become our information or history set and the game is played over the remaining elements. We call these problems the random order secretary problem with $p$-sampling (ROS$p$ for short) and the adversarial order secretary problem with $p$-sampling (AOS$p$ for short). Our main result is to obtain best possible algorithms for both problems and all values of $p$. As $p$ grows to 1 the obtained guarantees converge to the optimal guarantees in the full information case. In the adversarial order setting, the best possible algorithm turns out to be a simple fixed threshold algorithm in which the optimal threshold is a function of $p$ only. In the random order setting we prove that the best possible algorithm is characterized by a fixed sequence of time thresholds, dictating at which point in time we should start accepting a value that is both a maximum of the online sequence and has a given ranking within the sampled elements.
Determining the Intrinsic Structure of Public Software Development History
Venue: arXiv cs.SE 2020
Year: 2020

Background. Collaborative software development has produced a wealth of version control system (VCS) data that can now be analyzed in full. Little is known about the intrinsic structure of the entire corpus of publicly available VCS as an interconnected graph. Understanding its structure is needed to determine the best approach to analyze it in full and to avoid methodological pitfalls when doing so. Objective. We intend to determine the most salient network topol-ogy properties of public software development history as captured by VCS. We will explore: degree distributions, determining whether they are scale-free or not; distribution of connect component sizes; distribution of shortest path lengths.Method. We will use Software Heritage-which is the largest corpus of public VCS data-compress it using webgraph compression techniques, and analyze it in-memory using classic graph algorithms. Analyses will be performed both on the full graph and on relevant subgraphs. Limitations. The study is exploratory in nature; as such no hypotheses on the findings is stated at this time. Chosen graph algorithms are expected to scale to the corpus size, but it will need to be confirmed experimentally. External validity will depend on how representative Software Heritage is of the software commons.
Do not trust the neighbors! Adversarial Metric Learning for Self-Supervised Scene Flow Estimation
Venue: arXiv cs.CV 2020
Year: 2020

Scene flow is the task of estimating 3D motion vectors to individual points of a dynamic 3D scene. Motion vectors have shown to be beneficial for downstream tasks such as action classification and collision avoidance. However, data collected via LiDAR sensors and stereo cameras are computation and labor intensive to precisely annotate for scene flow. We address this annotation bottleneck on two ends. We propose a 3D scene flow benchmark and a novel self-supervised setup for training flow models. The benchmark consists of datasets designed to study individual aspects of flow estimation in progressive order of complexity, from a single object in motion to real-world scenes. Furthermore, we introduce Adversarial Metric Learning for self-supervised flow estimation. The flow model is fed with sequences of point clouds to perform flow estimation. A second model learns a latent metric to distinguish between the points translated by the flow estimations and the target point cloud. This latent metric is learned via a Multi-Scale Triplet loss, which uses intermediary feature vectors for the loss calculation. We use our proposed benchmark to draw insights about the performance of the baselines and of different models when trained using our setup. We find that our setup is able to keep motion coherence and preserve local geometries, which many self-supervised baselines fail to grasp. Dealing with occlusions, on the other hand, is still an open challenge.
Bridging the Technology Gap Between Industry and Semantic Web: Generating Databases and Server Code From RDF
Venue: arXiv cs.DB 2020
Year: 2020

Despite great advances in the area of Semantic Web, industry rather seldom adopts Semantic Web technologies and their storage and query concepts. Instead, relational databases (RDB) are often deployed to store business-critical data, which are accessed via REST interfaces. Yet, some enterprises would greatly benefit from Semantic Web related datasets which are usually represented with the Resource Description Framework (RDF). To bridge this technology gap, we propose a fully automatic approach that generates suitable RDB models with REST APIs to access them. In our evaluation, generated databases from different RDF datasets are examined and compared. Our findings show that the databases sufficiently reflect their counterparts while the API is able to reproduce rather simple SPARQL queries. Potentials for improvements are identified, for example, the reduction of data redundancies in generated databases.
Performance of Transfer Learning Model vs. Traditional Neural Network in Low System Resource Environment
Venue: arXiv cs.CL 2020
Year: 2020

Recently, the use of pre-trained model to build neural network based on transfer learning methodology is increasingly popular. These pre-trained models present the benefit of using less computing resources to train model with smaller amount of training data. The rise of state-of-the-art models such as BERT, XLNet and GPT boost accuracy and benefit as a base model for transfer leanring. However, these models are still too complex and consume many computing resource to train for transfer learning with low GPU memory. We will compare the performance and cost between lighter transfer learning model and purposely built neural network for NLP application of text classification and NER model.
The Person Index Challenge: Extraction of Persons from Messy, Short Texts
Venue: arXiv cs.CL 2020
Year: 2020

When persons are mentioned in texts with their first name, last name and/or middle names, there can be a high variation which of their names are used, how their names are ordered and if their names are abbreviated. If multiple persons are mentioned consecutively in very different ways, especially short texts can be perceived as "messy". Once ambiguous names occur, associations to persons may not be inferred correctly. Despite these eventualities, in this paper we ask how well an unsupervised algorithm can build a person index from short texts. We define a person index as a structured table that distinctly catalogs individuals by their names. First, we give a formal definition of the problem and describe a procedure to generate ground truth data for future evaluations. To give a first solution to this challenge, a baseline approach is implemented. By using our proposed evaluation strategy, we test the performance of the baseline and suggest further improvements. For future research the source code is publicly available.
pymgrid: An Open-Source Python Microgrid Simulator for Applied Artificial Intelligence Research
Venue: arXiv cs.AI 2020
Year: 2020

Microgrids, self contained electrical grids that are capable of disconnecting from the main grid, hold potential in both tackling climate change mitigation via reducing CO2 emissions and adaptation by increasing infrastructure resiliency. Due to their distributed nature, microgrids are often idiosyncratic; as a result, control of these systems is nontrivial. While microgrid simulators exist, many are limited in scope and in the variety of microgrids they can simulate. We propose pymgrid, an open-source Python package to generate and simulate a large number of microgrids, and the first open-source tool that can generate more than 600 different microgrids. pymgrid abstracts most of the domain expertise, allowing users to focus on control algorithms. In particular, pymgrid is built to be a reinforcement learning (RL) platform, and includes the ability to model microgrids as Markov decision processes. pymgrid also introduces two pre-computed list of microgrids, intended to allow for research reproducibility in the microgrid setting.
Domain Adaptive Knowledge Distillation for Driving Scene Semantic Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Practical autonomous driving systems face two crucial challenges: memory constraints and domain gap issues. In this paper, we present a novel approach to learn domain adaptive knowledge in models with limited memory, thus bestowing the model with the ability to deal with these issues in a comprehensive manner. We term this as "Domain Adaptive Knowledge Distillation" and address the same in the context of unsupervised domain-adaptive semantic segmentation by proposing a multi-level distillation strategy to effectively distil knowledge at different levels. Further, we introduce a novel cross entropy loss that leverages pseudo labels from the teacher. These pseudo teacher labels play a multifaceted role towards: (i) knowledge distillation from the teacher network to the student network & (ii) serving as a proxy for the ground truth for target domain images, where the problem is completely unsupervised. We introduce four paradigms for distilling domain adaptive knowledge and carry out extensive experiments and ablation studies on real-to-real as well as synthetic-to-real scenarios. Our experiments demonstrate the profound success of our proposed method.
Subtensor Quantization for Mobilenets
Venue: arXiv cs.CV 2020
Year: 2020

Quantization for deep neural networks (DNN) have enabled developers to deploy models with less memory and more efficient low-power inference. However, not all DNN designs are friendly to quantization. For example, the popular Mobilenet architecture has been tuned to reduce parameter size and computational latency with separable depth-wise convolutions, but not all quantization algorithms work well and the accuracy can suffer against its float point versions. In this paper, we analyzed several root causes of quantization loss and proposed alternatives that do not rely on per-channel or training-aware approaches. We evaluate the image classification task on ImageNet dataset, and our post-training quantized 8-bit inference top-1 accuracy in within 0.7% of the floating point version.
Street to Cloud: Improving Flood Maps With Crowdsourcing and Semantic Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

To address the mounting destruction caused by floods in climate-vulnerable regions, we propose Street to Cloud, a machine learning pipeline for incorporating crowdsourced ground truth data into the segmentation of satellite imagery of floods. We propose this approach as a solution to the labor-intensive task of generating high-quality, hand-labeled training data, and demonstrate successes and failures of different plausible crowdsourcing approaches in our model. Street to Cloud leverages community reporting and machine learning to generate novel, near-real time insights into the extent of floods to be used for emergency response.
Hierarchical Complementary Learning for Weakly Supervised Object Localization
Venue: arXiv cs.CV 2020
Year: 2020

Weakly supervised object localization (WSOL) is a challenging problem which aims to localize objects with only image-level labels. Due to the lack of ground truth bounding boxes, class labels are mainly employed to train the model. This model generates a class activation map (CAM) which activates the most discriminate features. However, the main drawback of CAM is the ability to detect just a part of the object. To solve this problem, some researchers have removed parts from the detected object \cite{b1, b2, b4}, or the image \cite{b3}. The aim of removing parts from image or detected parts of the object is to force the model to detect the other features. However, these methods require one or many hyper-parameters to erase the appropriate pixels on the image, which could involve a loss of information. In contrast, this paper proposes a Hierarchical Complementary Learning Network method (HCLNet) that helps the CNN to perform better classification and localization of objects on the images. HCLNet uses a complementary map to force the network to detect the other parts of the object. Unlike previous works, this method does not need any extras hyper-parameters to generate different CAMs, as well as does not introduce a big loss of information. In order to fuse these different maps, two different fusion strategies known as the addition strategy and the l1-norm strategy have been used. These strategies allowed to detect the whole object while excluding the background. Extensive experiments show that HCLNet obtains better performance than state-of-the-art methods.
Indirection Stream Semantic Register Architecture for Efficient Sparse-Dense Linear Algebra
Venue: arXiv cs.AR 2020
Year: 2020

Sparse-dense linear algebra is crucial in many domains, but challenging to handle efficiently on CPUs, GPUs, and accelerators alike; multiplications with sparse formats like CSR and CSF require indirect memory lookups. In this work, we enhance a memory-streaming RISC-V ISA extension to accelerate sparse-dense products through streaming indirection. We present efficient dot, matrix-vector, and matrix-matrix product kernels using our hardware, enabling single-core FPU utilizations of up to 80% and speedups of up to 7.2x over an optimized baseline without extensions. A matrix-vector implementation on a multi-core cluster is up to 5.8x faster and 2.7x more energy-efficient with our kernels than an optimized baseline. We propose further uses for our indirection hardware, such as scatter-gather operations and codebook decoding, and compare our work to state-of-the-art CPU, GPU, and accelerator approaches, measuring a 2.8x higher peak FP64 utilization in CSR matrix-vector multiplication than a GTX 1080 Ti GPU running a cuSPARSE kernel.
JNLP Team: Deep Learning for Legal Processing in COLIEE 2020
Venue: arXiv cs.CL 2020
Year: 2020

We propose deep learning based methods for automatic systems of legal retrieval and legal question-answering in COLIEE 2020. These systems are all characterized by being pre-trained on large amounts of data before being finetuned for the specified tasks. This approach helps to overcome the data scarcity and achieve good performance, thus can be useful for tackling related problems in information retrieval, and decision support in the legal domain. Besides, the approach can be explored to deal with other domain specific problems.
Topic-Centric Unsupervised Multi-Document Summarization of Scientific and News Articles
Venue: arXiv cs.CL 2020
Year: 2020

Recent advances in natural language processing have enabled automation of a wide range of tasks, including machine translation, named entity recognition, and sentiment analysis. Automated summarization of documents, or groups of documents, however, has remained elusive, with many efforts limited to extraction of keywords, key phrases, or key sentences. Accurate abstractive summarization has yet to be achieved due to the inherent difficulty of the problem, and limited availability of training data. In this paper, we propose a topic-centric unsupervised multi-document summarization framework to generate extractive and abstractive summaries for groups of scientific articles across 20 Fields of Study (FoS) in Microsoft Academic Graph (MAG) and news articles from DUC-2004 Task 2. The proposed algorithm generates an abstractive summary by developing salient language unit selection and text generation techniques. Our approach matches the state-of-the-art when evaluated on automated extractive evaluation metrics and performs better for abstractive summarization on five human evaluation metrics (entailment, coherence, conciseness, readability, and grammar). We achieve a kappa score of 0.68 between two co-author linguists who evaluated our results. We plan to publicly share MAG-20, a human-validated gold standard dataset of topic-clustered research articles and their summaries to promote research in abstractive summarization.
To Close Is Easier Than To Open: Dual Parameterization To k-Median
Venue: arXiv cs.DS 2020
Year: 2020

The $k$-Median problem is one of the well-known optimization problems that formalize the task of data clustering. Here, we are given sets of facilities $F$ and clients $C$, and the goal is to open $k$ facilities from the set $F$, which provides the best division into clusters, that is, the sum of distances from each client to the closest open facility is minimized. In the Capacitated $k$-Median, the facilities are also assigned capacities specifying how many clients can be served by each facility. Both problems have been extensively studied from the perspective of approximation algorithms. Recently, several surprising results have come from the area of parameterized complexity, which provided better approximation factors via algorithms with running times of the form $f(k)\cdot poly(n)$. In this work, we extend this line of research by studying a different choice of parameterization. We consider the parameter $\ell = |F| - k$, that is, the number of facilities that remain closed. It turns out that such a parameterization reveals yet another behavior of $k$-Median. We observe that the problem is W[1]-hard but it admits a parameterized approximation scheme. Namely, we present an algorithm with running time $2^{O(\ell\log(\ell/ε))}\cdot poly(n)$ that achieves a $(1+ε)$-approximation. On the other hand, we show that under the assumption of Gap Exponential Time Hypothesis, one cannot extend this result to the capacitated version of the problem.
Combining GANs and AutoEncoders for Efficient Anomaly Detection
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we propose CBiGAN -- a novel method for anomaly detection in images, where a consistency constraint is introduced as a regularization term in both the encoder and decoder of a BiGAN. Our model exhibits fairly good modeling power and reconstruction consistency capability. We evaluate the proposed method on MVTec AD -- a real-world benchmark for unsupervised anomaly detection on high-resolution images -- and compare against standard baselines and state-of-the-art approaches. Experiments show that the proposed method improves the performance of BiGAN formulations by a large margin and performs comparably to expensive state-of-the-art iterative methods while reducing the computational cost. We also observe that our model is particularly effective in texture-type anomaly detection, as it sets a new state of the art in this category. Our code is available at https://github.com/fabiocarrara/cbigan-ad/.
A Model of Polarization on Social Media Caused by Empathy and Repulsion
Venue: arXiv cs.SI 2020
Year: 2020

In recent years, the ease with which social media can be accessed has led to the unexpected problem of a shrinkage in information sources. This phenomenon is caused by a system that facilitates the connection of people with similar ideas and recommendation systems. Bias in the selection of information sources promotes polarization that divides people into multiple groups with opposing views and creates conflicts between opposing groups. This paper elucidates the mechanism of polarization by proposing a model of opinion formation in social media that considers users' reactions of empathy and repulsion. Based on the idea that opinion neutrality is only relative, this model offers a novel technology for dealing with polarization.
Assistive Diagnostic Tool for Brain Tumor Detection using Computer Vision
Venue: arXiv cs.CV 2020
Year: 2020

Today, over 700,000 people are living with brain tumors in the United States. Brain tumors can spread very quickly to other parts of the brain and the spinal cord unless necessary preventive action is taken. Thus, the survival rate for this disease is less than 40% for both men and women. A conclusive and early diagnosis of a brain tumor could be the difference between life and death for some. However, brain tumor detection and segmentation are tedious and time-consuming processes as it can only be done by radiologists and clinical experts. The use of computer vision techniques, such as Mask R Convolutional Neural Network (Mask R CNN), to detect and segment brain tumors can mitigate the possibility of human error while increasing prediction accuracy rates. The goal of this project is to create an assistive diagnostics tool for brain tumor detection and segmentation. Transfer learning was used with the Mask R CNN, and necessary parameters were accordingly altered, as a starting point. The model was trained with 20 epochs and later tested. The prediction segmentation matched 90% with the ground truth. This suggests that the model was able to perform at a high level. Once the model was finalized, the application running on Flask was created. The application will serve as a tool for medical professionals. It allows doctors to upload patient brain tumor MRI images in order to receive immediate results on the diagnosis and segmentation for each patient.