OpenRaiser/Intern-Atlas huggingface dataset:
Realistic Video Summarization through VISIOCITY: A New Benchmark and Evaluation Framework
Venue: arXiv cs.CV 2020
Year: 2020

Automatic video summarization is still an unsolved problem due to several challenges. We take steps towards making automatic video summarization more realistic by addressing them. Firstly, the currently available datasets either have very short videos or have few long videos of only a particular type. We introduce a new benchmarking dataset VISIOCITY which comprises of longer videos across six different categories with dense concept annotations capable of supporting different flavors of video summarization and can be used for other vision problems. Secondly, for long videos, human reference summaries are difficult to obtain. We present a novel recipe based on pareto optimality to automatically generate multiple reference summaries from indirect ground truth present in VISIOCITY. We show that these summaries are at par with human summaries. Thirdly, we demonstrate that in the presence of multiple ground truth summaries (due to the highly subjective nature of the task), learning from a single combined ground truth summary using a single loss function is not a good idea. We propose a simple recipe VISIOCITY-SUM to enhance an existing model using a combination of losses and demonstrate that it beats the current state of the art techniques when tested on VISIOCITY. We also show that a single measure to evaluate a summary, as is the current typical practice, falls short. We propose a framework for better quantitative assessment of summary quality which is closer to human judgment than a single measure, say F1. We report the performance of a few representative techniques of video summarization on VISIOCITY assessed using various measures and bring out the limitation of the techniques and/or the assessment mechanism in modeling human judgment and demonstrate the effectiveness of our evaluation framework in doing so.
A regularized deep matrix factorized model of matrix completion for image restoration
Venue: arXiv cs.LG 2020
Year: 2020

It has been an important approach of using matrix completion to perform image restoration. Most previous works on matrix completion focus on the low-rank property by imposing explicit constraints on the recovered matrix, such as the constraint of the nuclear norm or limiting the dimension of the matrix factorization component. Recently, theoretical works suggest that deep linear neural network has an implicit bias towards low rank on matrix completion. However, low rank is not adequate to reflect the intrinsic characteristics of a natural image. Thus, algorithms with only the constraint of low rank are insufficient to perform image restoration well. In this work, we propose a Regularized Deep Matrix Factorized (RDMF) model for image restoration, which utilizes the implicit bias of the low rank of deep neural networks and the explicit bias of total variation. We demonstrate the effectiveness of our RDMF model with extensive experiments, in which our method surpasses the state of art models in common examples, especially for the restoration from very few observations. Our work sheds light on a more general framework for solving other inverse problems by combining the implicit bias of deep learning with explicit regularization.
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining
Venue: arXiv cs.CV 2020
Year: 2020

This paper studies composer style classification of piano sheet music images. Previous approaches to the composer classification task have been limited by a scarcity of data. We address this issue in two ways: (1) we recast the problem to be based on raw sheet music images rather than a symbolic music format, and (2) we propose an approach that can be trained on unlabeled data. Our approach first converts the sheet music image into a sequence of musical "words" based on the bootleg feature representation, and then feeds the sequence into a text classifier. We show that it is possible to significantly improve classifier performance by first training a language model on a set of unlabeled data, initializing the classifier with the pretrained language model weights, and then finetuning the classifier on a small amount of labeled data. We train AWD-LSTM, GPT-2, and RoBERTa language models on all piano sheet music images in IMSLP. We find that transformer-based architectures outperform CNN and LSTM models, and pretraining boosts classification accuracy for the GPT-2 model from 46\% to 70\% on a 9-way classification task. The trained model can also be used as a feature extractor that projects piano sheet music into a feature space that characterizes compositional style.
On the Use of Interpretable Machine Learning for the Management of Data Quality
Venue: arXiv cs.LG 2020
Year: 2020

Data quality is a significant issue for any application that requests for analytics to support decision making. It becomes very important when we focus on Internet of Things (IoT) where numerous devices can interact to exchange and process data. IoT devices are connected to Edge Computing (EC) nodes to report the collected data, thus, we have to secure data quality not only at the IoT but also at the edge of the network. In this paper, we focus on the specific problem and propose the use of interpretable machine learning to deliver the features that are important to be based for any data processing activity. Our aim is to secure data quality, at least, for those features that are detected as significant in the collected datasets. We have to notice that the selected features depict the highest correlation with the remaining in every dataset, thus, they can be adopted for dimensionality reduction. We focus on multiple methodologies for having interpretability in our learning models and adopt an ensemble scheme for the final decision. Our scheme is capable of timely retrieving the final result and efficiently select the appropriate features. We evaluate our model through extensive simulations and present numerical results. Our aim is to reveal its performance under various experimental scenarios that we create varying a set of parameters adopted in our mechanism.
New Constructions of Optimal Cyclic (r,δ) Locally Repairable Codes from Their Zeros
Venue: arXiv cs.IT 2020
Year: 2020

An $(r, δ)$-locally repairable code ($(r, δ)$-LRC for short) was introduced by Prakash et al. \cite{Prakash2012} for tolerating multiple failed nodes in distributed storage systems, which was a generalization of the concept of $r$-LRCs produced by Gopalan et al. \cite{Gopalan2012}. An $(r, δ)$-LRC is said to be optimal if it achieves the Singleton-like bound. Recently, Chen et al. \cite{Chen2018} generalized the construction of cyclic $r$-LRCs proposed by Tamo et al. \cite{Tamo2015,Tamo2016} and constructed several classes of optimal $(r, δ)$-LRCs of length $n$ for $n\, |\, (q-1)$ or $n\,|\, (q+1)$, respectively in terms of a union of the set of zeros controlling the minimum distance and the set of zeros ensuring the locality. Following the work of \cite{Chen2018,Chen2019}, this paper first characterizes $(r, δ)$-locality of a cyclic code via its zeros. Then we construct several classes of optimal cyclic $(r, δ)$-LRCs of length $n$ for $n\, |\, (q-1)$ or $n\,|\, (q+1)$, respectively from the product of two sets of zeros. Our constructions include all optimal cyclic $(r,δ)$-LRCs proposed in \cite{Chen2018,Chen2019}, and our method seems more convenient to obtain optimal cyclic $(r, δ)$-LRCs with flexible parameters. Moreover, many optimal cyclic $(r,δ)$-LRCs of length $n$ for $n\, |\, (q-1)$ or $n\,|\, (q+1)$, respectively such that $(r+δ-1)\nmid n$ can be obtained from our method.
Face2Face: Real-time Face Capture and Reenactment of RGB Videos
Venue: arXiv cs.CV 2020
Year: 2020

We present Face2Face, a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.
Deterministic Distributed Expander Decomposition and Routing with Applications in Distributed Derandomization
Venue: arXiv cs.DS 2020
Year: 2020

There is a recent exciting line of work in distributed graph algorithms in the $\mathsf{CONGEST}$ model that exploit expanders. All these algorithms so far are based on two tools: expander decomposition and expander routing. An $(ε,ϕ)$-expander decomposition removes $ε$-fraction of the edges so that the remaining connected components have conductance at least $ϕ$, i.e., they are $ϕ$-expanders, and expander routing allows each vertex $v$ in a $ϕ$-expander to very quickly exchange $\text{deg}(v)$ messages with any other vertices, not just its local neighbors. In this paper, we give the first efficient deterministic distributed algorithms for both tools. We show that an $(ε,ϕ)$-expander decomposition can be deterministically computed in $\text{poly}(ε^{-1}) n^{o(1)}$ rounds for $ϕ= \text{poly}(ε) n^{-o(1)}$, and that expander routing can be performed deterministically in $\text{poly}(ϕ^{-1})n^{o(1)}$ rounds. Both results match previous bounds of randomized algorithms by [Chang and Saranurak, PODC 2019] and [Ghaffari, Kuhn, and Su, PODC 2017] up to subpolynomial factors. Consequently, we derandomize existing distributed algorithms that exploit expanders. We show that a minimum spanning tree on $n^{o(1)}$-expanders can be constructed deterministically in $n^{o(1)}$ rounds, and triangle detection and enumeration on general graphs can be solved deterministically in $O(n^{0.58})$ and $n^{2/3 + o(1)}$ rounds, respectively. We also give the first polylogarithmic-round randomized algorithm for constructing an $(ε,ϕ)$-expander decomposition in $\text{poly}(ε^{-1}, \log n)$ rounds for $ϕ= 1 / \text{poly}(ε^{-1}, \log n)$. The previous algorithm by [Chang and Saranurak, PODC 2019] needs $n^{Ω(1)}$ rounds for any $ϕ\ge 1/\text{poly}\log n$.
Force myography benchmark data for hand gesture recognition and transfer learning
Venue: arXiv cs.LG 2020
Year: 2020

Force myography has recently gained increasing attention for hand gesture recognition tasks. However, there is a lack of publicly available benchmark data, with most existing studies collecting their own data often with custom hardware and for varying sets of gestures. This limits the ability to compare various algorithms, as well as the possibility for research to be done without first needing to collect data oneself. We contribute to the advancement of this field by making accessible a benchmark dataset collected using a commercially available sensor setup from 20 persons covering 18 unique gestures, in the hope of allowing further comparison of results as well as easier entry into this field of research. We illustrate one use-case for such data, showing how we can improve gesture recognition accuracy by utilising transfer learning to incorporate data from multiple other persons. This also illustrates that the dataset can serve as a benchmark dataset to facilitate research on transfer learning algorithms.
Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data
Venue: arXiv cs.LG 2020
Year: 2020

There has been considerable and growing interest in applying machine learning for cyber defenses. One promising approach has been to apply natural language processing techniques to analyze logs data for suspicious behavior. A natural question arises to how robust these systems are to adversarial attacks. Defense against sophisticated attack is of particular concern for cyber defenses. In this paper, we develop a testing framework to evaluate adversarial robustness of machine learning cyber defenses, particularly those focused on log data. Our framework uses techniques from deep reinforcement learning and adversarial natural language processing. We validate our framework using a publicly available dataset and demonstrate that our adversarial attack does succeed against the target systems, revealing a potential vulnerability. We apply our framework to analyze the influence of different levels of dropout regularization and find that higher dropout levels increases robustness. Moreover 90% dropout probability exhibited the highest level of robustness by a significant margin, which suggests unusually high dropout may be necessary to properly protect against adversarial attacks.
Hybrid Deep Learning Gaussian Process for Diabetic Retinopathy Diagnosis and Uncertainty Quantification
Venue: arXiv cs.CV 2020
Year: 2020

Diabetic Retinopathy (DR) is one of the microvascular complications of Diabetes Mellitus, which remains as one of the leading causes of blindness worldwide. Computational models based on Convolutional Neural Networks represent the state of the art for the automatic detection of DR using eye fundus images. Most of the current work address this problem as a binary classification task. However, including the grade estimation and quantification of predictions uncertainty can potentially increase the robustness of the model. In this paper, a hybrid Deep Learning-Gaussian process method for DR diagnosis and uncertainty quantification is presented. This method combines the representational power of deep learning, with the ability to generalize from small datasets of Gaussian process models. The results show that uncertainty quantification in the predictions improves the interpretability of the method as a diagnostic support tool. The source code to replicate the experiments is publicly available at https://github.com/stoledoc/DLGP-DR-Diagnosis.
The BIRAFFE2 Experiment. Study in Bio-Reactions and Faces for Emotion-based Personalization for AI Systems
Venue: arXiv cs.HC 2020
Year: 2020

The paper describes BIRAFFE2 data set, which is a result of an affective computing experiment conducted between 2019 and 2020, that aimed to develop computer models for classification and recognition of emotion. Such work is important to develop new methods of natural Human-AI interaction. As we believe that models of emotion should be personalized by design, we present an unified paradigm allowing to capture emotional responses of different persons, taking individual personality differences into account. We combine classical psychological paradigms of emotional response collection with the newer approach, based on the observation of the computer game player. By capturing ones psycho-physiological reactions (ECG, EDA signal recording), mimic expressions (facial emotion recognition), subjective valence-arousal balance ratings (widget ratings) and gameplay progression (accelerometer and screencast recording), we provide a framework that can be easily used and developed for the purpose of the machine learning methods.
Deriving Differential Target Propagation from Iterating Approximate Inverses
Venue: arXiv cs.LG 2020
Year: 2020

We show that a particular form of target propagation, i.e., relying on learned inverses of each layer, which is differential, i.e., where the target is a small perturbation of the forward propagation, gives rise to an update rule which corresponds to an approximate Gauss-Newton gradient-based optimization, without requiring the manipulation or inversion of large matrices. What is interesting is that this is more biologically plausible than back-propagation yet may turn out to implicitly provide a stronger optimization procedure. Extending difference target propagation, we consider several iterative calculations based on local auto-encoders at each layer in order to achieve more precise inversions for more accurate target propagation and we show that these iterative procedures converge exponentially fast if the auto-encoding function minus the identity function has a Lipschitz constant smaller than one, i.e., the auto-encoder is coarsely succeeding at performing an inversion. We also propose a way to normalize the changes at each layer to take into account the relative influence of each layer on the output, so that larger weight changes are done on more influential layers, like would happen in ordinary back-propagation with gradient descent.
Fast, Structured Clinical Documentation via Contextual Autocomplete
Venue: arXiv cs.LG 2020
Year: 2020

We present a system that uses a learned autocompletion mechanism to facilitate rapid creation of semi-structured clinical documentation. We dynamically suggest relevant clinical concepts as a doctor drafts a note by leveraging features from both unstructured and structured medical data. By constraining our architecture to shallow neural networks, we are able to make these suggestions in real time. Furthermore, as our algorithm is used to write a note, we can automatically annotate the documentation with clean labels of clinical concepts drawn from medical vocabularies, making notes more structured and readable for physicians, patients, and future algorithms. To our knowledge, this system is the only machine learning-based documentation utility for clinical notes deployed in a live hospital setting, and it reduces keystroke burden of clinical concepts by 67% in real environments.
Improving probability selecting based weights for Satisfiability Problem
Venue: arXiv cs.AI 2020
Year: 2020

The Boolean Satisfiability problem (SAT) is important on artificial intelligence community and the impact of its solving on complex problems. Recently, great breakthroughs have been made respectively on stochastic local search (SLS) algorithms for uniform random k-SAT resulting in several state-of-the-art SLS algorithms Score2SAT, YalSAT, ProbSAT, CScoreSAT and on a hybrid algorithm for hard random SAT (HRS) resulting in one state-of-the-art hybrid algorithm SparrowToRiss. However, there is no an algorithm which can effectively solve both uniform random k-SAT and HRS. In this paper, we present a new SLS algorithm named SelectNTS for uniform random k-SAT and HRS. SelectNTS is an improved probability selecting based local search algorithm for SAT problem. The core of SelectNTS relies on new clause and variable selection heuristics. The new clause selection heuristic uses a new clause weighting scheme and a biased random walk. The new variable selection heuristic uses a probability selecting strategy with the variation of CC strategy based on a new variable weighting scheme. Extensive experimental results on the well-known random benchmarks instances from the SAT Competitions in 2017 and 2018, and on randomly generated problems, show that our algorithm outperforms state-of-the-art random SAT algorithms, and our SelectNTS can effectively solve both uniform random k-SAT and HRS.
Understanding the Stability of Deep Control Policies for Biped Locomotion
Venue: arXiv cs.GR 2020
Year: 2020

Achieving stability and robustness is the primary goal of biped locomotion control. Recently, deep reinforce learning (DRL) has attracted great attention as a general methodology for constructing biped control policies and demonstrated significant improvements over the previous state-of-the-art. Although deep control policies have advantages over previous controller design approaches, many questions remain unanswered. Are deep control policies as robust as human walking? Does simulated walking use similar strategies as human walking to maintain balance? Does a particular gait pattern similarly affect human and simulated walking? What do deep policies learn to achieve improved gait stability? The goal of this study is to answer these questions by evaluating the push-recovery stability of deep policies compared to human subjects and a previous feedback controller. We also conducted experiments to evaluate the effectiveness of variants of DRL algorithms.
Hierarchical Action Classification with Network Pruning
Venue: arXiv cs.CV 2020
Year: 2020

Research on human action classification has made significant progresses in the past few years. Most deep learning methods focus on improving performance by adding more network components. We propose, however, to better utilize auxiliary mechanisms, including hierarchical classification, network pruning, and skeleton-based preprocessing, to boost the model robustness and performance. We test the effectiveness of our method on four commonly used testing datasets: NTU RGB+D 60, NTU RGB+D 120, Northwestern-UCLA Multiview Action 3D, and UTD Multimodal Human Action Dataset. Our experiments show that our method can achieve either comparable or better performance on all four datasets. In particular, our method sets up a new baseline for NTU 120, the largest dataset among the four. We also analyze our method with extensive comparisons and ablation studies.
Correctness by construction for probabilistic programs
Venue: arXiv cs.LO 2020
Year: 2020

The "correct by construction" paradigm is an important component of modern Formal Methods, and here we use the probabilistic Guarded-Command Language $\mathit{pGCL}$ to illustrate its application to $\mathit{probabilistic}$ programming. $\mathit{pGCL}$ extends Dijkstra's guarded-command language $\mathit{GCL}$ with probabilistic choice, and is equipped with a correctness-preserving refinement relation $(\sqsubseteq)$ that enables compact, abstract specifications of probabilistic properties to be transformed gradually to concrete, executable code by applying mathematical insights in a systematic and layered way. Characteristically for "correctness by construction", as far as possible the reasoning in each refinement-step layer does not depend on earlier layers, and does not affect later ones. We demonstrate the technique by deriving a fair-coin implementation of any given discrete probability distribution. In the special case of simulating a fair die, our correct-by-construction algorithm turns out to be "within spitting distance" of Knuth and Yao's optimal solution.
Implications of Dissemination Strategies on the Security of Distributed Ledgers
Venue: arXiv cs.DC 2020
Year: 2020

This paper describes a simulation study on security attacks over Distributed Ledger Technologies (DLTs). We specifically focus on attacks at the underlying peer-to-peer layer of these systems, that is in charge of disseminating messages containing data and transaction to be spread among all participants. In particular, we consider the Sybil attack, according to which a malicious node creates many Sybils that drop messages coming from a specific attacked node, or even all messages from honest nodes. Our study shows that the selection of the specific dissemination protocol, as well as the amount of connections each peer has, have an influence on the resistance to this attack.