OpenRaiser/Intern-Atlas huggingface dataset:
TutorialVQA: Question Answering Dataset for Tutorial Videos
Venue: arXiv cs.CL 2020
Year: 2020

Despite the number of currently available datasets on video question answering, there still remains a need for a dataset involving multi-step and non-factoid answers. Moreover, relying on video transcripts remains an under-explored topic. To adequately address this, We propose a new question answering task on instructional videos, because of their verbose and narrative nature. While previous studies on video question answering have focused on generating a short text as an answer, given a question and video clip, our task aims to identify a span of a video segment as an answer which contains instructional details with various granularities. This work focuses on screencast tutorial videos pertaining to an image editing program. We introduce a dataset, TutorialVQA, consisting of about 6,000manually collected triples of (video, question, answer span). We also provide experimental results with several baselines algorithms using the video transcripts. The results indicate that the task is challenging and call for the investigation of new algorithms.
See and Read: Detecting Depression Symptoms in Higher Education Students Using Multimodal Social Media Data
Venue: arXiv cs.CL 2020
Year: 2020

Mental disorders such as depression and anxiety have been increasing at alarming rates in the worldwide population. Notably, the major depressive disorder has become a common problem among higher education students, aggravated, and maybe even occasioned, by the academic pressures they must face. While the reasons for this alarming situation remain unclear (although widely investigated), the student already facing this problem must receive treatment. To that, it is first necessary to screen the symptoms. The traditional way for that is relying on clinical consultations or answering questionnaires. However, nowadays, the data shared at social media is a ubiquitous source that can be used to detect the depression symptoms even when the student is not able to afford or search for professional care. Previous works have already relied on social media data to detect depression on the general population, usually focusing on either posted images or texts or relying on metadata. In this work, we focus on detecting the severity of the depression symptoms in higher education students, by comparing deep learning to feature engineering models induced from both the pictures and their captions posted on Instagram. The experimental results show that students presenting a BDI score higher or equal than 20 can be detected with 0.92 of recall and 0.69 of precision in the best case, reached by a fusion model. Our findings show the potential of large-scale depression screening, which could shed light upon students at-risk.
Reading the Manual: Event Extraction as Definition Comprehension
Venue: arXiv cs.CL 2020
Year: 2020

We ask whether text understanding has progressed to where we may extract event information through incremental refinement of bleached statements derived from annotation manuals. Such a capability would allow for the trivial construction and extension of an extraction framework by intended end-users through declarations such as, "Some person was born in some location at some time." We introduce an example of a model that employs such statements, with experiments illustrating we can extract events under closed ontologies and generalize to unseen event types simply by reading new definitions.
COSTRA 1.0: A Dataset of Complex Sentence Transformations
Venue: arXiv cs.CL 2020
Year: 2020

We present COSTRA 1.0, a dataset of complex sentence transformations. The dataset is intended for the study of sentence-level embeddings beyond simple word alternations or standard paraphrasing. This first version of the dataset is limited to sentences in Czech but the construction method is universal and we plan to use it also for other languages. The dataset consist of 4,262 unique sentences with average length of 10 words, illustrating 15 types of modifications such as simplification, generalization, or formal and informal language variation. The hope is that with this dataset, we should be able to test semantic properties of sentence embeddings and perhaps even to find some topologically interesting 'skeleton' in the sentence embedding space. A preliminary analysis using LASER, multi-purpose multi-lingual sentence embeddings suggests that the LASER space does not exhibit the desired properties.
Blockchain-Powered Collaboration in Heterogeneous Swarms of Robots
Venue: arXiv cs.CR 2020
Year: 2020

One of the key challenges in the collaboration within heterogeneous multi-robot systems is the optimization of the amount and type of data to be shared between robots with different sensing capabilities and computational resources. In this paper, we present a novel approach to managing collaboration terms in heterogeneous multi-robot systems with blockchain technology. Leveraging the extensive research of consensus algorithms in the blockchain domain, we exploit key technologies in this field to be integrated for consensus in robotic systems. We propose the utilization of proof of work systems to have an online estimation of the available computational resources at different robots. Furthermore, we define smart contracts that integrate information about the environment from different robots in order to evaluate and rank the quality and accuracy of each of the robots' sensor data. This means that the key parameters involved in heterogeneous robotic collaboration are integrated within the Blockchain and estimated at all robots equally without explicitly sharing information about the robots' hardware or sensors. Trustability is based on the verification of data samples that are submitted to the blockchain within each data exchange transaction and validated by other robots operating in the same environment. Initial results are reported which show the viability of the concepts presented in this paper.
Robust Online Model Adaptation by Extended Kalman Filter with Exponential Moving Average and Dynamic Multi-Epoch Strategy
Venue: arXiv cs.LG 2020
Year: 2020

High fidelity behavior prediction of intelligent agents is critical in many applications. However, the prediction model trained on the training set may not generalize to the testing set due to domain shift and time variance. The challenge motivates the adoption of online adaptation algorithms to update prediction models in real-time to improve the prediction performance. Inspired by Extended Kalman Filter (EKF), this paper introduces a series of online adaptation methods, which are applicable to neural network-based models. A base adaptation algorithm Modified EKF with forgetting factor (MEKF$_λ$) is introduced first, followed by exponential moving average filtering techniques. Then this paper introduces a dynamic multi-epoch update strategy to effectively utilize samples received in real time. With all these extensions, we propose a robust online adaptation algorithm: MEKF with Exponential Moving Average and Dynamic Multi-Epoch strategy (MEKF$_{\text{EMA-DME}}$). The proposed algorithm outperforms existing methods as demonstrated in experiments. The source code is open-sourced in the following link https://github.com/intelligent-control-lab/MEKF_MAME.
Learning with Multiplicative Perturbations
Venue: arXiv cs.LG 2020
Year: 2020

Adversarial Training (AT) and Virtual Adversarial Training (VAT) are the regularization techniques that train Deep Neural Networks (DNNs) with adversarial examples generated by adding small but worst-case perturbations to input examples. In this paper, we propose xAT and xVAT, new adversarial training algorithms, that generate \textbf{multiplicative} perturbations to input examples for robust training of DNNs. Such perturbations are much more perceptible and interpretable than their \textbf{additive} counterparts exploited by AT and VAT. Furthermore, the multiplicative perturbations can be generated transductively or inductively while the standard AT and VAT only support a transductive implementation. We conduct a series of experiments that analyze the behavior of the multiplicative perturbations and demonstrate that xAT and xVAT match or outperform state-of-the-art classification accuracies across multiple established benchmarks while being about 30\% faster than their additive counterparts. Furthermore, the resulting DNNs also demonstrate distinct weight distributions.
Domain-independent Dominance of Adaptive Methods
Venue: arXiv cs.LG 2020
Year: 2020

From a simplified analysis of adaptive methods, we derive AvaGrad, a new optimizer which outperforms SGD on vision tasks when its adaptability is properly tuned. We observe that the power of our method is partially explained by a decoupling of learning rate and adaptability, greatly simplifying hyperparameter search. In light of this observation, we demonstrate that, against conventional wisdom, Adam can also outperform SGD on vision tasks, as long as the coupling between its learning rate and adaptability is taken into account. In practice, AvaGrad matches the best results, as measured by generalization accuracy, delivered by any existing optimizer (SGD or adaptive) across image classification (CIFAR, ImageNet) and character-level language modelling (Penn Treebank) tasks.
Computing syzygies in finite dimension using fast linear algebra
Venue: arXiv cs.SC 2020
Year: 2020

We consider the computation of syzygies of multivariate polynomials in a finite-dimensional setting: for a $\mathbb{K}[X_1,\dots,X_r]$-module $\mathcal{M}$ of finite dimension $D$ as a $\mathbb{K}$-vector space, and given elements $f_1,\dots,f_m$ in $\mathcal{M}$, the problem is to compute syzygies between the $f_i$'s, that is, polynomials $(p_1,\dots,p_m)$ in $\mathbb{K}[X_1,\dots,X_r]^m$ such that $p_1 f_1 + \dots + p_m f_m = 0$ in $\mathcal{M}$. Assuming that the multiplication matrices of the $r$ variables with respect to some basis of $\mathcal{M}$ are known, we give an algorithm which computes the reduced Gröbner basis of the module of these syzygies, for any monomial order, using $O(m D^{ω-1} + r D^ω\log(D))$ operations in the base field $\mathbb{K}$, where $ω$ is the exponent of matrix multiplication. Furthermore, assuming that $\mathcal{M}$ is itself given as $\mathcal{M} = \mathbb{K}[X_1,\dots,X_r]^n/\mathcal{N}$, under some assumptions on $\mathcal{N}$ we show that these multiplication matrices can be computed from a Gröbner basis of $\mathcal{N}$ within the same complexity bound. In particular, taking $n=1$, $m=1$ and $f_1=1$ in $\mathcal{M}$, this yields a change of monomial order algorithm along the lines of the FGLM algorithm with a complexity bound which is sub-cubic in $D$.
A Maximin Optimal Online Power Control Policy for Energy Harvesting Communications
Venue: arXiv cs.IT 2020
Year: 2020

A general theory of online power control for discrete-time battery limited energy harvesting communications is developed, which leads to, among other things, an explicit characterization of a maximin optimal policy. This policy only requires the knowledge of the (effective) mean of the energy arrival process and maximizes the minimum asymptotic expected average reward (with the minimization taken over all energy arrival distributions of a given (effective) mean). Moreover, it is universally near optimal and has a strictly better worst-case performance as well as a strictly improved lower multiplicative factor in comparison with the fixed fraction policy proposed by Shaviv and Özgür when the objective is to maximize the throughput over an additive white Gaussian noise channel. The competitiveness of this maximin optimal policy is also demonstrated via numerical examples.
Mining Domain Knowledge: Improved Framework towards Automatically Standardizing Anatomical Structure Nomenclature in Radiotherapy
Venue: arXiv cs.CV 2020
Year: 2020

The automatic standardization of nomenclature for anatomical structures in radiotherapy (RT) clinical data is a critical prerequisite for data curation and data-driven research in the era of big data and artificial intelligence, but it is currently an unmet need. Existing methods either cannot handle cross-institutional datasets or suffer from heavy imbalance and poor-quality delineation in clinical RT datasets. To solve these problems, we propose an automated structure nomenclature standardization framework, 3D Non-local Network with Voting (3DNNV). This framework consists of an improved data processing strategy, namely, adaptive sampling and adaptive cropping (ASAC) with voting, and an optimized feature extraction module. The framework simulates clinicians' domain knowledge and recognition mechanisms to identify small-volume organs at risk (OARs) with heavily imbalanced data better than other methods. We used partial data from an open-source head-and-neck cancer dataset to train the model, then tested the model on three cross-institutional datasets to demonstrate its generalizability. 3DNNV outperformed the baseline model, achieving higher average true positive rates (TPR) overall categories on the three test datasets (+8.27%, +2.39%, and +5.53%, respectively). More importantly, the 3DNNV outperformed the baseline on the test dataset, 28.63% to 91.17%, in terms of F1 score for a small-volume OAR with only 9 training samples. The results show that 3DNNV can be applied to identify OARs, even error-prone ones. Furthermore, we discussed the limitations and applicability of the framework in practical scenarios. The framework we developed can assist in standardizing structure nomenclature to facilitate data-driven clinical research in cancer radiotherapy.
HABNet: Machine Learning, Remote Sensing Based Detection and Prediction of Harmful Algal Blooms
Venue: arXiv cs.LG 2020
Year: 2020

This paper describes the application of machine learning techniques to develop a state-of-the-art detection and prediction system for spatiotemporal events found within remote sensing data; specifically, Harmful Algal Bloom events (HABs). We propose an HAB detection system based on: a ground truth historical record of HAB events, a novel spatiotemporal datacube representation of each event (from MODIS and GEBCO bathymetry data) and a variety of machine learning architectures utilising state-of-the-art spatial and temporal analysis methods based on Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) components together with Random Forest and Support Vector Machine (SVM) classification methods. This work has focused specifically on the case study of the detection of Karenia Brevis Algae (K. brevis) HAB events within the coastal waters of Florida (over 2850 events from 2003 to 2018; an order of magnitude larger than any previous machine learning detection study into HAB events). The development of multimodal spatiotemporal datacube data structures and associated novel machine learning methods give a unique architecture for the automatic detection of environmental events. Specifically, when applied to the detection of HAB events it gives a maximum detection accuracy of 91% and a Kappa coefficient of 0.81 for the Florida data considered. A HAB forecast system was also developed where a temporal subset of each datacube was used to predict the presence of a HAB in the future. This system was not significantly less accurate than the detection system being able to predict with 86% accuracy up to 8 days in the future.
Scratch that! An Evolution-based Adversarial Attack against Neural Networks
Venue: arXiv cs.NE 2020
Year: 2020

We study black-box adversarial attacks for image classifiers in a constrained threat model, where adversaries can only modify a small fraction of pixels in the form of scratches on an image. We show that it is possible for adversaries to generate localized \textit{adversarial scratches} that cover less than $5\%$ of the pixels in an image and achieve targeted success rates of $98.77\%$ and $97.20\%$ on ImageNet and CIFAR-10 trained ResNet-50 models, respectively. We demonstrate that our scratches are effective under diverse shapes, such as straight lines or parabolic B\a'ezier curves, with single or multiple colors. In an extreme condition, in which our scratches are a single color, we obtain a targeted attack success rate of $66\%$ on CIFAR-10 with an order of magnitude fewer queries than comparable attacks. We successfully launch our attack against Microsoft's Cognitive Services Image Captioning API and propose various mitigation strategies.
Power of Pre-Processing: Production Scheduling with Variable Energy Pricing and Power-Saving States
Venue: arXiv cs.DS 2020
Year: 2020

This paper addresses a single machine scheduling problem with non-preemptive jobs to minimize the total electricity cost. Two latest trends in the area of the energy-aware scheduling are considered, namely the variable energy pricing and the power-saving states of a machine. Scheduling of the jobs and the machine states are considered jointly to achieve the highest possible savings. Although this problem has been previously addressed in the literature, the reported results of the state-of-the-art method show that the optimal solutions can be found only for instances with up to 35 jobs and 209 intervals within 3 hours of computation. We propose an elegant pre-processing technique called SPACES for computing the optimal switching of the machine states with respect to the energy costs. The optimal switchings are associated with the shortest paths in an interval-state graph that describes all possible transitions between the machine states in time. This idea allows us to implement efficient integer linear programming and constraint programming models of the problem while preserving the optimality. The efficiency of the models lies in the simplification of the optimal switching representation. The results of the experiments show that our approach outperforms the existing state-of-the-art exact method. On a set of benchmark instances with varying sizes and different state transition graphs, the proposed approach finds the optimal solutions even for the large instances with up to 190 jobs and 1277 intervals within an hour of computation.
MetalGAN: Multi-Domain Label-Less Image Synthesis Using cGANs and Meta-Learning
Venue: arXiv cs.LG 2020
Year: 2020

Image synthesis is currently one of the most addressed image processing topic in computer vision and deep learning fields of study. Researchers have tackled this problem focusing their efforts on its several challenging problems, e.g. image quality and size, domain and pose changing, architecture of the networks, and so on. Above all, producing images belonging to different domains by using a single architecture is a very relevant goal for image generation. In fact, a single multi-domain network would allow greater flexibility and robustness in the image synthesis task than other approaches. This paper proposes a novel architecture and a training algorithm, which are able to produce multi-domain outputs using a single network. A small portion of a dataset is intentionally used, and there are no hard-coded labels (or classes). This is achieved by combining a conditional Generative Adversarial Network (cGAN) for image generation and a Meta-Learning algorithm for domain switch, and we called our approach MetalGAN. The approach has proved to be appropriate for solving the multi-domain problem and it is validated on facial attribute transfer, using CelebA dataset.
Online Password Guessability via Multi-Dimensional Rank Estimation
Venue: arXiv cs.CR 2020
Year: 2020

Human-chosen passwords are the a dominant form of authentication systems. Passwords strength estimators are used to help users avoid picking weak passwords by predicting how many attempts a password cracker would need until it finds a given password. In this paper we propose a novel password strength estimator, called PESrank, which accurately models the behavior of a powerful password cracker. PESrank calculates the rank of a given password in an optimal descending order of likelihood. PESrank estimates a given password's rank in fractions of a second---without actually enumerating the passwords---so it is practical for online use. It also has a training time that is drastically shorter than previous methods. Moreover, PESrank is efficiently tweakable to allow model personalization in fractions of a second, without the need to retrain the model; and it is explainable: it is able to provide information on why the password has its calculated rank, and gives the user insight on how to pick a better password. Our idea is to cast the question of password rank estimation in a probabilistic framework used in side-channel cryptanalysis. We view each password as a point in a $d$-dimensional search space, and learn the probability distribution of each dimension separately. The dimensions represent the base word, plus a dimension for each possible transformation such as adding a suffix or using a capitalization pattern. Using this model, password strength estimation is analogous to side-channel rank estimation. We implemented PERrank in Python and conducted an extensive evaluation study of it. We also integrated it into the registration page of a course at our university. Even with a model based on 905 million passwords, the response time was well under 1 second, with up to a 1-bit accuracy margin between the upper bound and the lower bound on the rank.
Duty to Warn in Strategic Games
Venue: arXiv cs.AI 2020
Year: 2020

The paper investigates the second-order blameworthiness or duty to warn modality "one coalition knew how another coalition could have prevented an outcome". The main technical result is a sound and complete logical system that describes the interplay between the distributed knowledge and the duty to warn modalities.
Advanced attribute-based protocol based on the modified secret sharing scheme
Venue: arXiv cs.CR 2020
Year: 2020

We construct a new protocol for attribute-based encryption with the use of the modification of the standard secret sharing scheme. In the suggested modification of the secret sharing scheme, only one master key for each user is required that is achieved by linearly enlarging public parameters in the access formula. We then use this scheme for designing an attribute-based encryption protocol related to some access structure in terms of attributes. We demonstrate that the universe of possible attributes does not affect the resulting efficiency of the scheme. The security proofs for both constructions are provided.
Optimal state replication in stateful data planes
Venue: arXiv cs.NI 2020
Year: 2020

In SDN stateful data planes, switches can execute algorithms to process traffic based on local states. This approach permits to offload decisions from the controller to the switches, thus to reduce the latency to react to network events. We consider distributed network applications that process traffic at each switch based on local replicas of network-wide states. Replicating a state across multiple switches poses many challenges, because the number of state replicas and their placement affects both the data traffic and the synchronization traffic among the replicas. In our work, we formulate the optimal placement problem for replicated states, taking into account the data traffic routing, to ensure that the network applications affect the proper traffic flows, and the synchronization traffic between replicas, to ensure state coherence. Due to the high complexity required to find the optimal solution, we propose an approximated algorithm able to scale to large network instances. We show numerically that this algorithm, despite its simplicity, approximates quite well the optimal solution. We also show the beneficial effects of replicating the states with respect to the single-replica scenario, so far considered in the literature. Finally, we provide an asymptotic analysis to find the optimal number of replicas.