OpenRaiser/Intern-Atlas huggingface dataset:
Frame-To-Frame Consistent Semantic Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we aim for temporally consistent semantic segmentation throughout frames in a video. Many semantic segmentation algorithms process images individually which leads to an inconsistent scene interpretation due to illumination changes, occlusions and other variations over time. To achieve a temporally consistent prediction, we train a convolutional neural network (CNN) which propagates features through consecutive frames in a video using a convolutional long short term memory (ConvLSTM) cell. Besides the temporal feature propagation, we penalize inconsistencies in our loss function. We show in our experiments that the performance improves when utilizing video information compared to single frame prediction. The mean intersection over union (mIoU) metric on the Cityscapes validation set increases from 45.2 % for the single frames to 57.9 % for video data after implementing the ConvLSTM to propagate features trough time on the ESPNet. Most importantly, inconsistency decreases from 4.5 % to 1.3 % which is a reduction by 71.1 %. Our results indicate that the added temporal information produces a frame-to-frame consistent and more accurate image understanding compared to single frame processing. Code and videos are available at https://github.com/mrebol/f2f-consistent-semantic-segmentation
Age of Information-Reliability Trade-offs in Energy Harvesting Sensor Networks
Venue: arXiv cs.IT 2020
Year: 2020

Age of Information (AoI) is a recently defined quantity, which measures the freshness of information in a communication scheme. In this paper, we analyze a network that consists of a sensor node, an energy source and a receiver. The energy source is broadcasting energy and the sensor is charging its battery using energy-harvesting technologies. Whenever the battery gets fully charged, the sensor measures some quantity (called its status) from an environment, and (or) transmits its status to the receiver. The full analysis of AoI of this network, in the setting when each status is sent once, is given previously. However, that approach does not present a reliability guarantee better than the success probability of one transmission. In this paper, we present a closed form expression for the AoI of a deterministic and a randomized scheme that guarantee a desired probability of successful transmission for each status, alongside with a zero-error scheme. Furthermore, we define a novel notion called AoI-reliability trade-off and present the AoI-reliability trade-offs of our schemes. Additionally, we show that numerical results match our theoretical findings.
Cluster-Based Cooperative Digital Over-the-Air Aggregation for Wireless Federated Edge Learning
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we study a federated learning system at the wireless edge that uses over-the-air computation (AirComp). In such a system, users transmit their messages over a multi-access channel concurrently to achieve fast model aggregation. Recently, an AirComp scheme based on digital modulation has been proposed featuring one-bit gradient quantization and truncated channel inversion at users and a majority-voting based decoder at the fusion center (FC). We propose an improved digital AirComp scheme to relax its requirements on the transmitters, where users perform phase correction and transmit with full power. To characterize the decoding failure probability at the FC, we introduce the normalized detection signal-to-noise ratio (SNR), which can be interpreted as the effective participation rate of users. To mitigate wireless fading, we further propose a cluster-based system and design the relay selection scheme based on the normalized detection SNR. By local data fusion within each cluster and relay selection, our scheme can fully exploit spatial diversity to increase the effective number of voting users and accelerate model convergence.
Swipe dynamics as a means of authentication: results from a Bayesian unsupervised approach
Venue: arXiv cs.CR 2020
Year: 2020

The field of behavioural biometrics stands as an appealing alternative to more traditional biometric systems due to the ease of use from a user perspective and potential robustness to presentation attacks. This paper focuses its attention to a specific type of behavioural biometric utilising swipe dynamics, also referred to as touch gestures. In touch gesture authentication, a user swipes across the touchscreen of a mobile device to perform an authentication attempt. A key characteristic of touch gesture authentication and new behavioural biometrics in general is the lack of available data to train and validate models. From a machine learning perspective, this presents the classic curse of dimensionality problem and the methodology presented here focuses on Bayesian unsupervised models as they are well suited to such conditions. This paper presents results from a set of experiments consisting of 38 sessions with labelled victim as well as blind and over-the-shoulder presentation attacks. Three models are compared using this dataset; two single-mode models: a shrunk covariance estimate and a Bayesian Gaussian distribution, as well as a Bayesian non-parametric infinite mixture of Gaussians, modelled as a Dirichlet Process. Equal error rates (EER) for the three models are compared and attention is paid to how these vary across the two single-mode models at differing numbers of enrolment samples.
Value driven Analysis Framework of Service Ecosystem Evolution Mechanism
Venue: arXiv cs.OH 2020
Year: 2020

With the development of cloud computing, service computing, IoT(Internet of Things) and mobile Internet, the diversity and sociality of services are increasingly apparent. To meet the customized user demands, Service Ecosystem is emerging as a complex social-technology system, which is formed with various IT services through cross-border integration. However, how to analyze and promote the evolution mechanism of service ecosystem is still a serious challenge in the field, which is of great significance to achieve the expected system evolution trends. Based on this, this paper proposes a value-driven analysis framework of service ecosystem, including value creation, value operation, value realization and value distribution. In addition, a computational experiment system is established to verify the effectiveness of the analysis framework, which stimulates the effect of different operation strategies on the value network in the service ecosystem. The result shows that our analysis framework can provide new means and ideas for the analysis of service ecosystem evolution, and can also support the design of operation strategies. Index
Multifidelity Data Fusion via Gradient-Enhanced Gaussian Process Regression
Venue: arXiv cs.CE 2020
Year: 2020

We propose a data fusion method based on multi-fidelity Gaussian process regression (GPR) framework. This method combines available data of the quantity of interest (QoI) and its gradients with different fidelity levels, namely, it is a Gradient-enhanced Cokriging method (GE-Cokriging). It provides the approximations of both the QoI and its gradients simultaneously with uncertainty estimates. We compare this method with the conventional multi-fidelity Cokriging method that does not use gradients information, and the result suggests that GE-Cokriging has a better performance in predicting both QoI and its gradients. Moreover, GE-Cokriging even shows better generalization result in some cases where Cokriging performs poorly due to the singularity of the covariance matrix. We demonstrate the application of GE-Cokriging in several practical cases including reconstructing the trajectories and velocity of an underdamped oscillator with respect to time simultaneously, and investigating the sensitivity of power factor of a load bus with respect to varying power inputs of a generator bus in a large scale power system. We also show that though GE-Cokriging method requires a little bit higher computational cost than Cokriging method, the result of accuracy comparison shows that this cost is usually worth it.
Ubicomp Digital 2020 -- Handwriting classification using a convolutional recurrent network
Venue: arXiv cs.LG 2020
Year: 2020

The Ubicomp Digital 2020 -- Time Series Classification Challenge from STABILO is a challenge about multi-variate time series classification. The data collected from 100 volunteer writers, and contains 15 features measured with multiple sensors on a pen. In this paper,we use a neural network to classify the data into 52 classes, that is lower and upper cases of Arabic letters. The proposed architecture of the neural network a is CNN-LSTM network. It combines convolutional neural network (CNN) for short term context with along short term memory layer (LSTM) for also long term dependencies. We reached an accuracy of 68% on our writer exclusive test set and64.6% on the blind challenge test set resulting in the second place.
Evolutionary Grammar-Based Fuzzing
Venue: arXiv cs.SE 2020
Year: 2020

A fuzzer provides randomly generated inputs to a targeted software to expose erroneous behavior. To efficiently detect defects, generated inputs should conform to the structure of the input format and thus, grammars can be used to generate syntactically correct inputs. In this context, fuzzing can be guided by probabilities attached to competing rules in the grammar, leading to the idea of probabilistic grammar-based fuzzing. However, the optimal assignment of probabilities to individual grammar rules to effectively expose erroneous behavior for individual systems under test is an open research question. In this paper, we present EvoGFuzz, an evolutionary grammar-based fuzzing approach to optimize the probabilities to generate test inputs that may be more likely to trigger exceptional behavior. The evaluation shows the effectiveness of EvoGFuzz in detecting defects compared to probabilistic grammar-based fuzzing (baseline). Applied to ten real-world applications with common input formats (JSON, JavaScript, or CSS3), the evaluation shows that EvoGFuzz achieved a significantly larger median line coverage for all subjects by up to 48% compared to the baseline. Moreover, EvoGFuzz managed to expose 11 unique defects, from which five have not been detected by the baseline.
An Investigation on Deep Learning with Beta Stabilizer
Venue: arXiv cs.LG 2020
Year: 2020

Artificial neural networks (ANN) have been used in many applications such like handwriting recognition and speech recognition. It is well-known that learning rate is a crucial value in the training procedure for artificial neural networks. It is shown that the initial value of learning rate can confoundedly affect the final result and this value is always set manually in practice. A new parameter called beta stabilizer has been introduced to reduce the sensitivity of the initial learning rate. But this method has only been proposed for deep neural network (DNN) with sigmoid activation function. In this paper we extended beta stabilizer to long short-term memory (LSTM) and investigated the effects of beta stabilizer parameters on different models, including LSTM and DNN with relu activation function. It is concluded that beta stabilizer parameters can reduce the sensitivity of learning rate with almost the same performance on DNN with relu activation function and LSTM. However, it is shown that the effects of beta stabilizer on DNN with relu activation function and LSTM are fewer than the effects on DNN with sigmoid activation function.
A Self-Assessing Compilation Based Search Approach for Analytical Research and Data Retrieval
Venue: arXiv cs.IR 2020
Year: 2020

While meta-analytic research is performed, it becomes time-consuming to filter through the sheer amount of sources made available by individual databases and search engines and therefore degrades the specificity of source analysis. This study sought to predict the feasibility of a research-oriented searching algorithm across all topics and a search technique to combat flaws in dealing with large datasets by automating three key components of meta-analysis: a query-based search associated with the intended research topic, selecting given sources and determining their relevance to the original query, and extracting applicable information including excerpts and citations. The algorithm was evaluated using 5 key historical topics, and results were broken down into 4 categories: the total number of relevant sources retrieved, the efficiency given a particular search, the total time it takes to finish a complete cycle, and the quality of the extracted sources when compared to results from current searching methods. Although results differed through several searches, on average, the program collected a total of 126 sources per search with an average efficiency of 19.55 sources per second which, when compared and qualitatively evaluated for definitive results, indicates that an algorithm developed across all subject areas will make progress in future research methods.
ContentWise Impressions: An Industrial Dataset with Impressions Included
Venue: arXiv cs.IR 2020
Year: 2020

In this article, we introduce the ContentWise Impressions dataset, a collection of implicit interactions and impressions of movies and TV series from an Over-The-Top media service, which delivers its media contents over the Internet. The dataset is distinguished from other already available multimedia recommendation datasets by the availability of impressions, i.e., the recommendations shown to the user, its size, and by being open-source. We describe the data collection process, the preprocessing applied, its characteristics, and statistics when compared to other commonly used datasets. We also highlight several possible use cases and research questions that can benefit from the availability of user impressions in an open-source dataset. Furthermore, we release software tools to load and split the data, as well as examples of how to use both user interactions and impressions in several common recommendation algorithms.
Late Temporal Modeling in 3D CNN Architectures with BERT for Action Recognition
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we combine 3D convolution with late temporal modeling for action recognition. For this aim, we replace the conventional Temporal Global Average Pooling (TGAP) layer at the end of 3D convolutional architecture with the Bidirectional Encoder Representations from Transformers (BERT) layer in order to better utilize the temporal information with BERT's attention mechanism. We show that this replacement improves the performances of many popular 3D convolution architectures for action recognition, including ResNeXt, I3D, SlowFast and R(2+1)D. Moreover, we provide the-state-of-the-art results on both HMDB51 and UCF101 datasets with 85.10% and 98.69% top-1 accuracy, respectively. The code is publicly available.
Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm
Venue: arXiv cs.LG 2020
Year: 2020

Drawing an analogy with automatic image completion systems, we propose Music SketchNet, a neural network framework that allows users to specify partial musical ideas guiding automatic music generation. We focus on generating the missing measures in incomplete monophonic musical pieces, conditioned on surrounding context, and optionally guided by user-specified pitch and rhythm snippets. First, we introduce SketchVAE, a novel variational autoencoder that explicitly factorizes rhythm and pitch contour to form the basis of our proposed model. Then we introduce two discriminative architectures, SketchInpainter and SketchConnector, that in conjunction perform the guided music completion, filling in representations for the missing measures conditioned on surrounding context and user-specified snippets. We evaluate SketchNet on a standard dataset of Irish folk music and compare with models from recent works. When used for music completion, our approach outperforms the state-of-the-art both in terms of objective metrics and subjective listening tests. Finally, we demonstrate that our model can successfully incorporate user-specified snippets during the generation process.
NLPDove at SemEval-2020 Task 12: Improving Offensive Language Detection with Cross-lingual Transfer
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes our approach to the task of identifying offensive languages in a multilingual setting. We investigate two data augmentation strategies: using additional semi-supervised labels with different thresholds and cross-lingual transfer with data selection. Leveraging the semi-supervised dataset resulted in performance improvements compared to the baseline trained solely with the manually-annotated dataset. We propose a new metric, Translation Embedding Distance, to measure the transferability of instances for cross-lingual data selection. We also introduce various preprocessing steps tailored for social media text along with methods to fine-tune the pre-trained multilingual BERT (mBERT) for offensive language identification. Our multilingual systems achieved competitive results in Greek, Danish, and Turkish at OffensEval 2020.
Modular Constraint Solver Cooperation via Abstract Interpretation
Venue: arXiv cs.AI 2020
Year: 2020

Cooperation among constraint solvers is difficult because different solving paradigms have different theoretical foundations. Recent works have shown that abstract interpretation can provide a unifying theory for various constraint solvers. In particular, it relies on abstract domains which capture constraint languages as ordered structures. The key insight of this paper is viewing cooperation schemes as abstract domains combinations. We propose a modular framework in which solvers and cooperation schemes can be seamlessly added and combined. This differs from existing approaches such as SMT where the cooperation scheme is usually fixed (e.g., Nelson-Oppen). We contribute to two new cooperation schemes: (i) interval propagators completion that allows abstract domains to exchange bound constraints, and (ii) delayed product which exchanges over-approximations of constraints between two abstract domains. Moreover, the delayed product is based on delayed goal of logic programming, and it shows that abstract domains can also capture control aspects of constraint solving. Finally, to achieve modularity, we propose the shared product to combine abstract domains and cooperation schemes. Our approach has been fully implemented, and we provide various examples on the flexible job shop scheduling problem. Under consideration for acceptance in TPLP.
Intensional Datatype Refinement
Venue: arXiv cs.PL 2020
Year: 2020

The pattern-match safety problem is to verify that a given functional program will never crash due to non-exhaustive patterns in its function definitions. We present a refinement type system that can be used to solve this problem. The system extends ML-style type systems with algebraic datatypes by a limited form of structural subtyping and environment-level intersection. We describe a fully automatic, sound and complete type inference procedure for this system which, under reasonable assumptions, is worst-case linear-time in the program size. Compositionality is essential to obtaining this complexity guarantee. A prototype implementation for Haskell is able to analyse a selection of packages from the Hackage database in a few hundred milliseconds.
Learning from a Complementary-label Source Domain: Theory and Algorithms
Venue: arXiv cs.LG 2020
Year: 2020

In unsupervised domain adaptation (UDA), a classifier for the target domain is trained with massive true-label data from the source domain and unlabeled data from the target domain. However, collecting fully-true-label data in the source domain is high-cost and sometimes impossible. Compared to the true labels, a complementary label specifies a class that a pattern does not belong to, hence collecting complementary labels would be less laborious than collecting true labels. Thus, in this paper, we propose a novel setting that the source domain is composed of complementary-label data, and a theoretical bound for it is first proved. We consider two cases of this setting, one is that the source domain only contains complementary-label data (completely complementary unsupervised domain adaptation, CC-UDA), and the other is that the source domain has plenty of complementary-label data and a small amount of true-label data (partly complementary unsupervised domain adaptation, PC-UDA). To this end, a complementary label adversarial network} (CLARINET) is proposed to solve CC-UDA and PC-UDA problems. CLARINET maintains two deep networks simultaneously, where one focuses on classifying complementary-label source data and the other takes care of source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines on handwritten-digits-recognition and objects-recognition tasks.
Tracking Skin Colour and Wrinkle Changes During Cosmetic Product Trials Using Smartphone Images
Venue: arXiv cs.CV 2020
Year: 2020

Background: To explore how the efficacy of product trials for skin cosmetics can be improved through the use of consumer-level images taken by volunteers using a conventional smartphone. Materials and Methods: 12 women aged 30 to 60 years participated in a product trial and had close-up images of the cheek and temple regions of their faces taken with a high-resolution Antera 3D CS camera at the start and end of a 4-week period. Additionally, they each had ``selfies'' of the same regions of their faces taken regularly throughout the trial period. Automatic image analysis to identify changes in skin colour used three kinds of colour normalisation and analysis for wrinkle composition identified edges and calculated their magnitude. Results: Images taken at the start and end of the trial acted as baseline ground truth for normalisation of smartphone images and showed large changes in both colour and wrinkle magnitude during the trial for many volunteers. Conclusions: Results demonstrate that regular use of selfie smartphone images within trial periods can add value to interpretation of the efficacy of the trial.
A System for Worldwide COVID-19 Information Aggregation
Venue: arXiv cs.CL 2020
Year: 2020

The global pandemic of COVID-19 has made the public pay close attention to related news, covering various domains, such as sanitation, treatment, and effects on education. Meanwhile, the COVID-19 condition is very different among the countries (e.g., policies and development of the epidemic), and thus citizens would be interested in news in foreign countries. We build a system for worldwide COVID-19 information aggregation containing reliable articles from 10 regions in 7 languages sorted by topics. Our reliable COVID-19 related website dataset collected through crowdsourcing ensures the quality of the articles. A neural machine translation module translates articles in other languages into Japanese and English. A BERT-based topic-classifier trained on our article-topic pair dataset helps users find their interested information efficiently by putting articles into different categories.