OpenRaiser/Intern-Atlas huggingface dataset:
Retrieving and Highlighting Action with Spatiotemporal Reference
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we present a framework that jointly retrieves and spatiotemporally highlights actions in videos by enhancing current deep cross-modal retrieval methods. Our work takes on the novel task of action highlighting, which visualizes where and when actions occur in an untrimmed video setting. Action highlighting is a fine-grained task, compared to conventional action recognition tasks which focus on classification or window-based localization. Leveraging weak supervision from annotated captions, our framework acquires spatiotemporal relevance maps and generates local embeddings which relate to the nouns and verbs in captions. Through experiments, we show that our model generates various maps conditioned on different actions, in which conventional visual reasoning methods only go as far as to show a single deterministic saliency map. Also, our model improves retrieval recall over our baseline without alignment by 2-3% on the MSR-VTT dataset.
Do Code Review Measures Explain the Incidence of Post-Release Defects?
Venue: arXiv cs.SE 2020
Year: 2020

Aim: In contrast to studies of defects found during code review, we aim to clarify whether code reviews measures can explain the prevalence of post-release defects. Method: We replicate a study by McIntoshet. al that uses additive regression to model the relationship between defects and code reviews. To increase external validity, we apply the same methodology on a new software project. We discuss our findings with the first author of the original study, McIntosh. We then investigate how to reduce the impact of correlated predictors in the variable selection process and how to increase understanding of the inter-relationships among the predictors by employing Bayesian Network (BN) models. Context: As in the original study, we use the same measures authors obtained for Qt project in the original study. We mine data from version control and issue tracker of Google Chrome and operationalize measures that are close analogs to the large collection of code, process, and code review measures used in the replicated the study. Results: Both the data from the original study and the Chrome data showed high instability of the influence of code review measures on defects with the results being highly sensitive to variable selection procedure. Models without code review predictors had as good or better fit than those with review predictors. Replication, however, confirms with the bulk of prior work showing that prior defects, module size, and authorship have the strongest relationship to post-release defects. The application of BN models helped explain the observed instability by demonstrating that the review-related predictors do not affect post-release defects directly and showed indirect effects. For example, changes that have no review discussion tend to be associated with files that have had many prior defects which in turn increase the number of post-release defects.
Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech
Venue: arXiv cs.CL 2020
Year: 2020

Accent conversion (AC) transforms a non-native speaker's accent into a native accent while maintaining the speaker's voice timbre. In this paper, we propose approaches to improving accent conversion applicability, as well as quality. First of all, we assume no reference speech is available at the conversion stage, and hence we employ an end-to-end text-to-speech system that is trained on native speech to generate native reference speech. To improve the quality and accent of the converted speech, we introduce reference encoders which make us capable of utilizing multi-source information. This is motivated by acoustic features extracted from native reference and linguistic information, which are complementary to conventional phonetic posteriorgrams (PPGs), so they can be concatenated as features to improve a baseline system based only on PPGs. Moreover, we optimize model architecture using GMM-based attention instead of windowed attention to elevate synthesized performance. Experimental results indicate when the proposed techniques are applied the integrated system significantly raises the scores of acoustic quality (30$\%$ relative increase in mean opinion score) and native accent (68$\%$ relative preference) while retaining the voice identity of the non-native speaker.
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes
Venue: arXiv cs.LG 2020
Year: 2020

Accurate mortality prediction allows Intensive Care Units (ICUs) to adequately benchmark clinical practice and identify patients with unexpected outcomes. Traditionally, simple statistical models have been used to assess patient death risk, many times with sub-optimal performance. On the other hand deep learning holds promise to positively impact clinical practice by leveraging medical data to assist diagnosis and prediction, including mortality prediction. However, as the question of whether powerful Deep Learning models attend correlations backed by sound medical knowledge when generating predictions remains open, additional interpretability tools are needed to foster trust and encourage the use of AI by clinicians. In this work we show a Deep Learning model trained on MIMIC-III to predict mortality using raw nursing notes, together with visual explanations for word importance. Our model reaches a ROC of 0.8629 (+/-0.0058), outperforming the traditional SAPS-II score and providing enhanced interpretability when compared with similar Deep Learning approaches.
An examination of applicability of face recognition sensors in public facilities
Venue: arXiv cs.CY 2020
Year: 2020

This study aimed to explore the usability and applicability of face recognition sensors in public spaces to collect customer footfall data, which could then be analysed and evaluated for facility design and planning. Nine OMRON sensors were provided for the project and installed at five locations in a public facility for three months. The project was carried out by a local consortium with the cooperation of local technology-based Small Medium-sized Enterprises (SMEs), business organisations, and a local university. Collected data were analysed to develop a report with diagrams, and reveal issues and potential for practical application in the future.
Towards Friendly Mixed Initiative Procedural Content Generation: Three Pillars of Industry
Venue: arXiv cs.HC 2020
Year: 2020

While the games industry is moving towards procedural content generation (PCG) with tools available under popular platforms such as Unreal, Unity or Houdini, and video game titles like No Man's Sky and Horizon Zero Dawn taking advantage of PCG, the gap between academia and industry is as wide as it has ever been, in terms of communication and sharing methods. One of the authors, has worked on both sides of this gap and in an effort to shorten it and increase the synergy between the two sectors, has identified three design pillars for PCG using mixed-initiative interfaces. The three pillars are Respect Designer Control, Respect the Creative Process and Respect Existing Work Processes. Respecting designer control is about creating a tool that gives enough control to bring out the designer's vision. Respecting the creative process concerns itself with having a feedback loop that is short enough, that the creative process is not disturbed. Respecting existing work processes means that a PCG tool should plug in easily to existing asset pipelines. As academics and communicators, it is surprising that publications often do not describe ways for developers to use our work or lack considerations for how a piece of work might fit into existing content pipelines.
Uncertainty Estimation in Deep 2D Echocardiography Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

2D echocardiography is the most common imaging modality for cardiovascular diseases. The portability and relatively low-cost nature of Ultrasound (US) enable the US devices needed for performing echocardiography to be made widely available. However, acquiring and interpreting cardiac US images is operator dependent, limiting its use to only places where experts are present. Recently, Deep Learning (DL) has been used in 2D echocardiography for automated view classification, and structure and function assessment. Although these recent works show promise in developing computer-guided acquisition and automated interpretation of echocardiograms, most of these methods do not model and estimate uncertainty which can be important when testing on data coming from a distribution further away from that of the training data. Uncertainty estimates can be beneficial both during the image acquisition phase (by providing real-time feedback to the operator on acquired image's quality), and during automated measurement and interpretation. The performance of uncertainty models and quantification metric may depend on the prediction task and the models being compared. Hence, to gain insight of uncertainty modelling for left ventricular segmentation from US images, we compare three ensembling based uncertainty models quantified using four different metrics (one newly proposed) on state-of-the-art baseline networks using two publicly available echocardiogram datasets. We further demonstrate how uncertainty estimation can be used to automatically reject poor quality images and improve state-of-the-art segmentation results.
Machine Learning-based Cardinality Estimation in DBMS on Pre-Aggregated Data
Venue: arXiv cs.DB 2020
Year: 2020

Cardinality estimation is a fundamental task in database query processing and optimization. As shown in recent papers, machine learning (ML)-based approaches can deliver more accurate cardinality estimations than traditional approaches. However, a lot of example queries have to be executed during the model training phase to learn a data-dependent ML model leading to a very time-consuming training phase. Many of those example queries use the same base data, have the same query structure, and only differ in their predicates. Thus, index structures appear to be an ideal optimization technique at first glance. However, their benefit is limited. To speed up this model training phase, our core idea is to determine a predicate-independent pre-aggregation of the base data and to execute the example queries over this pre-aggregated data. Based on this idea, we present a specific aggregate-enabled training phase for ML-based cardinality estimation approaches in this paper. As we are going to show with different workloads in our evaluation, we are able to achieve an average speedup of 63 with our aggregate-enabled training phase.
Adversarial Alignment of Multilingual Models for Extracting Temporal Expressions from Text
Venue: arXiv cs.CL 2020
Year: 2020

Although temporal tagging is still dominated by rule-based systems, there have been recent attempts at neural temporal taggers. However, all of them focus on monolingual settings. In this paper, we explore multilingual methods for the extraction of temporal expressions from text and investigate adversarial training for aligning embedding spaces to one common space. With this, we create a single multilingual model that can also be transferred to unseen languages and set the new state of the art in those cross-lingual transfer experiments.
Embeddings as representation for symbolic music
Venue: arXiv cs.CL 2020
Year: 2020

A representation technique that allows encoding music in a way that contains musical meaning would improve the results of any model trained for computer music tasks like generation of melodies and harmonies of better quality. The field of natural language processing has done a lot of work in finding a way to capture the semantic meaning of words and sentences, and word embeddings have successfully shown the capabilities for such a task. In this paper, we experiment with embeddings to represent musical notes from 3 different variations of a dataset and analyze if the model can capture useful musical patterns. To do this, the resulting embeddings are visualized in projections using the t-SNE technique.
Tree-Chain: A Fast Lightweight Consensus Algorithm for IoT Applications
Venue: arXiv cs.CR 2020
Year: 2020

Blockchain has received tremendous attention in non-monetary applications including the Internet of Things (IoT) due to its salient features including decentralization, security, auditability, and anonymity. Most conventional blockchains rely on computationally expensive consensus algorithms, have limited throughput, and high transaction delays. In this paper, we propose tree-chain a scalable fast blockchain instantiation that introduces two levels of randomization among the validators: i) transaction level where the validator of each transaction is selected randomly based on the most significant characters of the hash function output (known as consensus code), and ii) blockchain level where validator is randomly allocated to a particular consensus code based on the hash of their public key. Tree-chain introduces parallel chain branches where each validator commits the corresponding transactions in a unique ledger. Implementation results show that tree-chain is runnable on low resource devices and incurs low processing overhead, achieving near real-time transaction settlement.
Graphs with no induced house nor induced hole have the de Bruijn-Erdős property
Venue: arXiv cs.DM 2020
Year: 2020

A set of n points in the plane which are not all collinear defines at least n distinct lines. Chen and Chvátal conjectured in 2008 that a similar result can be achieved in the broader context of finite metric spaces. This conjecture remains open even for graph metrics. In this article we prove that graphs with no induced house nor induced cycle of length at least~5 verify the desired property. We focus on lines generated by vertices at distance at most 2, define a new notion of ``good pairs'' that might have application in larger families, and finally use a discharging technique to count lines in irreducible graphs.
Monads and "do" notation in the Wolfram Language
Venue: arXiv cs.PL 2020
Year: 2020

This paper describes a categorical interpretation of the Wolfram Language and introduces a simple implementation of monadic types and the "do" notation. The monadic style of programming combined with the many built in functions of the Wolfram Language has potential to be a powerful tool in writing Wolfram Language code. Additionally, using pure functions and the "do" notation can result in programs that are very predictable and easy to parallelize.
Built Infrastructure Monitoring and Inspection Using UAVs and Vision-based Algorithms
Venue: arXiv cs.CV 2020
Year: 2020

This study presents an inspecting system using real-time control unmanned aerial vehicles (UAVs) to investigate structural surfaces. The system operates under favourable weather conditions to inspect a target structure, which is the Wentworth light rail base structure in this study. The system includes a drone, a GoPro HERO4 camera, a controller and a mobile phone. The drone takes off the ground manually in the testing field to collect the data requiring for later analysis. The images are taken through HERO 4 camera and then transferred in real time to the remote processing unit such as a ground control station by the wireless connection established by a Wi-Fi router. An image processing method has been proposed to detect defects or damages such as cracks. The method based on intensity histogram algorithms to exploit the pixel group related to the crack contained in the low intensity interval. Experiments, simulation and comparisons have been conducted to evaluate the performance and validity of the proposed system.
On the Separability Problem of String Constraints
Venue: arXiv cs.FL 2020
Year: 2020

We address the separability problem for straight-line string constraints. The separability problem for languages of a class C by a class S asks: given two languages A and B in C, does there exist a language I in S separating A and B (i.e., I is a superset of A and disjoint from B)? The separability of string constraints is the same as the fundamental problem of interpolation for string constraints. We first show that regular separability of straight line string constraints is undecidable. Our second result is the decidability of the separability problem for straight-line string constraints by piece-wise testable languages, though the precise complexity is open. In our third result, we consider the positive fragment of piece-wise testable languages as a separator, and obtain an EXPSPACE algorithm for the separability of a useful class of straight-line string constraints, and a PSPACE-hardness result.
A comparative study of machine learning techniques used in non-clinical systems for continuous healthcare of independent livings
Venue: arXiv cs.CY 2020
Year: 2020

New technologies are adapted to made progress in healthcare especially for independent livings. Medication at distance is leading to integrate technologies with medical. Machine learning methods in collaboration with wearable sensor network technology are used to find hidden patterns in data, detect patient movements, observe habits of patient, analyze clinical data of patient, find intention of patients and make decision on the bases of gathered data. This research performs comparative study on non-clinical systems in healthcare for independent livings. In this study, these systems are sub-divided w.r.t their working into two types: single purpose systems and multi-purpose systems. Systems that are built for single specific purpose (e.g. detect fall, detect emergent state of chronic disease patient) and cannot support healthcare generically are known as single purpose systems, where multi-purpose systems are built to serve for multiple problems (e.g. heart attack etc.) by using single system. This study analyzes usages of machine learning techniques in healthcare systems for independent livings. Answer Set Programming (ASP), Artificial Neural Networks, Classification, Sampling and Rule Based Reasoning etc. are some state of art techniques used to determine emergent situations and observe changes in patient data. Among all methods, ASP logic is used most widely, it is due to its feature to deal with incomplete data. It is also observed that system using ANN shows better accuracy than other systems. It is observed that most of the systems created are for single purpose. In this work, 10 single purpose systems and 5 multi-purpose systems are studied. There is need to create more generic systems that can be used for patients with multiple diseases. Also most of the systems created are prototypical. There is need to create systems that can serve healthcare services in real world.
Webpage Segmentation for Extracting Images and Their Surrounding Contextual Information
Venue: arXiv cs.MM 2020
Year: 2020

Web images come in hand with valuable contextual information. Although this information has long been mined for various uses such as image annotation, clustering of images, inference of image semantic content, etc., insufficient attention has been given to address issues in mining this contextual information. In this paper, we propose a webpage segmentation algorithm targeting the extraction of web images and their contextual information based on their characteristics as they appear on webpages. We conducted a user study to obtain a human-labeled dataset to validate the effectiveness of our method and experiments demonstrated that our method can achieve better results compared to an existing segmentation algorithm.
Motif Discovery Algorithms in Static and Temporal Networks: A Survey
Venue: arXiv cs.SI 2020
Year: 2020

Motifs are the fundamental components of complex systems. The topological structure of networks representing complex systems and the frequency and distribution of motifs in these networks are intertwined. The complexities associated with graph and subgraph isomorphism problems, as the core of frequent subgraph mining, have direct impacts on the performance of motif discovery algorithms. To cope with these complexities, researchers have adopted different strategies for candidate generation and enumeration, and frequency computation. In the past few years, there has been an increasing interest in the analysis and mining of temporal networks. These networks, in contrast to their static counterparts, change over time in the form of insertion, deletion, or substitution of edges or vertices or their attributes. In this paper, we provide a survey of motif discovery algorithms proposed in the literature for mining static and temporal networks and review the corresponding algorithms based on their adopted strategies for candidate generation and frequency computation. As we witness the generation of a large amount of network data in social media platforms, bioinformatics applications, and communication and transportation networks and the advance in distributed computing and big data technology, we also conduct a survey on the algorithms proposed to resolve the CPU-bound and I/O bound problems in mining static and temporal networks.
Predictor Antenna Systems: Exploiting Channel State Information for Vehicle Communications
Venue: arXiv cs.IT 2020
Year: 2020

Vehicle communication is one of the most important use cases in the fifth generation of wireless networks (5G). The growing demand for quality of service (QoS) characterized by performance metrics, such as spectrum efficiency, peak data rate, and outage probability, is mainly limited by inaccurate prediction/estimation of channel state information (CSI) of the rapidly changing environment around moving vehicles. One way to increase the prediction horizon of CSI in order to improve the QoS is deploying predictor antennas (PAs). A PA system consists of two sets of antennas typically mounted on the roof of a vehicle, where the PAs positioned at the front of the vehicle are used to predict the CSI observed by the receive antennas (RAs) that are aligned behind the PAs. In realistic PA systems, however, the actual benefit is affected by a variety of factors, including spatial mismatch, antenna utilization, temporal correlation of scattering environment, and CSI estimation error. This thesis investigates different resource allocation schemes for the PA systems under practical constraints.
Images and Misinformation in Political Groups: Evidence from WhatsApp in India
Venue: arXiv cs.SI 2020
Year: 2020

WhatsApp is a key medium for the spread of news and rumors, often shared as images. We study a large collection of politically-oriented WhatsApp groups in India, focusing on the period leading up to the 2019 Indian national elections. By labeling samples of random and popular images, we find that around 13% of shared images are known misinformation and most fall into three types of images. Machine learning methods can be used to predict whether a viral image is misinformation, but are brittle to shifts in content over time.