OpenRaiser/Intern-Atlas huggingface dataset:
On-shelf Utility Mining of Sequence Data
Venue: arXiv cs.DB 2020
Year: 2020

Utility mining has emerged as an important and interesting topic owing to its wide application and considerable popularity. However, conventional utility mining methods have a bias toward items that have longer on-shelf time as they have a greater chance to generate a high utility. To eliminate the bias, the problem of on-shelf utility mining (OSUM) is introduced. In this paper, we focus on the task of OSUM of sequence data, where the sequential database is divided into several partitions according to time periods and items are associated with utilities and several on-shelf time periods. To address the problem, we propose two methods, OSUM of sequence data (OSUMS) and OSUMS+, to extract on-shelf high-utility sequential patterns. For further efficiency, we also designed several strategies to reduce the search space and avoid redundant calculation with two upper bounds time prefix extension utility (TPEU) and time reduced sequence utility (TRSU). In addition, two novel data structures were developed for facilitating the calculation of upper bounds and utilities. Substantial experimental results on certain real and synthetic datasets show that the two methods outperform the state-of-the-art algorithm. In conclusion, OSUMS may consume a large amount of memory and is unsuitable for cases with limited memory, while OSUMS+ has wider real-life applications owing to its high efficiency.
Towards Effective and Efficient Padding Machines for Tor
Venue: arXiv cs.CR 2020
Year: 2020

Tor recently integrated a circuit padding framework for creating padding machines: defenses that work by defining state machines that inject dummy traffic to protect against traffic analysis attacks like Website Fingerprinting (WF) attacks. In this paper, we explore the design of effective and efficient padding machines to defend against WF attacks. Through the use of carefully crafted datasets, a circuit padding simulator, genetic programming, and manual tuning of padding machines we explore different aspects of what makes padding machines effective and efficient defenses. Our final machine, named Interspace, is probabilistically-defined with a tweakable trade-off between efficiency and effectiveness against the state-of-the-art deep-learning WF attack Deep Fingerprinting by Sirinam et al. We show that Interspace can be both more effective and efficient than WTF-PAD by Juarez et al. the padding machine that inspired the design of Tor's circuit padding framework. We end this paper by showing how Interspace can be made less effective, identifying the promising tactic of probabilistically defined padding machines, and highlighting the need to further explore this tactic in more complex defenses.
A survey of benchmarking frameworks for reinforcement learning
Venue: arXiv cs.LG 2020
Year: 2020

Reinforcement learning has recently experienced increased prominence in the machine learning community. There are many approaches to solving reinforcement learning problems with new techniques developed constantly. When solving problems using reinforcement learning, there are various difficult challenges to overcome. To ensure progress in the field, benchmarks are important for testing new algorithms and comparing with other approaches. The reproducibility of results for fair comparison is therefore vital in ensuring that improvements are accurately judged. This paper provides an overview of different contributions to reinforcement learning benchmarking and discusses how they can assist researchers to address the challenges facing reinforcement learning. The contributions discussed are the most used and recent in the literature. The paper discusses the contributions in terms of implementation, tasks and provided algorithm implementations with benchmarks. The survey aims to bring attention to the wide range of reinforcement learning benchmarking tasks available and to encourage research to take place in a standardised manner. Additionally, this survey acts as an overview for researchers not familiar with the different tasks that can be used to develop and test new reinforcement learning algorithms.
High-Throughput Parallel Viterbi Decoder on GPU Tensor Cores
Venue: arXiv cs.DC 2020
Year: 2020

Many research works have been performed on implementation of Vitrerbi decoding algorithm on GPU instead of FPGA because this platform provides considerable flexibility in addition to great performance. Recently, the recently-introduced Tensor cores in modern GPU architectures provide incredible computing capability. This paper proposes a novel parallel implementation of Viterbi decoding algorithm based on Tensor cores in modern GPU architectures. The proposed parallel algorithm is optimized to efficiently utilize the computing power of Tensor cores. Experiments show considerable throughput improvements in comparison with previous works.
An Ethical Highlighter for People-Centric Dataset Creation
Venue: arXiv cs.CY 2020
Year: 2020

Important ethical concerns arising from computer vision datasets of people have been receiving significant attention, and a number of datasets have been withdrawn as a result. To meet the academic need for people-centric datasets, we propose an analytical framework to guide ethical evaluation of existing datasets and to serve future dataset creators in avoiding missteps. Our work is informed by a review and analysis of prior works and highlights where such ethical challenges arise.
Improving Layer-wise Adaptive Rate Methods using Trust Ratio Clipping
Venue: arXiv cs.LG 2020
Year: 2020

Training neural networks with large batch is of fundamental significance to deep learning. Large batch training remarkably reduces the amount of training time but has difficulties in maintaining accuracy. Recent works have put forward optimization methods such as LARS and LAMB to tackle this issue through adaptive layer-wise optimization using trust ratios. Though prevailing, such methods are observed to still suffer from unstable and extreme trust ratios which degrades performance. In this paper, we propose a new variant of LAMB, called LAMBC, which employs trust ratio clipping to stabilize its magnitude and prevent extreme values. We conducted experiments on image classification tasks such as ImageNet and CIFAR-10 and our empirical results demonstrate promising improvements across different batch sizes.
Multi-objective Neural Architecture Search with Almost No Training
Venue: arXiv cs.CV 2020
Year: 2020

In the recent past, neural architecture search (NAS) has attracted increasing attention from both academia and industries. Despite the steady stream of impressive empirical results, most existing NAS algorithms are computationally prohibitive to execute due to the costly iterations of stochastic gradient descent (SGD) training. In this work, we propose an effective alternative, dubbed Random-Weight Evaluation (RWE), to rapidly estimate the performance of network architectures. By just training the last linear classification layer, RWE reduces the computational cost of evaluating an architecture from hours to seconds. When integrated within an evolutionary multi-objective algorithm, RWE obtains a set of efficient architectures with state-of-the-art performance on CIFAR-10 with less than two hours' searching on a single GPU card. Ablation studies on rank-order correlations and transfer learning experiments to ImageNet have further validated the effectiveness of RWE.
Relation Clustering in Narrative Knowledge Graphs
Venue: arXiv cs.CL 2020
Year: 2020

When coping with literary texts such as novels or short stories, the extraction of structured information in the form of a knowledge graph might be hindered by the huge number of possible relations between the entities corresponding to the characters in the novel and the consequent hurdles in gathering supervised information about them. Such issue is addressed here as an unsupervised task empowered by transformers: relational sentences in the original text are embedded (with SBERT) and clustered in order to merge together semantically similar relations. All the sentences in the same cluster are finally summarized (with BART) and a descriptive label extracted from the summary. Preliminary tests show that such clustering might successfully detect similar relations, and provide a valuable preprocessing for semi-supervised approaches.
Automated acquisition of structured, semantic models of manipulation activities from human VR demonstration
Venue: arXiv cs.AI 2020
Year: 2020

In this paper we present a system capable of collecting and annotating, human performed, robot understandable, everyday activities from virtual environments. The human movements are mapped in the simulated world using off-the-shelf virtual reality devices with full body, and eye tracking capabilities. All the interactions in the virtual world are physically simulated, thus movements and their effects are closely relatable to the real world. During the activity execution, a subsymbolic data logger is recording the environment and the human gaze on a per-frame basis, enabling offline scene reproduction and replays. Coupled with the physics engine, online monitors (symbolic data loggers) are parsing (using various grammars) and recording events, actions, and their effects in the simulated world.
Connecting Context-specific Adaptation in Humans to Meta-learning
Venue: arXiv cs.AI 2020
Year: 2020

Cognitive control, the ability of a system to adapt to the demands of a task, is an integral part of cognition. A widely accepted fact about cognitive control is that it is context-sensitive: Adults and children alike infer information about a task's demands from contextual cues and use these inferences to learn from ambiguous cues. However, the precise way in which people use contextual cues to guide adaptation to a new task remains poorly understood. This work connects the context-sensitive nature of cognitive control to a method for meta-learning with context-conditioned adaptation. We begin by identifying an essential difference between human learning and current approaches to meta-learning: In contrast to humans, existing meta-learning algorithms do not make use of task-specific contextual cues but instead rely exclusively on online feedback in the form of task-specific labels or rewards. To remedy this, we introduce a framework for using contextual information about a task to guide the initialization of task-specific models before adaptation to online feedback. We show how context-conditioned meta-learning can capture human behavior in a cognitive task and how it can be scaled to improve the speed of learning in various settings, including few-shot classification and low-sample reinforcement learning. Our work demonstrates that guiding meta-learning with task information can capture complex, human-like behavior, thereby deepening our understanding of cognitive control.
Urban Twitter Networks and Communities: A Case Study of Microblogging in Athens
Venue: arXiv cs.SI 2020
Year: 2020

This paper examines the community formed by the Twitter users that used a city-level hashtag. In particular, we provide a network perspective of the city of Athens, Greece, as demonstrated by the analysis and visualization of the relevant Twitter hashtag data, in order to present both an overview and deeper insights at the microblogging practices of this geographic local network. Further analysis suggests that the Twitter community defined by the members of the network shows strong signs of a real-life community.
MITAO: a tool for enabling scholars in the Humanities to use Topic Modelling in their studies
Venue: arXiv cs.DL 2020
Year: 2020

Automatic text analysis methods, such as Topic Modelling, are gaining much attention in Humanities. However, scholars need to have extensive coding skills to use such methods appropriately. The need of having this technical expertise prevents the broad adoption of these methods in Humanities research. In this paper, to help scholars in the Humanities to use Topic Modelling having no or limited coding skills, we introduce MITAO, a web-based tool that allow the definition of a visual workflow which embeds various automatic text analysis operations and allows one to store and share both the workflow and the results of its execution to other researchers, which enables the reproducibility of the analysis. We present an example of an application of use of Topic Modelling with MITAO using a collection of English abstracts of the articles published in "Umanistica Digitale". The results returned by MITAO are shown with dynamic web-based visualizations, which allowed us to have preliminary insights about the evolution of the topics treated over the time in the articles published in "Umanistica Digitale". All the results along with the defined workflows are published and accessible for further studies.
Convolutional Neural Networks Towards Arduino Navigation of Indoor Environments
Venue: arXiv cs.RO 2020
Year: 2020

In this paper we propose a number of tested ways in which a low-budget demo car could be made to navigate an indoor environment. Canny Edge Detection, Supervised Floor Detection and Imitation Learning were used separately and are contrasted in their effectiveness. The equipment used in this paper approximated an autonomous robot configured to work with a mobile device for image processing. This paper does not provide definitive solutions and simply illustrates the approaches taken to successfully achieve autonomous navigation of indoor environments. The successes and failures of all approaches were recorded and elaborated on to give the reader an insight into the construction of such an autonomous robot.
IntegriScreen: Visually Supervising Remote User Interactions on Compromised Clients
Venue: arXiv cs.CR 2020
Year: 2020

Remote services and applications that users access via their local clients (laptops or desktops) usually assume that, following a successful user authentication at the beginning of the session, all subsequent communication reflects the user's intent. However, this is not true if the adversary gains control of the client and can therefore manipulate what the user sees and what is sent to the remote server. To protect the user's communication with the remote server despite a potentially compromised local client, we propose the concept of continuous visual supervision by a second device equipped with a camera. Motivated by the rapid increase of the number of incoming devices with front-facing cameras, such as augmented reality headsets and smart home assistants, we build upon the core idea that the user's actual intended input is what is shown on the client's screen, despite what ends up being sent to the remote server. A statically positioned camera enabled device can, therefore, continuously analyze the client's screen to enforce that the client behaves honestly despite potentially being malicious. We evaluate the present-day feasibility and deployability of this concept by developing a fully functional prototype, running a host of experimental tests on three different mobile devices, and by conducting a user study in which we analyze participants' use of the system during various simulated attacks. Experimental evaluation indeed confirms the feasibility of the concept of visual supervision, given that the system consistently detects over 98% of evaluated attacks, while study participants with little instruction detect the remaining attacks with high probability.
On Performance Comparison of Multi-Antenna HD-NOMA, SCMA and PD-NOMA Schemes
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we study the uplink channel throughput performance of a proposed novel multiple-antenna hybrid-domain non-orthogonal multiple access (MA-HD-NOMA) scheme. This scheme combines the conventional sparse code multiple access (SCMA) and power-domain NOMA (PD-NOMA) schemes in order to increase the number of users served as compared to conventional NOMA schemes and uses multiple antennas at the base station. To this end, a joint resource allocation problem for the MA-HD-NOMA scheme is formulated that maximizes the sum rate of the entire system. For a comprehensive comparison, the joint resource allocation problems for the multi-antenna SCMA (MA-SCMA) and multi-antenna PD-NOMA (MA-PD-NOMA) schemes with the same overloading factor are formulated as well. Each of the formulated problems is a mixed-integer non-convex program, and hence, we apply successive convex approximation (SCA)- and reweighted $\ell_1$ minimization-based approaches to obtain rapidly converging solutions. Numerical results reveal that the proposed MA-HD-NOMA scheme has superior performance compared to MA-SCMA and MA-PD-NOMA.
AdaBins: Depth Estimation using Adaptive Bins
Venue: arXiv cs.CV 2020
Year: 2020

We address the problem of estimating a high quality dense depth map from a single RGB input image. We start out with a baseline encoder-decoder convolutional neural network architecture and pose the question of how the global processing of information can help improve overall depth estimation. To this end, we propose a transformer-based architecture block that divides the depth range into bins whose center value is estimated adaptively per image. The final depth values are estimated as linear combinations of the bin centers. We call our new building block AdaBins. Our results show a decisive improvement over the state-of-the-art on several popular depth datasets across all metrics. We also validate the effectiveness of the proposed block with an ablation study and provide the code and corresponding pre-trained weights of the new state-of-the-art model.
i3DMM: Deep Implicit 3D Morphable Model of Human Heads
Venue: arXiv cs.CV 2020
Year: 2020

We present the first deep implicit 3D morphable model (i3DMM) of full heads. Unlike earlier morphable face models it not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair. We collect a new dataset consisting of 64 people with different expressions and hairstyles to train i3DMM. Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair. (ii) In contrast to mesh-based models it can be trained on merely rigidly aligned scans, without requiring difficult non-rigid registration. (iii) We design a novel architecture to decouple the shape model into an implicit reference shape and a deformation of this reference shape. With that, dense correspondences between shapes can be learned implicitly. (iv) This architecture allows us to semantically disentangle the geometry and color components, as color is learned in the reference space. Geometry is further disentangled as identity, expressions, and hairstyle, while color is disentangled as identity and hairstyle components. We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer. We will make our model publicly available.
Thermodynamic Consistent Neural Networks for Learning Material Interfacial Mechanics
Venue: arXiv cs.CE 2020
Year: 2020

For multilayer materials in thin substrate systems, interfacial failure is one of the most challenges. The traction-separation relations (TSR) quantitatively describe the mechanical behavior of a material interface undergoing openings, which is critical to understand and predict interfacial failures under complex loadings. However, existing theoretical models have limitations on enough complexity and flexibility to well learn the real-world TSR from experimental observations. A neural network can fit well along with the loading paths but often fails to obey the laws of physics, due to a lack of experimental data and understanding of the hidden physical mechanism. In this paper, we propose a thermodynamic consistent neural network (TCNN) approach to build a data-driven model of the TSR with sparse experimental data. The TCNN leverages recent advances in physics-informed neural networks (PINN) that encode prior physical information into the loss function and efficiently train the neural networks using automatic differentiation. We investigate three thermodynamic consistent principles, i.e., positive energy dissipation, steepest energy dissipation gradient, and energy conservative loading path. All of them are mathematically formulated and embedded into a neural network model with a novel defined loss function. A real-world experiment demonstrates the superior performance of TCNN, and we find that TCNN provides an accurate prediction of the whole TSR surface and significantly reduces the violated prediction against the laws of physics.
Monte Carlo Tree Search for a single target search game on a 2-D lattice
Venue: arXiv cs.LG 2020
Year: 2020

Monte Carlo Tree Search (MCTS) is a branch of stochastic modeling that utilizes decision trees for optimization, mostly applied to artificial intelligence (AI) game players. This project imagines a game in which an AI player searches for a stationary target within a 2-D lattice. We analyze its behavior with different target distributions and compare its efficiency to the Levy Flight Search, a model for animal foraging behavior. In addition to simulated data analysis we prove two theorems about the convergence of MCTS when computation constraints neglected.
Intrinsic Knowledge Evaluation on Chinese Language Models
Venue: arXiv cs.CL 2020
Year: 2020

Recent NLP tasks have benefited a lot from pre-trained language models (LM) since they are able to encode knowledge of various aspects. However, current LM evaluations focus on downstream performance, hence lack to comprehensively inspect in which aspect and to what extent have they encoded knowledge. This paper addresses both queries by proposing four tasks on syntactic, semantic, commonsense, and factual knowledge, aggregating to a total of $39,308$ questions covering both linguistic and world knowledge in Chinese. Throughout experiments, our probes and knowledge data prove to be a reliable benchmark for evaluating pre-trained Chinese LMs. Our work is publicly available at https://github.com/ZhiruoWang/ChnEval.
ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we introduce ProtoPShare, a self-explained method that incorporates the paradigm of prototypical parts to explain its predictions. The main novelty of the ProtoPShare is its ability to efficiently share prototypical parts between the classes thanks to our data-dependent merge-pruning. Moreover, the prototypes are more consistent and the model is more robust to image perturbations than the state of the art method ProtoPNet. We verify our findings on two datasets, the CUB-200-2011 and the Stanford Cars.
A smartphone based multi input workflow for non-invasive estimation of haemoglobin levels using machine learning techniques
Venue: arXiv cs.LG 2020
Year: 2020

We suggest a low cost, non invasive healthcare system that measures haemoglobin levels in patients and can be used as a preliminary diagnostic test for anaemia. A combination of image processing, machine learning and deep learning techniques are employed to develop predictive models to measure haemoglobin levels. This is achieved through the color analysis of the fingernail beds, palpebral conjunctiva and tongue of the patients. This predictive model is then encapsulated in a healthcare application. This application expedites data collection and facilitates active learning of the model. It also incorporates personalized calibration of the model for each patient, assisting in the continual monitoring of the haemoglobin levels of the patient. Upon validating this framework using data, it can serve as a highly accurate preliminary diagnostic test for anaemia.