OpenRaiser/Intern-Atlas huggingface dataset:
A hierarchical residual network with compact triplet-center loss for sketch recognition
Venue: arXiv cs.CV 2021
Year: 2021

With the widespread use of touch-screen devices, it is more and more convenient for people to draw sketches on screen. This results in the demand for automatically understanding the sketches. Thus, the sketch recognition task becomes more significant than before. To accomplish this task, it is necessary to solve the critical issue of improving the distinction of the sketch features. To this end, we have made efforts in three aspects. First, a novel multi-scale residual block is designed. Compared with the conventional basic residual block, it can better perceive multi-scale information and reduce the number of parameters during training. Second, a hierarchical residual structure is built by stacking multi-scale residual blocks in a specific way. In contrast with the single-level residual structure, the learned features from this structure are more sufficient. Last but not least, the compact triplet-center loss is proposed specifically for the sketch recognition task. It can solve the problem that the triplet-center loss does not fully consider too large intra-class space and too small inter-class space in sketch field. By studying the above modules, a hierarchical residual network as a whole is proposed for sketch recognition and evaluated on Tu-Berlin benchmark thoroughly. The experimental results show that the proposed network outperforms most of baseline methods and it is excellent among non-sequential models at present.
Code Comprehension Confounders: A Study of Intelligence and Personal
Venue: arXiv cs.SE 2021
Year: 2021

Literature and intuition suggest that a developer's intelligence and personality have an impact on their performance in comprehending source code. Researchers made this suggestion in the past when discussing threats to validity of their study results. However, the lack of studies investigating the relationship of intelligence and personality to performance in code comprehension makes scientifically sound reasoning about their influence difficult. We conduct the first empirical evaluation, a correlational study with undergraduates, to investigate the correlation of intelligence and personality with performance in code comprehension, that is with correctness in answering comprehension questions on code snippets. We found that personality traits are unlikely to impact code comprehension performance, at least not considered in isolation. Conscientiousness, in combination with other factors, however, explains some of the variance in code comprehension performance. For intelligence, significant small to moderate positive effects on code comprehension performance were found for three of four factors measured, i.e., fluid intelligence, visual perception, and cognitive speed. Crystallized intelligence has a positive but statistically insignificant effect on code comprehension performance. According to our results, several intelligence facets as well as the personality trait conscientiousness are potential confounders that should not be neglected in code comprehension studies of individual performance and should be controlled for via an appropriate study design. We call for the conduct of further studies on the relationship between intelligence and personality with code comprehension, in part because code comprehension involves more facets than we can measure in a single study and because our regression model explains only a small portion of the variance in code comprehension performance.
SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies
Venue: arXiv cs.RO 2021
Year: 2021

In this paper we present the first safe system for full control of self-driving vehicles trained from human demonstrations and deployed in challenging, real-world, urban environments. Current industry-standard solutions use rule-based systems for planning. Although they perform reasonably well in common scenarios, the engineering complexity renders this approach incompatible with human-level performance. On the other hand, the performance of machine-learned (ML) planning solutions can be improved by simply adding more exemplar data. However, ML methods cannot offer safety guarantees and sometimes behave unpredictably. To combat this, our approach uses a simple yet effective rule-based fallback layer that performs sanity checks on an ML planner's decisions (e.g. avoiding collision, assuring physical feasibility). This allows us to leverage ML to handle complex situations while still assuring the safety, reducing ML planner-only collisions by 95%. We train our ML planner on 300 hours of expert driving demonstrations using imitation learning and deploy it along with the fallback layer in downtown San Francisco, where it takes complete control of a real vehicle and navigates a wide variety of challenging urban driving scenarios.
Active Learning for Argument Mining: A Practical Approach
Venue: arXiv cs.CL 2021
Year: 2021

Despite considerable recent progress, the creation of well-balanced and diverse resources remains a time-consuming and costly challenge in Argument Mining. Active Learning reduces the amount of data necessary for the training of machine learning models by querying the most informative samples for annotation and therefore is a promising method for resource creation. In a large scale comparison of several Active Learning methods, we show that Active Learning considerably decreases the effort necessary to get good deep learning performance on the task of Argument Unit Recognition and Classification (AURC).
Meta Reinforcement Learning Based Sensor Scanning in 3D Uncertain Environments for Heterogeneous Multi-Robot Systems
Venue: arXiv cs.RO 2021
Year: 2021

We study a novel problem that tackles learning based sensor scanning in 3D and uncertain environments with heterogeneous multi-robot systems. Our motivation is two-fold: first, 3D environments are complex, the use of heterogeneous multi-robot systems intuitively can facilitate sensor scanning by fully taking advantage of sensors with different capabilities. Second, in uncertain environments (e.g. rescue), time is of great significance. Since the learning process normally takes time to train and adapt to a new environment, we need to find an effective way to explore and adapt quickly. To this end, in this paper, we present a meta-learning approach to improve the exploration and adaptation capabilities. The experimental results demonstrate our method can outperform other methods by approximately 15%-27% on success rate and 70%-75% on adaptation speed.
Weighted Secrecy Coverage Analysis and the Impact of Friendly Jamming over UAV-Enabled Networks
Venue: arXiv cs.IT 2021
Year: 2021

In 5G and beyond networks, Unmanned Aerial Vehicles (UAV) are an attractive solution to enhance the secrecy of a wireless systems by exploiting their predominant LOS links and spacial manoeuvrability to introduce a friendly jamming. In this work, we investigate the impact of two cooperative UAV-based jammers on the secrecy performance of a ground wireless wiretap channel by considering secrecy-area related metrics, the jamming coverage and jamming efficiency. Moreover, we propose a hybrid metric, the so-called Weighted Secrecy Coverage (WSC) that can be used as a metric for gaining insights on the optimal deployments of the UAV jammers to provide the best exploration of jamming signals. For evaluating these metrics, we derive a closed-form position-based metric, the secrecy improvement, and propose an analogous computationally simpler metric. Our simulations show that a balanced power allocation between the two UAVs leads to the best performances, as well as a symmetrical positioning behind the line of sight between the legitimate transmitter and receiver. Moreover, there exist an optimal UAV height for the jammers. Finally, we propose a sub-optimal and simpler problem for the maximisation of the WSC.
Literature Review on Endoscopic Robotic Systems in Ear and Sinus Surgery
Venue: arXiv cs.RO 2021
Year: 2021

In otolaryngologic surgery, endoscopy is increasingly used to provide a better view of hard-to-reach areas and to promote minimally invasive surgery. However, the need to manipulate the endoscope limits the surgeon's ability to operate with only one instrument at a time. Currently, several robotic systems are being developed, demonstrating the value of robotic assistance in microsurgery. The aim of this literature review is to present and classify current robotic systems that are used for otological and endonasal applications. For these solutions, an analysis of the functionalities in relation to the surgeon's needs will be carried out in order to produce a set of specifications for the creation of new robots.
Machine learning methods for prediction of cancer driver genes: a survey paper
Venue: arXiv cs.LG 2021
Year: 2021

Identifying the genes and mutations that drive the emergence of tumors is a major step to improve understanding of cancer and identify new directions for disease diagnosis and treatment. Despite the large volume of genomics data, the precise detection of driver mutations and their carrying genes, known as cancer driver genes, from the millions of possible somatic mutations remains a challenge. Computational methods play an increasingly important role in identifying genomic patterns associated with cancer drivers and developing models to predict driver events. Machine learning (ML) has been the engine behind many of these efforts and provides excellent opportunities for tackling remaining gaps in the field. Thus, this survey aims to perform a comprehensive analysis of ML-based computational approaches to identify cancer driver mutations and genes, providing an integrated, panoramic view of the broad data and algorithmic landscape within this scientific problem. We discuss how the interactions among data types and ML algorithms have been explored in previous solutions and outline current analytical limitations that deserve further attention from the scientific community. We hope that by helping readers become more familiar with significant developments in the field brought by ML, we may inspire new researchers to address open problems and advance our knowledge towards cancer driver discovery.
PSSPR: A Source Location Privacy Protection Scheme Based on Sector Phantom Routing in WSNs
Venue: arXiv cs.NI 2021
Year: 2021

Source location privacy (SLP) protection is an emerging research topic in wireless sensor networks (WSNs). Because the source location represents the valuable information of the target being monitored and tracked, it is of great practical significance to achieve a high degree of privacy of the source location. Although many studies based on phantom nodes have alleviates the protection of source location privacy to some extent. It is urgent to solve the problems such as complicate the ac path between nodes, improve the centralized distribution of Phantom nodes near the source nodes and reduce the network communication overhead. In this paper, PSSPR routing is proposed as a visable approach to address SLP issues. We use the coordinates of the center node V to divide sector-domain, which act as important role in generating a new phantom nodes. The phantom nodes perform specified routing policies to ensure that they can choose various locations. In addition, the directed random route can ensure that data packets avoid the visible range when they move to the sink node hop by hop. Thus, the source location is protected. Theoretical analysis and simulation experiments show that this protocol achieves higher security of source node location with less communication overhead.
Faster Improvement Rate Population Based Training
Venue: arXiv cs.NE 2021
Year: 2021

The successful training of neural networks typically involves careful and time consuming hyperparameter tuning. Population Based Training (PBT) has recently been proposed to automate this process. PBT trains a population of neural networks concurrently, frequently mutating their hyperparameters throughout their training. However, the decision mechanisms of PBT are greedy and favour short-term improvements which can, in some cases, lead to poor long-term performance. This paper presents Faster Improvement Rate PBT (FIRE PBT) which addresses this problem. Our method is guided by an assumption: given two neural networks with similar performance and training with similar hyperparameters, the network showing the faster rate of improvement will lead to a better final performance. Using this, we derive a novel fitness metric and use it to make some of the population members focus on long-term performance. Our experiments show that FIRE PBT is able to outperform PBT on the ImageNet benchmark and match the performance of networks that were trained with a hand-tuned learning rate schedule. We apply FIRE PBT to reinforcement learning tasks and show that it leads to faster learning and higher final performance than both PBT and random hyperparameter search.
Intelligence Complements from the Built Environment: A review of Smart Building Technologies for Cognitively Declined Occupants
Venue: arXiv cs.HC 2021
Year: 2021

Traditionally, caregivers, whether formal or informal, have taken the responsibility of providing assistance and care to patients with cognitive decline. Usually, both the caregiver and the patient are subjected to financial and emotional burdens, which impact the patient's life quality. To overcome this issue, Ambient Assistive Living (AAL) technologies have been adopted to replace the caregivers and complement patients' lack of intelligence. Technologies such as Internet of Things (IoT) and Artificial Intelligence (AI) have enabled intelligent ubiquitous learning for smart buildings to monitor the cognitively declined occupants and provide in-home assistive services and solutions. This paper aims to summarize and evaluate the intelligence complements provided by smart buildings that can increase the cognitively declined occupants' quality of life and autonomy. Through a systematic literature review, the authors find that most of the existing contributions are towards identifying the occupants' behavior, and thus, to determine corresponding assistive services and solutions. Five key research gaps are identified, including the lack of adequate adoption of technological interventions to fully support the occupants' autonomy and independence. The authors also propose a conceptual framework to highlight the research gaps in smart building applications for cognitively declined occupants and to map the future research directions.
A First-Occupancy Representation for Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Both animals and artificial agents benefit from state representations that support rapid transfer of learning across tasks and which enable them to efficiently traverse their environments to reach rewarding states. The successor representation (SR), which measures the expected cumulative, discounted state occupancy under a fixed policy, enables efficient transfer to different reward structures in an otherwise constant Markovian environment and has been hypothesized to underlie aspects of biological behavior and neural activity. However, in the real world, rewards may move or only be available for consumption once, may shift location, or agents may simply aim to reach goal states as rapidly as possible without the constraint of artificially imposed task horizons. In such cases, the most behaviorally-relevant representation would carry information about when the agent was likely to first reach states of interest, rather than how often it should expect to visit them over a potentially infinite time span. To reflect such demands, we introduce the first-occupancy representation (FR), which measures the expected temporal discount to the first time a state is accessed. We demonstrate that the FR facilitates exploration, the selection of efficient paths to desired states, allows the agent, under certain conditions, to plan provably optimal trajectories defined by a sequence of subgoals, and induces similar behavior to animals avoiding threatening stimuli.
Turning old models fashion again: Recycling classical CNN networks using the Lattice Transformation
Venue: arXiv cs.CV 2021
Year: 2021

In the early 1990s, the first signs of life of the CNN era were given: LeCun et al. proposed a CNN model trained by the backpropagation algorithm to classify low-resolution images of handwritten digits. Undoubtedly, it was a breakthrough in the field of computer vision. But with the rise of other classification methods, it fell out fashion. That was until 2012, when Krizhevsky et al. revived the interest in CNNs by exhibiting considerably higher image classification accuracy on the ImageNet challenge. Since then, the complexity of the architectures are exponentially increasing and many structures are rapidly becoming obsolete. Using multistream networks as a base and the feature infusion precept, we explore the proposed LCNN cross-fusion strategy to use the backbones of former state-of-the-art networks on image classification in order to discover if the technique is able to put these designs back in the game. In this paper, we showed that we can obtain an increase of accuracy up to 63.21% on the NORB dataset we comparing with the original structure. However, no technique is definitive. While our goal is to try to reuse previous state-of-the-art architectures with few modifications, we also expose the disadvantages of our explored strategy.
Symbolic Regression by Exhaustive Search: Reducing the Search Space Using Syntactical Constraints and Efficient Semantic Structure Deduplication
Venue: arXiv cs.LG 2021
Year: 2021

Symbolic regression is a powerful system identification technique in industrial scenarios where no prior knowledge on model structure is available. Such scenarios often require specific model properties such as interpretability, robustness, trustworthiness and plausibility, that are not easily achievable using standard approaches like genetic programming for symbolic regression. In this chapter we introduce a deterministic symbolic regression algorithm specifically designed to address these issues. The algorithm uses a context-free grammar to produce models that are parameterized by a non-linear least squares local optimization procedure. A finite enumeration of all possible models is guaranteed by structural restrictions as well as a caching mechanism for detecting semantically equivalent solutions. Enumeration order is established via heuristics designed to improve search efficiency. Empirical tests on a comprehensive benchmark suite show that our approach is competitive with genetic programming in many noiseless problems while maintaining desirable properties such as simple, reliable models and reproducibility.
A Contrastive Learning Approach to Auroral Identification and Classification
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised learning algorithms are beginning to achieve accuracies comparable to their supervised counterparts on benchmark computer vision tasks, but their utility for practical applications has not yet been demonstrated. In this work, we present a novel application of unsupervised learning to the task of auroral image classification. Specifically, we modify and adapt the Simple framework for Contrastive Learning of Representations (SimCLR) algorithm to learn representations of auroral images in a recently released auroral image dataset constructed using image data from Time History of Events and Macroscale Interactions during Substorms (THEMIS) all-sky imagers. We demonstrate that (a) simple linear classifiers fit to the learned representations of the images achieve state-of-the-art classification performance, improving the classification accuracy by almost 10 percentage points over the current benchmark; and (b) the learned representations naturally cluster into more clusters than exist manually assigned categories, suggesting that existing categorizations are overly coarse and may obscure important connections between auroral types, near-earth solar wind conditions, and geomagnetic disturbances at the earth's surface. Moreover, our model is much lighter than the previous benchmark on this dataset, requiring in the area of fewer than 25\% of the number of parameters. Our approach exceeds an established threshold for operational purposes, demonstrating readiness for deployment and utilization.
New Hybrid Techniques for Business Recommender Systems
Venue: arXiv cs.IR 2021
Year: 2021

Besides the typical applications of recommender systems in B2C scenarios such as movie or shopping platforms, there is a rising interest in transforming the human-driven advice provided e.g. in consultancy via the use of recommender systems. We explore the special characteristics of such knowledge-based B2B services and propose a process that allows to incorporate recommender systems into them. We suggest and compare several recommender techniques that allow to incorporate the necessary contextual knowledge (e.g. company demographics). These techniques are evaluated in isolation on a test set of business intelligence consultancy cases. We then identify the respective strengths of the different techniques and propose a new hybridisation strategy to combine these strengths. Our results show that the hybridisation leads to a substantial performance improvement over the individual methods.
Fine-tuning Vision Transformers for the Prediction of State Variables in Ising Models
Venue: arXiv cs.CV 2021
Year: 2021

Transformers are state-of-the-art deep learning models that are composed of stacked attention and point-wise, fully connected layers designed for handling sequential data. Transformers are not only ubiquitous throughout Natural Language Processing (NLP), but, recently, they have inspired a new wave of Computer Vision (CV) applications research. In this work, a Vision Transformer (ViT) is applied to predict the state variables of 2-dimensional Ising model simulations. Our experiments show that ViT outperform state-of-the-art Convolutional Neural Networks (CNN) when using a small number of microstate images from the Ising model corresponding to various boundary conditions and temperatures. This work opens the possibility of applying ViT to other simulations, and raises interesting research directions on how attention maps can learn about the underlying physics governing different phenomena.
Risk averse non-stationary multi-armed bandits
Venue: arXiv cs.LG 2021
Year: 2021

This paper tackles the risk averse multi-armed bandits problem when incurred losses are non-stationary. The conditional value-at-risk (CVaR) is used as the objective function. Two estimation methods are proposed for this objective function in the presence of non-stationary losses, one relying on a weighted empirical distribution of losses and another on the dual representation of the CVaR. Such estimates can then be embedded into classic arm selection methods such as epsilon-greedy policies. Simulation experiments assess the performance of the arm selection algorithms based on the two novel estimation approaches, and such policies are shown to outperform naive benchmarks not taking non-stationarity into account.