OpenRaiser/Intern-Atlas huggingface dataset:
Performance Analysis of 2-Step Random Access with CDMA in Machine-Type Communication
Venue: arXiv cs.IT 2020
Year: 2020

There is a growing interest in the transition from 4-step random access to 2-step random access in machine-type communication (MTC), since 2-step random access is well-suited to short message delivery in various Internet of Things (IoT) applications. In this paper, we study a 2-step random access approach that uses code division multiple access (CDMA) to form multiple channels for data packet transmissions with a spreading factor less than the number of channels. As a result, the length of data transmission phase in 2-step random access can be shorter at the cost of multiuser interference. To see how the decrease of the length of data transmission phase can improve the spectral efficiency, we derive the throughput as well as the spectral efficiency. From the results of analysis, we can show that the 2-step CDMA-based random access approach can have a higher spectral efficiency than conventional 2-step approach with orthogonal channel allocations, which means that the performance of MTC can be improved by successfully transmitting more data packets per unit time using CDMA. This is also confirmed by simulation results.
MS-GWNN:multi-scale graph wavelet neural network for breast cancer diagnosis
Venue: arXiv cs.CV 2020
Year: 2020

Breast cancer is one of the most common cancers in women worldwide, and early detection can significantly reduce the mortality rate of breast cancer. It is crucial to take multi-scale information of tissue structure into account in the detection of breast cancer. And thus, it is the key to design an accurate computer-aided detection (CAD) system to capture multi-scale contextual features in a cancerous tissue. In this work, we present a novel graph convolutional neural network for histopathological image classification of breast cancer. The new method, named multi-scale graph wavelet neural network (MS-GWNN), leverages the localization property of spectral graph wavelet to perform multi-scale analysis. By aggregating features at different scales, MS-GWNN can encode the multi-scale contextual interactions in the whole pathological slide. Experimental results on two public datasets demonstrate the superiority of the proposed method. Moreover, through ablation studies, we find that multi-scale analysis has a significant impact on the accuracy of cancer diagnosis.
Peacock Exploration: A Lightweight Exploration for UAV using Control-Efficient Trajectory
Venue: arXiv cs.RO 2020
Year: 2020

Unmanned Aerial Vehicles have received much attention in recent years due to its wide range of applications, such as exploration of an unknown environment to acquire a 3D map without prior knowledge of it. Existing exploration methods have been largely challenged by computationally heavy probabilistic path planning. Similarly, kinodynamic constraints or proper sensors considering the payload for UAVs were not considered. In this paper, to solve those issues and to consider the limited payload and computational resource of UAVs, we propose "Peacock Exploration": A lightweight exploration method for UAVs using precomputed minimum snap trajectories which look like a peacock's tail. Using the widely known, control efficient minimum snap trajectories and OctoMap, the UAV equipped with a RGB-D camera can explore unknown 3D environments without any prior knowledge or human-guidance with only O(logN) computational complexity. It also adopts the receding horizon approach and simple, heuristic scoring criteria. The proposed algorithm's performance is demonstrated by exploring a challenging 3D maze environment and compared with a state-of-the-art algorithm.
Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents
Venue: arXiv cs.CL 2020
Year: 2020

Goal-oriented conversational agents are becoming prevalent in our daily lives. For these systems to engage users and achieve their goals, they need to exhibit appropriate social behavior as well as provide informative replies that guide users through tasks. The first component of the research in this paper applies statistical modeling techniques to understand conversations between users and human agents for customer service. Analyses show that social language used by human agents is associated with greater users' responsiveness and task completion. The second component of the research is the construction of a conversational agent model capable of injecting social language into an agent's responses while still preserving content. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
Tips and Tricks for Webly-Supervised Fine-Grained Recognition: Learning from the WebFG 2020 Challenge
Venue: arXiv cs.CV 2020
Year: 2020

WebFG 2020 is an international challenge hosted by Nanjing University of Science and Technology, University of Edinburgh, Nanjing University, The University of Adelaide, Waseda University, etc. This challenge mainly pays attention to the webly-supervised fine-grained recognition problem. In the literature, existing deep learning methods highly rely on large-scale and high-quality labeled training data, which poses a limitation to their practicability and scalability in real world applications. In particular, for fine-grained recognition, a visual task that requires professional knowledge for labeling, the cost of acquiring labeled training data is quite high. It causes extreme difficulties to obtain a large amount of high-quality training data. Therefore, utilizing free web data to train fine-grained recognition models has attracted increasing attentions from researchers in the fine-grained community. This challenge expects participants to develop webly-supervised fine-grained recognition methods, which leverages web images in training fine-grained recognition models to ease the extreme dependence of deep learning methods on large-scale manually labeled datasets and to enhance their practicability and scalability. In this technical report, we have pulled together the top WebFG 2020 solutions of total 54 competing teams, and discuss what methods worked best across the set of winning teams, and what surprisingly did not help.
Image-to-Image Retrieval by Learning Similarity between Scene Graphs
Venue: arXiv cs.CV 2020
Year: 2020

As a scene graph compactly summarizes the high-level content of an image in a structured and symbolic manner, the similarity between scene graphs of two images reflects the relevance of their contents. Based on this idea, we propose a novel approach for image-to-image retrieval using scene graph similarity measured by graph neural networks. In our approach, graph neural networks are trained to predict the proxy image relevance measure, computed from human-annotated captions using a pre-trained sentence similarity model. We collect and publish the dataset for image relevance measured by human annotators to evaluate retrieval algorithms. The collected dataset shows that our method agrees well with the human perception of image similarity than other competitive baselines.
Resource Analysis of Ethereum 2.0 Clients
Venue: arXiv cs.CR 2020
Year: 2020

Scalability is a common issue among the most used permissionless blockchains, and several approaches have been proposed accordingly. As Ethereum is set to be a solid foundation for a decentralized Internet web, the need for tackling scalability issues while preserving the security of the network is an important challenge. In order to successfully deliver effective scaling solutions, Ethereum is on the path of a major protocol improvement called Ethereum 2.0 (Eth2), which implements sharding. As the change of consensus mechanism is an extremely delicate matter, this improvement will be achieved through different phases, the first of which is the implementation of the Beacon Chain. For this, a specification has been developed and multiple groups have implemented clients to run the new protocol. In this work, we analyse the resource usage behaviour of different clients running as Eth2 nodes, comparing their performance and analysing differences. Our results show multiple network perturbations and how different clients react to it.
Thermal Safety and Real-Time Predictability on Heterogeneous Embedded SoC Platforms
Venue: arXiv cs.DC 2020
Year: 2020

Recent embedded systems are designed with high-performance System-on-Chips (SoCs) to satisfy the computational needs of complex applications widely used in real life, such as airplane controllers, autonomous driving automobiles, medical devices, drones, and hand-held devices. Modern SoCs integrate multi-core CPUs and various types of accelerators including GPUs and DSPs. Uncontrolled heat dissipation is one of the main sources of interference that can adversely affect the reliability and real-time performance of safety-critical applications. The mechanisms currently available to protect SoCs from overheating, such as frequency throttling or core shutdown, may exacerbate the problem as they cause unpredictable delay and deadline misses. Dynamic changes in ambient temperature further increase the difficulty of solving this problem. This dissertation addresses the challenges caused by thermal interference in real-time mixed-criticality systems built with heterogeneous embedded SoC platforms. We propose a novel thermal-aware system framework with analytical timing and thermal models to guarantee safe execution of real-time tasks under the thermal constraints of a multi-core CPU/GPU integrated SoC. For mixed-criticality tasks, the proposed framework bounds the heat generation of the system at each criticality level and provides different levels of assurance against ambient temperature changes. In addition, we propose a data-driven thermal parameter estimation scheme that is directly applicable to MCSs built with commercial-off-the-shelf multi-core processors to obtain a precise thermal model without using special measurement instruments or access to proprietary information. The practicality and effectiveness of our solutions have been evaluated using real SoC platforms and our contributions will help develop systems with thermal safety and real-time predictability.
Supporting Human Memory by Reconstructing Personal Episodic Narratives from Digital Traces
Venue: arXiv cs.IR 2020
Year: 2020

Numerous applications capture in digital form aspects of people's lives. The resulting data, which we call Personal Digital Traces - PDTs, can be used to help reconstruct people's episodic memories and connect to their past personal events. This reconstruction has several applications, from helping patients with neurodegenerative diseases recall past events to gathering clues from multiple sources to identify recent contacts and places visited - a critical new application for the current health crisis. This paper takes steps towards integrating, connecting and summarizing the heterogeneous collection of data into episodic narratives using scripts - prototypical plans for everyday activities. Specifically, we propose a matching algorithm that groups several digital traces from many different sources into script instances (episodes), and we provide a technique for ranking the likelihood of candidate episodes. We report on the results of a study based on the personal data of real users, which gives evidence that our episode reconstruction technique 1) successfully integrates and combines traces from different sources into coherent episodes, and 2) augments users' memory of their past actions.
Analysis of the Scalability of a Deep-Learning Network for Steganography "Into the Wild"
Venue: arXiv cs.CR 2020
Year: 2020

Since the emergence of deep learning and its adoption in steganalysis fields, most of the reference articles kept using small to medium size CNN, and learn them on relatively small databases. Therefore, benchmarks and comparisons between different deep learning-based steganalysis algorithms, more precisely CNNs, are thus made on small to medium databases. This is performed without knowing: 1. if the ranking, with a criterion such as accuracy, is always the same when the database is larger, 2. if the efficiency of CNNs will collapse or not if the training database is a multiple of magnitude larger, 3. the minimum size required for a database or a CNN, in order to obtain a better result than a random guesser. In this paper, after a solid discussion related to the observed behaviour of CNNs as a function of their sizes and the database size, we confirm that the error's power-law also stands in steganalysis, and this in a border case, i.e. with a medium-size network, on a big, constrained and very diverse database.
Twin Neural Network Regression
Venue: arXiv cs.LG 2020
Year: 2020

We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are normally costly to produce, TNN regression intrinsically creates an ensemble of predictions of twice the size of the training set while only training a single neural network. Since ensembles have been shown to be more accurate than single models this property naturally transfers to TNN regression. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods. Furthermore, TNN regression is constrained by self-consistency conditions. We find that the violation of these conditions provides an estimate for the prediction uncertainty.
An LQR-assisted Control Algorithm for an Under-actuated In-pipe Robot in Water Distribution Systems
Venue: arXiv cs.RO 2020
Year: 2020

To address the operational challenges of in-pipe robots in large pipes of water distribution systems (WDS), in this research, a control algorithm is proposed for our previously designed robot [4]. Our size adaptable robot has an under-actuated modular design that can be used for both leak detection and quality monitoring. First, nonlinear dynamical governing equations of the robot are derived with the definition of two perpendicular planes, and two sets of states are defined for the robot for stabilization and mobilization. For stabilization, we calculated the auxiliary system matrices and designed a stabilizer controller based on the linear quadratic regulator (LQR) controller, and combined it with a proportional-integral-derivative (PID) based controller for mobilization. The controller scheme is validated with simulation in MATLAB in various operation conditions in three iterations. The simulation results show that the controller can stabilize the robot inside the pipe by converging the stabilizing states to zero and keeping them in zero with initial values between -25 degree and +25 degree and tracking velocities of 10cm/s, 30cm/s, and 50cm/s which makes the robot agile and dexterous for operation in pipelines.
Perimeter-defense Game between Aerial Defender and Ground Intruder
Venue: arXiv cs.RO 2020
Year: 2020

We study a variant of pursuit-evasion game in the context of perimeter defense. In this problem, the intruder aims to reach the base plane of a hemisphere without being captured by the defender, while the defender tries to capture the intruder. The perimeter-defense game was previously studied under the assumption that the defender moves on a circle. We extend the problem to the case where the defender moves on a hemisphere. To solve this problem, we analyze the strategies based on the breaching point at which the intruder tries to reach the target and predict the goal position, defined as optimal breaching point, that is achieved by the optimal strategies on both players. We provide the barrier that divides the state space into defender-winning and intruder-winning regions and prove that the optimal strategies for both players are to move towards the optimal breaching point. Simulation results are presented to demonstrate that the optimality of the game is given as a Nash equilibrium.
Unsupervised Real Time Prediction of Faults Using the Support Vector Machine
Venue: arXiv cs.LG 2020
Year: 2020

This paper aims at improving the classification accuracy of a Support Vector Machine (SVM) classifier with Sequential Minimal Optimization (SMO) training algorithm in order to properly classify failure and normal instances from oil and gas equipment data. Recent applications of failure analysis have made use of the SVM technique without implementing SMO training algorithm, while in our study we show that the proposed solution can perform much better when using the SMO training algorithm. Furthermore, we implement the ensemble approach, which is a hybrid rule based and neural network classifier to improve the performance of the SVM classifier (with SMO training algorithm). The optimization study is as a result of the underperformance of the classifier when dealing with imbalanced dataset. The selected best performing classifiers are combined together with SVM classifier (with SMO training algorithm) by using the stacking ensemble method which is to create an efficient ensemble predictive model that can handle the issue of imbalanced data. The classification performance of this predictive model is considerably better than the SVM with and without SMO training algorithm and many other conventional classifiers.
SGD Distributional Dynamics of Three Layer Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

With the rise of big data analytics, multi-layer neural networks have surfaced as one of the most powerful machine learning methods. However, their theoretical mathematical properties are still not fully understood. Training a neural network requires optimizing a non-convex objective function, typically done using stochastic gradient descent (SGD). In this paper, we seek to extend the mean field results of Mei et al. (2018) from two-layer neural networks with one hidden layer to three-layer neural networks with two hidden layers. We will show that the SGD dynamics is captured by a set of non-linear partial differential equations, and prove that the distributions of weights in the two hidden layers are independent. We will also detail exploratory work done based on simulation and real-world data.
Damaged Fingerprint Recognition by Convolutional Long Short-Term Memory Networks for Forensic Purposes
Venue: arXiv cs.CV 2020
Year: 2020

Fingerprint recognition is often a game-changing step in establishing evidence against criminals. However, we are increasingly finding that criminals deliberately alter their fingerprints in a variety of ways to make it difficult for technicians and automatic sensors to recognize their fingerprints, making it tedious for investigators to establish strong evidence against them in a forensic procedure. In this sense, deep learning comes out as a prime candidate to assist in the recognition of damaged fingerprints. In particular, convolution algorithms. In this paper, we focus on the recognition of damaged fingerprints by Convolutional Long Short-Term Memory networks. We present the architecture of our model and demonstrate its performance which exceeds 95% accuracy, 99% precision, and approaches 95% recall and 99% AUC.
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning algorithms are responsible for a technological revolution in a variety of tasks including image recognition or Go playing. Yet, why they work is not understood. Ultimately, they manage to classify data lying in high dimension -- a feat generically impossible due to the geometry of high dimensional space and the associated curse of dimensionality. Understanding what kind of structure, symmetry or invariance makes data such as images learnable is a fundamental challenge. Other puzzles include that (i) learning corresponds to minimizing a loss in high dimension, which is in general not convex and could well get stuck bad minima. (ii) Deep learning predicting power increases with the number of fitting parameters, even in a regime where data are perfectly fitted. In this manuscript, we review recent results elucidating (i,ii) and the perspective they offer on the (still unexplained) curse of dimensionality paradox. We base our theoretical discussion on the $(h,α)$ plane where $h$ is the network width and $α$ the scale of the output of the network at initialization, and provide new systematic measures of performance in that plane for MNIST and CIFAR 10. We argue that different learning regimes can be organized into a phase diagram. A line of critical points sharply delimits an under-parametrised phase from an over-parametrized one. In over-parametrized nets, learning can operate in two regimes separated by a smooth cross-over. At large initialization, it corresponds to a kernel method, whereas for small initializations features can be learnt, together with invariants in the data. We review the properties of these different phases, of the transition separating them and some open questions. Our treatment emphasizes analogies with physical systems, scaling arguments and the development of numerical observables to quantitatively test these results empirically.
Reachability relations of timed pushdown automata
Venue: arXiv cs.FL 2020
Year: 2020

Timed pushdown automata (TPDA) are an expressive formalism combining recursion with a rich logic of timing constraints. We prove that reachability relations of TPDA are expressible in linear arithmetic, a rich logic generalising Presburger arithmetic and rational arithmetic. The main technical ingredients are a novel quantifier elimination result for clock constraints (used to simplify the syntax of TPDA transitions), the use of clock difference relations to express reachability relations of the fractional clock values, and an application of Parikh's theorem to reconstruct the integral clock values.
Managed Information: A New Abstraction Mechanism for Handling Information in Software-as-a-Service
Venue: arXiv cs.SE 2020
Year: 2020

Management of information is an important aspect of every application. This includes, for example, protecting user data against breaches (like the one reported in the news about 50 million Facebook profiles being harvested for Cambridge Analytica), complying with data protection laws and regulations (like EU's new General Data Protection Regulation), coping with large databases, and retaining user data across software versions. Today, every application needs to cope with such concerns by itself and on its own. In this paper we introduce Managed Information (MI), an abstraction mechanism for managing extra-functional data related concerns, similar to how managed memory today abstracts away many memory related concerns. MI limits the access applications have to user data, which, in return, relieves them from responsibility over it. This is achieved by hosting them on a Managed Information Platform (MIP), and implementing their logic in a language that supports MI. As evidence for the feasibility of MI we describe the design and implementation of such a platform. For demonstration of MI, we describe a simple social network application built with it. The implementation is open source.