OpenRaiser/Intern-Atlas huggingface dataset:
A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Real-world data usually have high dimensionality and it is important to mitigate the curse of dimensionality. High-dimensional data are usually in a coherent structure and make the data in relatively small true degrees of freedom. There are global and local dimensionality reduction methods to alleviate the problem. Most of existing methods for local dimensionality reduction obtain an embedding with the eigenvalue or singular value decomposition, where the computational complexities are very high for a large amount of data. Here we propose a novel local nonlinear approach named Vec2vec for general purpose dimensionality reduction, which generalizes recent advancements in embedding representation learning of words to dimensionality reduction of matrices. It obtains the nonlinear embedding using a neural network with only one hidden layer to reduce the computational complexity. To train the neural network, we build the neighborhood similarity graph of a matrix and define the context of data points by exploiting the random walk properties. Experiments demenstrate that Vec2vec is more efficient than several state-of-the-art local dimensionality reduction methods in a large number of high-dimensional data. Extensive experiments of data classification and clustering on eight real datasets show that Vec2vec is better than several classical dimensionality reduction methods in the statistical hypothesis test, and it is competitive with recently developed state-of-the-art UMAP.
Designing and Implementing e-justice Systems: An Information Systems Approach to Regional Trial Court Case Docket Management in Northern Mindanao, Philippines
Venue: arXiv cs.CY 2021
Year: 2021

Computer-based information systems for case management are still at an early stage of adoption in many trial courts in the Philippines. In most cases, information system implemented is the case docket using the official record book on which cases are written and inventory of cases and reports are generated. This is a standalone system that often face data processing, data security and case management challenges. However, there are examples of Information systems in overcoming these pitfalls and producing innovative solutions that surpass data management practices in in many trial courts in the country. One such case is the Regional Trial Court Branch 23 of Cagayan de Oro City in Northern Mindanao, Philippines. A project named Web-based Case Docket Information System (WCDIS) has been designed and developed for the court branch. This system uses a framework known as System Development Life Cycle (SDLC) which is a guide for the design and development. This paper also discusses the key system functionalities and the implementation methodology, including both the benefits and shortcomings of this approach, with the goal of applying lessons learned for future installations. Foremost among the successes of this project is its ability to increase efficiency and reliability in completing court transactions.
Metapaths guided Neighbors aggregated Network for?Heterogeneous Graph Reasoning
Venue: arXiv cs.AI 2021
Year: 2021

Most real-world datasets are inherently heterogeneous graphs, which involve a diversity of node and relation types. Heterogeneous graph embedding is to learn the structure and semantic information from the graph, and then embed it into the low-dimensional node representation. Existing methods usually capture the composite relation of a heterogeneous graph by defining metapath, which represent a semantic of the graph. However, these methods either ignore node attributes, or discard the local and global information of the graph, or only consider one metapath. To address these limitations, we propose a Metapaths-guided Neighbors-aggregated Heterogeneous Graph Neural Network(MHN) to improve performance. Specially, MHN employs node base embedding to encapsulate node attributes, BFS and DFS neighbors aggregation within a metapath to capture local and global information, and metapaths aggregation to combine different semantics of the heterogeneous graph. We conduct extensive experiments for the proposed MHN on three real-world heterogeneous graph datasets, including node classification, link prediction and online A/B test on Alibaba mobile application. Results demonstrate that MHN performs better than other state-of-the-art baselines.
Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Study
Venue: arXiv cs.CL 2021
Year: 2021

Unsupervised pretraining is an integral part of many natural language processing systems, and transfer learning with language models has achieved remarkable results in many downstream tasks. In the clinical application of medical code assignment, diagnosis and procedure codes are inferred from lengthy clinical notes such as hospital discharge summaries. However, it is not clear if pretrained models are useful for medical code prediction without further architecture engineering. This paper conducts a comprehensive quantitative analysis of various contextualized language models' performance, pretrained in different domains, for medical code assignment from clinical notes. We propose a hierarchical fine-tuning architecture to capture interactions between distant words and adopt label-wise attention to exploit label information. Contrary to current trends, we demonstrate that a carefully trained classical CNN outperforms attention-based models on a MIMIC-III subset with frequent codes. Our empirical findings suggest directions for improving the medical code assignment application.
Integrated Age Estimation Mechanism
Venue: arXiv cs.CV 2021
Year: 2021

Machine-learning-based age estimation has received lots of attention. Traditional age estimation mechanism focuses estimation age error, but ignores that there is a deviation between the estimated age and real age due to disease. Pathological age estimation mechanism the author proposed before introduces age deviation to solve the above problem and improves classification capability of the estimated age significantly. However,it does not consider the age estimation error of the normal control (NC) group and results in a larger error between the estimated age and real age of NC group. Therefore, an integrated age estimation mechanism based on Decision-Level fusion of error and deviation orientation model is proposed to solve the problem.Firstly, the traditional age estimation and pathological age estimation mechanisms are weighted together.Secondly, their optimal weights are obtained by minimizing mean absolute error (MAE) between the estimated age and real age of normal people. In the experimental section, several representative age-related datasets are used for verification of the proposed method. The results show that the proposed age estimation mechanism achieves a good tradeoff effect of age estimation. It not only improves the classification ability of the estimated age, but also reduces the age estimation error of the NC group. In general, the proposed age estimation mechanism is effective. Additionally, the mechanism is a framework mechanism that can be used to construct different specific age estimation algorithms, contributing to relevant research.
SDN Controller Load Balancing Based on Reinforcement Learning
Venue: arXiv cs.NI 2021
Year: 2021

Aiming at the local overload of multi-controller deployment in software-defined networks, a load balancing mechanism of SDN controller based on reinforcement learning is designed. The initial paired migrate-out domain and migrate-in domain are obtained by calculating the load ratio deviation between the controllers, a preliminary migration triplet, contains migration domain mentioned above and a group of switches which are subordinated to the migrate-out domain, makes the migration efficiency reach the local optimum. Under the constraint of the best efficiency of migration in the whole and without migration conflict, selecting multiple sets of triples based on reinforcement learning, as the final migration of this round to attain the global optimal controller load balancing with minimum cost. The experimental results illustrate that the mechanism can make full use of the controllers' resources, quickly balance the load between controllers, reduce unnecessary migration overhead and get a faster response rate of the packet-in request.
Symbolic Reinforcement Learning for Safe RAN Control
Venue: arXiv cs.AI 2021
Year: 2021

In this paper, we demonstrate a Symbolic Reinforcement Learning (SRL) architecture for safe control in Radio Access Network (RAN) applications. In our automated tool, a user can select a high-level safety specifications expressed in Linear Temporal Logic (LTL) to shield an RL agent running in a given cellular network with aim of optimizing network performance, as measured through certain Key Performance Indicators (KPIs). In the proposed architecture, network safety shielding is ensured through model-checking techniques over combined discrete system models (automata) that are abstracted through reinforcement learning. We demonstrate the user interface (UI) helping the user set intent specifications to the architecture and inspect the difference in allowed and blocked actions.
Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verification
Venue: arXiv cs.LG 2021
Year: 2021

Bound propagation based incomplete neural network verifiers such as CROWN are very efficient and can significantly accelerate branch-and-bound (BaB) based complete verification of neural networks. However, bound propagation cannot fully handle the neuron split constraints introduced by BaB commonly handled by expensive linear programming (LP) solvers, leading to loose bounds and hurting verification efficiency. In this work, we develop $β$-CROWN, a new bound propagation based method that can fully encode neuron splits via optimizable parameters $β$ constructed from either primal or dual space. When jointly optimized in intermediate layers, $β$-CROWN generally produces better bounds than typical LP verifiers with neuron split constraints, while being as efficient and parallelizable as CROWN on GPUs. Applied to complete robustness verification benchmarks, $β$-CROWN with BaB is up to three orders of magnitude faster than LP-based BaB methods, and is notably faster than all existing approaches while producing lower timeout rates. By terminating BaB early, our method can also be used for efficient incomplete verification. We consistently achieve higher verified accuracy in many settings compared to powerful incomplete verifiers, including those based on convex barrier breaking techniques. Compared to the typically tightest but very costly semidefinite programming (SDP) based incomplete verifiers, we obtain higher verified accuracy with three orders of magnitudes less verification time. Our algorithm empowered the $α,\!β$-CROWN (alpha-beta-CROWN) verifier, the winning tool in VNN-COMP 2021. Our code is available at http://PaperCode.cc/BetaCROWN
Real-Time Surface Fitting to RGBD Sensor Data
Venue: arXiv cs.CV 2021
Year: 2021

This article describes novel approaches to quickly estimate planar surfaces from RGBD sensor data. The approach manipulates the standard algebraic fitting equations into a form that allows many of the needed regression variables to be computed directly from the camera calibration information. As such, much of the computational burden required by a standard algebraic surface fit can be pre-computed. This provides a significant time and resource savings, especially when many surface fits are being performed which is often the case when RGBD point-cloud data is being analyzed for normal estimation, curvature estimation, polygonization or 3D segmentation applications. Using an integral image implementation, the proposed approaches show a significant increase in performance compared to the standard algebraic fitting approaches.
Compiler-Guided Throughput Scheduling for Many-core Machines
Venue: arXiv cs.DC 2021
Year: 2021

Typical schedulers in multi-tenancy environments make use of reactive, feedback-oriented mechanisms based on performance counters to avoid resource contention but suffer from detection lag and loss of performance. In this paper, we address these limitations by exploring the utility of predictive analysis through dynamic forecasting of applications' resource-heavy regions during its execution. Our compiler framework classifies loops in programs and leverages traditional compiler analysis along with learning mechanisms to quantify their behaviour. Based on the predictability of their execution time, it then inserts different types of beacons at their entry/exit points. The information produced by beacons in multiple processes is aggregated and analyzed by the proactive scheduler to respond to the anticipated workload requirements. For throughput environments, we develop a framework that demonstrates high-quality predictions and improvements in throughput over CFS by 76.78% on an average and up to 3.2x on Amazon Graviton2 Machine on consolidated workloads across 45 benchmarks.
Research Software Sustainability and Citation
Venue: arXiv cs.SE 2021
Year: 2021

Software citation contributes to achieving software sustainability in two ways: It provides an impact metric to incentivize stakeholders to make software sustainable. It also provides references to software used in research, which can be reused and adapted to become sustainable. While software citation faces a host of technical and social challenges, community initiatives have defined the principles of software citation and are working on implementing solutions.
Have I been here before? Learning to Close the Loop with LiDAR Data in Graph-Based SLAM
Venue: arXiv cs.RO 2021
Year: 2021

This work presents an extension of graph-based SLAM methods to exploit the potential of 3D laser scans for loop detection. Every high-dimensional point cloud is replaced by a compact global descriptor, whereby a trained detector decides whether a loop exists. Searching for loops is performed locally in a variable space to consider the odometry drift. Since closing a wrong loop has fatal consequences, an extensive verification is performed before acceptance. The proposed algorithm is implemented as an extension of the widely used state-of-the-art library RTAB-Map, and several experiments show the improvement: During SLAM with a mobile service robot in changing indoor and outdoor campus environments, our approach improves RTAB-Map regarding total number of closed loops. Especially in the presence of significant environmental changes, which typically lead to failure, localization becomes possible by our extension. Experiments with a car in traffic (KITTI benchmark) show the general applicability of our approach. These results are comparable to the state-of-the-art LiDAR method LOAM. The developed ROS package is freely available.
Intraclass clustering: an implicit learning ability that regularizes DNNs
Venue: arXiv cs.LG 2021
Year: 2021

Several works have shown that the regularization mechanisms underlying deep neural networks' generalization performances are still poorly understood. In this paper, we hypothesize that deep neural networks are regularized through their ability to extract meaningful clusters among the samples of a class. This constitutes an implicit form of regularization, as no explicit training mechanisms or supervision target such behaviour. To support our hypothesis, we design four different measures of intraclass clustering, based on the neuron- and layer-level representations of the training data. We then show that these measures constitute accurate predictors of generalization performance across variations of a large set of hyperparameters (learning rate, batch size, optimizer, weight decay, dropout rate, data augmentation, network depth and width).
ArCode: A Tool for Supporting Comprehension andImplementation of Architectural Concerns
Venue: arXiv cs.SE 2021
Year: 2021

Integrated development environments (IDE) play an important role in supporting developers during program comprehension and completion. Many of these supportive features focus on low-level programming and debugging activities. Unfortunately, there is less support in understanding and implementing architectural concerns in the form of patterns, tactics and/or other concerns. In this paper we present ArCode, a tool designed as a plugin for a popular IDE, IntelliJ IDEA. ArCode is able to learn correct ways of using frameworks' API to implement architectural concerns such as Authentication and Authorization. Analyzing the programmer's code, this tool is able to find deviations from correct implementation and provide fix recommendations alongside with graphical demonstrations to better communicate the recommendations with the developers. We showcase how programmers can benefit from ArCode by providing an API misuse detection and API recommendation scenario for a famous Java framework, Java Authentication and Authorization (JAAS) security framework.
Wandering and getting lost: the architecture of an app activating local communities on dementia issues
Venue: arXiv cs.SE 2021
Year: 2021

We describe the architecture of Sammen Om Demens (SOD), an application for portable devices aiming at helping persons with dementia when wandering and getting lost through the involvement of caregivers, family members, and ordinary citizens who volunteer. To enable the real-time detection of a person with dementia that has lost orientation, we transfer location data at high frequency from a frontend on the smartphone of a person with dementia to a backend system. The backend system must be able to cope with the high throughput data and carry out possibly heavy computations for the detection of anomalous behavior via artificial intelligence techniques. This sets certain performance and architectural requirements on the design of the backend. In the paper, we discuss our design and implementation choices for the backend of SOD that involve microservices and serverless services to achieve efficiency and scalability. We give evidence of the achieved goals by deploying the SOD backend on a public cloud and measuring the performance on simulated load tests.
Improved Coherence Index-Based Bound in Compressive Sensing
Venue: arXiv cs.IT 2021
Year: 2021

Within the Compressive Sensing (CS) paradigm, sparse signals can be reconstructed based on a reduced set of measurements. Reliability of the solution is determined by the uniqueness condition. With its mathematically tractable and feasible calculation, coherence index is one of very few CS metrics with a considerable practical importance. In this paper, we propose an improvement of the coherence based uniqueness relation for the matching pursuit algorithms. Starting from a simple and intuitive derivation of the standard uniqueness condition based on the coherence index, we derive a less conservative coherence index-based lower bound for signal sparsity. The results are generalized to the uniqueness condition of the $l_0$-norm minimization for a signal represented in two orthonormal bases.
A conditional, a fuzzy and a probabilistic interpretation of self-organising maps
Venue: arXiv cs.AI 2021
Year: 2021

In this paper we establish a link between fuzzy and preferential semantics for description logics and Self-Organising Maps, which have been proposed as possible candidates to explain the psychological mechanisms underlying category generalisation. In particular, we show that the input/output behavior of a Self-Organising Map after training can be described by a fuzzy description logic interpretation as well as by a preferential interpretation, based on a concept-wise multipreference semantics, which takes into account preferences with respect to different concepts and has been recently proposed for ranked and for weighted defeasible description logics. Properties of the network can be proven by model checking on the fuzzy or on the preferential interpretation. Starting from the fuzzy interpretation, we also provide a probabilistic account for this neural network model.
Scaffold Embeddings: Learning the Structure Spanned by Chemical Fragments, Scaffolds and Compounds
Venue: arXiv cs.LG 2021
Year: 2021

Molecules have seemed like a natural fit to deep learning's tendency to handle a complex structure through representation learning, given enough data. However, this often continuous representation is not natural for understanding chemical space as a domain and is particular to samples and their differences. We focus on exploring a natural structure for representing chemical space as a structured domain: embedding drug-like chemical space into an enumerable hypergraph based on scaffold classes linked through an inclusion operator. This paper shows how molecules form classes of scaffolds, how scaffolds relate to each in a hypergraph, and how this structure of scaffolds is natural for drug discovery workflows such as predicting properties and optimizing molecular structures. We compare the assumptions and utility of various embeddings of molecules, such as their respective induced distance metrics, their extendibility to represent chemical space as a structured domain, and the consequences of utilizing the structure for learning tasks.
For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functions
Venue: arXiv cs.LG 2021
Year: 2021

It is well established that training deep neural networks gives useful representations that capture essential features of the inputs. However, these representations are poorly understood in theory and practice. In the context of supervised learning an important question is whether these representations capture features informative for classification, while filtering out non-informative noisy ones. We explore a formalization of this question by considering a generative process where each class is associated with a high-dimensional manifold and different classes define different manifolds. Under this model, each input is produced using two latent vectors: (i) a "manifold identifier" $γ$ and; (ii)~a "transformation parameter" $θ$ that shifts examples along the surface of a manifold. E.g., $γ$ might represent a canonical image of a dog, and $θ$ might stand for variations in pose, background or lighting. We provide theoretical and empirical evidence that neural representations can be viewed as LSH-like functions that map each input to an embedding that is a function of solely the informative $γ$ and invariant to $θ$, effectively recovering the manifold identifier $γ$. An important consequence of this behavior is one-shot learning to unseen classes.
Modern Dimension Reduction
Venue: arXiv cs.LG 2021
Year: 2021

Data are not only ubiquitous in society, but are increasingly complex both in size and dimensionality. Dimension reduction offers researchers and scholars the ability to make such complex, high dimensional data spaces simpler and more manageable. This Element offers readers a suite of modern unsupervised dimension reduction techniques along with hundreds of lines of R code, to efficiently represent the original high dimensional data space in a simplified, lower dimensional subspace. Launching from the earliest dimension reduction technique principal components analysis and using real social science data, I introduce and walk readers through application of the following techniques: locally linear embedding, t-distributed stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection, self-organizing maps, and deep autoencoders. The result is a well-stocked toolbox of unsupervised algorithms for tackling the complexities of high dimensional data so common in modern society. All code is publicly accessible on Github.