OpenRaiser/Intern-Atlas huggingface dataset:
Using DevOps Toolchains in Agile Model-Driven Engineering
Venue: arXiv cs.SE 2021
Year: 2021

For Model-Driven Engineering (MDE) to become Agile, it is has to be usable with and for DevOps as the technical basis of Agility. We describe our experiences in implementing and applying the BB8 architecture that provides a means to reuse Eclipse-based MDE toolkits within an integrating workflow system. We describe the use of the architecture with in the Gitlab DevOps system and discuss future research fields that emerge as the result of our implementation.
Using mixup as regularization and tuning hyper-parameters for ResNets
Venue: arXiv cs.CV 2021
Year: 2021

While novel computer vision architectures are gaining traction, the impact of model architectures is often related to changes or exploring in training methods. Identity mapping-based architectures ResNets and DenseNets have promised path-breaking results in the image classification task and are go-to methods for even now if the data given is fairly limited. Considering the ease of training with limited resources this work revisits the ResNets and improves the ResNet50 \cite{resnets} by using mixup data-augmentation as regularization and tuning the hyper-parameters.
On the Capacity of MISO Optical Intensity Channels With Per-Antenna Intensity Constraints
Venue: arXiv cs.IT 2021
Year: 2021

This paper investigates the capacity of general multiple-input single-output (MISO) optical intensity channels (OICs) under per-antenna peak- and average-intensity constraints. We first consider the MISO equal-cost constrained OIC (EC-OIC), where, apart from the peak-intensity constraint, average intensities of inputs are equal to arbitrarily preassigned constants. The second model of our interest is the MISO bounded-cost constrained OIC (BC-OIC), where, as compared with the EC-OIC, average intensities of inputs are no larger than arbitrarily preassigned constants. By leveraging tools from quantile functions, stop-loss transform and convex ordering of nonnegative random variables, we prove two decomposition theorems for bounded and nonnegative random variables, based on which we equivalently transform both the EC-OIC and the BC-OIC into respective single-input single-output channels under a peak-intensity and several stop-loss mean constraints. Capacity lower and upper bounds for both channels are established, based on which the asymptotic capacity at high and low signal-to-noise-ratio are determined.
A Contextual Latent Space Model: Subsequence Modulation in Melodic Sequence
Venue: arXiv cs.LG 2021
Year: 2021

Some generative models for sequences such as music and text allow us to edit only subsequences, given surrounding context sequences, which plays an important part in steering generation interactively. However, editing subsequences mainly involves randomly resampling subsequences from a possible generation space. We propose a contextual latent space model (CLSM) in order for users to be able to explore subsequence generation with a sense of direction in the generation space, e.g., interpolation, as well as exploring variations -- semantically similar possible subsequences. A context-informed prior and decoder constitute the generative model of CLSM, and a context position-informed encoder is the inference model. In experiments, we use a monophonic symbolic music dataset, demonstrating that our contextual latent space is smoother in interpolation than baselines, and the quality of generated samples is superior to baseline models. The generation examples are available online.
Link Analysis meets Ontologies: Are Embeddings the Answer?
Venue: arXiv cs.LG 2021
Year: 2021

The increasing amounts of semantic resources offer valuable storage of human knowledge; however, the probability of wrong entries increases with the increased size. The development of approaches that identify potentially spurious parts of a given knowledge base is thus becoming an increasingly important area of interest. In this work, we present a systematic evaluation of whether structure-only link analysis methods can already offer a scalable means to detecting possible anomalies, as well as potentially interesting novel relation candidates. Evaluating thirteen methods on eight different semantic resources, including Gene Ontology, Food Ontology, Marine Ontology and similar, we demonstrated that structure-only link analysis could offer scalable anomaly detection for a subset of the data sets. Further, we demonstrated that by considering symbolic node embedding, explanations of the predictions (links) could be obtained, making this branch of methods potentially more valuable than the black-box only ones. To our knowledge, this is currently one of the most extensive systematic studies of the applicability of different types of link analysis methods across semantic resources from different domains.
StrokeNet: Stroke Assisted and Hierarchical Graph Reasoning Networks
Venue: arXiv cs.CV 2021
Year: 2021

Scene text detection is still a challenging task, as there may be extremely small or low-resolution strokes, and close or arbitrary-shaped texts. In this paper, StrokeNet is proposed to effectively detect the texts by capturing the fine-grained strokes, and infer structural relations between the hierarchical representation in the graph. Different from existing approaches that represent the text area by a series of points or rectangular boxes, we directly localize strokes of each text instance through Stroke Assisted Prediction Network (SAPN). Besides, Hierarchical Relation Graph Network (HRGN) is adopted to perform relational reasoning and predict the likelihood of linkages, effectively splitting the close text instances and grouping node classification results into arbitrary-shaped text region. We introduce a novel dataset with stroke-level annotations, namely SynthStroke, for offline pre-training of our model. Experiments on wide-ranging benchmarks verify the State-of-the-Art performance of our method. Our dataset and code will be available.
A new dynamical model for solving rotation averaging problem
Venue: arXiv cs.CV 2021
Year: 2021

The paper analyzes the rotation averaging problem as a minimization problem for a potential function of the corresponding gradient system. This dynamical system is one generalization of the famous Kuramoto model on special orthogonal group SO(3), which is known as the non-Abelian Kuramoto model. We have proposed a novel method for finding weighted and unweighted rotation average. In order to verify the correctness of our algorithms, we have compared the simulation results with geometric and projected average using real and random data sets. In particular, we have discovered that our method gives approximately the same results as geometric average.
Automaton of molecular perceptions in biochemical reactions
Venue: arXiv cs.CE 2021
Year: 2021

Local interactions among biomolecules, and the role played by their environment, have gained increasing attention in modelling biochemical reactions. By defining the automaton of molecular perceptions, we explore an agent-based representation of the behaviour of biomolecules in living cells. Our approach considers the capability of a molecule to perceive its surroundings a key property of bimolecular interactions, which we investigate from a theoretical perspective. Graph-based reaction systems are then leveraged to abstract enzyme regulation as a result of the influence exerted by the environment on a catalysed reaction. By combining these methods, we aim at overcoming some limitations of current kinetic models, which do not take into account local molecular interactions and the way they are affected by the reaction environment.
Variance Reduction in Deep Learning: More Momentum is All You Need
Venue: arXiv cs.LG 2021
Year: 2021

Variance reduction (VR) techniques have contributed significantly to accelerating learning with massive datasets in the smooth and strongly convex setting (Schmidt et al., 2017; Johnson & Zhang, 2013; Roux et al., 2012). However, such techniques have not yet met the same success in the realm of large-scale deep learning due to various factors such as the use of data augmentation or regularization methods like dropout (Defazio & Bottou, 2019). This challenge has recently motivated the design of novel variance reduction techniques tailored explicitly for deep learning (Arnold et al., 2019; Ma & Yarats, 2018). This work is an additional step in this direction. In particular, we exploit the ubiquitous clustering structure of rich datasets used in deep learning to design a family of scalable variance reduced optimization procedures by combining existing optimizers (e.g., SGD+Momentum, Quasi Hyperbolic Momentum, Implicit Gradient Transport) with a multi-momentum strategy (Yuan et al., 2019). Our proposal leads to faster convergence than vanilla methods on standard benchmark datasets (e.g., CIFAR and ImageNet). It is robust to label noise and amenable to distributed optimization. We provide a parallel implementation in JAX.
Real-time intelligent big data processing: technology, platform, and applications
Venue: arXiv cs.DC 2021
Year: 2021

Human beings keep exploring the physical space using information means. Only recently, with the rapid development of information technologies and the increasing accumulation of data, human beings can learn more about the unknown world with data-driven methods. Given data timeliness, there is a growing awareness of the importance of real-time data. There are two categories of technologies accounting for data processing: batching big data and streaming processing, which have not been integrated well. Thus, we propose an innovative incremental processing technology named after Stream Cube to process both big data and stream data. Also, we implement a real-time intelligent data processing system, which is based on real-time acquisition, real-time processing, real-time analysis, and real-time decision-making. The real-time intelligent data processing technology system is equipped with a batching big data platform, data analysis tools, and machine learning models. Based on our applications and analysis, the real-time intelligent data processing system is a crucial solution to the problems of the national society and economy.
A Variant RSA Acceleration with Parallelization
Venue: arXiv cs.DC 2021
Year: 2021

The standard RSA relies on multiple big-number modular exponentiation operations and longer key-length is required for better protection. This imposes a hefty time penalty for encryption and decryption. In this study, we analyzed and developed an improved parallel algorithm (PMKRSA) based on the idea of splitting the plaintext into multiple chunks and encrypt the chunks using multiple key-pairs. The algorithm in our new scheme is so natural for parallelized implementation that we also investigated its parallelization in a GPU environment. In the following, the structure of our new scheme is outlined, and its correctness is proved mathematically. Then, with the algorithm implemented and optimized on both CPU and CPU+GPU platforms, we showed that our algorithm shortens the computational time considerably, and it has a security advantage over the standard RSA as it is invulnerable to the common attacks. Finally, we also proved the feasibility of using our algorithm to encrypt large files through simulation. The results show that over the set of file size: 1 MB, 10 MB, 25 MB, 50 MB, 100 MB, the average encryption and decryption time of the CPU version is 0.2476s and 9.4476s, and for the CPU+GPU version, it is 0.0009s and 0.0618s, respectively.
Reviewing continual learning from the perspective of human-level intelligence
Venue: arXiv cs.LG 2021
Year: 2021

Humans' continual learning (CL) ability is closely related to Stability Versus Plasticity Dilemma that describes how humans achieve ongoing learning capacity and preservation for learned information. The notion of CL has always been present in artificial intelligence (AI) since its births. This paper proposes a comprehensive review of CL. Different from previous reviews that mainly focus on the catastrophic forgetting phenomenon in CL, this paper surveys CL from a more macroscopic perspective based on the Stability Versus Plasticity mechanism. Analogous to biological counterpart, "smart" AI agents are supposed to i) remember previously learned information (information retrospection); ii) infer on new information continuously (information prospection:); iii) transfer useful information (information transfer), to achieve high-level CL. According to the taxonomy, evaluation metrics, algorithms, applications as well as some open issues are then introduced. Our main contributions concern i) rechecking CL from the level of artificial general intelligence; ii) providing a detailed and extensive overview on CL topics; iii) presenting some novel ideas on the potential development of CL.
Object Recognition by a Minimally Pre-Trained System in the Process of Environment Exploration
Venue: arXiv cs.AI 2021
Year: 2021

We update the method of describing and assessing the process of the study of an abstract environment by a system, proposed earlier. We do not model any biological cognition mechanisms and consider the system as an agent equipped with an information processor (or a group of such agents), which makes a move in the environment, consumes information supplied by the environment, and gives out the next move (hence, the process is considered as a game). The system moves in an unknown environment and should recognize new objects located in it. In this case, the system should build comprehensive images of visible things and memorize them if necessary (and it should also choose the current goal set). The main problems here are object recognition, and the informational reward rating in the game. Thus, the main novelty of the paper is a new method of evaluating the amount of visual information about the object as the reward. In such a system, we suggest using a minimally pre-trained neural network to be responsible for the recognition: at first, we train the network only for Biederman geons (geometrical primitives). The geons are generated programmatically and we demonstrate that such a trained network recognizes geons in real objects quite well. We also offer to generate, procedurally, new objects from geon schemes (geon combinations in images) obtained from the environment and to store them in a database. In this case, we do not obtain new information about an object (i.e., our reward is maximal, thus the game and the object cognition process stop) when we stop getting new schemes of this kind. These schemes are generated from geons connected with the object. In the case of a possibly known item, the informational reward is maximal when we have no more detection uncertainty for any of the objects.
Learning Symbolic Rules for Reasoning in Quasi-Natural Language
Venue: arXiv cs.LG 2021
Year: 2021

Symbolic reasoning, rule-based symbol manipulation, is a hallmark of human intelligence. However, rule-based systems have had limited success competing with learning-based systems outside formalized domains such as automated theorem proving. We hypothesize that this is due to the manual construction of rules in past attempts. In this work, we ask how we can build a rule-based system that can reason with natural language input but without the manual construction of rules. We propose MetaQNL, a "Quasi-Natural" language that can express both formal logic and natural language sentences, and MetaInduce, a learning algorithm that induces MetaQNL rules from training data consisting of questions and answers, with or without intermediate reasoning steps. Our approach achieves state-of-the-art accuracy on multiple reasoning benchmarks; it learns compact models with much less data and produces not only answers but also checkable proofs. Further, experiments on a real-world morphological analysis benchmark show that it is possible for our method to handle noise and ambiguity. Code will be released at https://github.com/princeton-vl/MetaQNL.
Network Effects Research: A Systematic Review of Theoretical Mechanisms and Measures
Venue: arXiv cs.SI 2021
Year: 2021

This article contributes to the network effectiveness literature by identifying the theoretical mechanisms and network measures scholars in public administration and policy use to draw inferences between network structures and network effects. We conducted a systematic review of empirical network effects research in 40 journals of public administration and policy from 1998 to 2019. We reviewed and coded 89 articles and described the main social theories used in the network effectiveness literature and the associated mechanisms that translate network structures to network effects. We also explain how scholars operationalize those theoretical mechanisms through network measures. Overall, our findings reflect that there is a limited use of social theories for the explanation of network effects and in some cases, there is an inconsistent use of network measures associated with theories. Moreover, we identify the main challenges related to network effects, including the difficulty of isolating specific mechanisms related to a particular social theory, the use of network structures both as a mechanism and as a measure, and the lack of data to examine network dynamics and coevolution.
Formalization of Bond Graph using Higher-order-logic Theorem Proving
Venue: arXiv cs.LO 2021
Year: 2021

Bond graph is a unified graphical approach for describing the dynamics of complex engineering and physical systems and is widely adopted in a variety of domains, such as, electrical, mechanical, medical, thermal and fluid mechanics. Traditionally, these dynamics are analyzed using paper-and-pencil proof methods and computer-based techniques. However, both of these techniques suffer from their inherent limitations, such as human-error proneness, approximations of results and enormous computational requirements. Thus, these techniques cannot be trusted for performing the bond graph based dynamical analysis of systems from the safety-critical domains like robotics and medicine. Formal methods, in particular, higher-order-logic theorem proving, can overcome the shortcomings of these traditional methods and provide an accurate analysis of these systems. It has been widely used for analyzing the dynamics of engineering and physical systems. In this paper, we propose to use higher-order-logic theorem proving for performing the bond graph based analysis of the physical systems. In particular, we provide formalization of bond graph, which mainly includes functions that allow conversion of a bond graph to its corresponding mathematical model (state-space model) and the verification of its various properties, such as, stability. To illustrate the practical effectiveness of our proposed approach, we present the formal stability analysis of a prosthetic mechatronic hand using HOL Light theorem prover. Moreover, to help non-experts in HOL, we encode our formally verified stability theorems in MATLAB to perform the stability analysis of an anthropomorphic prosthetic mechatronic hand.
Efficient and Optimal Algorithms for Contextual Dueling Bandits under Realizability
Venue: arXiv cs.LG 2021
Year: 2021

We study the $K$-armed contextual dueling bandit problem, a sequential decision making setting in which the learner uses contextual information to make two decisions, but only observes \emph{preference-based feedback} suggesting that one decision was better than the other. We focus on the regret minimization problem under realizability, where the feedback is generated by a pairwise preference matrix that is well-specified by a given function class $\mathcal F$. We provide a new algorithm that achieves the optimal regret rate for a new notion of best response regret, which is a strictly stronger performance measure than those considered in prior works. The algorithm is also computationally efficient, running in polynomial time assuming access to an online oracle for square loss regression over $\mathcal F$. This resolves an open problem of Dudík et al. [2015] on oracle efficient, regret-optimal algorithms for contextual dueling bandits.
Explicit solution of divide-and-conquer dividing by a half recurrences with polynomial independent term
Venue: arXiv cs.DM 2021
Year: 2021

Divide-and-conquer dividing by a half recurrences, of the form $x_n =a\cdot x_{\left\lceil{n}/{2}\right\rceil}+a\cdot x_{\left\lfloor{n}/{2}\right\rfloor}+p(n)$, $n\geq 2$, appear in many areas of applied mathematics, from the analysis of algorithms to the optimization of phylogenetic balance indices. The Master Theorems that solve these equations do not provide the solution's explicit expression, only its big-$Θ$ order of growth. In this paper we give an explicit expression (in terms of the binary decomposition of $n$) for the solution $x_n$ of a recurrence of this form, with given initial condition $x_1$, when the independent term $p(n)$ is a polynomial in $\lceil{n}/{2}\rceil$ and $\lfloor{n}/{2}\rfloor$.
Compressed Sensing Channel Estimation for OTFS Modulation in Non-Integer Delay-Doppler Domain
Venue: arXiv cs.IT 2021
Year: 2021

This paper introduces a Compressed Sensing (CS) estimation scheme for Orthogonal Time Frequency Space (OTFS) channels with sparse multipath. The OTFS waveform represents signals in a two dimensional Delay-Doppler (DD) orthonormal basis. The proposed model does not require the assumption that the delays are integer multiples of the sampling period. The analysis shows that non-integer delay and Doppler shifts in the channel cannot be accurately modelled by integer approximations. An Orthogonal Matching Pursuit with Binary-division Refinement (OMPBR) estimation algorithm is proposed. The proposed estimator finds the best channel approximation over a continuous DD dictionary without integer approximations. This results in a significant reduction of the estimation normalized mean squared error with reasonable computational complexity.
NAM: Normalization-based Attention Module
Venue: arXiv cs.CV 2021
Year: 2021

Recognizing less salient features is the key for model compression. However, it has not been investigated in the revolutionary attention mechanisms. In this work, we propose a novel normalization-based attention module (NAM), which suppresses less salient weights. It applies a weight sparsity penalty to the attention modules, thus, making them more computational efficient while retaining similar performance. A comparison with three other attention mechanisms on both Resnet and Mobilenet indicates that our method results in higher accuracy. Code for this paper can be publicly accessed at https://github.com/Christian-lyc/NAM.
Introduction to Presentation Attack Detection in Iris Biometrics and Recent Advances
Venue: arXiv cs.CV 2021
Year: 2021

Iris recognition technology has attracted an increasing interest in the last decades in which we have witnessed a migration from research laboratories to real world applications. The deployment of this technology raises questions about the main vulnerabilities and security threats related to these systems. Among these threats presentation attacks stand out as some of the most relevant and studied. Presentation attacks can be defined as presentation of human characteristics or artifacts directly to the capture device of a biometric system trying to interfere its normal operation. In the case of the iris, these attacks include the use of real irises as well as artifacts with different level of sophistication such as photographs or videos. This chapter introduces iris Presentation Attack Detection (PAD) methods that have been developed to reduce the risk posed by presentation attacks. First, we summarise the most popular types of attacks including the main challenges to address. Secondly, we present a taxonomy of Presentation Attack Detection methods as a brief introduction to this very active research area. Finally, we discuss the integration of these methods into Iris Recognition Systems according to the most important scenarios of practical application.