OpenRaiser/Intern-Atlas huggingface dataset:
Solving the Same-Different Task with Convolutional Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning demonstrated major abilities in solving many kinds of different real-world problems in computer vision literature. However, they are still strained by simple reasoning tasks that humans consider easy to solve. In this work, we probe current state-of-the-art convolutional neural networks on a difficult set of tasks known as the same-different problems. All the problems require the same prerequisite to be solved correctly: understanding if two random shapes inside the same image are the same or not. With the experiments carried out in this work, we demonstrate that residual connections, and more generally the skip connections, seem to have only a marginal impact on the learning of the proposed problems. In particular, we experiment with DenseNets, and we examine the contribution of residual and recurrent connections in already tested architectures, ResNet-18, and CorNet-S respectively. Our experiments show that older feed-forward networks, AlexNet and VGG, are almost unable to learn the proposed problems, except in some specific scenarios. We show that recently introduced architectures can converge even in the cases where the important parts of their architecture are removed. We finally carry out some zero-shot generalization tests, and we discover that in these scenarios residual and recurrent connections can have a stronger impact on the overall test accuracy. On four difficult problems from the SVRT dataset, we can reach state-of-the-art results with respect to the previous approaches, obtaining super-human performances on three of the four problems.
Intelligent Reflecting Surface Enhanced Multi-UAV NOMA Networks
Venue: arXiv cs.IT 2021
Year: 2021

Intelligent reflecting surface (IRS) enhanced multi-unmanned aerial vehicle (UAV) non-orthogonal multiple access (NOMA) networks are investigated. A new transmission framework is proposed, where multiple UAV-mounted base stations employ NOMA to serve multiple groups of ground users with the aid of an IRS. The three-dimensional (3D) placement and transmit power of UAVs, the reflection matrix of the IRS, and the NOMA decoding orders among users are jointly optimized for maximization of the sum rate of considered networks. To tackle the formulated mixed-integer non-convex optimization problem with coupled variables, a block coordinate descent (BCD)-based iterative algorithm is developed. Specifically, the original problem is decomposed into three subproblems, which are alternatingly solved by exploiting the penalty method and the successive convex approximation technique. The proposed BCD-based algorithm is demonstrated to be able to obtain a stationary point of the original problem with polynomial time complexity. Numerical results show that: 1) the proposed NOMA-IRS scheme for multi-UAV networks achieves a higher sum rate compared to the benchmark schemes, i.e., orthogonal multiple access (OMA)-IRS and NOMA without IRS; 2) the use of IRS is capable of providing performance gain for multi-UAV networks by both enhancing channel qualities of UAVs to their served users and mitigating the inter-UAV interference; and 3) optimizing the UAV placement can make the sum rate gain brought by NOMA more distinct due to the flexible decoding order design.
The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
Venue: arXiv cs.HC 2021
Year: 2021

We present an in-depth analysis of the impact of multi-word suggestion choices from a neural language model on user behaviour regarding input and text composition in email writing. Our study for the first time compares different numbers of parallel suggestions, and use by native and non-native English writers, to explore a trade-off of "efficiency vs ideation", emerging from recent literature. We built a text editor prototype with a neural language model (GPT-2), refined in a prestudy with 30 people. In an online study (N=156), people composed emails in four conditions (0/1/3/6 parallel suggestions). Our results reveal (1) benefits for ideation, and costs for efficiency, when suggesting multiple phrases; (2) that non-native speakers benefit more from more suggestions; and (3) further insights into behaviour patterns. We discuss implications for research, the design of interactive suggestion systems, and the vision of supporting writers with AI instead of replacing them.
Predicting Autism Spectrum Disorder Using Machine Learning Classifiers
Venue: arXiv cs.LG 2021
Year: 2021

Autism Spectrum Disorder (ASD) is on the rise and constantly growing. Earlier identify of ASD with the best outcome will allow someone to be safe and healthy by proper nursing. Humans can hardly estimate the present condition and stage of ASD by measuring primary symptoms. Therefore, it is being necessary to develop a method that will provide the best outcome and measurement of ASD. This paper aims to show several measurements that implemented in several classifiers. Among them, Support Vector Machine (SVM) provides the best result and under SVM, there are also some kernels to perform. Among them, the Gaussian Radial Kernel gives the best result. The proposed classifier achieves 95% accuracy using the publicly available standard ASD dataset.
User-Level Memory Scheduler for Optimizing Application Performance in NUMA-Based Multicore Systems
Venue: arXiv cs.DC 2021
Year: 2021

Multicore CPU architectures have been established as a structure for general-purpose systems for high-performance processing of applications. Recent multicore CPU has evolved as a system architecture based on non-uniform memory architecture. For the technique of using the kernel space that shifts the tasks to the ideal memory node, the characteristics of the applications of the user-space cannot be considered. Therefore, kernel level approaches cannot execute memory scheduling to recognize the importance of user applications. Moreover, users need to run applications after sufficiently understanding the multicore CPU based on non-uniform memory architecture to ensure the high performance of the user's applications. This paper presents a user-space memory scheduler that allocates the ideal memory node for tasks by monitoring the characteristics of non-uniform memory architecture. From our experiment, the proposed system improved the performance of the application by up to 25% compared to the existing system.
A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Graph representation of structured data can facilitate the extraction of stereoscopic features, and it has demonstrated excellent ability when working with deep learning systems, the so-called Graph Neural Networks (GNNs). Choosing a promising architecture for constructing GNNs can be transferred to a hyperparameter optimisation problem, a very challenging task due to the size of the underlying search space and high computational cost for evaluating candidate GNNs. To address this issue, this research presents a novel genetic algorithm with a hierarchical evaluation strategy (HESGA), which combines the full evaluation of GNNs with a fast evaluation approach. By using full evaluation, a GNN is represented by a set of hyperparameter values and trained on a specified dataset, and root mean square error (RMSE) will be used to measure the quality of the GNN represented by the set of hyperparameter values (for regression problems). While in the proposed fast evaluation process, the training will be interrupted at an early stage, the difference of RMSE values between the starting and interrupted epochs will be used as a fast score, which implies the potential of the GNN being considered. To coordinate both types of evaluations, the proposed hierarchical strategy uses the fast evaluation in a lower level for recommending candidates to a higher level, where the full evaluation will act as a final assessor to maintain a group of elite individuals. To validate the effectiveness of HESGA, we apply it to optimise two types of deep graph neural networks. The experimental results on three benchmark datasets demonstrate its advantages compared to Bayesian hyperparameter optimization.
Rikudo is NP-complete
Venue: arXiv cs.DM 2021
Year: 2021

Rikudo is a number-placement puzzle, where the player is asked to complete a Hamiltonian path on a hexagonal grid, given some clues (numbers already placed and edges of the path). We prove that the game is complete for NP, even if the puzzle has no hole. When all odd numbers are placed it is in P, whereas it is still NP-hard when all numbers of the form $3k+1$ are placed.
Robotic Knee Tracking Control to Mimic the Intact Human Knee Profile Based on Actor-critic Reinforcement Learning
Venue: arXiv cs.RO 2021
Year: 2021

We address a state-of-the-art reinforcement learning (RL) control approach to automatically configure robotic prosthesis impedance parameters to enable end-to-end, continuous locomotion intended for transfemoral amputee subjects. Specifically, our actor-critic based RL provides tracking control of a robotic knee prosthesis to mimic the intact knee profile. This is a significant advance from our previous RL based automatic tuning of prosthesis control parameters which have centered on regulation control with a designer prescribed robotic knee profile as the target. In addition to presenting the complete tracking control algorithm based on direct heuristic dynamic programming (dHDP), we provide an analytical framework for the tracking controller with constrained inputs. We show that our proposed tracking control possesses several important properties, such as weight convergence of the learning networks, Bellman (sub)optimality of the cost-to-go value function and control input, and practical stability of the human-robot system under input constraint. We further provide a systematic simulation of the proposed tracking control using a realistic human-robot system simulator, the OpenSim, to emulate how the dHDP enables level ground walking, walking on different terrains and at different paces. These results show that our proposed dHDP based tracking control is not only theoretically suitable, but also practically useful.
Vertical federated learning based on DFP and BFGS
Venue: arXiv cs.LG 2021
Year: 2021

As data privacy is gradually valued by people, federated learning(FL) has emerged because of its potential to protect data. FL uses homomorphic encryption and differential privacy encryption on the promise of ensuring data security to realize distributed machine learning by exchanging encrypted information between different data providers. However, there are still many problems in FL, such as the communication efficiency between the client and the server and the data is non-iid. In order to solve the two problems mentioned above, we propose a novel vertical federated learning framework based on the DFP and the BFGS(denoted as BDFL), then apply it to logistic regression. Finally, we perform experiments using real datasets to test efficiency of BDFL framework.
Complementary Waveforms for Range-Doppler Sidelobe Suppression Based on a Null Space Approach
Venue: arXiv cs.IT 2021
Year: 2021

While Doppler resilient complementary waveforms have previously been considered to suppress range sidelobes within a Doppler interval of interest in radar systems, their capability of Doppler resilience has not been fully utilized. In this paper, a new construction of Doppler resilient complementary waveforms based on a null space is proposed. With this new construction, one can flexibly include a specified Doppler interval of interest or even an overall Doppler interval into a term which results in range sidelobes. We can force this term to zero, which can be solved to obtain a null space. From the null space, the characteristic vector to control the transmission of basic Golay waveforms, and the coefficients of the receiver filter for Golay complementary waveform can be extracted. Besides, based on the derived null space, two challenging non-convex optimization problems are formulated and solved for maximizing the signal-to-noise ratio (SNR). Moreover, the coefficients of the receiver filter and the characteristic vector can be applied to fully polarimetric radar systems to achieve nearly perfect Doppler resilient performance, and hence fully suppress the inter-antenna interferences.
Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Despite the close connection between exploration and sample efficiency, most state of the art reinforcement learning algorithms include no considerations for exploration beyond maximizing the entropy of the policy. In this work we address this seeming missed opportunity. We observe that the most common formulation of directed exploration in deep RL, known as bonus-based exploration (BBE), suffers from bias and slow coverage in the few-sample regime. This causes BBE to be actively detrimental to policy learning in many control tasks. We show that by decoupling the task policy from the exploration policy, directed exploration can be highly effective for sample-efficient continuous control. Our method, Decoupled Exploration and Exploitation Policies (DEEP), can be combined with any off-policy RL algorithm without modification. When used in conjunction with soft actor-critic, DEEP incurs no performance penalty in densely-rewarding environments. On sparse environments, DEEP gives a several-fold improvement in data efficiency due to better exploration.
Unsupervised clustering of series using dynamic programming
Venue: arXiv cs.LG 2021
Year: 2021

We are interested in clustering parts of a given single multi-variate series in an unsupervised manner. We would like to segment and cluster the series such that the resulting blocks present in each cluster are coherent with respect to a known model (e.g. physics model). Data points are said to be coherent if they can be described using this model with the same parameters. We have designed an algorithm based on dynamic programming with constraints on the number of clusters, the number of transitions as well as the minimal size of a block such that the clusters are coherent with this process. We present an use-case: clustering of petrophysical series using the Waxman-Smits equation.
Formal Definitions of Memory Consistency Models
Venue: arXiv cs.DC 2021
Year: 2021

Shared Memory is a mechanism that allows several processes to communicate with each other by accessing -- writing or reading -- a set of variables that they have in common. A Consistency Model defines how each process observes the state of the Memory, according to the accesses performed by it and by the rest of the processes in the system. Therefore, it determines what value a read returns when a given process issues it. This implies that there must be an agreement among all, or among processes in different subsets, on the order in which all or a subset of the accesses happened. It is clear that a higher quantity of accesses or proceses taking part in the agreement makes it possibly harder or slower to be achieved. This is the main reason for which a number of Consistency Models for Shared Memory have been introduced. This paper is a handy summary of [2] and [3] where consistency models (Sequential, Causal, PRAM, Cache, Processors, Slow), including synchronized ones (Weak, Release, Entry), were formally defined. This provides a better understanding of those models and a way to reason and compare them through a concise notation. There are many papers on this subject in the literature such as [11] with which this work shares some concepts.
Automorphism Groups and Isometries for Cyclic Orbit Codes
Venue: arXiv cs.IT 2021
Year: 2021

We study orbit codes in the field extension ${\mathbb F}_{q^n}$. First we show that the automorphism group of a cyclic orbit code is contained in the normalizer of the Singer subgroup if the orbit is generated by a subspace that is not contained in a proper subfield of ${\mathbb F}_{q^n}$. We then generalize to orbits under the normalizer of the Singer subgroup. In that situation some exceptional cases arise and some open cases remain. Finally we characterize linear isometries between such codes.
ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings
Venue: arXiv cs.LG 2021
Year: 2021

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding's dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.
3D Underactuated Bipedal Walking via H-LIP based Gait Synthesis and Stepping Stabilization
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we holistically present a Hybrid-Linear Inverted Pendulum (H-LIP) based approach for synthesizing and stabilizing 3D foot-underactuated bipedal walking, with an emphasis on thorough hardware realization. The H-LIP is proposed to capture the essential components of the underactuated and actuated part of the robotic walking. The robot walking gait is then directly synthesized based on the H-LIP. We comprehensively characterize the periodic orbits of the H-LIP and provably derive the stepping stabilization via its step-to-step (S2S) dynamics, which is then utilized to approximate the S2S dynamics of the horizontal state of the center of mass (COM) of the robotic walking. The approximation facilities a H-LIP based stepping controller to provide desired step sizes to stabilize the robotic walking. By realizing the desired step sizes, the robot achieves dynamic and stable walking. The approach is fully evaluated in both simulation and experiment on the 3D underactuated bipedal robot Cassie, which demonstrates dynamic walking behaviors with both high versatility and robustness.
FlowReg: Fast Deformable Unsupervised Medical Image Registration using Optical Flow
Venue: arXiv cs.CV 2021
Year: 2021

We propose FlowReg, a deep learning-based framework for unsupervised image registration for neuroimaging applications. The system is composed of two architectures that are trained sequentially: FlowReg-A which affinely corrects for gross differences between moving and fixed volumes in 3D followed by FlowReg-O which performs pixel-wise deformations on a slice-by-slice basis for fine tuning in 2D. The affine network regresses the 3D affine matrix based on a correlation loss function that enforces global similarity. The deformable network operates on 2D image slices based on the optical flow network FlowNet-Simple but with three loss components. The photometric loss minimizes pixel intensity differences differences, the smoothness loss encourages similar magnitudes between neighbouring vectors, and a correlation loss that is used to maintain the intensity similarity between fixed and moving image slices. The proposed method is compared to four open source registration techniques ANTs, Demons, SE, and Voxelmorph. In total, 4643 FLAIR MR imaging volumes are used from dementia and vascular disease cohorts, acquired from over 60 international centres with varying acquisition parameters. A battery of quantitative novel registration validation metrics are proposed that focus on the structural integrity of tissues, spatial alignment, and intensity similarity. Experimental results show FlowReg (FlowReg-A+O) performs better than iterative-based registration algorithms for intensity and spatial alignment metrics with a Pixelwise Agreement of 0.65, correlation coefficient of 0.80, and Mutual Information of 0.29. Among the deep learning frameworks, FlowReg-A or FlowReg-A+O provided the highest performance over all but one of the metrics. Results show that FlowReg is able to obtain high intensity and spatial similarity while maintaining the shape and structure of anatomy and pathology.
Relational Type Theory (All Proofs)
Venue: arXiv cs.LO 2021
Year: 2021

This paper introduces Relational Type Theory (RelTT), a new approach to type theory with extensionality principles, based on a relational semantics for types. The type constructs of the theory are those of System F plus relational composition, converse, and promotion of application of a term to a relation. A concise realizability semantics is presented for these types. The paper shows how a number of constructions of traditional interest in type theory are possible in RelTT, including eta-laws for basic types, inductive types with their induction principles, and positive-recursive types. A crucial role is played by a lemma called Identity Inclusion, which refines the Identity Extension property familiar from the semantics of parametric polymorphism. The paper concludes with a type system for RelTT, paving the way for implementation.
Corrective Information Does Not Necessarily Curb Social Disruption
Venue: arXiv cs.SI 2021
Year: 2021

The spread of misinformation can cause social confusion. The authenticity of information on a social networking service (SNS) is unknown, and false information can be easily spread. Consequently, many studies have been conducted on methods to control the spread of misinformation on social networking sites. However, few studies have examined the impact of the spread of misinformation and its corrections on society. This study models the impact of the reduction of misinformation and the diffusion of corrective information on social disruption, and it identifies the features of this impact. In this study, we analyzed misinformation regarding the shortage of toilet paper during the 2020 COVID-19 epidemic, its corrections, and the excessive purchasing caused by this information. First, we analyze the amount of misinformation and corrective information spread on SNS, and we create a regression model to estimate the real-world impact of misinformation and its correction. This model is used to analyze the change in real-world impact corresponding to the change in the diffusion of misinformation and corrective information. Our analysis shows that the corrective information was spread to a much greater extent than the misinformation. In addition, our model reveals that the corrective information was what caused the excessive purchasing behavior. As a result of our further analysis, we found that the amount of diffusion of corrective information required to minimize the impact on the real world depends on the amount of the diffusion of misinformation.