OpenRaiser/Intern-Atlas huggingface dataset:
KNH: Multi-View Modeling with K-Nearest Hyperplanes Graph for Misinformation Detection
Venue: arXiv cs.LG 2021
Year: 2021

Graphs are one of the most efficacious structures for representing datapoints and their relations, and they have been largely exploited for different applications. Previously, the higher-order relations between the nodes have been modeled by a generalization of graphs known as hypergraphs. In hypergraphs, the edges are defined by a set of nodes i.e., hyperedges to demonstrate the higher order relationships between the data. However, there is no explicit higher-order generalization for nodes themselves. In this work, we introduce a novel generalization of graphs i.e., K-Nearest Hyperplanes graph (KNH) where the nodes are defined by higher order Euclidean subspaces for multi-view modeling of the nodes. In fact, in KNH, nodes are hyperplanes or more precisely m-flats instead of datapoints. We experimentally evaluate the KNH graph on two multi-aspect datasets for misinformation detection. The experimental results suggest that multi-view modeling of articles using KNH graph outperforms the classic KNN graph in terms of classification performance.
Investigating and Recommending Co-Changed Entities for JavaScript Programs
Venue: arXiv cs.SE 2021
Year: 2021

JavaScript (JS) is one of the most popular programming languages due to its flexibility and versatility, but maintaining JS code is tedious and error-prone. In our research, we conducted an empirical study to characterize the relationship between co-changed software entities (e.g., functions and variables), and built a machine learning (ML)-based approach to recommend additional entity to edit given developers' code changes. Specifically, we first crawled 14,747 commits in 10 open-source projects; for each commit, we created one or more change dependency graphs (CDGs) to model the referencer-referencee relationship between co-changed entities. Next, we extracted the common subgraphs between CDGs to locate recurring co-change patterns between entities. Finally, based on those patterns, we extracted code features from co-changed entities and trained an ML model that recommends entities-to-change given a program commit. According to our empirical investigation, (1) three recurring patterns commonly exist in all projects; (2) 80%--90% of co-changed function pairs either invoke the same function(s), access the same variable(s), or contain similar statement(s); (3) our ML-based approach CoRec recommended entity changes with high accuracy (73%--78%). CoRec complements prior work because it suggests changes based on program syntax, textual similarity, as well as software history; it achieved higher accuracy than two existing tools in our evaluation.
A Hidden Challenge of Link Prediction: Which Pairs to Check?
Venue: arXiv cs.SI 2021
Year: 2021

The traditional setup of link prediction in networks assumes that a test set of node pairs, which is usually balanced, is available over which to predict the presence of links. However, in practice, there is no test set: the ground-truth is not known, so the number of possible pairs to predict over is quadratic in the number of nodes in the graph. Moreover, because graphs are sparse, most of these possible pairs will not be links. Thus, link prediction methods, which often rely on proximity-preserving embeddings or heuristic notions of node similarity, face a vast search space, with many pairs that are in close proximity, but that should not be linked. To mitigate this issue, we introduce LinkWaldo, a framework for choosing from this quadratic, massively-skewed search space of node pairs, a concise set of candidate pairs that, in addition to being in close proximity, also structurally resemble the observed edges. This allows it to ignore some high-proximity but low-resemblance pairs, and also identify high-resemblance, lower-proximity pairs. Our framework is built on a model that theoretically combines Stochastic Block Models (SBMs) with node proximity models. The block structure of the SBM maps out where in the search space new links are expected to fall, and the proximity identifies the most plausible links within these blocks, using locality sensitive hashing to avoid expensive exhaustive search. LinkWaldo can use any node representation learning or heuristic definition of proximity, and can generate candidate pairs for any link prediction method, allowing the representation power of current and future methods to be realized for link prediction in practice. We evaluate LinkWaldo on 13 networks across multiple domains, and show that on average it returns candidate sets containing 7-33% more missing and future links than both embedding-based and heuristic baselines' sets.
PSA-Net: Deep Learning based Physician Style-Aware Segmentation Network for Post-Operative Prostate Cancer Clinical Target Volume
Venue: arXiv cs.CV 2021
Year: 2021

Automatic segmentation of medical images with DL algorithms has proven to be highly successful. With most of these algorithms, inter-observer variation is an acknowledged problem, leading to sub-optimal results. This problem is even more significant in post-operative clinical target volume (post-op CTV) segmentation due to the absence of macroscopic visual tumor in the image. This study, using post-op CTV segmentation as the test bed, tries to determine if physician styles are consistent and learnable, if there is an impact of physician styles on treatment outcome and toxicity; and how to explicitly deal with physician styles in DL algorithms to facilitate its clinical acceptance. A classifier is trained to identify which physician has contoured the CTV from just the contour and corresponding CT scan, to determine if physician styles are consistent and learnable. Next, we evaluate if adapting automatic segmentation to physician styles would be clinically feasible based on a lack of difference between outcomes. For modeling different physician styles of CTV segmentation, a concept called physician style-aware (PSA) segmentation is proposed which is an encoder-multidecoder network trained with perceptual loss. With the proposed physician style-aware network (PSA-Net), Dice similarity coefficient (DSC) accuracy increases on an average of 3.4% for all physicians from a general model that is not style adapted. We show that stylistic contouring variations also exist between institutions that follow the same segmentation guidelines and show the effectiveness of the proposed method in adapting to new institutional styles. We observed an accuracy improvement of 5% in terms of DSC when adapting to the style of a separate institution.
Roman Domination of the Comet, Double Comet, and Comb Graphs
Venue: arXiv cs.DM 2021
Year: 2021

One of the well-known measurements of vulnerability in graph theory is domination. There are many kinds of dominating and relative types of sets in graphs. However, we are going to focus on Roman domination, which is a type of domination that came up with an article by Ian Stewart. The Roman domination numbers of the comet, double comet, and comb graphs are given in this paper.
Training Larger Networks for Deep Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

The success of deep learning in the computer vision and natural language processing communities can be attributed to training of very deep neural networks with millions or billions of parameters which can then be trained with massive amounts of data. However, similar trend has largely eluded training of deep reinforcement learning (RL) algorithms where larger networks do not lead to performance improvement. Previous work has shown that this is mostly due to instability during training of deep RL agents when using larger networks. In this paper, we make an attempt to understand and address training of larger networks for deep RL. We first show that naively increasing network capacity does not improve performance. Then, we propose a novel method that consists of 1) wider networks with DenseNet connection, 2) decoupling representation learning from training of RL, 3) a distributed training method to mitigate overfitting problems. Using this three-fold technique, we show that we can train very large networks that result in significant performance gains. We present several ablation studies to demonstrate the efficacy of the proposed method and some intuitive understanding of the reasons for performance gain. We show that our proposed method outperforms other baseline algorithms on several challenging locomotion tasks.
Have Attention Heads in BERT Learned Constituency Grammar?
Venue: arXiv cs.CL 2021
Year: 2021

With the success of pre-trained language models in recent years, more and more researchers focus on opening the "black box" of these models. Following this interest, we carry out a qualitative and quantitative analysis of constituency grammar in attention heads of BERT and RoBERTa. We employ the syntactic distance method to extract implicit constituency grammar from the attention weights of each head. Our results show that there exist heads that can induce some grammar types much better than baselines, suggesting that some heads act as a proxy for constituency grammar. We also analyze how attention heads' constituency grammar inducing (CGI) ability changes after fine-tuning with two kinds of tasks, including sentence meaning similarity (SMS) tasks and natural language inference (NLI) tasks. Our results suggest that SMS tasks decrease the average CGI ability of upper layers, while NLI tasks increase it. Lastly, we investigate the connections between CGI ability and natural language understanding ability on QQP and MNLI tasks.
Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach
Venue: arXiv cs.NI 2021
Year: 2021

We consider a many-to-one wireless architecture for federated learning at the network edge, where multiple edge devices collaboratively train a model using local data. The unreliable nature of wireless connectivity, together with constraints in computing resources at edge devices, dictates that the local updates at edge devices should be carefully crafted and compressed to match the wireless communication resources available and should work in concert with the receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent algorithms for such settings. Specifically, for the wireless edge employing over-the-air computing, a common subset of k-coordinates of the gradient updates across edge devices are selected by the receiver in each iteration, and then transmitted simultaneously over k sub-carriers, each experiencing time-varying channel conditions. We characterize the impact of communication error and compression, in terms of the resulting gradient bias and mean squared error, on the convergence of the proposed algorithms. We then study learning-driven communication error minimization via joint optimization of power allocation and learning rates. Our findings reveal that optimal power allocation across different sub-carriers should take into account both the gradient values and channel conditions, thus generalizing the widely used water-filling policy. We also develop sub-optimal distributed solutions amenable to implementation.
Temporal-Amount Snapshot MultiGraph for Ethereum Transaction Tracking
Venue: arXiv cs.CR 2021
Year: 2021

With the wide application of blockchain in the financial field, the rise of various types of cybercrimes has brought great challenges to the security of blockchain. In order to better understand this emerging market and explore more efficient countermeasures for effective supervision, it is imperative to track transactions on blockchain-based systems. Due to the openness of Ethereum, we can easily access the publicly available transaction records, model them as a complex network, and further study the problem of transaction tracking via link prediction, which provides a deeper understanding of Ethereum transactions from a network perspective. Specifically, we introduce an embedding based link prediction framework that is composed of temporal-amount snapshot multigraph (TASMG) and present temporal-amount walk (TAW). By taking the realistic rules and features of transaction networks into consideration, we propose TASMG to model Ethereum transaction records as a temporal-amount network and then present TAW to effectively embed accounts via their transaction records, which integrates temporal and amount information of the proposed network. Experimental results demonstrate the superiority of the proposed framework in learning more informative representations and could be an effective method for transaction tracking.
Transferring Domain Knowledge with an Adviser in Continuous Tasks
Venue: arXiv cs.AI 2021
Year: 2021

Recent advances in Reinforcement Learning (RL) have surpassed human-level performance in many simulated environments. However, existing reinforcement learning techniques are incapable of explicitly incorporating already known domain-specific knowledge into the learning process. Therefore, the agents have to explore and learn the domain knowledge independently through a trial and error approach, which consumes both time and resources to make valid responses. Hence, we adapt the Deep Deterministic Policy Gradient (DDPG) algorithm to incorporate an adviser, which allows integrating domain knowledge in the form of pre-learned policies or pre-defined relationships to enhance the agent's learning process. Our experiments on OpenAi Gym benchmark tasks show that integrating domain knowledge through advisers expedites the learning and improves the policy towards better optima.
Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet
Venue: arXiv cs.CL 2021
Year: 2021

Recent years have seen a proliferation of attention mechanisms and the rise of Transformers in Natural Language Generation (NLG). Previously, state-of-the-art NLG architectures such as RNN and LSTM ran into vanishing gradient problems; as sentences grew larger, distance between positions remained linear, and sequential computation hindered parallelization since sentences were processed word by word. Transformers usher in a new era. In this paper, we explore three major Transformer-based models, namely GPT, BERT, and XLNet, that carry significant implications for the field. NLG is a burgeoning area that is now bolstered with rapid developments in attention mechanisms. From poetry generation to summarization, text generation derives benefit as Transformer-based language models achieve groundbreaking results.
Deterministic CONGEST Algorithm for MDS on Bounded Arboricity Graphs
Venue: arXiv cs.DS 2021
Year: 2021

We provide a deterministic CONGEST algorithm to constant factor approximate the minimum dominating set on graphs of bounded arboricity in $O(\log n)$ rounds. This improves over the well-known randomized algorithm of Lenzen and Wattenhofer[DISC2010] by making it a deterministic algorithm.
Restore from Restored: Single-image Inpainting
Venue: arXiv cs.CV 2021
Year: 2021

Recent image inpainting methods show promising results due to the power of deep learning, which can explore external information available from a large training dataset. However, many state-of-the-art inpainting networks are still limited in exploiting internal information available in the given input image at test time. To mitigate this problem, we present a novel and efficient self-supervised fine-tuning algorithm that can adapt the parameters of fully pre-trained inpainting networks without using ground-truth target images. We update the parameters of the pre-trained state-of-the-art inpainting networks by utilizing existing self-similar patches within the given input image without changing network architecture and improve the inpainting quality by a large margin. Qualitative and quantitative experimental results demonstrate the superiority of the proposed algorithm, and we achieve state-of-the-art inpainting results on publicly available numerous benchmark datasets.
Boosting Deep Transfer Learning for COVID-19 Classification
Venue: arXiv cs.CV 2021
Year: 2021

COVID-19 classification using chest Computed Tomography (CT) has been found pragmatically useful by several studies. Due to the lack of annotated samples, these studies recommend transfer learning and explore the choices of pre-trained models and data augmentation. However, it is still unknown if there are better strategies than vanilla transfer learning for more accurate COVID-19 classification with limited CT data. This paper provides an affirmative answer, devising a novel `model' augmentation technique that allows a considerable performance boost to transfer learning for the task. Our method systematically reduces the distributional shift between the source and target domains and considers augmenting deep learning with complementary representation learning techniques. We establish the efficacy of our method with publicly available datasets and models, along with identifying contrasting observations in the previous studies.
Meta-Path-Free Representation Learning on Heterogeneous Networks
Venue: arXiv cs.SI 2021
Year: 2021

Real-world networks and knowledge graphs are usually heterogeneous networks. Representation learning on heterogeneous networks is not only a popular but a pragmatic research field. The main challenge comes from the heterogeneity -- the diverse types of nodes and edges. Besides, for a given node in a HIN, the significance of a neighborhood node depends not only on the structural distance but semantics. How to effectively capture both structural and semantic relations is another challenge. The current state-of-the-art methods are based on the algorithm of meta-path and therefore have a serious disadvantage -- the performance depends on the arbitrary choosing of meta-path(s). However, the selection of meta-path(s) is experience-based and time-consuming. In this work, we propose a novel meta-path-free representation learning on heterogeneous networks, namely Heterogeneous graph Convolutional Networks (HCN). The proposed method fuses the heterogeneity and develops a $k$-strata algorithm ($k$ is an integer) to capture the $k$-hop structural and semantic information in heterogeneous networks. To the best of our knowledge, this is the first attempt to break out of the confinement of meta-paths for representation learning on heterogeneous networks. We carry out extensive experiments on three real-world heterogeneous networks. The experimental results demonstrate that the proposed method significantly outperforms the current state-of-the-art methods in a variety of analytic tasks.
Towards an accountable Internet of Things: A call for reviewability
Venue: arXiv cs.CY 2021
Year: 2021

As the IoT becomes increasingly ubiquitous, concerns are being raised about how IoT systems are being built and deployed. Connected devices will generate vast quantities of data, which drive algorithmic systems and result in real-world consequences. Things will go wrong, and when they do, how do we identify what happened, why they happened, and who is responsible? Given the complexity of such systems, where do we even begin? This chapter outlines aspects of accountability as they relate to IoT, in the context of the increasingly interconnected and data-driven nature of such systems. Specifically, we argue the urgent need for mechanisms - legal, technical, and organisational - that facilitate the review of IoT systems. Such mechanisms work to support accountability, by enabling the relevant stakeholders to better understand, assess, interrogate and challenge the connected environments that increasingly pervade our world.
Practical application of the multi-model approach in the study of complex systems
Venue: arXiv cs.OH 2021
Year: 2021

Different kinds of models are used to study various natural and technical phenomena. Usually, the researcher is limited to using a certain kind of model approach, not using others (or even not realizing the existence of other model approaches). The authors believe that a complete study of a certain phenomenon should cover several model approaches. The paper describes several model approaches which we used in the study of the random early detection algorithm for active queue management. Both the model approaches themselves and their implementation and the results obtained are described.
Online matching in lossless expanders
Venue: arXiv cs.DS 2021
Year: 2021

Bauwens and Zimand [BZ 2019] have shown that lossless expanders have an interesting online matching property. The result appears in an implicit form in [BZ 2019]. We present an explicit version of this property which is directly amenable to typical applications, prove it in a self-contained manner that clarifies the role of some parameters, and give two applications. A $(K, ε)$ lossless expander is a bipartite graph such that any subset $S$ of size at most $K$ of nodes on the left side of the bipartition has at least $(1-ε) D |S|$ neighbors, where $D$ is the left degree.The main result is that any such graph, after a slight modification, admits $(1-O(ε)D, 1)$ online matching up to size $K$. This means that for any sequence $S=(x_1, \ldots, x_K)$ of nodes on the left side of the bipartition, one can assign in an online manner to each node $x_i$ in $S$ a set $A_i$ consisting of $(1-O(ε))$ fraction of its neighbors so that the sets $A_1, \ldots, A_K$ are pairwise disjoint. "Online manner" refers to the fact that, for every $i$, the set of nodes assigned to $x_i$ only depends on the nodes assigned to $x_1, \ldots, x_{i-1}$. The first application concerns storage schemes for representing a set $S$, so that a membership query "Is $x \in S$?" can be answered probabilistically by reading a single bit. All the previous one-probe storage schemes were for a static set $S$. We show that a lossless expander can be used to construct a one-probe storage scheme for dynamic sets, i.e., sets in which elements can be inserted and deleted without affecting the representation of other elements. The second application is about non-blocking networks.
A Federated Data-Driven Evolutionary Algorithm
Venue: arXiv cs.NE 2021
Year: 2021

Data-driven evolutionary optimization has witnessed great success in solving complex real-world optimization problems. However, existing data-driven optimization algorithms require that all data are centrally stored, which is not always practical and may be vulnerable to privacy leakage and security threats if the data must be collected from different devices. To address the above issue, this paper proposes a federated data-driven evolutionary optimization framework that is able to perform data driven optimization when the data is distributed on multiple devices. On the basis of federated learning, a sorted model aggregation method is developed for aggregating local surrogates based on radial-basis-function networks. In addition, a federated surrogate management strategy is suggested by designing an acquisition function that takes into account the information of both the global and local surrogate models. Empirical studies on a set of widely used benchmark functions in the presence of various data distributions demonstrate the effectiveness of the proposed framework.
Design Iterations for Passive Aerial Manipulator
Venue: arXiv cs.RO 2021
Year: 2021

Grabbing a manoeuvring target using drones is a challenging problem. This paper presents the design, development, and prototyping of a novel aerial manipulator for target interception. It is a single Degree of Freedom (DoF) manipulator with passive basket-type end-effector. The proposed design is energy efficient, light weight and suitable for aerial grabbing applications. The detailed design of the proposed manipulation mechanism and a novel in-flight extending propeller guard, is reported in this paper.