OpenRaiser/Intern-Atlas huggingface dataset:
Unit Disk Visibility Graphs
Venue: arXiv cs.CG 2021
Year: 2021

We study unit disk visibility graphs, where the visibility relation between a pair of geometric entities is defined by not only obstacles, but also the distance between them. That is, two entities are not mutually visible if they are too far apart, regardless of having an obstacle between them. This particular graph class models real world scenarios more accurately compared to the conventional visibility graphs. We first define and classify the unit disk visibility graphs, and then show that the 3-coloring problem is NP-complete when unit disk visibility model is used for a set of line segments (which applies to a set of points) and for a polygon with holes.
Self-Labeling of Fully Mediating Representations by Graph Alignment
Venue: arXiv cs.LG 2021
Year: 2021

To be able to predict a molecular graph structure ($W$) given a 2D image of a chemical compound ($U$) is a challenging problem in machine learning. We are interested to learn $f: U \rightarrow W$ where we have a fully mediating representation $V$ such that $f$ factors into $U \rightarrow V \rightarrow W$. However, observing V requires detailed and expensive labels. We propose graph aligning approach that generates rich or detailed labels given normal labels $W$. In this paper we investigate the scenario of domain adaptation from the source domain where we have access to the expensive labels $V$ to the target domain where only normal labels W are available. Focusing on the problem of predicting chemical compound graphs from 2D images the fully mediating layer is represented using the planar embedding of the chemical graph structure we are predicting. The use of a fully mediating layer implies some assumptions on the mechanism of the underlying process. However if the assumptions are correct it should allow the machine learning model to be more interpretable, generalize better and be more data efficient at training time. The empirical results show that, using only 4000 data points, we obtain up to 4x improvement of performance after domain adaptation to target domain compared to pretrained model only on the source domain. After domain adaptation, the model is even able to detect atom types that were never seen in the original source domain. Finally, on the Maybridge data set the proposed self-labeling approach reached higher performance than the current state of the art.
Beyond Low-Pass Filters: Adaptive Feature Propagation on Graphs
Venue: arXiv cs.LG 2021
Year: 2021

Graph neural networks (GNNs) have been extensively studied for prediction tasks on graphs. As pointed out by recent studies, most GNNs assume local homophily, i.e., strong similarities in local neighborhoods. This assumption however limits the generalizability power of GNNs. To address this limitation, we propose a flexible GNN model, which is capable of handling any graphs without being restricted by their underlying homophily. At its core, this model adopts a node attention mechanism based on multiple learnable spectral filters; therefore, the aggregation scheme is learned adaptively for each graph in the spectral domain. We evaluated the proposed model on node classification tasks over eight benchmark datasets. The proposed model is shown to generalize well to both homophilic and heterophilic graphs. Further, it outperforms all state-of-the-art baselines on heterophilic graphs and performs comparably with them on homophilic graphs.
The Complete Affine Automorphism Group of Polar Codes
Venue: arXiv cs.IT 2021
Year: 2021

Recently, a permutation-based successive cancellation (PSC) decoding framework for polar codes attaches much attention. It decodes several permuted codewords with independent successive cancellation (SC) decoders. Its latency thus can be reduced to that of SC decoding. However, the PSC framework is ineffective for permutations falling into the lower-triangular affine (LTA) automorphism group, as they are invariant under SC decoding. As such, a larger block lower-triangular affine (BLTA) group that contains SC-variant permutations was discovered for decreasing polar codes. But it was unknown whether BLTA equals the complete automorphism group. In this paper, we prove that BLTA equals the complete automorphisms of decreasing polar codes that can be formulated as affine trasformations.
Understanding Internet of Things Malware by Analyzing Endpoints in their Static Artifacts
Venue: arXiv cs.CR 2021
Year: 2021

The lack of security measures among the Internet of Things (IoT) devices and their persistent online connection gives adversaries a prime opportunity to target them or even abuse them as intermediary targets in larger attacks such as distributed denial-of-service (DDoS) campaigns. In this paper, we analyze IoT malware and focus on the endpoints reachable on the public Internet, that play an essential part in the IoT malware ecosystem. Namely, we analyze endpoints acting as dropzones and their targets to gain insights into the underlying dynamics in this ecosystem, such as the affinity between the dropzones and their target IP addresses, and the different patterns among endpoints. Towards this goal, we reverse-engineer 2,423 IoT malware samples and extract strings from them to obtain IP addresses. We further gather information about these endpoints from public Internet-wide scanners, such as Shodan and Censys. For the masked IP addresses, we examine the Classless Inter-Domain Routing (CIDR) networks accumulating to more than 100 million (78.2% of total active public IPv4 addresses) endpoints. Our investigation from four different perspectives provides profound insights into the role of endpoints in IoT malware attacks, which deepens our understanding of IoT malware ecosystems and can assist future defenses.
Socio-Technical Grounded Theory for Software Engineering
Venue: arXiv cs.SE 2021
Year: 2021

Grounded Theory (GT), a sociological research method designed to study social phenomena, is increasingly being used to investigate the human and social aspects of software engineering (SE). However, being written by and for sociologists, GT is often challenging for a majority of SE researchers to understand and apply. Additionally, SE researchers attempting ad hoc adaptations of traditional GT guidelines for modern socio-technical (ST) contexts often struggle in the absence of clear and relevant guidelines to do so, resulting in poor quality studies. To overcome these research community challenges and leverage modern research opportunities, this paper presents Socio-Technical Grounded Theory (STGT) designed to ease application and achieve quality outcomes. It defines what exactly is meant by an ST research context and presents the STGT guidelines that expand GT's philosophical foundations, provide increased clarity and flexibility in its methodological steps and procedures, define possible scope and contexts of application, encourage frequent reporting of a variety of interim, preliminary, and mature outcomes, and introduce nuanced evaluation guidelines for different outcomes. It is hoped that the SE research community and related ST disciplines such as computer science, data science, artificial intelligence, information systems, human computer/robot/AI interaction, human-centered emerging technologies (and increasingly other disciplines being transformed by rapid digitalisation and AI-based augmentation), will benefit from applying STGT to conduct quality research studies and systematically produce rich findings and mature theories with confidence.
FRITL: A Hybrid Method for Causal Discovery in the Presence of Latent Confounders
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of estimating a particular type of linear non-Gaussian model. Without resorting to the overcomplete Independent Component Analysis (ICA), we show that under some mild assumptions, the model is uniquely identified by a hybrid method. Our method leverages the advantages of constraint-based methods and independent noise-based methods to handle both confounded and unconfounded situations. The first step of our method uses the FCI procedure, which allows confounders and is able to produce asymptotically correct results. The results, unfortunately, usually determine very few unconfounded direct causal relations, because whenever it is possible to have a confounder, it will indicate it. The second step of our procedure finds the unconfounded causal edges between observed variables among only those adjacent pairs informed by the FCI results. By making use of the so-called Triad condition, the third step is able to find confounders and their causal relations with other variables. Afterward, we apply ICA on a notably smaller set of graphs to identify remaining causal relationships if needed. Extensive experiments on simulated data and real-world data validate the correctness and effectiveness of the proposed method.
MedSelect: Selective Labeling for Medical Image Classification Combining Meta-Learning with Deep Reinforcement Learning
Venue: arXiv cs.CV 2021
Year: 2021

We propose a selective learning method using meta-learning and deep reinforcement learning for medical image interpretation in the setting of limited labeling resources. Our method, MedSelect, consists of a trainable deep learning selector that uses image embeddings obtained from contrastive pretraining for determining which images to label, and a non-parametric selector that uses cosine similarity to classify unseen images. We demonstrate that MedSelect learns an effective selection strategy outperforming baseline selection strategies across seen and unseen medical conditions for chest X-ray interpretation. We also perform an analysis of the selections performed by MedSelect comparing the distribution of latent embeddings and clinical features, and find significant differences compared to the strongest performing baseline. We believe that our method may be broadly applicable across medical imaging settings where labels are expensive to acquire.
An Embedding-based Joint Sentiment-Topic Model for Short Texts
Venue: arXiv cs.CL 2021
Year: 2021

Short text is a popular avenue of sharing feedback, opinions and reviews on social media, e-commerce platforms, etc. Many companies need to extract meaningful information (which may include thematic content as well as semantic polarity) out of such short texts to understand users' behaviour. However, obtaining high quality sentiment-associated and human interpretable themes still remains a challenge for short texts. In this paper we develop ELJST, an embedding enhanced generative joint sentiment-topic model that can discover more coherent and diverse topics from short texts. It uses Markov Random Field Regularizer that can be seen as a generalisation of skip-gram based models. Further, it can leverage higher-order semantic information appearing in word embedding, such as self-attention weights in graphical models. Our results show an average improvement of 10% in topic coherence and 5% in topic diversification over baselines. Finally, ELJST helps understand users' behaviour at more granular levels which can be explained. All these can bring significant values to the service and healthcare industries often dealing with customers.
Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond
Venue: arXiv cs.LG 2021
Year: 2021

Many techniques have been proposed for image reconstruction in medical imaging that aim to recover high-quality images especially from limited or corrupted measurements. Model-based reconstruction methods have been particularly popular (e.g., in magnetic resonance imaging and tomographic modalities) and exploit models of the imaging system's physics together with statistical models of measurements, noise and often relatively simple object priors or regularizers. For example, sparsity or low-rankness based regularizers have been widely used for image reconstruction from limited data such as in compressed sensing. Learning-based approaches for image reconstruction have garnered much attention in recent years and have shown promise across biomedical imaging applications. These methods include synthesis dictionary learning, sparsifying transform learning, and different forms of deep learning involving complex neural networks. We briefly discuss classical model-based reconstruction methods and then review reconstruction methods at the intersection of model-based and learning-based paradigms in detail. This review includes many recent methods based on unsupervised learning, and supervised learning, as well as a framework to combine multiple types of learned models together.
Deep Unsupervised Learning for Generalized Assignment Problems: A Case-Study of User-Association in Wireless Networks
Venue: arXiv cs.LG 2021
Year: 2021

There exists many resource allocation problems in the field of wireless communications which can be formulated as the generalized assignment problems (GAP). GAP is a generic form of linear sum assignment problem (LSAP) and is more challenging to solve owing to the presence of both equality and inequality constraints. We propose a novel deep unsupervised learning (DUL) approach to solve GAP in a time-efficient manner. More specifically, we propose a new approach that facilitates to train a deep neural network (DNN) using a customized loss function. This customized loss function constitutes the objective function and penalty terms corresponding to both equality and inequality constraints. Furthermore, we propose to employ a Softmax activation function at the output of DNN along with tensor splitting which simplifies the customized loss function and guarantees to meet the equality constraint. As a case-study, we consider a typical user-association problem in a wireless network, formulate it as GAP, and consequently solve it using our proposed DUL approach. Numerical results demonstrate that the proposed DUL approach provides near-optimal results with significantly lower time-complexity.
Does the geographic proximity effect on knowledge spillovers vary across research fields?
Venue: arXiv cs.DL 2021
Year: 2021

Policy makers are interested in the influence of geographic distance on knowledge flows, however these can be expected to vary across research fields. The effects of geographic distance on flows are analyzed by means of citations to scientific literature. The field of observation consists of the 2010-2012 Italian publications and relevant citations up to the close of 2017. The geographic proximity effect is analyzed at national, continental, and intercontinental level in 244 fields, and results as evident at national level and in some cases at continental level, but not at intercontinental level. For flows between Italian municipalities, citations decrease with distance in all fields. At continental level, four fields are identified having knowledge flows that grow with distance; at intercontinental level, this occurs in 26 fields. The influence of distance is more limited in the fields of Humanities and Social sciences, much more significant in the Sciences, mainly in the Natural sciences.
A novel methodology to assess the scientific standing of nations at field level
Venue: arXiv cs.DL 2021
Year: 2021

The formulation of national research policies would benefit greatly from reliable strategic analysis of the scientific infrastructure, aimed at identifying the relevant strengths and weaknesses at field level. Bibliometric methodologies thus far proposed in the literature are not completely satisfactory. This work proposes a novel "output-to-input-oriented" approach, which permits identification of research strengths and weaknesses on the basis of the ratios of top scientists and highly cited articles to research expenditures in each field. The proposed approach is applied to the Italian academic system. 2012-2016 scientific publications are analyzed, in the 218 research fields where bibliometric assessment is appropriate.
Leveraging pre-trained representations to improve access to untranscribed speech from endangered languages
Venue: arXiv cs.CL 2021
Year: 2021

Pre-trained speech representations like wav2vec 2.0 are a powerful tool for automatic speech recognition (ASR). Yet many endangered languages lack sufficient data for pre-training such models, or are predominantly oral vernaculars without a standardised writing system, precluding fine-tuning. Query-by-example spoken term detection (QbE-STD) offers an alternative for iteratively indexing untranscribed speech corpora by locating spoken query terms. Using data from 7 Australian Aboriginal languages and a regional variety of Dutch, all of which are endangered or vulnerable, we show that QbE-STD can be improved by leveraging representations developed for ASR (wav2vec 2.0: the English monolingual model and XLSR53 multilingual model). Surprisingly, the English model outperformed the multilingual model on 4 Australian language datasets, raising questions around how to optimally leverage self-supervised speech representations for QbE-STD. Nevertheless, we find that wav2vec 2.0 representations (either English or XLSR53) offer large improvements (56-86% relative) over state-of-the-art approaches on our endangered language datasets.
Local Explanations via Necessity and Sufficiency: Unifying Theory and Practice
Venue: arXiv cs.LG 2021
Year: 2021

Necessity and sufficiency are the building blocks of all successful explanations. Yet despite their importance, these notions have been conceptually underdeveloped and inconsistently applied in explainable artificial intelligence (XAI), a fast-growing research area that is so far lacking in firm theoretical foundations. Building on work in logic, probability, and causality, we establish the central role of necessity and sufficiency in XAI, unifying seemingly disparate methods in a single formal framework. We provide a sound and complete algorithm for computing explanatory factors with respect to a given context, and demonstrate its flexibility and competitive performance against state of the art alternatives on various tasks.
Secure Platform for Processing Sensitive Data on Shared HPC Systems
Venue: arXiv cs.CR 2021
Year: 2021

High performance computing clusters operating in shared and batch mode pose challenges for processing sensitive data. In the meantime, the need for secure processing of sensitive data on HPC system is growing. In this work we present a novel method for creating secure computing environments on traditional multi-tenant high-performance computing clusters. Our platform as a service provides a customizable, virtualized solution using PCOCC and SLURM to meet strict security requirements without modifying the exist-ing HPC infrastructure. We show how this platform has been used in real-world research applications from different research domains. The solution is scalable by design with low performance overhead and can be generalized for processing sensitive data on shared HPC systems imposing high security criteria
Genomic Encryption of Biometric Information for Privacy-Preserving Forensics
Venue: arXiv cs.CR 2021
Year: 2021

DNA fingerprinting is a cornerstone for human identification in forensics, where the sequence of highly polymorphic short tandem repeats (STRs) from an individual is compared against a DNA database. This presents significant privacy risks to individuals with DNA profiles in the database due to hacking by malicious attackers who may access the data and misuse it for secondary purposes. In this paper, we propose a novel cryptographic framework for jointly encrypting DNA-based fingerprints (STRs) with other biometric data, for example, facial images, such that the STRs and biometrics information of an individual are revealed only when a positive match is found, i.e. the STRs act as decryption keys. Specifically, when a search is performed on the encrypted database using STR sequences of an individual in the database, a perfect match generates the facial image and/ or other biometrics of the individual while the lack of a match returns a null result. By jointly encrypting DNA fingerprints and other biometrics using the unique STRs generated keys, our approach ensures perfect privacy of the encrypted information with decryption of only the record with STRs matching the query. This safeguards the information of other individuals in the same database. The proposed approach can also be used to securely authenticate the identity of individuals or biological material in scenarios beyond forensics including tracking the identity of samples for clinical genetics and cell therapies.
CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Current perception systems often carry multimodal imagers and sensors such as 2D cameras and 3D LiDAR sensors. To fuse and utilize the data for downstream perception tasks, robust and accurate calibration of the multimodal sensor data is essential. We propose a novel deep learning-driven technique (CalibDNN) for accurate calibration among multimodal sensor, specifically LiDAR-Camera pairs. The key innovation of the proposed work is that it does not require any specific calibration targets or hardware assistants, and the entire processing is fully automatic with a single model and single iteration. Results comparison among different methods and extensive experiments on different datasets demonstrates the state-of-the-art performance.
Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness
Venue: arXiv cs.LG 2021
Year: 2021

Adversarial attacks have rendered high security risks on modern deep learning systems. Adversarial training can significantly enhance the robustness of neural network models by suppressing the non-robust features. However, the models often suffer from significant accuracy loss on clean data. Ensemble training methods have emerged as promising solutions for defending against adversarial attacks by diversifying the vulnerabilities among the sub-models, simultaneously maintaining comparable accuracy as standard training. However, existing ensemble methods are with poor scalability, owing to the rapid complexity increase when including more sub-models in the ensemble. Moreover, in real-world applications, it is difficult to deploy an ensemble with multiple sub-models, owing to the tight hardware resource budget and latency requirement. In this work, we propose ensemble-in-one (EIO), a simple but efficient way to train an ensemble within one random gated network (RGN). EIO augments the original model by replacing the parameterized layers with multi-path random gated blocks (RGBs) to construct a RGN. By diversifying the vulnerability of the numerous paths within the RGN, better robustness can be achieved. It provides high scalability because the paths within an EIO network exponentially increase with the network depth. Our experiments demonstrate that EIO consistently outperforms previous ensemble training methods with even less computational overhead.