OpenRaiser/Intern-Atlas huggingface dataset:
Convolutional Cobweb: A Model of Incremental Learning from 2D Images
Venue: arXiv cs.CV 2022
Year: 2022

This paper presents a new concept formation approach that supports the ability to incrementally learn and predict labels for visual images. This work integrates the idea of convolutional image processing, from computer vision research, with a concept formation approach that is based on psychological studies of how humans incrementally form and use concepts. We experimentally evaluate this new approach by applying it to an incremental variation of the MNIST digit recognition task. We compare its performance to Cobweb, a concept formation approach that does not support convolutional processing, as well as two convolutional neural networks that vary in the complexity of their convolutional processing. This work represents a first step towards unifying modern computer vision ideas with classical concept formation research.
DDU-Net: Dual-Decoder-U-Net for Road Extraction Using High-Resolution Remote Sensing Images
Venue: arXiv cs.CV 2022
Year: 2022

Extracting roads from high-resolution remote sensing images (HRSIs) is vital in a wide variety of applications, such as autonomous driving, path planning, and road navigation. Due to the long and thin shape as well as the shades induced by vegetation and buildings, small-sized roads are more difficult to discern. In order to improve the reliability and accuracy of small-sized road extraction when roads of multiple sizes coexist in an HRSI, an enhanced deep neural network model termed Dual-Decoder-U-Net (DDU-Net) is proposed in this paper. Motivated by the U-Net model, a small decoder is added to form a dual-decoder structure for more detailed features. In addition, we introduce the dilated convolution attention module (DCAM) between the encoder and decoders to increase the receptive field as well as to distill multi-scale features through cascading dilated convolution and global average pooling. The convolutional block attention module (CBAM) is also embedded in the parallel dilated convolution and pooling branches to capture more attention-aware features. Extensive experiments are conducted on the Massachusetts Roads dataset with experimental results showing that the proposed model outperforms the state-of-the-art DenseUNet, DeepLabv3+ and D-LinkNet by 6.5%, 3.3%, and 2.1% in the mean Intersection over Union (mIoU), and by 4%, 4.8%, and 3.1% in the F1 score, respectively. Both ablation and heatmap analyses are presented to validate the effectiveness of the proposed model.
Epidemic Source Detection in Contact Tracing Networks: Epidemic Centrality in Graphs and Message-Passing Algorithms
Venue: arXiv cs.SI 2022
Year: 2022

We study the epidemic source detection problem in contact tracing networks modeled as a graph-constrained maximum likelihood estimation problem using the susceptible-infected model in epidemiology. Based on a snapshot observation of the infection subgraph, we first study finite degree regular graphs and regular graphs with cycles separately, thereby establishing a mathematical equivalence in maximal likelihood ratio between the case of finite acyclic graphs and that of cyclic graphs. In particular, we show that the optimal solution of the maximum likelihood estimator can be refined to distances on graphs based on a novel statistical distance centrality that captures the optimality of the nonconvex problem. An efficient contact tracing algorithm is then proposed to solve the general case of finite degree-regular graphs with multiple cycles. Our performance evaluation on a variety of graphs shows that our algorithms outperform the existing state-of-the-art heuristics using contact tracing data from the SARS-CoV 2003 and COVID-19 pandemics by correctly identifying the superspreaders on some of the largest superspreading infection clusters in Singapore and Taiwan.
DEFER: Distributed Edge Inference for Deep Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Modern machine learning tools such as deep neural networks (DNNs) are playing a revolutionary role in many fields such as natural language processing, computer vision, and the internet of things. Once they are trained, deep learning models can be deployed on edge computers to perform classification and prediction on real-time data for these applications. Particularly for large models, the limited computational and memory resources on a single edge device can become the throughput bottleneck for an inference pipeline. To increase throughput and decrease per-device compute load, we present DEFER (Distributed Edge inFERence), a framework for distributed edge inference, which partitions deep neural networks into layers that can be spread across multiple compute nodes. The architecture consists of a single "dispatcher" node to distribute DNN partitions and inference data to respective compute nodes. The compute nodes are connected in a series pattern where each node's computed result is relayed to the subsequent node. The result is then returned to the Dispatcher. We quantify the throughput, energy consumption, network payload, and overhead for our framework under realistic network conditions using the CORE network emulator. We find that for the ResNet50 model, the inference throughput of DEFER with 8 compute nodes is 53% higher and per node energy consumption is 63% lower than single device inference. We further reduce network communication demands and energy consumption using the ZFP serialization and LZ4 compression algorithms. We have implemented DEFER in Python using the TensorFlow and Keras ML libraries, and have released DEFER as an open-source framework to benefit the research community.
Unveiling music genre structure through common-interest communities
Venue: arXiv cs.SI 2022
Year: 2022

Using a dataset of more than 90,000 metal music reviews written by over 9,000 users in a period of 15 years, we analyse the genre structure of metal music with the aid of review text information. We model the relationships between genres using a user-oriented network, based on the written reviews. We then perform community detection and employ a network "averaging" method to obtain stable genre clusters, in order to analyse the structures of clusters both locally within each cluster and globally over the entire network. In addition to identifying the clusters, we use Dependency Parsing and modified Term Frequency - Inverse Document Frequency to extract significant and unique features of each cluster. These structures and review text information can allow us to understand how music audience (fans) perceive similar and different genres, and also assist in classifying different genres which share common-interest user communities, offering a more objective way in grouping music genres. Furthermore, the classification can also help recommendation engines provide more targeted suggestions of music, and potentially help musicians to select genre labels for their music, and design music to better cater to preferences of their audiences based on previous reviews.
Frequent Itemset-driven Search for Finding Minimum Node Separators in Complex Networks
Venue: arXiv cs.NE 2022
Year: 2022

Finding an optimal set of critical nodes in a complex network has been a long-standing problem in the fields of both artificial intelligence and operations research. Potential applications include epidemic control, network security, carbon emission monitoring, emergence response, drug design, and vulnerability assessment. In this work, we consider the problem of finding a minimal node separator whose removal separates a graph into multiple different connected components with fewer than a limited number of vertices in each component. To solve it, we propose a frequent itemset-driven search approach, which integrates the concept of frequent itemset mining in data mining into the well-known memetic search framework. Starting from a high-quality population built by the solution construction and population repair procedures, it iteratively employs the frequent itemset recombination operator (to generate promising offspring solution based on itemsets that frequently occur in high-quality solutions), tabu search-based simulated annealing (to find high-quality local optima), population repair procedure (to modify the population), and rank-based population management strategy (to guarantee a healthy population). Extensive evaluations on 50 widely used benchmark instances show that it significantly outperforms state-of-the-art algorithms. In particular, it discovers 29 new upper bounds and matches 18 previous best-known bounds. Finally, experimental analyses are performed to confirm the effectiveness of key algorithmic modules of the proposed method.
Evidence-aware Fake News Detection with Graph Neural Networks
Venue: arXiv cs.CL 2022
Year: 2022

The prevalence and perniciousness of fake news has been a critical issue on the Internet, which stimulates the development of automatic fake news detection in turn. In this paper, we focus on the evidence-based fake news detection, where several evidences are utilized to probe the veracity of news (i.e., a claim). Most previous methods first employ sequential models to embed the semantic information and then capture the claim-evidence interaction based on different attention mechanisms. Despite their effectiveness, they still suffer from two main weaknesses. Firstly, due to the inherent drawbacks of sequential models, they fail to integrate the relevant information that is scattered far apart in evidences for veracity checking. Secondly, they neglect much redundant information contained in evidences that may be useless or even harmful. To solve these problems, we propose a unified Graph-based sEmantic sTructure mining framework, namely GET in short. Specifically, different from the existing work that treats claims and evidences as sequences, we model them as graph-structured data and capture the long-distance semantic dependency among dispersed relevant snippets via neighborhood propagation. After obtaining contextual semantic information, our model reduces information redundancy by performing graph structure learning. Finally, the fine-grained semantic representations are fed into the downstream claim-evidence interaction module for predictions. Comprehensive experiments have demonstrated the superiority of GET over the state-of-the-arts.
Knowledge Tracing: A Survey
Venue: arXiv cs.CY 2022
Year: 2022

Humans ability to transfer knowledge through teaching is one of the essential aspects for human intelligence. A human teacher can track the knowledge of students to customize the teaching on students needs. With the rise of online education platforms, there is a similar need for machines to track the knowledge of students and tailor their learning experience. This is known as the Knowledge Tracing (KT) problem in the literature. Effectively solving the KT problem would unlock the potential of computer-aided education applications such as intelligent tutoring systems, curriculum learning, and learning materials' recommendation. Moreover, from a more general viewpoint, a student may represent any kind of intelligent agents including both human and artificial agents. Thus, the potential of KT can be extended to any machine teaching application scenarios which seek for customizing the learning experience for a student agent (i.e., a machine learning model). In this paper, we provide a comprehensive and systematic review for the KT literature. We cover a broad range of methods starting from the early attempts to the recent state-of-the-art methods using deep learning, while highlighting the theoretical aspects of models and the characteristics of benchmark datasets. Besides these, we shed light on key modelling differences between closely related methods and summarize them in an easy-to-understand format. Finally, we discuss current research gaps in the KT literature and possible future research and application directions.
ASOCEM: Automatic Segmentation Of Contaminations in cryo-EM
Venue: arXiv cs.CV 2022
Year: 2022

Particle picking is currently a critical step in the cryo-electron microscopy single particle reconstruction pipeline. Contaminations in the acquired micrographs severely degrade the performance of particle pickers, resulting is many ``non-particles'' in the collected stack of particles. In this paper, we present ASOCEM (Automatic Segmentation Of Contaminations in cryo-EM), an automatic method to detect and segment contaminations, which requires as an input only the approximated particle size. In particular, it does not require any parameter tuning nor manual intervention. Our method is based on the observation that the statistical distribution of contaminated regions is different from that of the rest of the micrograph. This nonrestrictive assumption allows to automatically detect various types of contaminations, from the carbon edges of the supporting grid to high contrast blobs of different sizes. We demonstrate the efficiency of our algorithm using various experimental data sets containing various types of contaminations. ASOCEM is integrated as part of the KLT picker \cite{ELDAR2020107473} and is available at \url{https://github.com/ShkolniskyLab/kltpicker2}.
Performance Analysis of IEEE 802.11p Preamble Insertion in C-V2X Sidelink Signals for Co-Channel Coexistence
Venue: arXiv cs.NI 2022
Year: 2022

Spectrum scarcity is one of the main challenges of future wireless technologies. When looking at vehicle-to-everything (V2X), this is amplified as spectrum sharing could impact road safety and traffic efficiency. It is therefore of particular importance to study solutions that allow the coexistence, in the same geographical area and in the same channels, of what are today the main V2X access technologies, namely IEEE 802.11p and sidelink LTE-V2X Mode 4. In this work, in addition to investigating the impact of the reciprocal interference, which we demonstrate to have a strong impact especially on the first and in congested channel conditions, a mitigation solution is extensively studied, which is based on the insertion of the IEEE 802.11p preamble at the beginning of the LTE-V2X sidelink transmission. The proposal, which is also under discussion within the standardization bodies, requires no modifications to the IEEE 802.11p protocol stack and minor changes to LTE-V2X sidelink. This solution is directly applicable to upcoming IEEE 802.11bd and extendable to NR-V2X sidelink. The paper shows, through analysis and simulations in free-flow and dense scenarios, that the proposal allows for a mitigation of collisions caused by co-channel coexistence under low to high-load channel conditions and that the improvement is also granted in congested cases when combined with additional countermeasures. Regarding the latter aspect, in particular, different approaches are compared, demonstrating that acting on the congestion control mechanisms is a simple but effective solution.
Attentional Feature Refinement and Alignment Network for Aircraft Detection in SAR Imagery
Venue: arXiv cs.CV 2022
Year: 2022

Aircraft detection in Synthetic Aperture Radar (SAR) imagery is a challenging task in SAR Automatic Target Recognition (SAR ATR) areas due to aircraft's extremely discrete appearance, obvious intraclass variation, small size and serious background's interference. In this paper, a single-shot detector namely Attentional Feature Refinement and Alignment Network (AFRAN) is proposed for detecting aircraft in SAR images with competitive accuracy and speed. Specifically, three significant components including Attention Feature Fusion Module (AFFM), Deformable Lateral Connection Module (DLCM) and Anchor-guided Detection Module (ADM), are carefully designed in our method for refining and aligning informative characteristics of aircraft. To represent characteristics of aircraft with less interference, low-level textural and high-level semantic features of aircraft are fused and refined in AFFM throughly. The alignment between aircraft's discrete back-scatting points and convolutional sampling spots is promoted in DLCM. Eventually, the locations of aircraft are predicted precisely in ADM based on aligned features revised by refined anchors. To evaluate the performance of our method, a self-built SAR aircraft sliced dataset and a large scene SAR image are collected. Extensive quantitative and qualitative experiments with detailed analysis illustrate the effectiveness of the three proposed components. Furthermore, the topmost detection accuracy and competitive speed are achieved by our method compared with other domain-specific,e.g., DAPN, PADN, and general CNN-based methods,e.g., FPN, Cascade R-CNN, SSD, RefineDet and RPDet.
WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data
Venue: arXiv cs.AI 2022
Year: 2022

Detecting relevant changes in dynamic time series data in a timely manner is crucially important for many data analysis tasks in real-world settings. Change point detection methods have the ability to discover changes in an unsupervised fashion, which represents a desirable property in the analysis of unbounded and unlabeled data streams. However, one limitation of most of the existing approaches is represented by their limited ability to handle multivariate and high-dimensional data, which is frequently observed in modern applications such as traffic flow prediction, human activity recognition, and smart grids monitoring. In this paper, we attempt to fill this gap by proposing WATCH, a novel Wasserstein distance-based change point detection approach that models an initial distribution and monitors its behavior while processing new data points, providing accurate and robust detection of change points in dynamic high-dimensional data. An extensive experimental evaluation involving a large number of benchmark datasets shows that WATCH is capable of accurately identifying change points and outperforming state-of-the-art methods.
What is the mission of innovation?
Venue: arXiv cs.SI 2022
Year: 2022

Governments and organizations recognize the need to revisit a mission-driven innovation amidst national and organizational innovation policy formulations. Notwithstanding a fertile research agenda on mission statements (hereafter mission(s)), several lines of inquiry remain open, such as crossnational and multisectorial studies and an examination of research knowledge intensive institutions. In this article, we identify similarities and differences in the content of missions from government, private, higher education, and health research knowledge intensive institutions in a sample of over 1,900 institutions from 89 countries through the deployment of sentiment analysis, readability, and lexical diversity; semantic networks; and a similarity computation between document corpus. We found that missions of research knowledge intensive institutions are challenging to read texts with lower lexical diversity that favors positive rather than negative words. In stark contrast to this, the non-profit sector is consonant in multiple dimensions in its use of Corporate Social Responsibility jargon. The lexical appearance of research in the missions varies according to mission sectorial context, and each sector has a cluster specific focus. Utilizing the mission as a strategic planning tool in higher income regions might serve to explain corpora similarities shared by sectors and continents.
Network Slicing-Based Customization of 5G Mobile Services
Venue: arXiv cs.NI 2022
Year: 2022

Through network slicing, different requirements of different applications and services can be met. These requirements can be in terms of latency, bandwidth, mobility support, defining service area, as well as security. Through fine and dynamic tuning of network slices, services can have their delivery platforms constantly customized according to their changing needs. In this article, we present our implementation of an E2E network slice orchestration platform, evaluate its performance in terms of dynamic deployment of network slices in an E2E fashion, and discuss how its functionality can be enhanced to better customize the network slices according to the needs of their respective services.
Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis
Venue: arXiv cs.CL 2022
Year: 2022

Social media data such as Twitter messages ("tweets") pose a particular challenge to NLP systems because of their short, noisy, and colloquial nature. Tasks such as Named Entity Recognition (NER) and syntactic parsing require highly domain-matched training data for good performance. To date, there is no complete training corpus for both NER and syntactic analysis (e.g., part of speech tagging, dependency parsing) of tweets. While there are some publicly available annotated NLP datasets of tweets, they are only designed for individual tasks. In this study, we aim to create Tweebank-NER, an English NER corpus based on Tweebank V2 (TB2), train state-of-the-art (SOTA) Tweet NLP models on TB2, and release an NLP pipeline called Twitter-Stanza. We annotate named entities in TB2 using Amazon Mechanical Turk and measure the quality of our annotations. We train the Stanza pipeline on TB2 and compare with alternative NLP frameworks (e.g., FLAIR, spaCy) and transformer-based models. The Stanza tokenizer and lemmatizer achieve SOTA performance on TB2, while the Stanza NER tagger, part-of-speech (POS) tagger, and dependency parser achieve competitive performance against non-transformer models. The transformer-based models establish a strong baseline in Tweebank-NER and achieve the new SOTA performance in POS tagging and dependency parsing on TB2. We release the dataset and make both the Stanza pipeline and BERTweet-based models available "off-the-shelf" for use in future Tweet NLP research. Our source code, data, and pre-trained models are available at: \url{https://github.com/social-machines/TweebankNLP}.
TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data
Venue: arXiv cs.LG 2022
Year: 2022

Efficient anomaly detection and diagnosis in multivariate time-series data is of great importance for modern industrial applications. However, building a system that is able to quickly and accurately pinpoint anomalous observations is a challenging problem. This is due to the lack of anomaly labels, high data volatility and the demands of ultra-low inference times in modern applications. Despite the recent developments of deep learning approaches for anomaly detection, only a few of them can address all of these challenges. In this paper, we propose TranAD, a deep transformer network based anomaly detection and diagnosis model which uses attention-based sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD uses focus score-based self-conditioning to enable robust multi-modal feature extraction and adversarial training to gain stability. Additionally, model-agnostic meta learning (MAML) allows us to train the model using limited data. Extensive empirical studies on six publicly available datasets demonstrate that TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training. Specifically, TranAD increases F1 scores by up to 17%, reducing training times by up to 99% compared to the baselines.