OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Relation Discovery and Utilization in Multi-Entity Time Series Forecasting
Venue: arXiv cs.LG 2022
Year: 2022

Time series forecasting plays a key role in a variety of domains. In a lot of real-world scenarios, there exist multiple forecasting entities (e.g. power station in the solar system, stations in the traffic system). A straightforward forecasting solution is to mine the temporal dependency for each individual entity through 1d-CNN, RNN, transformer, etc. This approach overlooks the relations between these entities and, in consequence, loses the opportunity to improve performance using spatial-temporal relation. However, in many real-world scenarios, beside explicit relation, there could exist crucial yet implicit relation between entities. How to discover the useful implicit relation between entities and effectively utilize the relations for each entity under various circumstances is crucial. In order to mine the implicit relation between entities as much as possible and dynamically utilize the relation to improve the forecasting performance, we propose an attentional multi-graph neural network with automatic graph learning (A2GNN) in this work. Particularly, a Gumbel-softmax based auto graph learner is designed to automatically capture the implicit relation among forecasting entities. We further propose an attentional relation learner that enables every entity to dynamically pay attention to its preferred relations. Extensive experiments are conducted on five real-world datasets from three different domains. The results demonstrate the effectiveness of A2GNN beyond several state-of-the-art methods.
Analysis of Personal Data Visualization Reviews On mHealth Apps (short paper)
Venue: arXiv cs.HC 2022
Year: 2022

Mobile devices, specifically, smartphones proved easy and quick access to data visualisations throughout various tracking apps. Mobile health (mHealth) apps have given non-expert users access to data visualisation to track their activities and health-related issues such as heart tracking and medication. However, no work is done on user experience or perception of data visualisations in mHealth apps. App reviews offer an indirect anchor for researchers to examine how non-expert users perceive and interact with data visualisations and identify the key challenges and recommendations. This paper introduces an analysis of app reviews on data visualisations reported on a dataset of 250 mHealth apps on the Google Play Store. We identified 8,406 comments related to data visualisations. 919 neutral comments, 1,557 negative comments and 5,930 positive comments. From analysing the user reviews, functional requirements turned out to be the most common problem across these app reviews, followed by the look and feel and then data problems. A complete set of data visualisations seem to be the most well-received capability of mHealth apps. We used these comments to develop classification and data visualisation guidelines when developing mobile data visualisations.
Physics-Informed Graph Learning
Venue: arXiv cs.LG 2022
Year: 2022

An expeditious development of graph learning in recent years has found innumerable applications in several diversified fields. Of the main associated challenges are the volume and complexity of graph data. The graph learning models suffer from the inability to efficiently learn graph information. In order to indemnify this inefficacy, physics-informed graph learning (PIGL) is emerging. PIGL incorporates physics rules while performing graph learning, which has enormous benefits. This paper presents a systematic review of PIGL methods. We begin with introducing a unified framework of graph learning models followed by examining existing PIGL methods in relation to the unified framework. We also discuss several future challenges for PIGL. This survey paper is expected to stimulate innovative research and development activities pertaining to PIGL.
HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels
Venue: arXiv cs.CV 2022
Year: 2022

This paper proposes an Information Bottleneck theory based filter pruning method that uses a statistical measure called Mutual Information (MI). The MI between filters and class labels, also called \textit{Relevance}, is computed using the filter's activation maps and the annotations. The filters having High Relevance (HRel) are considered to be more important. Consequently, the least important filters, which have lower Mutual Information with the class labels, are pruned. Unlike the existing MI based pruning methods, the proposed method determines the significance of the filters purely based on their corresponding activation map's relationship with the class labels. Architectures such as LeNet-5, VGG-16, ResNet-56\textcolor{myblue}{, ResNet-110 and ResNet-50 are utilized to demonstrate the efficacy of the proposed pruning method over MNIST, CIFAR-10 and ImageNet datasets. The proposed method shows the state-of-the-art pruning results for LeNet-5, VGG-16, ResNet-56, ResNet-110 and ResNet-50 architectures. In the experiments, we prune 97.98 \%, 84.85 \%, 76.89\%, 76.95\%, and 63.99\% of Floating Point Operation (FLOP)s from LeNet-5, VGG-16, ResNet-56, ResNet-110, and ResNet-50 respectively.} The proposed HRel pruning method outperforms recent state-of-the-art filter pruning methods. Even after pruning the filters from convolutional layers of LeNet-5 drastically (i.e. from 20, 50 to 2, 3, respectively), only a small accuracy drop of 0.52\% is observed. Notably, for VGG-16, 94.98\% parameters are reduced, only with a drop of 0.36\% in top-1 accuracy. \textcolor{myblue}{ResNet-50 has shown a 1.17\% drop in the top-5 accuracy after pruning 66.42\% of the FLOPs.} In addition to pruning, the Information Plane dynamics of Information Bottleneck theory is analyzed for various Convolutional Neural Network architectures with the effect of pruning.
Sobolev Transport: A Scalable Metric for Probability Measures with Graph Metrics
Venue: arXiv cs.LG 2022
Year: 2022

Optimal transport (OT) is a popular measure to compare probability distributions. However, OT suffers a few drawbacks such as (i) a high complexity for computation, (ii) indefiniteness which limits its applicability to kernel machines. In this work, we consider probability measures supported on a graph metric space and propose a novel Sobolev transport metric. We show that the Sobolev transport metric yields a closed-form formula for fast computation and it is negative definite. We show that the space of probability measures endowed with this transport distance is isometric to a bounded convex set in a Euclidean space with a weighted $\ell_p$ distance. We further exploit the negative definiteness of the Sobolev transport to design positive-definite kernels, and evaluate their performances against other baselines in document classification with word embeddings and in topological data analysis.
Thinking the Fusion Strategy of Multi-reference Face Reenactment
Venue: arXiv cs.CV 2022
Year: 2022

In recent advances of deep generative models, face reenactment -manipulating and controlling human face, including their head movement-has drawn much attention for its wide range of applicability. Despite its strong expressiveness, it is inevitable that the models fail to reconstruct or accurately generate unseen side of the face of a given single reference image. Most of existing methods alleviate this problem by learning appearances of human faces from large amount of data and generate realistic texture at inference time. Rather than completely relying on what generative models learn, we show that simple extension by using multiple reference images significantly improves generation quality. We show this by 1) conducting the reconstruction task on publicly available dataset, 2) conducting facial motion transfer on our original dataset which consists of multi-person's head movement video sequences, and 3) using a newly proposed evaluation metric to validate that our method achieves better quantitative results.
RuCLIP -- new models and experiments: a technical report
Venue: arXiv cs.CV 2022
Year: 2022

In the report we propose six new implementations of ruCLIP model trained on our 240M pairs. The accuracy results are compared with original CLIP model with Ru-En translation (OPUS-MT) on 16 datasets from different domains. Our best implementations outperform CLIP + OPUS-MT solution on most of the datasets in few-show and zero-shot tasks. In the report we briefly describe the implementations and concentrate on the conducted experiments. Inference execution time comparison is also presented in the report.
Authenticated time for detecting GNSS attacks
Venue: arXiv cs.CR 2022
Year: 2022

Information cross-validation can be a powerful tool to detect manipulated, dubious GNSS data. A promising approach is to leverage time obtained over networks a mobile device can connect to, and detect discrepancies between the GNSS-provided time and the network time. The challenge lies in having reliably both accurate and trustworthy network time as the basis for the GNSS attack detection. Here, we provide a concrete proposal that leverages, together with the network time servers, the nearly ubiquitous IEEE 802.11 (Wi-Fi) infrastructure. Our framework supports application-layer, secure and robust real time broadcasting by Wi-Fi Access Points (APs), based on hash chains and infrequent digital signatures verification to minimize computational and communication overhead, allowing mobile nodes to efficiently obtain authenticated and rich time information as they roam. We pair this method with Network Time Security (NTS), for enhanced resilience through multiple sources, available, ideally, simultaneously. We analyze the performance of our scheme in a dedicated setup, gauging the overhead for authenticated time data (Wi-Fi timestamped beacons and NTS). The results show that it is possible to provide security for the external to GNSS time sources, with minimal overhead for authentication and integrity, even when the GNSS-equipped nodes are mobile, and thus have short interactions with the Wi-Fi infrastructure and possibly intermittent Internet connectivity, as well as limited resources.
NU HLT at CMCL 2022 Shared Task: Multilingual and Crosslingual Prediction of Human Reading Behavior in Universal Language Space
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we present a unified model that works for both multilingual and crosslingual prediction of reading times of words in various languages. The secret behind the success of this model is in the preprocessing step where all words are transformed to their universal language representation via the International Phonetic Alphabet (IPA). To the best of our knowledge, this is the first study to favorable exploit this phonological property of language for the two tasks. Various feature types were extracted covering basic frequencies, n-grams, information theoretic, and psycholinguistically-motivated predictors for model training. A finetuned Random Forest model obtained best performance for both tasks with 3.8031 and 3.9065 MAE scores for mean first fixation duration (FFDAvg) and mean total reading time (TRTAvg) respectively.
Stateful Structural Operational Semantics
Venue: arXiv cs.LO 2022
Year: 2022

Compositionality of denotational semantics is an important concern in programming semantics. Mathematical operational semantics in the sense of Turi and Plotkin guarantees compositionality, but seen from the point of view of stateful computation it applies only to very fine-grained equivalences that essentially assume unrestricted interference by the environment between any two statements. We introduce the more restrictive stateful SOS rule format for stateful languages. We show that compositionality of two more coarse-grained semantics, respectively given by assuming read-only interference or no interference between steps, remains an undecidable property even for stateful SOS. However, further restricting the rule format in a manner inspired by the cool GSOS formats of Bloom and van Glabbeek, we obtain the streamlined and cool stateful SOS formats, which respectively guarantee compositionality of the two more abstract equivalences.
Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking
Venue: arXiv cs.CV 2022
Year: 2022

Active Multi-Object Tracking (AMOT) is a task where cameras are controlled by a centralized system to adjust their poses automatically and collaboratively so as to maximize the coverage of targets in their shared visual field. In AMOT, each camera only receives partial information from its observation, which may mislead cameras to take locally optimal action. Besides, the global goal, i.e., maximum coverage of objects, is hard to be directly optimized. To address the above issues, we propose a coordinate-aligned multi-camera collaboration system for AMOT. In our approach, we regard each camera as an agent and address AMOT with a multi-agent reinforcement learning solution. To represent the observation of each agent, we first identify the targets in the camera view with an image detector, and then align the coordinates of the targets in 3D environment. We define the reward of each agent based on both global coverage as well as four individual reward terms. The action policy of the agents is derived with a value-based Q-network. To the best of our knowledge, we are the first to study the AMOT task. To train and evaluate the efficacy of our system, we build a virtual yet credible 3D environment, named "Soccer Court", to mimic the real-world AMOT scenario. The experimental results show that our system achieves a coverage of 71.88%, outperforming the baseline method by 8.9%.
Variation Aware Training of Hybrid Precision Neural Networks with 28nm HKMG FeFET Based Synaptic Core
Venue: arXiv cs.ET 2022
Year: 2022

This work proposes a hybrid-precision neural network training framework with an eNVM based computational memory unit executing the weighted sum operation and another SRAM unit, which stores the error in weight update during back propagation and the required number of pulses to update the weights in the hardware. The hybrid training algorithm for MLP based neural network with 28 nm ferroelectric FET (FeFET) as synaptic devices achieves inference accuracy up to 95% in presence of device and cycle variations. The architecture is primarily evaluated using behavioral or macro-model of FeFET devices with experimentally calibrated device variations and we have achieved accuracies compared to floating-point implementations.
Remaining Useful Life Prediction Using Temporal Deep Degradation Network for Complex Machinery with Attention-based Feature Extraction
Venue: arXiv cs.LG 2022
Year: 2022

The precise estimate of remaining useful life (RUL) is vital for the prognostic analysis and predictive maintenance that can significantly reduce failure rate and maintenance costs. The degradation-related features extracted from the sensor streaming data with neural networks can dramatically improve the accuracy of the RUL prediction. The Temporal deep degradation network (TDDN) model is proposed to make the RUL prediction with the degradation-related features given by the one-dimensional convolutional neural network (1D CNN) feature extraction and attention mechanism. 1D CNN is used to extract the temporal features from the streaming sensor data. Temporal features have monotonic degradation trends from the fluctuating raw sensor streaming data. Attention mechanism can improve the RUL prediction performance by capturing the fault characteristics and the degradation development with the attention weights. The performance of the TDDN model is evaluated on the public C-MAPSS dataset and compared with the existing methods. The results show that the TDDN model can achieve the best RUL prediction accuracy in complex conditions compared to current machine learning models. The degradation-related features extracted from the high-dimension sensor streaming data demonstrate the clear degradation trajectories and degradation stages that enable TDDN to predict the turbofan-engine RUL accurately and efficiently.
Subtyping brain diseases from imaging data
Venue: arXiv cs.LG 2022
Year: 2022

The imaging community has increasingly adopted machine learning (ML) methods to provide individualized imaging signatures related to disease diagnosis, prognosis, and response to treatment. Clinical neuroscience and cancer imaging have been two areas in which ML has offered particular promise. However, many neurologic and neuropsychiatric diseases, as well as cancer, are often heterogeneous in terms of their clinical manifestations, neuroanatomical patterns or genetic underpinnings. Therefore, in such cases, seeking a single disease signature might be ineffectual in delivering individualized precision diagnostics. The current chapter focuses on ML methods, especially semi-supervised clustering, that seek disease subtypes using imaging data. Work from Alzheimer Disease and its prodromal stages, psychosis, depression, autism, and brain cancer are discussed. Our goal is to provide the readers with a broad overview in terms of methodology and clinical applications.
UAV/HAP-Assisted Vehicular Edge Computing in 6G: Where and What to Offload?
Venue: arXiv cs.NI 2022
Year: 2022

In the context of 6th generation (6G) networks, vehicular edge computing (VEC) is emerging as a promising solution to let battery-powered ground vehicles with limited computing and storage resources offload processing tasks to more powerful devices. Given the dynamic vehicular environment, VEC systems need to be as flexible, intelligent, and adaptive as possible. To this aim, in this paper we study the opportunity to realize VEC via non-terrestrial networks (NTNs), where ground vehicles offload resource-hungry tasks to Unmanned Aerial Vehicles (UAVs), High Altitude Platforms (HAPs), or a combination of the two. We define an optimization problem in which tasks are modeled as a Poisson arrival process, and apply queuing theory to find the optimal offloading factor in the system. Numerical results show that aerial-assisted VEC is feasible even in dense networks, provided that high-capacity HAP/UAV platforms are available.
A Deep Reinforcement Learning based Approach for NOMA-based Random Access Network with Truncated Channel Inversion Power Control
Venue: arXiv cs.NI 2022
Year: 2022

As a main use case of 5G and Beyond wireless network, the ever-increasing machine type communications (MTC) devices pose critical challenges over MTC network in recent years. It is imperative to support massive MTC devices with limited resources. To this end, Non-orthogonal multiple access (NOMA) based random access network has been deemed as a prospective candidate for MTC network. In this paper, we propose a deep reinforcement learning (RL) based approach for NOMA-based random access network with truncated channel inversion power control. Specifically, each MTC device randomly selects a pre-defined power level with a certain probability for data transmission. Devices are using channel inversion power control yet subject to the upper bound of the transmission power. Due to the stochastic feature of the channel fading and the limited transmission power, devices with different achievable power levels have been categorized as different types of devices. In order to achieve high throughput with considering the fairness between all devices, two objective functions are formulated. One is to maximize the minimum long-term expected throughput of all MTC devices, the other is to maximize the geometric mean of the long-term expected throughput for all MTC devices. A Policy based deep reinforcement learning approach is further applied to tune the transmission probabilities of each device to solve the formulated optimization problems. Extensive simulations are conducted to show the merits of our proposed approach.
Organ Shape Sensing using Pneumatically Attachable Flexible Rails in Robotic-Assisted Laparoscopic Surgery
Venue: arXiv cs.RO 2022
Year: 2022

In robotic-assisted partial nephrectomy, surgeons remove a part of a kidney often due to the presence of a mass. A drop-in ultrasound probe paired to a surgical robot is deployed to execute multiple swipes over the kidney surface to localise the mass and define the margins of resection. This sub-task is challenging and must be performed by a highly skilled surgeon. Automating this sub-task may reduce cognitive load for the surgeon and improve patient outcomes. The overall goal of this work is to autonomously move the ultrasound probe on the surface of the kidney taking advantage of the use of the Pneumatically Attachable Flexible (PAF) rail system, a soft robotic device used for organ scanning and repositioning. First, we integrate a shape-sensing optical fibre into the PAF rail system to evaluate the curvature of target organs in robotic-assisted laparoscopic surgery. Then, we investigate the impact of the stiffness of the material of the PAF rail on the curvature sensing accuracy, considering that soft targets are present in the surgical field. Finally, we use shape sensing to plan the trajectory of the da Vinci surgical robot paired with a drop-in ultrasound probe and autonomously generate an Ultrasound scan of a kidney phantom.
Generating Synthetic Mobility Networks with Generative Adversarial Networks
Venue: arXiv cs.LG 2022
Year: 2022

The increasingly crucial role of human displacements in complex societal phenomena, such as traffic congestion, segregation, and the diffusion of epidemics, is attracting the interest of scientists from several disciplines. In this article, we address mobility network generation, i.e., generating a city's entire mobility network, a weighted directed graph in which nodes are geographic locations and weighted edges represent people's movements between those locations, thus describing the entire mobility set flows within a city. Our solution is MoGAN, a model based on Generative Adversarial Networks (GANs) to generate realistic mobility networks. We conduct extensive experiments on public datasets of bike and taxi rides to show that MoGAN outperforms the classical Gravity and Radiation models regarding the realism of the generated networks. Our model can be used for data augmentation and performing simulations and what-if analysis.
Outing Power Outages: Real-time and Predictive Socio-demographic Analytics for New York City
Venue: arXiv cs.CY 2022
Year: 2022

Electrical outages continue to occur despite technological innovations and improvements to electric power distribution infrastructure. In this paper, we describe a tool that was designed to acquire and collect data on electric power outages in New York City since July 2020. The electrical outages are then displayed on a front-end application, which is publicly available. We use the collected outage data to analyze these outages and their socio-economic impacts on electricity vulnerable population groups. We determined that there was a slightly negative linear relationship between income and number of outages. Finally, a Markov Influence Graph was created to better understand the spatial and temporal relationships between outages.