Reviewerly/exHarmony huggingface dataset:
Graph Communal Contrastive Learning

Graph representation learning is crucial for many real-world applications (e.g. social relation analysis). A fundamental problem for graph representation learning is how to effectively learn representations without human labeling, which is usually costly and time-consuming. Graph contrastive learning (GCL) addresses this problem by pulling the positive node pairs (or similar nodes) closer while pushing the negative node pairs (or dissimilar nodes) apart in the representation space. Despite the success of the existing GCL methods, they primarily sample node pairs based on the node-level proximity yet the community structures have rarely been taken into consideration. As a result, two nodes from the same community might be sampled as a negative pair. We argue that the community information should be considered to identify node pairs in the same communities, where the nodes insides are semantically similar. To address this issue, we propose a novel Graph Communal Contrastive Learning (gCooL) framework to jointly learn the community partition and learn node representations in an end-to-end fashion. Specifically, the proposed gCooL consists of two components: a Dense Community Aggregation (DeCA) algorithm for community detection and a Reweighted Self-supervised Cross-contrastive (ReSC) training scheme to utilize the community information. Additionally, the real-world graphs are complex and often consist of multiple views. In this paper, we demonstrate that the proposed gCooL can also be naturally adapted to multiplex graphs. Finally, we comprehensively evaluate the proposed gCooL on a variety of real-world graphs. The experimental results show that the gCooL outperforms the state-of-the-art methods.
Meta-Knowledge Learning and Domain Adaptation for Unseen Background Subtraction

Background subtraction is a classic video processing task pervading in numerous visual applications such as video surveillance and traffic monitoring. Given the diversity and variability of real application scenes, an ideal background subtraction model should be robust to various scenarios. Even though deep-learning approaches have demonstrated unprecedented improvements, they often fail to generalize to unseen scenarios, thereby less suitable for extensive deployment. In this work, we propose to tackle cross-scene background subtraction via a two-phase framework that includes meta-knowledge learning and domain adaptation. Specifically, as we observe that meta-knowledge (i.e., scene-independent common knowledge) is the cornerstone for generalizing to unseen scenes, we draw on traditional frame differencing algorithms and design a deep difference network (DDN) to encode meta-knowledge especially temporal change knowledge from various cross-scene data (source domain) without intermittent foreground motion pattern. In addition, we explore a self-training domain adaptation strategy based on iterative evolution. With iteratively updated pseudo-labels, the DDN is continuously fine-tuned and evolves progressively toward unseen scenes (target domain) in an unsupervised fashion. Our framework could be easily deployed on unseen scenes without relying on their annotations. As evidenced by our experiments on the CDnet2014 dataset, it brings a significant improvement to background subtraction. Our method has a favorable processing speed (70 fps) and outperforms the best unsupervised algorithm and top supervised algorithm designed for unseen scenes by 9% and 3%, respectively.
AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally Consistent Video Semantic Segmentation

In video segmentation, generating temporally consistent results across frames is as important as achieving frame-wise accuracy. This paper presents an efficient, intuitive, and unsupervised online adaptation method, AuxAdapt, for improving the temporal consistency of most neural network models. It does not require optical flow and only takes one pass of the video. Since inconsistency mainly arises from the model’s uncertainty in its output, we propose an adaptation scheme where the model learns from its own segmentation decisions as it streams a video, which allows producing more confident and temporally consistent labeling for similarly-looking pixels across frames. For stability and efficiency, we leverage a small auxiliary segmentation network (AuxNet) to assist with this adaptation. More specifically, AuxNet readjusts the decision of the original segmentation network (Main-Net) by adding its own estimations to that of MainNet. At every frame, only AuxNet is updated via back-propagation while keeping MainNet fixed. We extensively evaluate our test-time adaptation approach on standard video benchmarks, including Cityscapes, CamVid, and KITTI. The results demonstrate that our approach provides label-wise accurate, temporally consistent, and computationally efficient adaptation.
SegNet-based first-break picking via seismic waveform classification directly from shot gathers with sparsely distributed traces

Manually picking regularly and densely distributed first breaks (FBs) are critical for shallow velocity-model building in seismic data processing. However, it is time consuming. We employ the fully-convolutional SegNet to address this issue and present a fast automatic seismic waveform classification method to pick densely-sampled FBs directly from common-shot gathers with sparsely distributed traces. Through feeding a large number of representative shot gathers with missing traces and the corresponding binary labels segmented by manually interpreted fully-sampled FBs, we can obtain a well-trained SegNet model. When any unseen gather including the one with irregular trace spacing is inputted, the SegNet can output the probability distribution of different categories for waveform classification. Then FBs can be picked by locating the boundaries between one class on post-FBs data and the other on pre-FBs background. Two land datasets with each over 2000 shots are adopted to illustrate that one well-trained 25-layer SegNet can favorably classify waveform and further pick fully-sampled FBs verified by the manually-derived ones, even when the proportion of randomly missing traces reaches 50%, 21 traces are missing consecutively, or traces are missing regularly.
MDA-Net: Multiscale dual attention-based network for breast lesion segmentation using ultrasound images

Accurate breast lesion segmentation is a great help in the initial stage of breast cancer treatment planning. Ultrasound is considered the safe and cheapest method for the breast screening process. However, ultrasound images inherently contain speckle noise, unclear boundaries, and complex shapes, making it more challenging for automatic segmentation methods. This work proposes a multiscale dual attention-based network (MDA-Net) for concurrent segmentation of breast lesions images. The multiscale fusion (MF) block is introduced that addresses the classical fixed receptive field issues, and helps to extract more semantic features and aims to achieve more features diversity. A dual-attention (dA) is also proposed, which is a hybrid of channel-based attention (cA) and lesion attention (lA) blocks that improves the feature representation capability and adaptatively learns a discriminative representation of high-level features. As a result, a combination of two attention blocks helped the proposed network to concentrate on a more relevant field of view of targets. The MDA-Net is extensively tested on both self-collected private datasets and two public UDIAT, BUSIS datasets. Furthermore, our method is also evaluated on MRI datasets to observe the broad applicability of our method in a different imaging modality. The MDA-Net has achieved the DSC of 87.68%, 91.85%, 90.41%, 83.47% on UDIAT, BUSIS, Private, and RIDER breast MRI datasets (p-value < 0.05 with paired t-test). Our MDA-Net implementation code and pretrained models are released at GitHub: https://github.com/ahmedeqbal/MDA-Net.
A machine learning approach for semi-automatic assessment of IADL dependence in older adults with wearable sensors

The assessment of dependence in older adults currently requires a manual collection of data taken from questionnaires. This process is time consuming for the clinicians and intrudes the daily life of the elderly. This paper aims to semi-automate the acquisition and analysis of health data to assess and predict the dependence in older adults while executing one instrumental activity of daily living (IADL).In a mobile-health (m-health) scenario, we analyze whether the acquisition of data through wearables during the performance of IADLs, and with the help of machine learning techniques could replace the traditional questionnaires to evaluate dependence. To that end, we collected data from wearables, while older adults do the shopping activity. A trial supervisor (TS) labelled the different shopping stages (SS) in the collected data. We performed data pre-processing techniques over those SS and analyzed them with three machine learning algorithms: k-Nearest Neighbors (k-NN), Random Forest (RF) and Support Vector Machines (SVM).Our results confirm that it is possible to replace the traditional questionnaires with wearable data. In particular, the best learning algorithm we tried reported an accuracy of 97% in the assessment of dependence. We tuned the hyperparameters of this algorithm and used embedded feature selection technique to get the best performance with a subset of only 10 features out of the initial 85. This model considers only features extracted from four sensors of a single wearable: accelerometer, heart rate, electrodermal activity and temperature. Although these features are not observational, our current proposal is semi-automatic, because it needs a TS labelling the SS (with a smartphone application). In the future, this labelling process could be automatic as well.Our method can semi-automatically assess the dependence, without disturbing daily activities of elderly people. This method can save clinicians' time in the evaluation of dependence in older adults and reduce healthcare costs.
Real-time optical reconstruction for a three-dimensional light-field display based on path-tracing and CNN super-resolution

Three-Dimensional (3D) light-field display plays a vital role in realizing 3D display. However, the real-time high quality 3D light-field display is difficult, because super high-resolution 3D light field images are hard to be achieved in real-time. Although extensive research has been carried out on fast 3D light-field image generation, no single study exists to satisfy real-time 3D image generation and display with super high-resolution such as 7680×4320. To fulfill real-time 3D light-field display with super high-resolution, a two-stage 3D image generation method based on path tracing and image super-resolution (SR) is proposed, which takes less time to render 3D images than previous methods. In the first stage, path tracing is used to generate low-resolution 3D images with sparse views based on Monte-Carlo integration. In the second stage, a lite SR algorithm based on a generative adversarial network (GAN) is presented to up-sample the low-resolution 3D images to high-resolution 3D images of dense views with photo-realistic image quality. To implement the second stage efficiently and effectively, the elemental images (EIs) are super-resolved individually for better image quality and geometry accuracy, and a foreground selection scheme based on ray casting is developed to improve the rendering performance. Finally, the output EIs from CNN are used to recompose the high-resolution 3D images. Experimental results demonstrate that real-time 3D light-field display over 30fps at 8K resolution can be realized, while the structural similarity (SSIM) can be over 0.90. It is hoped that the proposed method will contribute to the field of real-time 3D light-field display.
Transfer Learning for Mining Feature Requests and Bug Reports from Tweets and App Store Reviews

Identifying feature requests and bug reports in user comments holds great potential for development teams. However, automated mining of RE-related information from social media and app stores is challenging since (1) about 70% of user comments contain noisy, irrelevant information, (2) the amount of user comments grows daily making manual analysis unfeasible, and (3) user comments are written in different languages. Existing approaches build on traditional machine learning (ML) and deep learning (DL), but fail to detect feature requests and bug reports with high Recall and acceptable Precision which is necessary for this task. In this paper, we investigate the potential of transfer learning (TL) for the classification of user comments. Specifically, we train both monolingual and multilingual BERT models and compare the performance with state-of-the-art methods. We found that monolingual BERT models outperform existing baseline methods in the classification of English App Reviews as well as English and Italian Tweets. However, we also observed that the application of heavyweight TL models does not necessarily lead to better performance. In fact, our multilingual BERT models perform worse than traditional ML methods.
DARDet: A Dense Anchor-Free Rotated Object Detector in Aerial Images

Rotated object detection in aerial images has received increasing attention for a wide range of applications. However, it is also a challenging task due to the huge variations of scale, rotation, aspect ratio, and densely arranged targets. Most existing methods heavily rely on a large number of pre-defined anchors with different scales, angles, and aspect ratios, and are optimized with a distance loss. Therefore, these methods are sensitive to anchor hyper-parameters and easily suffer from performance degradation caused by boundary discontinuity. To handle this problem, in this paper, we propose a dense anchor-free rotated object detector (DARDet) for rotated object detection in aerial images. Our DARDet directly predicts five parameters of rotated boxes at each foreground pixel of feature maps. We design a new alignment convolution module to extracts aligned features and introduce a PIoU loss for precise and stable regression. Our method achieves state-of-the-art performance on three commonly used aerial objects datasets (i.e., DOTA, HRSC2016, and UCAS-AOD) while keeping high efficiency. Code is available at https://github.com/zf020114/DARDet.
Majorities help minorities: Hierarchical structure guided transfer learning for few-shot fault recognition

To ensure the operational safety and reliability, fault recognition of complex systems is becoming an essential process in industrial systems. However, the existing recognition methods mainly focus on common faults with enough data, which ignore that many faults are lack of samples in engineering practice. Transfer learning can be helpful, but irrelevant knowledge transfer can cause performance degradation, especially in complex systems. To address the above problem, a hierarchy guided transfer learning framework (HGTL) is proposed in this paper for fault recognition with few-shot samples. Firstly, we fuse domain knowledge, label semantics and inter-class distance to calculate the affinity between categories, based on which a category hierarchical tree is constructed by hierarchical clustering. Then, guided by the hierarchical structure, the samples in most similar majority classes are selected from the source domain to pre-train the hierarchical feature learning network (HFN) and extract the transferable fault information. For the fault knowledge extracted from the child nodes of one parent node are similar and can be transferred with each other, so the trained HFN can extract better features of few samples classes with the help of the information from similar faults, and used to address few-shot fault recognition problems. Finally, a dataset of a nuclear power system with 65 categories and the widely used Tennessee Eastman dataset are analyzed respectively via the proposed method, as well as state-of-the-art recognition methods for comparison. The experimental results demonstrate the effectiveness and superiority of the proposed method in fault recognition with few-shot problem.
Decentralized Learning for Optimality in Stochastic Dynamic Teams and Games With Local Control and Global State Information

Stochastic dynamic teams and games are rich models for decentralized systems and challenging testing grounds for multiagent learning. Previous work that guaranteed team optimality assumed stateless dynamics, or an explicit coordination mechanism, or joint-control sharing. In this article, we present an algorithm with guarantees of convergence to team optimal policies in teams and common interest games. The algorithm is a two-timescale method that uses a variant of Q-learning on the finer timescale to perform policy evaluation while exploring the policy space on the coarser timescale. Agents following this algorithm are “independent learners”: they use only local controls, local cost realizations, and global state information, without access to controls of other agents. The results presented here are the first, to the best of our knowledge, to give formal guarantees of convergence to team optimality using independent learners in stochastic dynamic teams and common interest games.
RFID and camera fusion for recognition of human-object interactions

Recognition of human-object interactions is practically important in various human-centric sensing scenarios such as smart supermarket, factory, and home. This paper proposes an RF-Camera system by fusing RFID and Computer Vision (CV) techniques, which is the first work to recognize the human gestural interactions with physical objects in multi-subject and multi-object scenarios. In RF-Camera, we first propose a dimension reduction method to transform the subject's 3D hand trajectory captured by depth camera to a 2D image, using which the subject's gesture can be recognized. We also propose a method to extract the facial image of target subject from an image that may contain irrelevant subjects, thereby further recognizing his/her identity. Finally, we model the physical movements of the held object's tag and further predict the tag phase data, by comparing which with real phase data of each tag human-object matching can be discovered. When implementing RF-Camera, three technical challenges need to be addressed. (i) To remove noisy data corresponding to irrelevant actions from raw sensing data, we propose a state transition diagram to determine the boundary of effective data. (ii) To predict phase data of the held target tag with unknown hand-tag offset, we quantify target tag trajectory by adding a variable hand-tag vector to captured hand trajectory. (iii) To ensure high reading rates of target tags in tag-dense scenarios, we propose a CV-assisted RFID scheduling method, in which analytics on CV data can help schedule RFID readings. We conduct extensive experiments to evaluate the performance of RF-Camera. Experimental results demonstrate that RF-Camera can recognize the gestural actions, human identity and human-object matching with an average accuracy higher than 90% in most cases.
Sabotaging the harvesting of correlations from quantum fields

We study the non-perturbative harvesting of classical and quantum correlations between two parties coupled to a quantum field. First, we consider a scenario with an arbitrary number of two-level systems that couple to a quantum field locally in time. Then, we study the impact of the presence of additional detectors (interlopers) on the ability for two target detectors (Alice and Bob) to acquire correlations through their interaction with the field. We analyze the harvesting of different correlation measures in this non-perturbative regime and we demonstrate that even a single interloper can completely sabotage all correlation harvesting between Alice and Bob by acting on the causal past of one of them. Specifically, we show that the interloper is able to interact with the field so that the field itself `floods' one of the parties with entropy. This prevents Alice and Bob from acquiring any correlations. Furthermore, we show that this kind of attack cannot be defended against.
End-to-End Learning of Fused Image and Non-Image Features for Improved Breast Cancer Classification from MRI

Breast cancer diagnosis is inherently multimodal. To assess a patient’s cancer status, physicians integrate imaging findings with a variety of clinical risk factor data. Despite this, deep learning approaches for automatic breast cancer classification often only utilize image data or non-image clinical data, but not both simultaneously. In this work, we implemented and compared strategies for the fusion of imaging and tabular non-image data in an end-to-end trainable manner, evaluating fusion at different stages in the model (fusing intermediate features vs. output probabilities) and with different operations (concatenation vs. addition vs. multiplication). This retrospective study utilized dynamic contrast-enhanced MRI (DCE-MRI) data from 10,185 breast MRI examinations of 5,248 women. DCE-MRIs were reduced to 2D maximum intensity projections, split into single-breast images, then linked to a set of 18 non-image features including clinical indication and mam-mographic breast density. We first trained unimodal baseline models on images alone and non-image data alone. We then developed three multimodal fusion models that learn jointly from image and non-image data, evaluating performance by area under the receiver operating characteristic curve (AUC) and specificity at 95% sensitivity. The image-only baseline achieved an AUC of 0.849 (95% CI: 0.834, 0.864) and specificity at 95% sensitivity of 30.1% (95% CI: 23.1%, 37.0%), while the best-performing fusion model achieved an AUC of 0.898 (95% CI: 0.885, 0.909) and specificity of 49.1% (95% CI: 38.8%, 55.3%). Furthermore, all three fusion methods significantly outperformed both unimodal baselines with respect to AUC and specificity at 95% sensitivity. This work demonstrates in our dataset for breast cancer classification that incorporating non-image data with images can significantly improve predictive performance and that fusion of intermediate learned features is superior to fusion of final probabilities.
Sovrin Network for Decentralized Digital Identity: Analysing a Self-Sovereign Identity System Based on Distributed Ledger Technology

Digital identity is the key to the evolving digital society and economy. Since the inception of digital identity, numerous Identity Management (IDM) systems have been developed to manage digital identity depending on the requirements of the individual and that of organisations. This evolution of IDM systems has provided an incremental process leading to the granting of control of identity ownership and personal data to its user, thus producing an IDM which is more user-centric with enhanced security and privacy. A recently promising IDM known as Self-Sovereign Identity (SSI) has the potential to provide this sovereignty to the identity owner. The Sovrin Network is an emerging SSI service utility enabling self-sovereign identity for all, therefore, its assessment has to be carefully considered with reference to its architecture, working, functionality, strengths and limitations. This paper presents an analysis of the Sovrin Network based on aforementioned features. Firstly, it presents the architecture and components of the Sovrin Network. Secondly, it illustrates the working of the Sovrin Network and performs a detailed analysis of its various functionalities and metrics. Finally, based on the detailed analysis, it presents the strengths and limitations of the Sovrin Network.