Reviewerly/exHarmony huggingface dataset:
Semi-supervised Active Learning for Semi-supervised Models: Exploit Adversarial Examples with Graph-based Virtual Labels

The performance of computer vision models significantly improves with more labeled data. However, the acquisition of labeled data is limited by the high cost. To mitigate the reliance on large labeled datasets, active learning (AL) and semi-supervised learning (SSL) are frequently adopted. Although current mainstream methods begin to combine SSL and AL (SSL-AL) to excavate the diverse expressions of unlabeled samples, these methods’ fully supervised task models are still trained only with labeled data. Besides, these method’s SSL-AL frameworks suffer from mismatch problems. Here, we propose a graph-based SSL-AL framework to unleash the SSL task models’ power and make an effective SSL-AL interaction. In the framework, SSL leverages graph-based label propagation to deliver virtual labels to unlabeled samples, rendering AL samples’ structural distribution and boosting AL. AL finds samples near the clusters’ boundary to help SSL perform better label propagation by exploiting adversarial examples. The information exchange in the closed-loop realizes mutual enhancement of SSL and AL. Experimental results show that our method outperforms the state-of-the-art methods against classification and segmentation benchmarks.
Burn-to-Claim: An asset transfer protocol for blockchain interoperability

The future of multi-blockchain architecture depends on the emergence of new protocols that enable consensus between trustless cross-blockchain participants. However, interoperability between blockchains remains a research challenge. The existing interoperability approaches provide integration through solutions using a middleware system, making it difficult to gain confidence in the security and correctness of the process. A cross-blockchain protocol must provide a self-verifiable state proof that encodes trust in the transfer process to guarantee consensus. Inspired by the burn-address concept, we propose a Burn-to-Claim cross-blockchain protocol to exchange assets between two networks. The proposed protocol transfers assets from one blockchain system to another so that the asset is burned from the source blockchain and recreated on the destination blockchain. Our protocol makes use of digital signatures, hash-time-locks and integration mechanisms to perform cross-blockchain transactions in a distributed manner. We theoretically prove that the proposed cross-blockchain protocol transfers assets securely and correctly. In addition, the experimental results demonstrate the feasibility of the Burn-to-Claim protocol when used in an application environment.
Homomorphic Encryption Based Privacy-Preservation for IoMT

Healthcare applications store private user data on cloud servers and perform computation operations that support several patient diagnoses. Growing cyber-attacks on hospital systems result in user data being held at ransom. Furthermore, mathematical operations on data stored in the Cloud are exposed to untrusted external entities that sell private data for financial gain. In this paper, we propose a privacy-preserving scheme using homomorphic encryption to secure medical plaintext data from being accessed by attackers. Secret sharing distributes computations to several virtual nodes on the edge and masks all arithmetic operations, preventing untrusted cloud servers from learning the tasks performed on the encrypted patient data. Virtual edge nodes benefit from cloud computing resources to accomplish computing-intensive mathematical functions and reduce latency in device–edge node data transmission. A comparative analysis with existing studies demonstrates that homomorphically encrypted data stored at the edge preserves data privacy and integrity. Furthermore, secret sharing-based multi-node computation using virtual nodes ensures data confidentiality from untrusted cloud networks.
From Distance Learning to Integrated Digital Learning: A Fuzzy Cognitive Analysis Focused on Engagement, Motivation, and Participation During COVID-19 Pandemic

Abstract This work focuses on Distance Learning during the COVID-19 pandemic to improve students’ motivation, participation, and engagement, trying to contain the drop-out phenomenon. Distance Learning at the time of COVID-19 is an educational methodology and it can be considered the only occasion to keep an educational connection between students and teachers. Experimentation in a Mathematics STEM class was carried out evaluating the impact of Distance Learning on students’ levels of motivation, participation, and engagement, computed through a Fuzzy Cognitive Map. Specifically, it was performed on some affective and interaction parameters derived from using an adaptive e-learning platform and from the answers of a semi-structured questionnaire. The results, which have been analysed through Technological Pedagogical Content Knowledge and Instrumental Genesis theories, show on one hand that Distance Learning is valid as an additional and support methodology but, on the other hand, they highlight the ineffectiveness of completely remote teaching. Therefore, a teaching method that integrates moments of distance teaching with activities carried out in the presence, in the classroom, or in other university environments, is hoped to be used as soon as the emergency is over: a mix of styles, a fluid flow of knowledge between the physical classroom and the virtual classroom. We will call this Integrated Digital Learning.
Research of Chinese intangible cultural heritage knowledge graph construction and attribute value extraction with graph attention network

• We construct a Chinese intangible cultural heritage (ICH) knowledge graph (KG). • We propose a general ICH KG construction framework. • We offer a novel ICH attribute value extraction (AVE) model. • We provide an ICH annotation method through distant supervision. • The proposed KG construction framework and AVE model can be used in other domains. The development of digital technology promotes the construction of the Intangible cultural heritage (ICH) database but the data is still unorganized and not linked well, which makes the public hard to master the overall knowledge of the ICH. An ICH knowledge graph (KG) can help the public to understand the ICH and facilitate the protection of the ICH. However, a general framework of ICH KG construction is lacking now. In this study, we take the Chinese ICH (nation-level) as an example and propose a framework to build a Chinese ICH KG combining multiple data sources from Baike and the official website, which can extend the scale of the KG. Besides, the data of ICH grows daily, requiring us to design an efficient model to extract the knowledge from the data to update the KG in time. The built KG is based on the triple 〈 entity, attribute, attribute value〉 and we introduce the attribute value extraction (AVE) task. However, the public Chinese ICH annotated AVE corpus is lacking. To solve that, we construct a Chinese ICH AVE corpus based on the Distant Supervision (DS) automatically rather than employing traditional manual annotation. Currently, AVE is usually seen as the sequence tagging task. In this paper, we take the ICH AVE as a node classification task and propose an AVE model BGC, combining the BiLSTM and graph attention network, which can fuse and utilize the word-level and character-level information by means of the ICH lexicon generated from the KG. We conduct extensive experiments and compare the proposed model with other state-of-the-art models. Experimental results show that the proposed model is of superiority.
USCL: Pretraining Deep Ultrasound Image Diagnosis Model Through Video Contrastive Representation Learning

Most deep neural networks (DNNs) based ultrasound (US) medical image analysis models use pretrained backbones (e.g., ImageNet) for better model generalization. However, the domain gap between natural and medical images causes an inevitable performance bottleneck. To alleviate this problem, an US dataset named US-4 is constructed for direct pretraining on the same domain. It contains over 23,000 images from four US video sub-datasets. To learn robust features from US-4, we propose an US semi-supervised contrastive learning method, named USCL, for pretraining. In order to avoid high similarities between negative pairs as well as mine abundant visual features from limited US videos, USCL adopts a sample pair generation method to enrich the feature involved in a single step of contrastive optimization. Extensive experiments on several downstream tasks show the superiority of USCL pretraining against ImageNet pretraining and other state-of-the-art (SOTA) pretraining approaches. In particular, USCL pretrained backbone achieves fine-tuning accuracy of over 94% on POCUS dataset, which is 10% higher than 84% of the ImageNet pretrained model. The source codes of this work are available at https://github.com/983632847/USCL.
Generative Self-training for Cross-Domain Unsupervised Tagged-to-Cine MRI Synthesis

Self-training based unsupervised domain adaptation (UDA) has shown great potential to address the problem of domain shift, when applying a trained deep learning model in a source domain to unlabeled target domains. However, while the self-training UDA has demonstrated its effectiveness on discriminative tasks, such as classification and segmentation, via the reliable pseudo-label selection based on the softmax discrete histogram, the self-training UDA for generative tasks, such as image synthesis, is not fully investigated. In this work, we propose a novel generative self-training (GST) UDA framework with continuous value prediction and regression objective for cross-domain image synthesis. Specifically, we propose to filter the pseudo-label with an uncertainty mask, and quantify the predictive confidence of generated images with practical variational Bayes learning. The fast test-time adaptation is achieved by a round-based alternative optimization scheme. We validated our framework on the tagged-to-cine magnetic resonance imaging (MRI) synthesis problem, where datasets in the source and target domains were acquired from different scanners or centers. Extensive validations were carried out to verify our framework against popular adversarial training UDA methods. Results show that our GST, with tagged MRI of test subjects in new target domains, improved the synthesis quality by a large margin, compared with the adversarial training UDA methods.
Self-supervised Visual Representation Learning for Histopathological Images

Self-supervised learning provides a possible solution to extract effective visual representations from unlabeled histopathological images. However, existing methods either fail to make good use of domain-specific knowledge, or rely on side information like spatial proximity and magnification. In this paper, we propose CS-CO, a hybrid self-supervised visual representation learning method tailored for histopathological images, which integrates advantages of both generative and discriminative models. The proposed method consists of two self-supervised learning stages: cross-stain prediction (CS) and contrastive learning (CO), both of which are designed based on domain-specific knowledge and do not require side information. A novel data augmentation approach, stain vector perturbation, is specifically proposed to serve contrastive learning. Experimental results on the public dataset NCT-CRC-HE-100K demonstrate the superiority of the proposed method for histopathological image visual representation. Under the common linear evaluation protocol, our method achieves 0.915 eight-class classification accuracy with only 1,000 labeled data, which is about 1.3% higher than the fully-supervised ResNet18 classifier trained with the whole 89,434 labeled training data. Our code is available at https://github.com/easonyang1996/CS-CO.
Improving Neural Network Efficiency via Post-training Quantization with Adaptive Floating-Point

Model quantization has emerged as a mandatory technique for efficient inference with advanced Deep Neural Networks (DNN) by representing model parameters with fewer bits. Nevertheless, prior model quantization either suffers from the inefficient data encoding method thus leading to noncompetitive model compression rate, or requires time-consuming quantization aware training process. In this work, we propose a novel Adaptive Floating-Point (AFP) as a variant of standard IEEE-754 floating-point format, with flexible configuration of exponent and mantissa segments. Leveraging the AFP for model quantization (i.e., encoding the parameter) could significantly enhance the model compression rate without accuracy degradation and model re-training. We also want to highlight that our proposed AFP could effectively eliminate the computationally intensive de-quantization step existing in the dynamic quantization technique adopted by the famous machine learning frameworks (e.g., pytorch, tensorRT, etc.). Moreover, we develop a framework to automatically optimize and choose the adequate AFP configuration for each layer, thus maximizing the compression efficacy. Our experiments indicate that AFP-encoded ResNet-50/MobileNet-v2 only has ∼0.04/0.6% accuracy degradation w.r.t its full-precision counterpart. It outperforms the state-of-the-art works by 1.1% in accuracy using the same bit-width while reducing the energy consumption by 11.2×, which is quite impressive for inference. Code is released at: https://github.com/MXHX7199/ICCV_2021_AFP
Real-world Video Super-resolution: A Benchmark Dataset and A Decomposition based Learning Scheme

Video super-resolution (VSR) aims to improve the spatial resolution of low-resolution (LR) videos. Existing VSR methods are mostly trained and evaluated on synthetic datasets, where the LR videos are uniformly downsampled from their high-resolution (HR) counterparts by some simple operators (e.g., bicubic downsampling). Such simple synthetic degradation models, however, cannot well describe the complex degradation processes in real-world videos, and thus the trained VSR models become ineffective in real-world applications. As an attempt to bridge the gap, we build a real-world video super-resolution (RealVSR) dataset by capturing paired LR-HR video sequences using the multi-camera system of iPhone 11 Pro Max. Since the LR-HR video pairs are captured by two separate cameras, there are inevitably certain misalignment and luminance/color differences between them. To more robustly train the VSR model and recover more details from the LR inputs, we convert the LR-HR videos into YCbCr space and decompose the luminance channel into a Laplacian pyramid, and then apply different loss functions to different components. Experiments validate that VSR models trained on our RealVSR dataset demonstrate better visual quality than those trained on synthetic datasets under real-world settings. They also exhibit good generalization capability in cross-camera tests. The dataset and code can be found at https://github.com/IanYeung/RealVSR.
Shape-aware Multi-Person Pose Estimation from Multi-View Images

In this paper we contribute a simple yet effective approach for estimating 3D poses of multiple people from multi-view images. Our proposed coarse-to-fine pipeline first aggregates noisy 2D observations from multiple camera views into 3D space and then associates them into individual instances based on a confidence-aware majority voting technique. The final pose estimates are attained from a novel optimization scheme which links high-confidence multi-view 2D observations and 3D joint candidates. More-over, a statistical parametric body model such as SMPL is leveraged as a regularizing prior for these 3D joint candidates. Specifically, both 3D poses and SMPL parameters are optimized jointly in an alternating fashion. Here the parametric models help in correcting implausible 3D pose estimates and filling in missing joint detections while updated 3D poses in turn guide obtaining better SMPL estimations. By linking 2D and 3D observations, our method is both accurate and generalizes to different data sources because it better decouples the final 3D pose from the interperson constellation and is more robust to noisy 2D detections. We systematically evaluate our method on public datasets and achieve state-of-the-art performance. The code and video will be available on the project page: https://ait.ethz.ch/projects/2021/multi-human-pose/.
LA-HCN: Label-based Attention for Hierarchical Multi-label Text Classification Neural Network

Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However, very few studies have focused on hierarchical feature extraction and explore the association between the hierarchical labels and the text. In this paper, we propose a Label-based Attention for Hierarchical Multi-label Text Classification Neural Network (LA-HCN), where the novel label-based attention module is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Besides, hierarchical information is shared across levels while preserving the hierarchical label-based information. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications. In our experiments, LA-HCN outperforms other state-of-the-art neural network-based HMTC algorithms on four public HMTC datasets. The ablation study also demonstrates the effectiveness of the proposed label-based attention module as well as the novel local and global embeddings and classifications. By visualizing the learned attention (words), we find that LA-HCN is able to extract meaningful information corresponding to the different labels which provides explainability that may be helpful for the human analyst.
Adaptive Kalman Filtering for Recursive Both Additive Noise and Multiplicative Noise

This paper focuses on adaptive Kalman filtering problem for linear systems with unknown covariances of both dynamic multiplicative noise (multiplicative measurement noise) and additive noises (additive process and measurement noises). A recursive-noise adaptive Kalman filter is proposed to estimate both states and covariances of noises by using the varaitional Bayesian (VB) inference and an indirect method. First, we characterize inverse Wishart priors for both measurement noise covariance and process noise covariance and employ the Student's t-distribution to represent the likelihood function, which is non-Gaussian and affected by mixing multiplicative noise and additive measurement noise. Then, an adaptive Kalman filtering for recursive both noise covariance matrices and dynamic state, is proposed following VB inference. Performance analysis for VB procedures and the proposed filter is provided to ensure the convergence and stability. A target tracking example is provided to validate the effectiveness of the proposed filtering algorithm.
Simulated-multifractal models: A futuristic review of multifractal modeling in geochemical anomaly classification

Various statistical methods, including fractal/multifractal techniques, have been applied to clustering of geochemical samples based on their elemental concentrations. Most classification models provide a single classification map whose precision (stability) and accuracy are unknown. Classification models based on more than one approach permit assessment of such uncertainty, but the effect of different assumptions in the model may be difficult to assess. Alternatively, the precision can be assessed using simulation methods on a given modelling approach, such as sequential Gaussian simulation (SGSIM) with a large number of realizations with changes in classification (number of clusters or allocation of samples to specific clusters). In this research, novel robust simulated-multifractal classification models have been introduced and reviewed based on several traditional as well as newly established multifractal models for further consideration in geochemical anomaly classification.
LegalGNN: Legal Information Enhanced Graph Neural Network for Recommendation

Recommendation in legal scenario (Legal-Rec) is a specialized recommendation task that aims to provide potential helpful legal documents for users. While there are mainly three differences compared with traditional recommendation: (1) Both the structural connections and textual contents of legal information are important in the Legal-Rec scenario, which means feature fusion is very important here. (2) Legal-Rec users prefer the newest legal cases (the latest legal interpretation and legal practice), which leads to a severe new-item problem. (3) Different from users in other scenarios, most Legal-Rec users are expert and domain-related users. They often concentrate on several topics and have more stable information needs. So it is important to accurately model user interests here. To the best of our knowledge, existing recommendation work cannot handle these challenges simultaneously. To address these challenges, we propose a legal information enhanced graph neural network–based recommendation framework (LegalGNN). First, a unified legal content and structure representation model is designed for feature fusion, where the Heterogeneous Legal Information Network (HLIN) is constructed to connect the structural features (e.g., knowledge graph) and contextual features (e.g., the content of legal documents) for training. Second, to model user interests, we incorporate the queries users issued in legal systems into the HLIN and link them with both retrieved documents and inquired users. This extra information is not only helpful for estimating user preferences, but also valuable for cold users/items (with less interaction history) in this scenario. Third, a graph neural network with relational attention mechanism is applied to make use of high-order connections in HLIN for Legal-Rec. Experimental results on a real-world legal dataset verify that LegalGNN outperforms several state-of-the-art methods significantly. As far as we know, LegalGNN is the first graph neural model for legal recommendation.
Integration of Patch Features Through Self-supervised Learning and Transformer for Survival Analysis on Whole Slide Images

Survival prediction using whole slide images (WSIs) can provide guidance for better treatment of diseases and patient care. Previous methods usually extract and process only image features from patches of WSIs. However, they ignore the significant role of spatial information of patches and the correlation between the patches of WSIs. Furthermore, those methods extract the patch features through the model pre-trained on ImageNet, overlooking the huge gap between WSIs and natural images. Therefore, we propose a new method, called SeTranSurv, for survival prediction. SeTranSurv extracts patch features from WSIs through self-supervised learning and adaptively aggregates these features according to their spatial information and correlation between patches using the Transformer. Experiments on three large cancer datasets indicate the effectiveness of our model. More importantly, SeTranSurv has better interpretability in locating important patterns and features that contribute to accurate cancer survival prediction.