Reviewerly/exHarmony huggingface dataset:
Beyond the <i>AJR</i>: Applying Screening Algorithms (Artificially) Intelligently

Commentary on Landy R, Wang VL, Baldwin DR, et al. Recalibration of a deep learning model for low-dose computed tomographic images to inform lung cancer screening intervals. JAMA Netw Open 2023; 6:e233273; https://doi.org/10.1001/jamanetworkopen.2023.3273. Abstract available at pubmed.ncbi.nlm.nih.gov/36929398/
Single-Stage Broad Multi-Instance Multi-Label Learning (BMIML) With Diverse Inter-Correlations and Its Application to Medical Image Classification

In real applications, one object (e.g., image) can be described by multiple instances (e.g., image patches) and simultaneously associated with multiple labels. Such applications can be formulated as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-instance multi-label learning</i> (MIML) problems and have been extensively studied currently. Existing MIML methods are useful in many applications but most of which suffer from relatively low accuracy and training efficiency due to: i) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the inter-label correlations</i> are neglected; ii) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the inter-instance correlations</i> cannot be learned directly with other types of correlations due to the missing instance labels; iii) diverse inter-correlations (e.g., inter-label correlations, inter-instance correlations) can only be learned in multiple stages. To resolve these issues, a new single-stage framework called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">broad multi-instance multi-label learning</i> (BMIML) is proposed. In BMIML, there are three innovative modules: i) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">auto-weighted label enhancement learning</i> (AWLEL) based on broad learning system (BLS) is designed, which simultaneously captures the inter-label correlations while traditional BLS cannot; ii) A specific MIML neural network, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scalable multi-instance probabilistic regression</i> (SMIPR) is constructed to effectively estimate the inter-instance correlations, which can provide additional probabilistic information for learning; iii) Finally, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interactive decision optimization</i> (IDO) is designed to combine and optimize the results from AWLEL and SMIPR and form a single-stage framework. Consequently, BMIML can achieve simultaneous learning of diverse inter-correlations between whole images, instances, and labels in single stage for higher classification accuracy and much faster training time. Experiments show that BMIML is highly competitive to (or even better than) existing methods in accuracy and much faster.
Unknown Threats Detection Methods of Smart Contracts

With the explosive growth of blockchain platforms and applications, security threats of blockchain also occur frequently. As a decentralized application deployed on the blockchain, smart contracts help the blockchain realize safe and efficient information storage, asset management, and value transfer. Therefore, smart contracts play a vital role in the security of the blockchain. In recent years, security threats against smart contracts have increased, not only causing huge economic losses but also impacting the credit system of the blockchain. Therefore, many researchers have carried out corresponding research on the security threats of smart contracts. Common threat detection methods include formal verification, symbolic execution, fuzzing, etc. Most of these methods are only for known threats, while there is not much work on detecting unknown threats. In order to better deal with unknown threats, we present a review of the typical smart contract security events in recent years, analyze the security threats from contract coding, Ethereum virtual machine, and blockchain characteristics. Further, we compare and summarize the latest unknown threat detection methods. Then, to address the problem that very few unknown threat samples are available, a detection method based on a few-shot learning is proposed.
Semantic Correlation Attention-Based Multiorder Multiscale Feature Fusion Network for Human Motion Prediction

Human motion prediction is to predict future human states based on the observed human states. However, current research ignores the semantic correlations between body parts (joints and bones) in the observed human states and motion time; thus, the prediction accuracy is limited. To address this issue, we propose a novel semantic correlation attention-based multiorder multiscale feature fusion network (SCAFF), which includes an encoder and a decoder. In the encoder, a multiorder difference calculation module (MODC) is designed to calculate the multiorder difference information of joint and bone attributes in the observed human states. Then, multiple semantic correlation attention-based graph calculation operators (SCA-GCOs) are stacked to extract the multiscale features of the multiorder difference information. Each SCA-GCO captures joint and bone dependencies of the multiorder difference information, refines them with a semantic correlation attention module (SCAM), and captures temporal dynamics of the refined joint and bone dependencies as the output features. Note that SCAM learns a semantic attention mask describing the semantic correlations between body parts and motion time for feature refinement. Afterward, multiple multiorder feature fusion modules (MOFFs) and multiscale feature fusion modules (MSFFs) are designed to fuse the multiscale features of the multiorder difference information extracted by multiple SCA-GCOs, thus obtaining the motion features of the observed human states. Based on the obtained motion features, the decoder recurrently recruits a composite gated recurrent module (CGRM) and multilayer perceptrons (MLPs) to predict future human states. As far as we know, this is the first attempt to consider the semantic correlations between body parts and motion time in human motion prediction. The results on public datasets demonstrate that SCAFF outperforms existing models.
Parameterized Convex Universal Approximators for Decision-Making Problems

Parameterized max-affine (PMA) and parameterized log-sum-exp (PLSE) networks are proposed for general decision-making problems.The proposed approximators generalize existing convex approximators, namely, max-affine (MA) and log-sum-exp (LSE) networks, by considering function arguments of condition and decision variables and replacing the network parameters of MA and LSE networks with continuous functions with respect to the condition variable.The universal approximation theorem of PMA and PLSE is proven, which implies that PMA and PLSE are shape-preserving universal approximators for parameterized convex continuous functions.Practical guidelines for incorporating deep neural networks within PMA and PLSE networks are provided.A numerical simulation is performed to demonstrate the performance of the proposed approximators.The simulation results support that PLSE outperforms other existing approximators in terms of minimizer and optimal value errors with scalable and efficient computation for high-dimensional cases.
The <i>Transform-and-Perform</i> Framework: Explainable Deep Learning Beyond Classification

In recent years, visual analytics (VA) has shown promise in alleviating the challenges of interpreting black-box deep learning (DL) models. While the focus of VA for explainable DL has been mainly on classification problems, DL is gaining popularity in high-dimensional-to-high-dimensional (H-H) problems such as image-to-image translation. In contrast to classification, H-H problems have no explicit instance groups or classes to study. Each output is continuous, high-dimensional, and changes in an unknown non-linear manner with changes in the input. These unknown relations between the input, model and output necessitate the user to analyze them in conjunction, leveraging symmetries between them. Since classification tasks do not exhibit some of these challenges, most existing VA systems and frameworks allow limited control of the components required to analyze models beyond classification. Hence, we identify the need for and present a unified conceptual framework, the Transform-and-Perform framework (T&P), to facilitate the design of VA systems for DL model analysis focusing on H-H problems. T&P provides a checklist to structure and identify workflows and analysis strategies to design new VA systems, and understand existing ones to uncover potential gaps for improvements. The goal is to aid the creation of effective VA systems that support the structuring of model understanding and identifying actionable insights for model improvements. We highlight the growing need for new frameworks like T&P with a real-world image-to-image translation application. We illustrate how T&P effectively supports the understanding and identification of potential gaps in existing VA systems.
CGOF++: Controllable 3D Face Synthesis With Conditional Generative Occupancy Fields

Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e.g., controlling the shapes, expressions, textures, and poses of the generated face images. However, previous methods focus on controllable 2D image generative models, which are prone to producing inconsistent face images under large expression and pose changes. In this paper, we propose a new NeRF-based conditional 3D face synthesis framework, which enables 3D controllability over the generated face images by imposing explicit 3D conditions from 3D face priors. At its core is a conditional Generative Occupancy Field (cGOF++) that effectively enforces the shape of the generated face to conform to a given 3D Morphable Model (3DMM) mesh, built on top of EG3D [1], a recent tri-plane-based generative model. To achieve accurate control over fine-grained 3D face shapes of the synthesized images, we additionally incorporate a 3D landmark loss as well as a volume warping loss into our synthesis framework. Experiments validate the effectiveness of the proposed method, which is able to generate high-fidelity face images and shows more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods.
Secure Task Distribution With Verifiable Re-Encryption in Mobile-Crowdsensing-Assisted Emergency IoT System

Extreme events (such as earthquakes, hurricanes, etc.) pose a dual challenge to the reliability and serviceability of IoT systems. With regard to this challenge, by publishing some tasks and then encouraging the public to assist in real-time data collection through their mobile terminals (namely, the mobile crowdsourcing-assisted IoT systems), is expected to play an important role in secondary disaster prevention and personnel rescue in extreme events. However, it has weaknesses in terms of security, flexibility, and efficiency. As an elegant solution, identity-based broadcast proxy re-encryption (PR-IBBE) enables flexible access authorization sharing and efficient broadcast distribution of encrypted tasks via the cloud. However, their security relies on fully-trusted or semi-trusted cloud assumptions, which are hard to be implemented in real-world scenarios. And the cloud is more vulnerable in an emergency event since there is a lack of effective management. Motivated by that, we propose the verifiable PR-IBBE (VPR-IBBE) scheme, which realizes a cross-domain identity-based broadcast task file secure authorization access, and empowers the verifiability and reputability of re-encrypted ciphertext under the untrusted cloud setting. This mechanism ensures that the relevance between the re-encrypted ciphertext and the original ciphertext can be publically verified, so the cloud can defend itself if there is a malicious accusation of forging the re-encrypted ciphertext. Through rigorous formal security proofs, we demonstrate that VPR-IBBE attains the indistinguishability of ciphertext against selective identity chosen ciphertext attack (IND-sID-CPA), and is also resistant to the collusion attack between the untrusted cloud and the cooperative performer. Theoretical comparison and experimental results demonstrate the practicability of our VPR-IBBE scheme, as well as the superiority over representative related works.
Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition

Human pose estimation (HPE) is a fundamental yet promising visual recognition problem. Existing popular methods (e.g., Hourglass and its variants) either attempt to directly add local features element-wisely, or (e.g., vision transformers) try to learn the global relationships among different human parts. However, it remains an open problem to effectively integrate the local-global representations for accurate HPE. In this work, we design four feature fusion strategies on the hierarchical ResNet structure, including direct channel concatenation, element-wise addition and two parallel structures. Both two parallel structures adopt the naive self-attention encoder to model global dependencies. The difference between them is that one adopts the original ResNet BottleNeck while the other employs a spatial-attention module (named SSF) to learn the local patterns. Experiments on COCO Keypoint 2017 show that our SSF for HPE (named SSPose) achieves the best average precision with acceptable computational cost among the compared state-of-the-art methods. In addition, we build a lightweight running dataset to verify the effectiveness of SSPose. Based solely on the keypoints estimated by our SSPose, we propose a regression model to identify valid running movements without training any other classifiers. Our source codes and running dataset are publicly available.
Artificial Intelligence and Pathomics

Artificial intelligence (AI) has the potential to transform pathologic diagnosis and cancer patient management as a predictive and prognostic biomarker. AI-based systems can be used to examine digitally scanned histopathology slides and differentiate benign from malignant cells and low from high grade. Deep learning models can analyze patient data from individual or multimodal combinations and identify patterns to be used to predict the response to different therapeutic options, the risk of recurrence or progression, and the prognosis of the newly diagnosed patient. AI-based models will improve treatment planning for patients with prostate cancer and improve the efficiency and cost-effectiveness of the pathology laboratory.
Federated Learning and NFT-Based Privacy-Preserving Medical-Data-Sharing Scheme for Intelligent Diagnosis in Smart Healthcare

Historical medical data of the patients have an important place in the healthcare industry for providing the best care to patients through intelligent health diagnosis and prediction of diseases. The existing intelligent health diagnosis systems collect data from medical institutions or laboratories and then use machine learning algorithms to predict diseases. But, in most cases, the medical institutions have incomplete medical data of the patients since a patient may consult different specialists (from various hospitals) during the treatment process. To overcome this problem, we build a smart and secure federated learning framework for intelligent health diagnosis with a blockchain-based incentive mechanism and NFT(Non-Fungible Tokens)-based marketplace. We make use of NFTs to develop clear demarkations on the ownership and accessibility of the data of patients. We create an NFT marketplace that manages access to the historical medical data of patients. A comprehensive incentive mechanism based on several factors, including the quality and relevance of the data, the frequency, and regularity of data uploading, etc., is incorporated to encourage and penalize the patients based on their contributions to the global model. We used the Polyak-averaging technique for aggregating local models to form a global model. The extensive analysis shows that the proposed model achieves comparable performance with the centralized machine learning models while affording better security and access to better data. The results also show the efficacy of the proposed blockchain-based incentive mechanism.
Fedcs: Efficient communication scheduling in decentralized federated learning

Decentralized federated learning is a training approach that prioritizes user data privacy protection, while also offering improved scalability and robustness. However, as the number of edge devices participating in training increases, a significant communication overhead arises among devices located in different geographical locations. Therefore, designing a well-thought-out gradient synchronization strategy is crucial for minimizing the overall communication overhead of training. To tackle this issue, this article introduces a 2D-Ring network structure based parameter synchronization strategy and a 2D-attention-based device placement algorithm, aiming to minimize communication overhead. The parameter synchronization strategy devises a two-layer circular communication architecture for the devices involved in training, thereby reducing the overall frequency of parameter synchronization in decentralized federated learning. By taking into account the total communication overhead and the device placement strategy, an optimization problem is formulated. Specifically, a 2D-attention neural network is constructed to optimize the device placement solution based on 2D-Ring network structure, leading to reduced communication overhead. Moreover, an evaluation model is designed to assess the communication overhead in a complex decentralized system during federated training. This enables precise determination of the total communication overhead throughout the training process, providing valuable insights for devising the device placement strategy. Extensive simulations confirm that the proposed approach achieves a substantial reductions of 55% and 64% in the total communication overhead for decentralized federated learning training with 50 and 100 devices, respectively.
Towards Scalable Multi-View Clustering via Joint Learning of Many Bipartite Graphs

This paper focuses on two limitations to previous multi-view clustering approaches. First, they frequently suffer from quadratic or cubic computational complexity, which restricts their feasibility for large-scale datasets. Second, they often rely on a single graph on each view, yet lack the ability to jointly explore many versatile graph structures for enhanced multi-view information exploration. In light of this, this paper presents a new Scalable Multi-view Clustering via Many Bipartite graphs (SMCMB) approach, which is capable of jointly learning and fusing many bipartite graphs from multiple views while maintaining high efficiency for very large-scale datasets. Different from the one-anchor-set-per-view paradigm, we first produce multiple diversified anchor sets on each view and thus obtain many anchor sets on multiple views, based on which the anchor-based subspace representation learning is enforced and many bipartite graphs are simultaneously learned. Then these bipartite graphs are efficiently partitioned to produce the base clusterings, which are further re-formulated into a unified bipartite graph for the final clustering. Note that SMCMB has almost linear time and space complexity. Extensive experiments on twenty general-scale and large-scale multi-view datasets confirm its superiority in scalability and robustness over the state-of-the-art.
Combining variational mode decomposition with regularisation techniques to denoise MRI data

In this paper, we propose a novel method for removing noise from MRI data by exploiting regularisation techniques combined with variational mode decomposition. Variational mode decomposition is a new decomposition technique for sparse decomposition of a 1D or 2D signal into a set of modes. In turn, regularisation is a method that can translate the ill-posed problem (e.g., image denoising) into a well-posed problem. The proposed method aims to remove the noise from the image in two steps. In the first step, the MR imaging data are decomposed by the 2D variational mode decomposition algorithm. In the second step, for effective suppression of Rician noise from MRI data, we used the fused lasso signal approximator with all modes acquired from the medical scan. The performance of the proposed approach was compared with state-of-the-art reference methods based on different metrics, that is, the peak signal-to-noise ratio, the structural similarity index metrics, the high-frequency error norm, the quality index based on local variance, and the sharpness index. The experiments were performed on the basis of both simulated and real images. The presented results prove the high denoising performance of the proposed algorithm; particularly under heavy noise conditions.
CT-Net: Asymmetric compound branch Transformer for medical image segmentation

The Transformer architecture has been widely applied in the field of image segmentation due to its powerful ability to capture long-range dependencies. However, its ability to capture local features is relatively weak and it requires a large amount of data for training. Medical image segmentation tasks, on the other hand, demand high requirements for local features and are often applied to small datasets. Therefore, existing Transformer networks show a significant decrease in performance when applied directly to this task. To address these issues, we have designed a new medical image segmentation architecture called CT-Net. It effectively extracts local and global representations using an asymmetric asynchronous branch parallel structure, while reducing unnecessary computational costs. In addition, we propose a high-density information fusion strategy that efficiently fuses the features of two branches using a fusion module of only 0.05M. This strategy ensures high portability and provides conditions for directly applying transfer learning to solve dataset dependency issues. Finally, we have designed a parameter-adjustable multi-perceptive loss function for this architecture to optimize the training process from both pixel-level and global perspectives. We have tested this network on 5 different tasks with 9 datasets, and compared to SwinUNet, CT-Net improves the IoU by 7.3% and 1.8% on Glas and MoNuSeg datasets respectively. Moreover, compared to SwinUNet, the average DSC on the Synapse dataset is improved by 3.5%.