Reviewerly/exHarmony huggingface dataset:
Identification of Robust Features for Classifying Spam and Ham Images using Transfer Learning

The utilization of transfer learning for image classification has gathered popularity and satisfactory results according to recent works. Nonetheless, exploring the extraction of image-based features prior to training have not been thoroughly investigated. In this work, we perform the classification of Spam versus Ham images through transfer learning using pre-trained neural network models, such as AlexNet and GoogleNet. Alongside this, we explore and compare the results obtained from image-based features, including Discrete Cosine Transform (DCT), Histograms, and Fast Fourier Transform (FFT), in order to identify the optimum approach. Experimental results suggest an accuracy of 98.56% for the top performing methodology, involving the use of DCT features with the GoogleNet model.
Adding Hardware Security into IoT-Blockchain Platforms

Blockchain is a technology that could help secure Internet of Things (IoT) systems and allow for easy interactions and communications among IoT devices without compromising their security. However, many nodes in IoT blockchain platforms lack adequate security at the hardware level; this is an important security aspect that can enhance both node security and overall trust in the platform. In this paper, we evaluate the viability of integrating a hardware secure module (HSM) and a hardware wallet (HW) into IoT blockchain applications. We provide a performance evaluation regarding the energy consumption of nodes that are equipped with HSM and HW and conduct security analyses. Finally, we discuss an example application, namely, secure community energy trading based on the deployment of an honesty-based distributed proof of authority (HDPoA) consensus algorithm where all nodes are equipped with an HSM and HW.
An Interactive IoT-Blockchain System for Big Data Management

An IoT-blockchain system is proposed to handle big data generated by a distributed network of sensors and controllers in an interactive manner. The system is designed using the Ethereum platform, which utilizes smart contracts, programmed in Solidity, to execute and manage data generated by IoT sensors and devices such as Raspberry Pi 4 and add-on hardware security modules. The proposed system runs a number of applications hosted by a local machine used to validate transactions. It then sends data to the rest of the network through InterPlanetary File System (IPFS) and Ethereum Swarm, forming a closed IoT ecosystem mastered by blockchain where a number of distributed IoT devices communicate and interact, thus forming a closed controlled environment. A prototype with four IoT handling units distributed over a wide geographical space was deployed to examine its feasibility, performance, and costs. Initial results indicate that big IoT data retrieval, storage is feasible, and interactivity is possible when meeting certain conditions of speed, cost and throughput.
Captive Animal Stress Study by Video Analysis

A fusion technique to synthesize the trajectory of a captive panda and its movement analysis to detect stress episodes are presented. The scene is a zoo environment that reproduces the natural habitat of a panda. It is visualized by four cameras. There were implemented the following steps: a) panda localization for each camera; b) trajectory synthesis from all views trajectories and c) trajectory filtering and analysis. Five categories of stress episodes characterized by movement patterns are identified.
MGTUNet: An new UNet for colon nuclei instance segmentation and quantification

Colorectal cancer (CRC) is among the top three malignant tumor types in terms of morbidity and mortality. Histopathological images are the gold standard for diagnosing colon cancer. Cellular nuclei instance segmentation and classification, and nuclear component regression tasks can aid in the analysis of the tumor microenvironment in colon tissue. Traditional methods are still unable to handle both types of tasks end-to-end at the same time, and have poor prediction accuracy and high application costs. This paper proposes a new UNet model for handling nuclei based on the UNet framework, called MGTUNet, which uses Mish, Group normalization and transposed convolution layer to improve the segmentation model, and a ranger optimizer to adjust the SmoothL1Loss values. Secondly, it uses different channels to segment and classify different types of nucleus, ultimately completing the nuclei instance segmentation and classification task, and the nuclei component regression task simultaneously. Finally, we did extensive comparison experiments using eight segmentation models. By comparing the three evaluation metrics and the parameter sizes of the models, MGTUNet obtained 0.6254 on PQ, 0.6359 on mPQ, and 0.8695 on R2. Thus, the experiments demonstrated that MGTUNet is now a state-of-the-art method for quantifying histopathological images of colon cancer.
A rehabilitation activity monitoring method based on Shallow-CNN

This paper proposes a shallow convolutional neural network (CNN) model to improve the efficiency and accuracy of real-time human activity recognition (HAR). In the traditional convolutional network, an Mix-Patch-Layer (MPL) block based on the attention mechanism is added to enhance the expressiveness of the network extracted features. This block makes the features in the network focus on the information between different parts of itself, which makes up for the loss of global information in temporal data features. Experiments show that the block can improve real-time human recognition accuracy and efficiency with a shallow network.
AUTOMED: Automated Medical Risk Predictive Modeling on Electronic Health Records

Electronic health records (EHR) have been widely applied to various tasks in the medical domain such as risk predictive modeling, which aims to predict further health conditions by analyzing patients' historical EHR. Existing work mainly focuses on modeling the sequential and temporal characteristics of EHR data with advanced deep learning techniques. However, the network architectures of these models are all manually designed based on experts' prior knowledge, which largely impedes non-experts from exploring this task. To address this issue, in this paper, we propose a novel automated risk prediction model named AutoMed to automatically search the optimal model architecture for modeling the complex EHR data and improving the performance of the risk prediction task. In particular, we follow the idea of neural architecture search to design a search space that contains three separate searchable modules. Two of them are used for analyzing sequential and temporal features of EHR data, respectively. The third is to automatically fuse both features together. Besides these three modules, AutoMed contains an embedding module and a prediction module. All the three searchable modules are jointly optimized in the search stage to derive the optimal model architecture. In such a way, the model design can be automatically achieved with few human interventions. Experimental results on three real-world datasets show that AutoMed outperforms state-of-the-art baselines in terms of PR-AUC, F1, and Cohen's Kappa. Moreover, the ablation study shows that AutoMed can obtain reasonable model architectures and offer useful insights to the future risk prediction model design.
Classifying Cervical Histopathological Whole Slide Images via Deep Multi-Instance Transfer Learning

The cervical histopathology analysis result is the gold standard for cervical cancer diagnosis. Conventional histopathological examination depends on pathologists’ observation under microscope, which is notoriously labor-intensive and subjective. The popularization of digital pathology technology makes the collection of the cervical histopathological whole slide images (WSIs) more convenient, so it has become possible to develop computer-aided diagnosis methods for cervical cancer. In this work, we first collected the cervical histopathological WSIs from 917 patients with pathological diagnosis through a retrospective study, of which 286 WSIs contained annotations of several lesion areas that were manually outlined by the pathologists. Then we proposed a method for classifying cervical histopathological WSIs by combining deep multi-instance transfer learning (DMITL) and support vector machine (SVM). The DMITL aimed for learning the representations of the WSIs, and the SVM was used for building the classification model of the WSIs. We generated the training and test sets based on our collected WSIs to train and evaluate our method. The validation results have shown that the good performance of our proposed method.
Temporal Hypergraph for Personalized Clinical Pathway Recommendation

Clinical pathway recommendation aims to recommend a set of treatment modalities or treatment procedures for a patient. In order to achieve personalized clinical pathway recommendation, more and more researchers try to mine similar clinical paths from the statistical features of patient-related clinical data (such as electronic health records), while ignoring the high-order interactions between patient-related medical entities, and the time-series change pattern of this high-order interaction relationship, resulting in the inability of existing methods to fully describe patient characteristics and to accurately recommend personalized clinical pathways. To solve the above problems, we propose a new personalized clinical pathway recommendation model TempHRec. To model the complex high-order relationship in clinical pathway, hypergraph technology is introduced to solve the problem that clinical events are correlated at the same time window. On this basis, we propose a temporal hypergraph, to construct a hypergraph for each timestamp with the help of a sliding time window to capture the timing information at the clinical pathway. Extensive experimental results on real-world datasets show that the proposed model achieves the best results compared to baseline methods.
A novel YOLO V5 framework for infrared image recognition

When it comes to face detection, problems of different illumination angles and uneven illumination distribution will inevitably produce some influences on the detection of human faces. In light of mentioned situation, this paper put forward an new framework of infrared face recognition algorithm based on YOLOv5 framework. Because of the small scale of indoor face dataset, the post-training model often overfits the brochure from the training set, leading to reduce generalization capability and other problems. Here, homomorphic filtering technology is used to increase the limited data sets. Then, the infrared face recognition model is trained based on YOLOv5. Experimental results indicate that the detection model achieves an accuracy of 93.55% and an F1-Sore of 87.37, which can adapt to detection tasks under different lighting changes and different shooting angles.
Deep Multi-Instance Learning with Adaptive Recurrent Pooling for Medical Image Classification

Recently, deep multi-instance neural networks have been successfully applied for medical image classification, where only image-level labels rather than fine-grained patch-level labels are available for use. One key issue for these multi-instance neural networks is how to aggregate all patch (instance) features into an entire image (bag) representation, referred to as multi-instance pooling, e.g., max, mean, and attention based pooling. Nevertheless, these multi-instance pooling operations do not take the structural information within an image into account. This is obviously inappropriate for medical image classification since there often exist certain dependencies among regional patches/lesions. We propose an adaptive recurrent pooling based deep multi-instance neural network in this paper. In this network, we first extract multi-view global structural features from every bag using the self-attention mechanism, and then aggregate these multi-view features into a whole bag representation based on the adaptive recurrent pooling operation in order to further capture the contextual information within the bag. Moreover, we introduce the cross-normalization operation used in the Unit Force Operated Vision Transformer into the self-attention module to reduce its computational complexity. We have experimentally evaluated the performance of the proposed network on three medical image datasets, namely UCSB breast, Messidor, and Colon cancer. The results demonstrate the advantage of our network over current state-of-the-art deep multi-instance networks in terms of classification accuracy and interpretability.
Few-Shot Radiology Report Generation via Knowledge Transfer and Multi-modal Alignment

Automatic radiology report generation aims at generating informative text from the given medical image, which could assist diagnosis and lighten the workload of radiologists. While some models have been proposed to study on this task, few of them paid attention to the radiology report generation for rare diseases, except for RareGen which solved this problem by enhancing the semantic representations of rare diseases. However, there still exist several problems to be addressed. The first lies in that modeling the correlations among diseases by current studies can result in the problem of frequency bias, which can affect the detection of rare diseases. The second lies in that how to get better representations of disease regions, so as to benefit their corresponding report generation in the decoding stage. To tackle these challenges, we propose a new few-shot radiology report generation model, namely FS-Gen. FS-Gen is assembled with one module for more effective detection of rare diseases in the encoding stage, and the other module for the better representation generation of disease regions in the decoding stage. Specifically, in the encoding stage, a cascade visual enhancement module is proposed to strengthen the correlations among diseases, without incurring the problem of frequency bias. On the other hand, in the decoding stage, a co-referential aligned topic generation module is introduced to simultaneously capture the location and semantic information of disease regions, by aligning the multimodal representations. Extensive experiments are conducted on real-world medical image datasets to demonstrate the effectiveness of our model.
Impact of Concatenation of Digital Craniocaudal Mammography Images on a Deep-Learning Breast-Density Classifier Using Inception-V3 and ViT

Breast density is an indicator of a patient’s predisposed risk of breast cancer. Although not fully understood, increased breast density increases the likelihood of developing breast cancer. Accurate assessment of breast density from mammogram images is a challenging task for the radiologist. A patient’s breast density is assigned to one of four categories outlined by Breast Imaging and Reporting Data Systems (BIRADS). There have been efforts to identify automated approaches to assist radiologists in the classification of a patient’s breast density. The interest in using deep learning to fulfill this need for an automated approach has seen a significant increase in recent years. The preprocessing techniques used to develop these deep learning approaches often have a profound impact on the model’s accuracy and clinical viability. In this paper, we outline a novel image preprocessing technique where we concatenate individual mammogram images and compare the results using this technique between Inception-v3 and a vision transformer (ViT). The results are compared using the area under (AUC) the receiver operator characteristics (ROC) curves and traditional accuracy metrics.
Patient Condition Change Network for Safe Medication Recommendation

As an important task of natural language processing, medication recommendation aims to recommend medication combinations according to the electronic health record, which can also be regarded as a multi-label classification task. But patients often have multiple diseases simultaneously, and the model must consider drug-drug interactions (DDI) of medication combinations when recommending medications, making medication recommendation more difficult. There is little existing work to explore the changes in patient conditions. However, these changes may point to future trends in patient conditions that are critical for reducing DDI rates in recommended drug combinations. In this paper, we proposed the Patient Condition Change Network (PCCNet), which models the current core medications of patient by mining the temporal and spatial changes of patient medication order and patient condition vector, and allocates some auxiliary medications as the currently recommended medication combination. The experimental results show that the proposed model greatly reduces the recommended DDI of medications while achieving results no lower than the state-of-the-art results. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> The code is available at https://github.com/master032/PCCNet
Gemini: a Real-time Video Analytics System with Dual Computing Resource Control

Edge-side real-time video analytics systems recognize spatial or temporal events (e.g., vehicle counting) in a video stream. To meet the delay requirement, existing systems in smart edge cameras conduct video preprocessing to filter out unnecessary frames and model inference using appropriately selected neural network (NN) models. Video preprocessing is instruction-intensive computing (IIC) and executed by the CPU of the edge camera, and model inference is data-intensive computing (DIC) and executed by the GPU of the edge camera. In this paper, we show that the analytics accuracy of existing systems can largely vary in fields. The root cause is that video analytics applications have different contents, which result in dynamic IIC and DIC workloads. Unfortunately, intelligent cameras in fields have fixed CPU and GPU resources and cannot effectively adapt to workload dynamics. We develop Gemini, a new real-time video analytics system enhanced by a dual-image FPGA. The newly developed dual-image FPGAs can be pre-configured with two FPGA images with a key advantage of negligible image switching time. We thus pre-configure one CPU image and one GPU image and elastically multiplex the dual CPU-GPU resources in the time dimension. The Gemini system design requires both hardware and software revisions. We overcame a challenge that the application development on different dual-image FPGAs is hardware-dependent. We develop a new abstraction of hardware functions to make the Gemini system hardware-agnostic. It is also a challenge to adapt to the dynamic workloads and optimize video analytics accuracy. We develop a bandit learning approach to capture content dynamics and conduct dual computing resource control. We implement Gemini and show that Gemini can improve the analytics accuracy to 90.35 %. We further evaluate Gemini by a case study where we use Gemini to support an intrusion detection application, and Gemini shows consistent high analytics accuracy.
Expert with Outlier Exposure for Continual Learning of New Diseases

Current intelligent diagnosis systems struggle to continually learn to diagnose more and more diseases due to catastrophic forgetting of old knowledge when learning new knowledge. Although storing small old data for subsequent continual learning can effectively help alleviate the forgetting issue, the heavy data imbalance between old classes and to-be-learned new classes in classifier training often causes biased prediction towards the new classes just learned by the updated classifier. In this study, an outlier detection technique is novelly applied to train an additional expert classifier for new classes to help alleviate the class imbalance issue and discriminate the learned new classes from old classes during inference (instead of the training phase). Specially, the stored small data of old classes are considered as outliers during training the expert classifier, such that the output probability distributions from the expert classifier are expected to be obviously different between test data of the old classes and those of the new classes. Such difference between old classes and new classes can be used to fine-tune the original output from the updated classifier which is responsible for prediction of all learned (old and new) classes. During inference, a novel ensemble strategy is proposed to combine the predictions from the updated classifier, the expert classifier, and the previously learned old classifier. The proposed learning and inference framework can be easily combined with existing continual learning strategies. Empirical evaluations on three medical image datasets and one natural image dataset show that the proposed framework can effectively improve continual learning performance.
Image Enhancement Methods for Anterior Segment Photographed Images

In the digital age, the use of smartphone digital camera images as an imaging modality for early eye disease screening and detection has been widely explored. Nevertheless, the process of taking pictures often raises some issues that may be related to the dynamic range limitations of cameras and uneven lighting, which may cause uneven illumination in the images. Therefore, image enhancement is further needed to improve the visual quality of the images and to provide sufficient detail for basic computer vision tasks including segmentation. This work studies the effect of image enhancement methods on the segmentation results of anterior segment photographed image (ASPI) ocular regions with nonuniform illumination. During the pre-processing stage, the image enhancement methods are performed on the nonuniform illumination ASPI. Those images are later trained using a deep learning approach for semantic segmentation. Based on the network performance, the Multiscale Retinex with Chromaticity Preservation (MSRCP) method gives the best segmentation results for all classes with mean accuracy of 0.921, mean IoU of 0.805 and mean BFScore of 0.717. In addition, the image quality assessment is performed on the ASPI with image enhancement methods. It is found that the images using MSRCP method had the lowest values for Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE), Naturalness Image Quality Evaluator (NIQE), and Perception-based Image Quality Evaluator (PIQE), indicating the highest image quality compared to the other methods. For future work, the segmented region of the eyes, including the sclera, iris, and pupil, can be utilized as a region of interest (ROI) to detect various anterior eye diseases.