Reviewerly/exHarmony huggingface dataset:
Ensembling to Leverage the Interpretability of Medical Image Analysis Systems

Along with the increase in the accuracy of artificial intelligence systems, complexity has also risen. Despite high accuracy, high-risk decision-making requires explanations about the model’s decision, which often take the form of saliency maps. This work examines the efficacy of ensembling deep convolutional neural networks to leverage explanations, under the concept that ensemble models are combinatory informed. A novel approach is presented for aggregating saliency maps derived from multiple base models, as an alternative way of combining the different perspectives that several competent models offer. The proposed methodology lowers computation costs, while allowing for the combinations of maps of various origins. Following a saliency map evaluation scheme, four tests are performed over three image datasets, two medical image datasets and one generic. The results suggest that interpretability is improved by combining information through the aggregation scheme. The discussion that follows provides insights into the inner workings behind the results, such as the specific combination of the interpretability and ensemble methods, and offers useful suggestions for future work.
Sensor Selection by Greedy Method for Linear Dynamical Systems: Comparative Study on Fisher-Information-Matrix, Observability-Gramian and Kalman-Filter-Based Indices

Objective functions for sensor selection are investigated in linear time-invariant systems with a large number of sensor candidates.This study compared the performance of sensor sets obtained using three types of D-optimality-based indices as objective functions for sensor selection based on the greedy method.The compared indices are computed based on the snapshot-to-snapshot Fisher information matrix, the observability Gramian and the Kalman filter-based matrix.Both random systems and systems with eigenmodes are considered, indices for selecting the best-performing sensor set for each are identified, as well as computational complexity and corresponding wall clock times.The sensor optimized for each index works best for that index, as expected.We also clarified the trend of the sensor sets selected by the greedy method based on each objective function in terms of the other objective function.The compared indices are computed based on the snapshot-to-snapshot Fisher information matrix, the observability Gramian and the Kalman filter-based matrix.Both random systems and systems with eigenmodes are considered, indices for selecting the best-performing sensor set for each are identified, as well as computational complexity and corresponding wall clock times. INDEX TERMSGreedy method, optimal design of experiment, sensor selection, linear time-invariant problem, Kalman filter I. INTRODUCTION 1 C APTURING the internal states of various dynamical 2 systems as easily as possible is one of the fundamen-3 tal challenges of science and engineering.Various physical
A Self-Adaptive Object Detection Network for Aerial Images Based on Feature Enhancement

Object detection in aerial images has received increasing attention for its widely applications. However, it is still a challenging task when dealing with difficult cases, such as the large variations of object sizes, the complex backgrounds in the large-scale views, as well as small targets packed in dense. In this regard, we propose a self-adaptive object detection network for aerial images based on feature enhancement, including region crop module with soft-attention (RCP), feature enhancement module (FET), self adaptive feature extraction module (SAE). Firstly, RCP module with soft-attention is explored to roughly crop the dense subregion and sparse subregion into patches accoridng to the variance of feature maps for the following feature extraction and detection. Secondly, FET module is proposed to acquire more semantic details by feature enhancement, which makes up the information loss during downsampling, especially for small objects in dense regions. Finally, SAE module is explored to effectively identify multiple target regions and single target regions in the dense patches. The similarity of adjacent dense areas in dense patches is calcluated, and the search range is gradually narrowed by continuously merging the areas with the largest similarity. The teacher-network and student-network are used to extract features for multiple target regions and single target regions, respectively. The proposed design improves the accuracy of real-time detection under the drone’s perspective. A large number of experiments and comprehensive evaluations on the VisDrone2019-DET dataset have shown the effectiveness and adaptability of the proposed method. Our source codes have been available at https://github.com/zhaokai152.
MAPD: An FPGA-Based Real-Time Video Haze Removal Accelerator Using Mixed Atmosphere Prior

Real-time video dehazing plays a key role in helping autonomous driving detect pedestrians or obstacles in severe foggy weather to prevent potential hazards. Existing video dehazing methods achieve good restoration performance but still suffer from oversaturation and low dehazing speed, especially for high-definition (HD, high-resolution) videos. In this article, we propose a mixed atmosphere prior information video dehazing accelerator (MAPD) and implement it on field programmable gate array (FPGA) to achieve real-time haze removal for HD video. MAPD provides a mixed atmospheric light model by applying heterogeneous atmospheric light in the foreground area to balance brightness deviation, and maintaining the global atmospheric light in the background region. Considering the parallel characteristics of FPGA, MAPD leverages the redundant information between adjacent frames to accelerate the dehazing process and designs an indirect transmission estimation to decrease resource consumption. For comparison, we also implement six dehazing solutions (DCP, color ellipsoid prior (CEP), RDCP, FFVD, MHVD, and REFD) on FPGA, and deploy a graphics processing unit (GPU)-based method <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(D^{4})$ </tex-math></inline-formula> on the platform with Nvidia 3080 GPU. Experiments using two widely used benchmarks show that MAPD increases the performance by up to 36.5%, 53.5%, 36.3%, 33.3%, 11.9%, and 23.3%, decreases resource consumption by up to 79.7%, 75.0%, 74.8%, 25.6%, 22.6%, and 73.9% and enhances FPS for HD videos by up to 241.6%, 145.9%, 151.7%, 68.6%, 50.6%, and 62.4%, compared with DCP, CEP, RDCP, FFVD, MHVD, and REFD. Compared to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D^{4}$ </tex-math></inline-formula> , MAPD also promotes the dehazing performance by up to 21.8%, and increases FPS by up to 487.0%.
Remote Sensing Image Registration Based Upon Extensive Convolutional Architecture With Transfer Learning and Network Pruning

Accurate registration of remote sensing images through automatic pipelines remains challenging. While bottlenecks have deferred the advancement of traditional approaches, more attention has been attracted on the incorporation of deep learning knowledge into the image registration process. This paper develops an efficient remote sensing image registration framework based upon a convolutional neural network (CNN) architecture, which is called the geometric correlation regression with dense feature network (GcrDfNet). To acquire deep features of remote sensing images, the DenseNet associated with partial transfer learning and partial parameter fine-tuning is exploited. The feature maps derived from the sensed and reference images are further analyzed using a geometric matching model followed by linear regression to compute their correlation and to estimate the transformation coefficients. Subsequently, a network pruning scheme is investigated to diminish the model structure while moderately escalating the registration accuracy. A wide variety of multitemporal and multispectral remote sensing images with distinctive scenarios were employed to evaluate the proposed image registration system. The ensemble parameter compression ratio was approximately 2.12 while slightly reducing the registration error. Experimental results indicated that our GcrDfNet outperformed the traditional and deep learning-based state-of-the-art methods both qualitatively and quantitatively. It is believed that this new image registration model is promising in many remote sensing image processing and analysis applications.
SCAUIE-Net: Underwater Image Enhancement Method Based on Spatial and Channel Attention

Underwater image enhancement is a Low-Level Vision task that plays an important role in marine resource development, but the light absorption and scattering cause severe underwater image quality degradation. To solve these problems, this paper proposes a neural network based on a spatial and channel attention module that reinforces the network’s attention to channel and spatial information. The network’s Confidence Generator can precisely extract feature maps from multi-scale underwater images. Meanwhile, we propose a new training loss function by mixing perceptual, MS-SSIM and MAE loss functions to further improve the contrast in high-frequency, colors and luminance. For training, this paper also uses a feature fusion strategy: Firstly, augmenting the training underwater images by Gamma Correction, White Balance and Histogram Equalization algorithms to remove color cast, lighten up dark regions and improve the contrast. Then, fusing the enhancing images with confidence maps predicted from the Generator. The network was validated in the UIEB dataset and obtains efficient improvements on Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) metrics, yielding a PSNR of 22.9286 and SSIM of 0.9290. Experimental results on real-world underwater images demonstrate that the proposed method performs well on different underwater scenes.
Improved Image Classification With Token Fusion

In this paper, we propose a method to improve image classification performance using the fusion of CNN and transformer structure. In the case of CNN, information about a local area on an image can be extracted well, but global information extraction is limited. On the other hand, the transformer has an advantage in global information extraction, but it requires much memory compared to CNN. We apply CNN on an image and consider the feature vector of each pixel on the resulting feature map by CNN as a token. At the same time, the image is divided into patches, and each patch is considered a token, like a transformer. Tokens by CNN and transformer have advantages in extracting local and global information, respectively. We assume that the combination of these two types of tokens will have an improved characteristic, and we show it through experiments. We propose three methods to fuse tokens having different characteristics: (1) late token fusion with parallel structure, (2) early token fusion (3) token fusion in layer-by-layer. The proposed method shows the best classification performance in experiments using ImageNet-1K.
Endeavors for Quality Education in Koshi Province (A Practicum Report)

This practicum exercise was completed at the Ministry of Social Development, Koshi Province (formerly Province 1). The report is a result of quantitative analyses of policies and programs of the Ministry of Social Development in Koshi Province executed in the year 2018-19. It investigates the progress made towards fulfilling two targets(4.6 and 4.a) of SDG 4, which are related to literacy rates and building effective learning environment in schools. The report concludes that the increase in enrollment is due to policy interventions of the provincial government in 2018-19, and the grants for the construction of separate girls' toilets seem to play the most significant role in the surge of enrollments. However, the literacy rate in males is significantly higher than that in females in the year 2021 which is an issue to be further analyzed through the lens of equity in policy outcomes.
Object-Based Hybrid Deep Learning Technique for Recognition of Sequential Actions

Using different objects or tools to perform activities in a step-by-step manner is a common practice in various settings, including workplaces, households, and recreational activities.However, this approach can pose several challenges and potential hazards if the correct sequence of actions is not followed and the object or tool is not used in the appropriate sequence; therefore, it must be addressed to ensure safety and efficiency.These issues have garnered significant attention in recent years.Previous research has relied on using body keypoints to detect actions, but not the objects or tools used during activity.As a result, the lack of a system to identify the target objects or tools being used while performing tasks increases the risk of accidents and mishaps during the process.This study suggests a possible solution to the aforementioned issue by introducing a model that is both efficient and durable.The model utilizes video data to monitor and identify daily activities, as well as the objects involved in the process, thus enabling real-time feedback and alerts to enhance safety and productivity.The suggested model separates the overall recognition process into two components.Firstly, it utilizes the advanced BlazePose architecture for pose estimation, and interpolates any undetected and wrong-detected landmarks to enhance the precision of the posture estimation.After this, the features are forwarded to a long short-term memory network to identify the actions performed during the activity.Secondly, the model also employs an enhanced YOLOv4 algorithm for object detection, to accurately identify the objects used in the course of the activity.Finally, a durable and efficient activity recognition model has been developed, which achieves 95.91% accuracy rate in identifying actions, a mean average precision score of 97.68% for detecting objects, and overall activity recognition model that is capable of processing at a rate of 10.47 frames per second.
An Analytic End-to-End Collaborative Deep Learning Algorithm

In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning is used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset is presented to demonstrate the performance of the proposed approach. In addition, an online kinematics control task of a UR5e robot arm is formulated to illustrate the regression approximation and online updating ability of the proposed algorithm.
Revisiting Gradient Regularization: Inject Robust Saliency-Aware Weight Bias for Adversarial Defense

Despite regularizing the Jacobians of neural networks to enhance model robustness has directly theoretical correlation with model prediction stability, a large defense performance gap exists when compared to the empirically perturbation-based adversarial training e.g. PGD-based, which enjoys nice discriminative saliency maps as well. To mitigate this issue, in this paper we first analyze the dilemma that the gradient map of its resulting model has no content hierarchy to mark out salient profile of input, as a negative signal of the obstructive for effective adversarial defense. Based on this, we argue that incorporating robust gradient-based saliency properties into regularized training may be helpful to reduce the performance gap. Specifically, we propose a simple method called Saliency-aware Gradient Regularization (SAGR), where a biased weight distribution strategy is introduced on positive gradient to structure and increase the impact of class-gradient components inside the Jacobian of model. The strategy maintains the dominant role of saliency-critical true-class gradient in learning process and differentiates diverse importance of gradient sensitivities that would localize input salient areas. Herein we interpret the sharpness of true-class sensitivity as robust recognition of more learning-relevant features e.g., regions containing dominant object in image for classification. Instead, false-class parts are considered as recognition-irrelevant nuisance factors e.g. the backgrounds, which are thus depressed with more strength. Experimental results demonstrate the efficacy of the proposed method and validate that distinguishment of sensitivities could further yield more robustness gain and sharper gradient saliency map.
On the Role of ViT and CNN in Semantic Communications: Analysis and Prototype Validation

Semantic communications have shown promising advancements by optimizing source and channel coding jointly. However, the dynamics of these systems remain understudied, limiting research and performance gains. Inspired by the robustness of Vision Transformers (ViTs) in handling image nuisances, we propose a ViT-based model for semantic communications. Our approach achieves a peak signal-to-noise ratio (PSNR) gain of +0.5 dB over convolutional neural network variants. We introduce novel measures, average cosine similarity and Fourier analysis, to analyze the inner workings of semantic communications and optimize the system’s performance. We also validate our approach through a real wireless channel prototype using software-defined radio (SDR). To the best of our knowledge, this is the first investigation of the fundamental workings of a semantic communications system, accompanied by the pioneering hardware implementation. To facilitate reproducibility and encourage further research, we provide open-source code, including neural network implementations and LabVIEW codes for SDR-based wireless transmission systems.
Task Alignment Interaction and Cross-Scale Guided Enhancement for Remote Sensing Object Detection

Object detection is a fundamental task in the analysis and interpretation of remote sensing images. However, compared to natural images, remote sensing images are characterized by broad diversity in object scales, fuzzy objects, and complex background, which bring great challenges to object detection. For overcoming the above problems, a task alignment interaction and cross-scale guidance enhancement network (TCNet) is proposed in this letter. Firstly, a generalized mean spatial pyramid pooling (GeMSPP) is designed and embedded in the backbone to adapt to changes of complex environment and reduce loss of features. Secondly, cross-scale guided enhancement network (CGEN) is proposed to generate high-quality non-aliasing multi-scale target features for each feature level by guiding the fusion of deep features and enhancing feature expression. Thirdly, Task alignment interactive head (TAIH) is adopted to enhance the classification and regression accuracy of the prediction box, so as to suppress background interference and highlight object features. Experiments conducted on public DIOR and RSOD datasets illustrate that the proposed modules can effectively improve the accuracy of detection and our network has superior performance compared with other state-of-the-art detectors.
<i>Mex</i>+<i>Sync</i>: Software Covert Channels Exploiting Mutual Exclusion and Synchronization

The mutual exclusion and synchronization mechanism (MESM) is a basic process management mechanism in modern operating systems. However, we first found that MESM has serious vulnerabilities. In this article, a new set of MESM-based software covert channels, named Mex+Sync, are proposed to transmit confidential information. Mex+Sync offer several advantages: 1) the covert channels are constructed at the software level and can be deployed on any hardware; 2) the closed shared resource ensures the quality of the channels with low interference and makes them hard to be detected; and 3) the attack utilizes the system’s software resources which abound and hence difficult to isolate. We build the covert channels on Linux and Windows with the following MESMs: <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">flock</monospace> , <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FileLockEX</monospace> , <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Mutex</monospace> , <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Semaphore</monospace> , <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event</monospace> , and <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">WaitableTimer</monospace> . Experimental results demonstrate that these covert channels can achieve transmission rate of 13.105, 12.383, and 6.552 kb/s, respectively, in the scenarios of local, cross-sandbox and cross-virtual machine, all with bit error rate under 1%. Meanwhile, we carefully analyze and verify the impact of noise on channel performance.
An automatic segmentation of breast ultrasound images using U-Net model

Medical imaging, like ultrasound, gives a good visual picture of how an organ works. However, a radiologist has a hard time and takes a long time to process these images, which delays the diagnosis. Several automated methods for detecting and segmenting breast lesions have been developed. Nevertheless, due to ultrasonic artifacts and the intricacy of lesion forms and locations, the segmentation of lesions or tumors from breast ultrasonography remains an open issue. Medical image segmentation has seen a breakthrough thanks to deep learning. U-Net is the most noteworthy deep network in this regard. The traditional U-Net design lacks precision when dealing with complex data sets, despite its exceptional performance in segmenting multimedia medical images. To reduce texture detail redundancy and avoid overfitting, we suggest developing the U-Net architecture by including dropout layers after each max pooling layer. Batchnormalization layers and a binary cross-entropy loss function were used to preserve breast tumor texture features and edge attributes while decreasing computational costs. We used the breast ultrasound dataset of 780 images with normal, benign, or malignant tumors. Our model showed superior segmentation results for breast ultrasound pictures compared to previous deep neural networks. Quantitative measures, accuracy, and IoU values were utilized to evaluate the suggested model?s effectiveness. The results were 99.34% and 99.60% for accuracy and IoU. The results imply that the augmented U-Net model that has been suggested has high diagnostic potential in the clinic since it can correctly segment breast lesions.