Reviewerly/exHarmony huggingface dataset:
Towards a data privacy-predictive performance trade-off

Machine learning is increasingly used in the most diverse applications and domains, whether in healthcare to predict pathologies or in the financial sector to detect fraud. Data utility is one of the linchpins for efficiency and accuracy in machine learning. However, when it contains personal information, full access may be restricted due to laws and regulations aiming to protect individuals' privacy. Therefore, data owners must ensure that any data shared guarantees such privacy. Removal or transforming private information (de-identification) are among the most common techniques. Intuitively, one can anticipate that reducing detail or distorting information would result in losses for predictive model performance, which might be reflected in a trade-off. However, previous work concerning classification tasks using de-identified data generally demonstrates that predictive performance can be preserved in specific applications, wherefore no trade-off is proven. In this paper, we aim to evaluate the existence of a trade-off between data privacy and predictive performance in classification tasks. We leverage a large set of privacy-preserving techniques and learning algorithms to assess re-identification ability and the impact of transformed variants on predictive performance. Unlike previous literature, we confirm that the higher the level of privacy (lower re-identification risk), the higher the impact on predictive performance, pointing towards clear evidence of a trade-off.
Human attention based movie summarization: Dataset and baseline model

A movie summarization model can automatically edit a condensed version of a movie by selecting keyframes. Some previous works have proposed some movie summarizers based on traditional methods or recent neural networks and achieved some progress. Despite the demonstrated successes, there are some limitations: (1) previous works mainly resort to hand-crafted heuristics and most of them are unsupervised; (2) currently there is no publicly suitable dataset available for the supervised movie summarization; (3) existing works only focus on the movies themselves while neglecting the audiences, who have the most to say in which part of the movie is more attractive. To break through the aforementioned limitations, we establish a movie summarization dataset Movie50 and propose a novel human attention based annotation pipeline. Furthermore, we propose the A/V-MSNet, an audiovisual neural network that takes advantage of spatio-temporal visual and auditory information to better simulate human attention as well as exploit more plentiful information. The network is designed, trained end-to-end, and evaluated on the public dataset and our dataset. Extensive experiments demonstrate the superiority of the proposed method.
Radio-Frequency Fingerprinting for Distributed IoT Networks: Authentication and QoS Optimization

With the advance of the Internet of Things (IoT), limited wireless resources cannot meet the requirements of all users. User authentication can identify the users who are not authenticated and only allocate the resources to authenticated users. However, traditional authentication methods require complicated computations that restrict the wide deployment of low-cost devices in IoT applications. This article proposes a federal learning-based authentication method with radio-frequency fingerprinting recognition under a distributed architecture, which is novel and different from the previous works since these works primarily focus on the network with a centralized node. We propose a distributed optimization problem in a hybrid local and edge computing scenario to minimize the network delay. By transforming the original problem into its Lagrange function and calculating its derivatives, we propose a method of solving the optimization problem and prove that the proposed method could reduce the unnecessary consumption of wireless resources and effectively improve the entire network's performance. The analytical results also indicate that the proposed method could outperform the benchmark authentication methods in reducing network delay.
Analyzing and Explaining Black-Box Models for Online Malware Detection

In recent years, a significant amount of research has focused on analyzing the effectiveness of machine learning (ML) models for malware detection. These approaches have ranged from methods such as decision trees and clustering to more complex approaches like support vector machine (SVM) and deep neural networks. In particular, neural networks have proven to be very effective in detecting complex and advanced malware. This, however, comes with a caveat. Neural networks are notoriously complex. Therefore, the decisions that they make are often just accepted without questioning why the model made that specific decision. The black box characteristic of neural networks has challenged researchers to explore methods to explain black-box models such as SVM and neural networks and their decision-making process. Transparency and explainability give the experts and malware analysts assurance and trustworthiness about the ML models’ decisions. In addition, it helps in generating comprehensive reports that can be used to enhance cyber threat intelligence sharing. As such, this much needed analysis drives our work in this paper to explore the explainability and interpretability of ML models in the field of online malware detection. In this paper, we used the Shapley Additive exPlanations (SHAP) explainability technique to achieve efficient performance in interpreting the outcome of different ML models such as SVM Linear, SVM-RBF (Radial Basis Function), Random Forest (RF), Feed-Forward Neural Net (FFNN), and Convolutional Neural Network (CNN) models trained on an online malware dataset. To explain the output of these models, explainability techniques such as KernalSHAP, TreeSHAP, and DeepSHAP are applied to the obtained results.
Energy-Efficient UAVs Coverage Path Planning Approach

Unmanned aerial vehicles (UAVs), commonly known as drones, have drawn significant consideration thanks to their agility, mobility, and flexibility features.They play a crucial role in modern reconnaissance, inspection, intelligence, and surveillance missions.Coverage path planning (CPP) which is one of the crucial aspects that determines an intelligent system's quality seeks an optimal trajectory to fully cover the region of interest (ROI).However, the flight time of the UAV is limited due to a battery limitation and may not cover the whole region, especially in large region.Therefore, energy consumption is one of the most challenging issues that need to be optimized.In this paper, we propose an energy-efficient coverage path planning algorithm to solve the CPP problem.The objective is to generate a collision-free coverage path that minimizes the overall energy consumption and guarantees covering the whole region.To do so, the flight path is optimized and the number of turns is reduced to minimize the energy consumption.The proposed approach first decomposes the ROI into a set of cells depending on a UAV camera footprint.Then, the coverage path planning problem is formulated, where the exact solution is determined using the CPLEX solver.For small-scale problems, the CPLEX shows a better solution in a reasonable time.However, the CPLEX solver fails to generate the solution within a reasonable time for large-scale problems.Thus, to solve the model for large-scale problems, simulated annealing for CPP is developed.The results show that heuristic approaches yield a better solution for large-scale problems within a much shorter execution time than the CPLEX solver.Finally, we compare the simulated annealing against the greedy algorithm.The results show that simulated annealing outperforms the greedy algorithm in generating better solution quality.
MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

We present MMNeRF, a simple yet powerful learning framework for highly photo-realistic novel view synthesis by learning Multi-modal and Multi-view features to guide neural radiance fields to a generic model. Novel view synthesis has achieved great improvement with the significant success of NeRF-series methods. However, how to make the method generic across scenes has always been a challenging task. A good idea is to introduce 2D image features as prior knowledge for adaptive modeling, yet RGB features lack geometry and 3D spatial information, which causes shape-radiance ambiguity issues and lead to blurry and low-resolution results in the synthesis images. We propose a multi-modal multi-view method to make up for the existing methods. Specifically, we introduce depth features besides RGB features into the model and effectively fuse these multi-modal features by modality-based attention. Furthermore, Our framework innovatively adopts the transformer encoder to fuse multi-view features and uses the transformer decoder to adaptively incorporate the target view with global memory. Extensive experiments are carried out on both categories-specific and category-agnostic benchmarks, and the results demonstrate that our MMNeRF achieves state-of-the-art neural rendering performance.
Deformable image registration based on single or multi-atlas methods for automatic muscle segmentation and the generation of augmented imaging datasets

Muscle segmentation is a process relied upon to gather medical image-based muscle characterisation, useful in directly assessing muscle volume and geometry, that can be used as inputs to musculoskeletal modelling pipelines. Manual or semi-automatic techniques are typically employed to segment the muscles and quantify their properties, but they require significant manual labour and incur operator repeatability issues. In this study an automatic process is presented, aiming to segment all lower limb muscles from Magnetic Resonance (MR) imaging data simultaneously using three-dimensional (3D) deformable image registration (single inputs or multi-atlas). Twenty-three of the major lower limb skeletal muscles were segmented from five subjects, with an average Dice similarity coefficient of 0.72, and average absolute relative volume error (RVE) of 12.7% (average relative volume error of -2.2%) considering the optimal subject combinations. The multi-atlas approach showed slightly better accuracy (average DSC: 0.73; average RVE: 1.67%). Segmented MR imaging datasets of the lower limb are not widely available in the literature, limiting the potential of new, probabilistic methods such as deep learning to be used in the context of muscle segmentation. In this work, Non-linear deformable image registration is used to generate 69 manually checked, segmented, 3D, artificial datasets, allowing access for future studies to use these new methods, with a large amount of reliable reference data.
Development of multiple <scp>AI</scp> pipelines that predict neoadjuvant chemotherapy response of breast cancer using H&amp;E‐stained tissues

In recent years, the treatment of breast cancer has advanced dramatically and neoadjuvant chemotherapy (NAC) has become a common treatment method, especially for locally advanced breast cancer. However, other than the subtype of breast cancer, no clear factor indicating sensitivity to NAC has been identified. In this study, we attempted to use artificial intelligence (AI) to predict the effect of preoperative chemotherapy from hematoxylin and eosin images of pathological tissue obtained from needle biopsies prior to chemotherapy. Application of AI to pathological images typically uses a single machine-learning model such as support vector machines (SVMs) or deep convolutional neural networks (CNNs). However, cancer tissues are extremely diverse and learning with a realistic number of cases limits the prediction accuracy of a single model. In this study, we propose a novel pipeline system that uses three independent models each focusing on different characteristics of cancer atypia. Our system uses a CNN model to learn structural atypia from image patches and SVM and random forest models to learn nuclear atypia from fine-grained nuclear features extracted by image analysis methods. It was able to predict the NAC response with 95.15% accuracy on a test set of 103 unseen cases. We believe that this AI pipeline system will contribute to the adoption of personalized medicine in NAC therapy for breast cancer.
Deep Encrypted Traffic Detection: An Anomaly Detection Framework for Encryption Traffic Based on Parallel Automatic Feature Extraction

With an increasing number of network attacks using encrypted communication, the anomaly detection of encryption traffic is of great importance to ensure reliable network operation. However, the existing feature extraction methods for encrypted traffic anomaly detection have difficulties in extracting features, resulting in their low efficiency. In this paper, we propose a framework of encrypted traffic anomaly detection based on parallel automatic feature extraction, called deep encrypted traffic detection (DETD). The proposed DETD uses a parallel small-scale multilayer stack autoencoder to extract local traffic features from encrypted traffic and then adopts an L1 regularization-based feature selection algorithm to select the most representative feature set for the final encrypted traffic anomaly detection task. The experimental results show that DETD has promising robustness in feature extraction, i.e., the feature extraction efficiency of DETD is 66% higher than that of the conventional stacked autoencoder, and the anomaly detection performance is as high as 99.998%, and thus DETD outperforms the deep full-range framework and other neural network anomaly detection algorithms.
A Survey on Attention mechanism in NLP

The Attention mechanism is to screen out a small amount of important information from a large amount of information and focus on this important information while ignoring most of the unimportant information, it is a resource allocation mechanism. The attention mechanism is mainly used in the field of Natural Language Processing to assign attention weight to text, the essence of this is to change from focusing on the whole to focusing on the key parts. This paper starts from the attention mechanism framework, Classification is conducted according to whether we can use the BackPropogation algorithm directly to calculate the gradient, whether the input is global or local participation, and the attention implementation function, which gives a systematic overview of the mainstream models of attention mechanisms. Finally, by combining the application of the attention mechanism in various fields, briefly explains the challenges of the attention mechanism.
VLCA: vision-language aligning model with cross-modal attention for bilingual remote sensing image captioning

In the field of satellite imagery, remote sensing image captioning (RSIC) is a hot topic with the challenge of overfitting and difficulty of image and text alignment. To address these issues, this paper proposes a vision-language aligning paradigm for RSIC to jointly represent vision and language. First, a new RSIC dataset DIOR-Captions is built for augmenting object detection in optical remote (DIOR) sensing images dataset with manually annotated Chinese and English contents. Second, a Vision-Language aligning model with Cross-modal Attention (VLCA) is presented to generate accurate and abundant bilingual descriptions for remote sensing images. Third, a cross-modal learning network is introduced to address the problem of visual-lingual alignment. Notably, VLCA is also applied to end-to-end Chinese captions generation by using the pre-training language model of Chinese. The experiments are carried out with various baselines to validate VLCA on the proposed dataset. The results demonstrate that the proposed algorithm is more descriptive and informative than existing algorithms in producing captions.
Highly reconfigurable oscillator-based Ising Machine through quasiperiodic modulation of coupling strength

Ising Machines (IMs) have the potential to outperform conventional Von-Neuman architectures in notoriously difficult optimization problems. Various IM implementations have been proposed based on quantum, optical, digital and analog CMOS, as well as emerging technologies. Networks of coupled electronic oscillators have recently been shown to exhibit characteristics required for implementing IMs. However, for this approach to successfully solve complex optimization problems, a highly reconfigurable implementation is needed. In this work, the possibility of implementing highly reconfigurable oscillator-based IMs is explored. An implementation based on quasiperiodically modulated coupling strength through a common medium is proposed and its potential is demonstrated through numerical simulations. Moreover, a proof-of-concept implementation based on CMOS coupled ring oscillators is proposed and its functionality is demonstrated. Simulation results show that our proposed architecture can consistently find the Max-Cut solution and demonstrate the potential to greatly simplify the physical implementation of highly reconfigurable oscillator-based IMs.
An innovative traffic light recognition method using vehicular ad-hoc networks

Car congestion is a pressing issue for everyone on the planet. Car congestion can be caused by accidents, traffic lights, rapid accelerations, deceleration, and hesitation of drivers, as well as a small low-carrying capacity road without bridges. Increasing road width and constructing roundabouts and bridges are solutions to car congestion, but the cost is significant. TLR (traffic light recognition) reduces accidents and traffic congestion caused by traffic lights (TLs). Image processing with convolutional neural network (CNN) lakes dealing with harsh weather. A semi-automatic annotation for traffic light detection employs a global navigation satellite system, raising the cost of automobiles. Data was not collected in harsh conditions, and tracking was not supported. Integrated channel feature tracking (ICFT) combines detection and tracking, but it does not support sharing information with neighbors. This study used vehicular ad-hoc networks (VANETs) for VANET traffic light recognition (VTLR). Information exchange as well as monitoring of the TL status, time remaining before a change, and recommended speeds are supported. Based on testing, it has been determined that VTLR performs better than semi-automatic annotation, image processing with CNN, and ICFT in terms of delay, success ratio, and the number of detections per second.
Measuring Similarity of Educational Items Using Data on Learners’ Performance and Behavioral Parameters: Application of New Models SCNN-Cosine and Fuzzy-Kappa

Currently, educational systems typically contain a wide range of items drawn from students' learning experiences, questions, problems, and other items.They are collected using modern computer-assisted learning platforms.These items can be grouped into Knowledge Components (KCs) using a similarity measure based on the learner's performance and the number of correct or incorrect answers.This approach leads to wrong assignment of items to the cluster with less interpretation of learners' skills.Consequently, looking for other performance parameters to enhance accuracy has become paramount.The subject's ability to respond to each item called latent trait includes hints, attempts and time response, which can ensure better accuracy and enhance clustering of different items into KCs.In this paper, we propose new similarity models based on combining Siamese Convolutional Neural Network (SCNN), cosine similarity and fuzzy logic systems for Kappa inter-rater agreement.These models aim to achieve efficient item grouping based on learners' performance and behavioral parameters.The proposed approach has been tested on Algebra and French Language datasets to experiment its performances.Experimental findings show the superiority of SCNN-Cosine in terms of clustering analysis measured by ARI, correlation, Calinski-Harabasz and Davies-Bouldin indices, less complexity and faster response, compared to Fuzzy-Kappa.
Technical note: colab_zirc_dims: a Google Colab-compatible toolset for automated and semi-automated measurement of mineral grains in laser ablation–inductively coupled plasma–mass spectrometry images using deep learning models

Abstract. Collecting grain measurements for large detrital zircon age datasets is a time-consuming task, but a growing number of studies suggest such data are essential to understanding the complex roles of grain size and morphology in grain transport and as indicators for grain provenance. We developed the colab_zirc_dims Python package to automate deep-learning-based segmentation and measurement of mineral grains from scaled images captured during laser ablation at facilities that use Chromium targeting software. The colab_zirc_dims package is implemented in a collection of highly interactive Jupyter notebooks that can be run either on a local computer or installation-free via Google Colab. These notebooks also provide additional functionalities for dataset preparation and for semi-automated grain segmentation and measurement using a simple graphical user interface. Our automated grain measurement algorithm approaches human measurement accuracy when applied to a manually measured n=5004 detrital zircon dataset. Errors and uncertainty related to variable grain exposure necessitate semi-automated measurement for production of publication-quality measurements, but we estimate that our semi-automated grain segmentation workflow will enable users to collect grain measurement datasets for large (n≥5000) applicable image datasets in under a day of work. We hope that the colab_zirc_dims toolset allows more researchers to augment their detrital geochronology datasets with grain measurements.
Predicting the Future Appearances of Lost Children for Information Forensics with Adaptive Discriminator-Based FLM GAN

This article proposes an adaptive discriminator-based GAN (generative adversarial network) model architecture with different scaling and augmentation policies to investigate and identify the cases of lost children even after several years (as human facial morphology changes after specific years). Uniform probability distribution with combined random and auto augmentation techniques to generate the future appearance of lost children’s faces are analyzed. X-flip and rotation are applied periodically during the pixel blitting to improve pixel-level accuracy. With an anisotropic scaling, the images were generated by the generator. Bilinear interpolation was carried out during up-sampling by setting the padding reflection during geometric transformation. The four nearest data points used to estimate such interpolation at a new point during Bilinear interpolation. The color transformation applied with the Luma flip on the rotation matrices spread log-normally for saturation. The luma-flip components use brightness and color information of each pixel as chrominance. The various scaling and modifications, combined with the StyleGan ADA architecture, were implemented using NVIDIA V100 GPU. The FLM method yields a BRISQUE score of between 10 and 30. The article uses MSE, RMSE, PSNR, and SSMIM parameters to compare with the state-of-the-art models. Using the Universal Quality Index (UQI), FLM model-generated output maintains a high quality. The proposed model obtains ERGAS (12 k–23 k), SCC (0.001–0.005), RASE (1 k–4 k), SAM (0.2–0.5), and VIFP (0.02–0.09) overall scores.