Reviewerly/exHarmony huggingface dataset:
Design of supervision-scalable learning systems: Methodology and performance benchmarking

The design of robust learning systems that offer stable performance under a wide range of supervision degrees is investigated in this work. We choose the image classification problem as an illustrative example and focus on the design of modularized systems that consist of three learning modules: representation learning, feature learning, and decision learning. We discuss ways to adjust each module so that the design is robust with respect to different training sample numbers. Based on these ideas, we propose two families of learning systems. One adopts the classical histogram of oriented gradients (HOG) features, while the other uses successive-subspace-learning (SSL) features. We test their performance against LeNet-5 and ResNet-18, two end-to-end optimized neural networks, for MNIST, Fashion-MNIST and CIFAR-10 datasets. The number of training samples per image class goes from the extremely weak supervision condition (i.e., one labeled sample per class) to the strong supervision condition (i.e., 4096 labeled samples per class) with a gradual transition in between (i.e., 2n, n=0,1,…,12). Experimental results show that the two families of modularized learning systems have more robust performance than LeNet-5 and ResNet-18. They both outperform the two deep learning networks by a large margin for small n and have performance comparable for large n.
clusterBMA: Bayesian model averaging for clustering

Various methods have been developed to combine inference across multiple sets of results for unsupervised clustering, within the ensemble clustering literature. The approach of reporting results from one 'best' model out of several candidate clustering models generally ignores the uncertainty that arises from model selection, and results in inferences that are sensitive to the particular model and parameters chosen. Bayesian model averaging (BMA) is a popular approach for combining results across multiple models that offers some attractive benefits in this setting, including probabilistic interpretation of the combined cluster structure and quantification of model-based uncertainty. In this work we introduce clusterBMA, a method that enables weighted model averaging across results from multiple unsupervised clustering algorithms. We use clustering internal validation criteria to develop an approximation of the posterior model probability, used for weighting the results from each model. From a combined posterior similarity matrix representing a weighted average of the clustering solutions across models, we apply symmetric simplex matrix factorisation to calculate final probabilistic cluster allocations. In addition to outperforming other ensemble clustering methods on simulated data, clusterBMA offers unique features including probabilistic allocation to averaged clusters, combining allocation probabilities from 'hard' and 'soft' clustering algorithms, and measuring model-based uncertainty in averaged cluster allocation. This method is implemented in an accompanying R package of the same name. We use simulated datasets to explore the ability of the proposed technique to identify robust integrated clusters with varying levels of separation between subgroups, and with varying numbers of clusters between models. Benchmarking accuracy against four other ensemble methods previously demonstrated to be highly effective in the literature, clusterBMA matches or exceeds the performance of competing approaches under various conditions of dimensionality and cluster separation. clusterBMA substantially outperformed other ensemble methods for high dimensional simulated data with low cluster separation, with 1.16 to 7.12 times better performance as measured by the Adjusted Rand Index. We also explore the performance of this approach through a case study that aims to identify probabilistic clusters of individuals based on electroencephalography (EEG) data. In applied settings for clustering individuals based on health data, the features of probabilistic allocation and measurement of model-based uncertainty in averaged clusters are useful for clinical relevance and statistical communication.
Adversarial attacks and defenses using feature-space stochasticity

Recent studies in deep neural networks have shown that injecting random noise in the input layer of the networks contributes towards ℓp-norm-bounded adversarial perturbations. However, to defend against unrestricted adversarial examples, most of which are not ℓp-norm-bounded in the input layer, such input-layer random noise may not be sufficient. In the first part of this study, we generated a novel class of unrestricted adversarial examples termed feature-space adversarial examples. These examples are far from the original data in the input space but adjacent to the original data in a hidden-layer feature space and far again in the output layer. In the second part of this study, we empirically showed that while injecting random noise in the input layer was unable to defend these feature-space adversarial examples, they were defended by injecting random noise in the hidden layer. These results highlight the novel benefit of stochasticity in higher layers, in that it is useful for defending against these feature-space adversarial examples, a class of unrestricted adversarial examples.
A Broad Ensemble Learning System for Drifting Stream Classification

In a data stream environment, classification models must effectively and efficiently handle concept drift.Ensemble methods are widely used for this purpose; however, the ones available in the literature either use a large data chunk to update the model or learn the data one by one.In the former, the model may miss the changes in the data distribution, while in the latter, the model may suffer from inefficiency and instability.To address these issues, we introduce a novel ensemble approach based on the Broad Learning System (BLS), where mini chunks are used at each update.BLS is an effective lightweight neural architecture recently developed for incremental learning.Although it is fast, it requires huge data chunks for effective updates and is unable to handle dynamic changes observed in data streams.Our proposed approach, named Broad Ensemble Learning System (BELS), uses a novel updating method that significantly improves best-in-class model accuracy.It employs an ensemble of output layers to address the limitations of BLS and handle drifts.Our model tracks the changes in the accuracy of the ensemble components and reacts to these changes.We present the mathematical derivation of BELS, perform comprehensive experiments with 35 datasets that demonstrate the adaptability of our model to various drift types, and provide its hyperparameter, ablation, and imbalanced dataset performance analysis.The experimental results show that the proposed approach outperforms 10 state-of-the-art baselines, and supplies an overall improvement of 18.59% in terms of average prequential accuracy.
Image restoration for optical synthetic aperture system via variational physics-informed network

Optical synthetic aperture with homogeneous circular sub-mirrors greatly improves the spatial resolution of space telescopes; however, the discrete and sparse characteristics of the sub-mirrors reduce the mid-frequency modulation transfer function (MTF), resulting in blurred images being obtained. In this paper, a method combining variational physics-informed with deep learning is presented, which shows blind image restoration without complex priors. The constraint effect of traditional maximum a posterior (MAP) framework is removed by variational inference framework, which is embedded into Variational Physics-informed Network (VPIN) to optimize neural network training. Residual dense blocks (RDBs) construction is contributed to image feature extraction. Networks with SSIM-corrected loss functions can be trained at the feature level to help with convergence. When SNR = 30 dB, the PSNR of Golay-6 remote sensing test set increases from 20.16 dB to 23.90 dB, SSIM is from 0.610 to 0.842, and MS-SSIM is from 0.930 to 0.955.
Low-mid adversarial perturbation against unauthorized face recognition system

In light of the growing concerns regarding the unauthorized use of facial recognition systems and its implications on individual privacy, the exploration of adversarial perturbations as a potential countermeasure has gained traction. However, challenges arise in effectively deploying this approach against unauthorized facial recognition systems due to the effects of JPEG compression on image distribution across the internet, which ultimately diminishes the efficacy of adversarial perturbations. Existing JPEG compression-resistant techniques struggle to strike a balance between resistance, transferability, and attack potency. To address these limitations, we propose a novel solution referred to as low frequency adversarial perturbation (LFAP). This method conditions the source model to leverage low-frequency characteristics through adversarial training. To further enhance the performance, we introduce an improved low-mid frequency adversarial perturbation (LMFAP) that incorporates mid-frequency components for an additive benefit. Our study encompasses a range of settings to replicate genuine application scenarios, including cross backbones, supervisory heads, training datasets, and testing datasets. Moreover, we evaluated our approaches on a commercial black-box API, Face++. The empirical results validate the cutting-edge performance achieved by our proposed solutions.
An intelligent hybrid approach for photovoltaic power forecasting using enhanced chaos game optimization algorithm and Locality sensitive hashing based Informer model

Highly accurate photovoltaic power forecasting is of great significance for increasing photovoltaic (PV) power generation on urban building rooftops. In this research context, a method based on variational mode decomposition (VMD), an enhanced chaos game optimization (ECGO) algorithm, and an improved Informer model is proposed for predicting PV power. First, the data are decomposed using VMD. Then the Informer model is improved using locality sensitive hashing (LSH) attention. In this study, an attempt is made to incorporate LSH to find the nearest few key vectors for query vectors, which enhances the accuracy of the model for prediction. The chaos game optimization (CGO) algorithm is then enhanced using the circle chaos initialization and golden sine algorithm. The Circle Chaos Initialization replaces the random initialization in the CGO algorithm, and the golden sine algorithm updates the position of the fourth seed in the CGO. The hyperparameters of the model proposed in this study were then optimized using an Enhanced Chaos Game Optimization (ECGO) algorithm. The experimental results show that the RMSE for April, May, and June data are improved by about 45.41%, 62.30%, and 50.52%, respectively, compared to Informer. In summary, the VMD-ECGO-LSH-Informer model can effectively improve the prediction accuracy of PV power generation, which provides important support and valuable contribution to improving the efficiency of rooftop PV power generation installations in urban buildings.
Cloud Security and Privacy

Abstract: Over the last decade, cloud computing has grown and evolved significantly. It provides cost-effective solutions as well as a wide range of adaptable services. Many businesses are using cloud services. Users mostly use cloud services to save data in a virtual environment. Users have no knowledge of or influence over what happens in the cloud, and nothing in the cloud is visible to them, which raises security and privacy problems. Malicious administrators can mess with the integrity, privacy, virtual machines, and volatile confidentiality of cloud services. Clouds are vulnerable to hacking, and their integrity, availability, and data security are all in jeopardy.
Pushing boundaries: An empirical view on the digital sovereignty of six governments in the midst of geopolitical tensions

In just a few years, the issue of "digital sovereignty" has emerged as an important security issue for governments across the globe, reflecting a growing unease about the security risks associated with government services that depend on foreign service providers for digital infrastructure and traffic routing. This work investigates to which extent government services and communication with citizens relies on infrastructure outside their own jurisdiction for six countries facing sensitive or sometimes even antagonistic relations with neighbors: India, the Netherlands, Pakistan, Taiwan, Ukraine, and the United Kingdom. By combining various methods (traceroute measurements, passive DNS data and geolocation), we determine where and how domains are hosted, as well as the network paths taken by citizens' traffic to them. We uncover different strategies and degrees of autonomy, as well as difficult tradeoffs between different risks to autonomy, some of which might be larger than the risks associated with the dependency on foreign providers. This includes transnational providers being used by all countries, with geopolitical rivals even being tenants on the same network and traffic between citizens and governments regularly traversing international borders. Furthermore, we compared our empirical findings to stated governmental policies and find that they are not always consistent.
Day-ahead seasonal solar radiation prediction, combining VMD and STACK algorithms

Abstract This article proposes a method for accurately predicting solar irradiance over a 24-hour horizon to forecast photovoltaic energy generation in a positive-energy building. In order to make this prediction, the input data are divided into seasons and preprocessed using the variational mode decomposition (seasonal-VMD) method. The VMD method is used for extracting high-bandwidth features from the input data, decomposing them into a finite number of smooth modes and focusing on specific frequency ranges. Hence, the accuracy of signal extraction using the VMD method can be improved by selecting particular parameters judiciously, which impacts the smoothing and frequency concentration of the extracted signal. In this regard, the salp swarm algorithm (SSA) is employed to identify the optimal VMD parameters that can be used to enhance extraction accuracy. In addition, the obtained residual between the observed solar irradiation data and their decomposed modes is treated to enhance the prediction process. A stacking algorithm (STACK) is used to predict the following 24-hour solar irradiance modes and the residual, which are finally summed to reconstruct the desired signal. The performances of the proposed prediction method are evaluated using two quantitative evaluation indices: the normalized root mean square percentage error (NRMSPE) and normalized mean absolute percentage error (NMAPE). The proposed model is trained on data collected for three years in Rabat (2019–22). The performance of the proposed model is evaluated by predicting the 24-hour solar irradiance for a different season. The proposed approach seasonal-VMD-STACK is compared with two other methods in the case of using VMD-based STACK without season partition and STACK method only. Moreover, the proposed method has exhibited stability and proven good results with an NRMSPE of 3.87% and an NMAPE of 1.58% for cloudy days during the test phase. The results demonstrate that residual preprocessing, seasonal input data partition and appropriate selection of VMD parameters improve the performance and accuracy of the prediction.
Conference key agreement in a quantum network

Abstract Quantum conference key agreement (QCKA) allows multiple users to establish a secure key from a shared multi-partite entangled state. In a quantum network, this protocol can be efficiently implemented using a single copy of a N-qubit Greenberger-Horne-Zeilinger (GHZ) state to distil a secure N-user conference key bit, whereas up to N-1 entanglement pairs are consumed in the traditional pair-wise protocol. We demonstrate the advantage provided by GHZ states in a testbed consisting of a photonic six-user quantum network, where four users can distil either a GHZ state or the required number of Bell pairs for QCKA using network routing techniques. In the asymptotic limit, we report a more than two-fold enhancement of the conference key rate when comparing the two protocols. We extrapolate our data set to show that the resource advantage for the GHZ protocol persists when taking into account finite-key effects.
Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detection

Most anomaly detection (AD) models are learned using only normal samples in an unsupervised way, which may result in ambiguous decision boundary and insufficient discriminability. In fact, a few anomaly samples are often available in real-world applications, the valuable knowledge of known anomalies should also be effectively exploited. However, utilizing a few known anomalies during training may cause another issue that the model may be biased by those known anomalies and fail to generalize to unseen anomalies. In this paper, we tackle supervised anomaly detection, i.e., we learn AD models using a few available anomalies with the objective to detect both the seen and unseen anomalies. We propose a novel explicit boundary guided semi-push-pull contrastive learning mechanism, which can enhance model's discriminability while mitigating the bias issue. Our approach is based on two core designs: First, we find an explicit and compact separating boundary as the guidance for further feature learning. As the boundary only relies on the normal feature distribution, the bias problem caused by a few known anomalies can be alleviated. Second, a boundary guided semi-push-pull loss is developed to only pull the normal features together while pushing the abnormal features apart from the separating boundary beyond a certain margin region. In this way, our model can form a more explicit and discriminative decision boundary to distinguish known and also unseen anomalies from normal samples more effectively. Code will be available at https://github.com/xcyao00/BGAD.
Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization

Recently, flat minima are proven to be effective for improving generalization and sharpness-aware minimization (SAM) achieves state-of-the-art performance. Yet the current definition of flatness discussed in SAM and its follow-ups are limited to the zeroth-order flatness (i.e., the worst-case loss within a perturbation radius). We show that the zeroth-order flatness can be insufficient to discriminate minima with low generalization error from those with high generalization error both when there is a single minimum or multiple minima within the given perturbation radius. Thus we present first-order flatness, a stronger measure of flatness focusing on the maximal gradient norm within a perturbation radius which bounds both the maximal eigenvalue of Hessian at local minima and the regularization function of SAM. We also present a novel training procedure named Gradient norm Aware Minimization (GAM) to seek minima with uniformly small curvature across all directions. Experimental results show that GAM improves the generalization of models trained with current optimizers such as SGD and Adam W on various datasets and networks. Furthermore, we show that GAM can help SAM find flatter minima and achieve better generalization. The code is available at https://github.com/xxgege/GAM.
Novel Integer Shmaliy Transform and New Multiparametric Piecewise Linear Chaotic Map for Joint Lossless Compression and Encryption of Medical Images in IoMTs

The discrete Shmaliy moment transform (DST) is a type of discrete orthogonal moment transform that is widely used in signal and image processing. However, DST is not suitable for lossless image applications due to its non-integer reversible nature. To overcome this limitation, we introduce the integer discrete Shmaliy transform (IDST) that performs integer-to-integer encoding, leading to a perfect and unique reconstruction of the input image. Next, a new 1D chaotic system model, the 1D multiparametric piecewise linear chaotic map (M-PWLCM), is presented as an extension of the existing 1D PWLCM. The M-PWLCM includes eight control parameters defined over an unlimited interval. To demonstrate the relevance of IDST and M-PWLCM in reversible image processing applications, they are used in a new scheme for lossless compression and encryption of medical images in the internet of medical things (IoMTs). On the one hand, the simulation results show that our scheme offers a good compression ratio and a higher level of security to resist differential attacks, brute force attacks and statistical attacks. On the other hand, the comparative analysis carried out shows the overall superiority of our scheme over similar state-of-the-art ones, both in achieving a higher compression ratio and better security when communicating medical images over unsecured IoMTs.
AdaTerm: Adaptive T-distribution estimated robust moments for Noise-Robust stochastic gradient optimization

With the increasing practicality of deep learning applications, practitioners are inevitably faced with datasets corrupted by noise from various sources such as measurement errors, mislabeling, and estimated surrogate inputs/outputs that can adversely impact the optimization results. It is a common practice to improve the optimization algorithm’s robustness to noise, since this algorithm is ultimately in charge of updating the network parameters. Previous studies revealed that the first-order moment used in Adam-like stochastic gradient descent optimizers can be modified based on the Student’s t-distribution. While this modification led to noise-resistant updates, the other associated statistics remained unchanged, resulting in inconsistencies in the assumed models. In this paper, we propose AdaTerm, a novel approach that incorporates the Student’s t-distribution to derive not only the first-order moment but also all the associated statistics. This provides a unified treatment of the optimization process, offering a comprehensive framework under the statistical model of the t-distribution for the first time. The proposed approach offers several advantages over previously proposed approaches, including reduced hyperparameters and improved robustness and adaptability. AdaTerm achieves this by considering the interdependence of gradient dimensions. In particular, upon detection, AdaTerm excludes aberrant gradients from the update process and enhances its robustness for subsequent updates. Conversely, it performs normal parameter updates when the gradients are statistically valid, allowing for flexibility in adapting its robustness. This noise-adaptive behavior contributes to AdaTerm’s exceptional learning performance, as demonstrated through various optimization problems with different and/or unknown noise ratios. Furthermore, we introduce a new technique for deriving a theoretical regret bound without relying on AMSGrad, providing a valuable contribution to the field.