Reviewerly/exHarmony huggingface dataset:
Multi-Sentence Complementarily Generation for Text-to-Image Synthesis

Generating realistic images based on text descriptions remains challenging in computer vision. Existing multi-stage generation methods are sufficient to generate high-resolution images. However, these methods mainly use one sentence to synthesize images, which are difficult to extract adequate semantic features, resulting in the generated images being far apart from ground-truth images. In this paper, we propose a Multi-Sentence Complementary Generative Adversarial Network, MSCGAN, which assists in generating accurate images by fusing the same semantics from different sentences and preserving their unique semantics. More specifically, the latest BERT model is employed to identify semantic features and a multi-semantic fusion module (MSFM) is designed to fuse the semantic features of different sentences. Besides, a pre-trained cross-modal contrast similarity model (CCSM) is developed to explore fine-grained loss on generated images. Moreover, a multi-sentence joint discriminator is designed to ensure that the generated images match all sentences. Experiments and ablation studies on CUB and MS-COCO datasets demonstrate the significant superiority of the proposed method compared to state-of-the-art methods.
Cloud Integration and Orchestration

The objective of this chapter is to introduce the basics of cloud integration and orchestration in multi-cloud environments. Through this chapter, the reader will come to why do we need multi-cloud environment and hybrid IT, its challenges and how cloud integration and orchestration help to resolve some of the challenges. The reader also comes to know about existing tools for cloud orchestrationCloud orchestration.
Intelligent Metaverse Scene Content Construction

The integration of artificial intelligence (AI) and virtual reality (VR) has expanded the possibilities of research in various scientific fields.Virtual reality simulations driven by artificial intelligence has revolutionary significance in the fields of education, healthcare, and entertainment.However, there is a lack of comprehensive investigation to systematically summarize the basic characteristics and development trajectory of metaverse visual content generated by artificial intelligence.This survey focused on the construction of intelligent metaverse scene content, aiming to bridge this gap by studying the application of artificial intelligence in content generation.It explores scene content generation, simulation biology, personalized content, and intelligent agents.By analyzing the current situation in this field and identifying common features, this survey provides a detailed description of the methods for constructing intelligent scenes in the metaverse.The main contribution of this study is a comprehensive analysis of the current situation of intelligent visual content production in the metaverse, highlighting emerging trends.It aims to motivate researchers to use artificial intelligence to enhance VR experience and promote the creation of immersive metaverse simulation scenarios.The discussion on methods for constructing intelligent scene content in the metaverse suggests that, in the era of intelligence, it has the potential to become the dominant approach for content creation in metaverse scenes.
Interpolative Metric Learning for Few-Shot Specific Emitter Identification

Specific emitter identification (SEI) is a potential physical layer authentication technology, which is one of the most critical complements of upper layer authentication. Radio frequency fingerprint (RFF)-based SEI is to distinguish one emitter from each other by immutable RF characteristics from electronic components. Due to the powerful ability of deep learning (DL) to extract hidden features and perform classification, it can extract highly separative features from massive signal samples, thus enabling SEI. Considering the condition of limited training samples, we propose a novel few-shot SEI (FS-SEI) method based on interpolative metric learning (InterML) which gets rid of the dependence on auxiliary dataset. Specifically, InterML is designed to mine more implicit samples in the sample space to improve generalization, and constrain the feature distance in the feature space to improve discriminability. The proposed InterML-based FS-SEI method is evaluated on a real-world Wi-Fi dataset. The simulation results show that the proposed method achieves better identification performance, higher feature discriminability and more stable performance than five latest FS-SEI methods. In the 10 shot scenario, the identification accuracy of InterML is 91.48%, compared to the comparison methods, the accuracy is improved by 0.62%–31.29%. Our code can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/chengJoin/InterML-for-FS-SEI</uri> .
Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Data

In this paper, we propose a feature affinity (FA) assisted knowledge distillation (KD) method to improve quantization-aware training of deep neural networks (DNN). The FA loss on intermediate feature maps of DNNs plays the role of teaching middle steps of a solution to a student instead of only giving final answers in the conventional KD where the loss acts on the network logits at the output level. Combining logit loss and FA loss, we found via convolutional network experiments on CIFAR-10/100, and Tiny ImageNet data sets that the quantized student network receives stronger supervision than from the labeled ground-truth data. The resulting FA quantization-distillation (FAQD), trained to convergence with a cosine annealing scheduler for 200 epochs, is capable of compressing models on label-free data up to or exceeding the accuracy levels of their full precision counterparts, which brings immediate practical benefits as pre-trained teacher models are readily available and unlabeled data are abundant. In contrast, data labeling is often laborious and expensive. Finally, we propose and prove error estimates for a fast feature affinity (FFA) loss function that accurately approximates FA loss at a lower order of computational complexity, which helps speed up training for high resolution image input. Source codes are available at: https://github.com/lzj994/FAQD.
Dual-Space Aggregation Learning and Random Erasure for Visible Infrared Person Re-Identification

Visible infrared person re-identification (VI Re-ID) is of particular importance for an intelligent safe-guard system, aiming to retrieve the same pedestrian from non-overlapping visible and infrared cameras. The VI Re-ID task is extremely challenging due to significant modality differences, high-sample noise, occlusions, etc. To address these issues, we explore a dual-space aggregation learning (DSAL) method that combines instance batch normalization (IBN) and residual shrinkage (RS) into a baseline model for feature learning and compression at the channel-level. The random erasing (RE) data augmentation method has been applied to preprocess the data. Experiments on two datasets demonstrate that: 1) IBN reduces shallow layer appearance differences and can bridge the gap between heterogeneous modalities; 2) The RS adaptive soft threshold sets the zero-domain features to zero to eliminate noise and clutter information, thereby enhancing the robustness of the network to noise; 3) RE data augmentation method significantly improves the model’s generalization ability. Particularly, the design of DSAL can be seamlessly embedded into other CNN frameworks as a bottleneck variant without additional computation costs. Compared with the strong baseline, on SYSU-MM01, Rank-1, mAP, and mINP significantly improved by 10.66%, 7.78%, and 5.91%, respectively. On RegDB, Rank-1, mAP, and mINP significantly improved by 16.40%, 13.83%, and 19.07%, respectively.
Progressive Hyperspectral Image Destriping With an Adaptive Frequencial Focus

Limited by the imaging paradigm, stripes is pervasive in remote sensing scenes, and its intensity, density, and periodicity differ dramatically among different imaging systems. Worse, it always co-exists with random noises caused by unstable imaging condition. However, current destriping methods are victim to undue ideal assumptions and fail to accurately eliminate stripes against diverse practical degradation, yielding excessive or inadequate destriping results. This study proposes a progressive hyperspectral destriping method with an adaptive frequency focus for accurate destriping and delicate restoration. Specifically, a hierarchical decomposition and reconstruction framework based on progressive wavelet learning encodes the degraded input to the frequency domain with smaller scales, easing the difficulty of restoration. Then, to avoid excessive or insufficient destriping, we devote specific efforts to finely separating noise and preserving details in the high-frequency domain. First, we devise a gradient-aware frequency attention block based on the prominent unidirectional pattern of stripes, empowering to adaptively assign weights according to their sensitivity to the spatial gradient. Second, we design a focal high-frequency loss item that is dynamically scaled according to feature distance in the high-frequency domain, profiting in identifying and preserving details. Extensive experiments conducted on data with synthetic stripes and realistic satellite scenes validate the superiority of the proposed method over the current state-of-the-art methods. The code is available at https://github.com/EtPan/PHID.
Disparity-Guided Light Field Image Super-Resolution via Feature Modulation and Recalibration

The disparity information reflects pixel-wise inter-view correlations among sub-aperture images (SAIs) of a light field (LF) image. Existing CNN-based methods for LF spatial super-resolution (SR) rarely utilize the disparities to incorporate the underlying inter-view correlations due to the lack of ground-truth disparity labels. In this paper, we incorporate a self-supervised training strategy to estimate disparity maps, and simultaneously propose a disparity-guided network for LF spatial SR (i.e., LF-DGNet), in which the estimated disparity maps are utilized to modulate the LF features. Specifically, we introduce a disparity-guided feature modulation (DGFM) module where the disparity maps are utilized to produce affine transformation parameters for each view-wise feature. We also propose a multi-view feature recalibration (MFR) module which consists of a triple attention (TA, i.e., spatial, channel, and angular attention) to enhance the ability of feature representation and further explore the intra- and inter-view correlations. Experimental results on public LF datasets including large-disparity LF images demonstrate that our proposed method achieves state-of-the-art performance both in quantitative results and visual quality. Our codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/GaoshengLiu/LF-DGNet</uri>
FedRecovery: Differentially Private Machine Unlearning for Federated Learning Frameworks

Over the past decades, the abundance of personal data has led to the rapid development of machine learning models and important advances in artificial intelligence (AI). However, alongside all the achievements, there are increasing privacy threats and security risks that may cause significant losses for data providers. Recent legislation requires that the private information about a user should be removed from a database as well as machine learning models upon certain deletion requests. While erasing data records from memory storage is straightforward, it is often challenging to remove the influence of particular data samples from a model that has already been trained. Machine unlearning is an emerging paradigm that aims to make machine learning models “forget” what they have learned about particular data. Nevertheless, the unlearning issue for federated learning has not been completely addressed due to its special working mode. First, existing solutions crucially rely on retraining-based model calibration, which is likely unavailable and can pose new privacy risks for federated learning frameworks. Second, today’s efficient unlearning strategies are mainly designed for convex problems, which are incapable of handling more complicated learning tasks like neural networks. To overcome these limitations, we took advantage of differential privacy and developed an efficient machine unlearning algorithm named FedRecovery. The FedRecovery erases the impact of a client by removing a weighted sum of gradient residuals from the global model, and tailors the Gaussian noise to make the unlearned model and retrained model statistically indistinguishable. Furthermore, the algorithm neither requires retraining-based fine-tuning nor needs the assumption of convexity. Theoretical analyses show the rigorous indistinguishability guarantee. Additionally, the experiment results on real-world datasets demonstrate that the FedRecovery is efficient and is able to produce a model that performs similarly to the retrained one.
When AI Meets Information Privacy: The Adversarial Role of AI in Data Sharing Scenario

Artificial intelligence (AI) is a transformative technology with a substantial number of practical applications in commercial sectors such as healthcare, finance, aviation, and smart cities. AI also has strong synergy with the information privacy (IP) domain from two distinct aspects: as a protection tool (i.e., safeguarding privacy), and as a threat tool (i.e., compromising privacy).In the former case, AI techniques are amalgamated with the traditional anonymization techniques to improve various key components of the anonymity process, and therefore, privacy is safeguarded effectively.In the latter case, some adversarial knowledge is aggregated with the help of AI techniques and subsequently used to compromise the privacy of individuals.To the best of our knowledge, threats posed by AI-generated knowledge such as synthetic data (SD) to information privacy are often underestimated, and most of the existing anonymization methods do not consider/model this SD-based knowledge that can be available to the adversary, leading to privacy breaches in some cases.In this paper, we highlight the role of AI as a threat tool (i.e., AI used to compromise an individual's privacy), with a special focus on SD that can serve as background knowledge leading to various kinds of privacy breaches.For instance, SD can encompass pertinent information (e.g., total # of attributes in data, distributions of sensitive information, category values of each attribute, minor and major values of some attributes, etc.) about real data that can offer a helpful hint to the adversary regarding the composition of anonymized data, that can subsequently lead to uncovering the identity or private information.We perform reasonable experiments on a real-life benchmark dataset to prove the pitfalls of AI in the data publishing scenario (when a database is either fully or partially released to public domains for conducting analytics).
Path Planning for Autonomous Systems Design: A Focus Genetic Algorithm for Complex Environments

Abstract Path planning has been a hot research topic in robotics and is a vital functionality for autonomous systems. As the time complexity of traditional path planning algorithms grows rapidly with the complexity of the problem, evolutionary algorithms are widely applied for their near-optimal solutions. However, evolutionary algorithms can be trapped in a local optimum or converge to infeasible solutions, especially for large search spaces. As the problem scale increases, evolutionary algorithms often cannot find feasible solutions with random exploration, making it extremely challenging to solve long-range path-planning problems in environments with obstacles of various shapes and sizes. For long-range path planning of an autonomous ship, the current downsampling map approach may result in the disappearance of rivers and make the problem unsolvable. This paper introduces a novel area-based collision assessment method for genetic algorithm (GA) that can always converge to feasible solutions with various waypoints in large-scale and obstacle-filled environments. Waypoint-based crossover and mutation operators are developed to allow GA to modify the length of the solution during planning. To avoid the premature problem of GA, the mutation process is replaced by a self-improving process to let the algorithm focus the operations on any potential solutions before discarding them in the selection process. The case studies show that the proposed GA-focus algorithm converges faster than RRT* and can be applied to various large-scale and challenging problems filled with obstacles of different shapes and sizes, and find high-quality solutions.
Disruption mitigation in the semiconductors supply chain by using public blockchains

Abstract Supply chain management has become increasingly complex, leading to information loss and delayed transmission, exacerbating the bullwhip effect. Furthermore, the need for more reliable information storage, traceability, and accountability has slowed supply chain management. In this context, blockchain technology (BCT) solves the bullwhip effect problem, as it supports distributed networking, information synchronization among nodes, digital encryption, traceable information, and unchangeable block content. This paper proposes using public blockchain to solve the bullwhip effect problem in the supply chain (SC), which has yet to be discussed in published academic research papers. To this end, a framework has been developed to determine the readiness of a supply chain for blockchain adoption. This framework includes a checklist to evaluate the motivation, timing, and appropriate platform for adopting blockchain. The framework has been validated using a case study of personal computer devices, which were affected by the Covid-19 pandemic and resulted in semiconductors supply chain disruption. The study involved polling four executives The study involved polling four executives (of large organizations in Egypt and some Middle Eastern countries) representing different stages in the supply chain, they are retail, distribution, manufacturing, and supply. The results showed that while the executives recognized the benefits of adopting blockchain, they were still determining if it was the right time for adoption and did not see the technology ready for their supply chains. Finally, some public blockchain platforms were introduced to help businesses choose the most suitable platform for adoption.
Reward poisoning attacks in deep reinforcement learning based on exploration strategies

In recent years, deep reinforcement learning (DRL) has been widely applied in various fields, and its vulnerability to adversarial samples, which is inherited from deep learning, has also received considerable attention for training robust DRL agents. However, most of existing attack approaches aim to perturb the observations of DRL agents, and poisoning attacks against DRL through perturbing the reward from the environment have been studied less. In this paper, we propose a novel reward poisoning adversarial attack method that employs the exploration strategy of reinforcement learning to help effectively attack a target. The adversary first builds a deep neural network to attain the exploration value of each state–action pair, i.e., the E-value, which is utilized to determine the appropriate time to attack the target and the sign of adversarial perturbation to generate adversary samples to the reward. The experimental results demonstrate that the reward poisoning attack algorithm based on E-values proposed in this paper enables an adversary to improve the efficiency of attacks with fewer adversarial samples.
Evaluation of Maestro, an extensible general-purpose data gathering and data classification platform

Various tools are available to help navigate and manage the vast amount of data on the Web that is constantly growing. However, automatically searching, gathering, and classifying data from multiple sources can be challenging. This research presents Maestro, a multiuser and multi-organization platform that enables users to create and manage specialized “search contexts” for streamlined this process. With Maestro, users can define and configure these search contexts to gather data from different web sources, process and filter the obtained data, classify it using machine learning models, and send the results to external services. Maestro platform is designed to be extensible, allowing new features to be easily and incrementally added without disrupting its core framework. The paper presents and examines two simple but illustrative examples showcasing the capabilities of Maestro: the Bird-species dataset and the Sound-to-Text conversion dataset. The article also evaluates Maestro's usability and extensibility through two user assessments. The first assessment focused on evaluating the user perception, including its usability and relevance, and the second on the easiness of adding new components to extend its functionality by developers. The results were positive and encouraging, with users praising the platform's ease of use and versatility.
Using Artificial Intelligence to Combat Money Laundering

Banks and other financial institutions handle sensitive records regarding people, trusts, and corporations.Money, as a sensitive and useful commodity, makes financial organizations valuable and prone to criminal elements.Common criminal activities that target the banking sector include money laundering, identity and personal records theft, and terrorism financing.These are global issues that have garnered the attention of international bodies and governments.One method proposed to deal with illicit finance and money laundering is artificial intelligence (AI).AI implements various algorithms and techniques to monitor customers, markets, and financial transactions that help identify various banking habits.Understanding clients' transactions and the nature of bank transfers enables AI to prevent and combat money laundering.This research offers an understanding of how artificial intelligence is used in the financial system to combat fraudulent activities such as money laundering.It is organized into five chapters covering various aspects of artificial intelligence and money laundering.
The Impact of Reflection Approximations on Visual Quality in Virtual Reality

Virtual Reality (VR) systems rely on established real-time rendering techniques to uphold a strict performance budget of a few milliseconds per frame. The utilized techniques are based on approximations that trade speed-up for the correctness of the generated depth cues. The visual impact of these trade-offs, however, remains largely unexplored in the context of VR. In this paper we focus specifically on the perception of distorted specular reflections. Our research goal is to quantify the impairment of common reflection approximations on the resulting visual quality in VR. To this end, a subjective quality assessment is conducted, where participants rate the visual difference and visual annoyance of multiple established reflection approximations in a side-by-side comparison to a raytraced reference solution. We introduce the first dataset of its kind, containing 9 specular reflection approximations evaluated for two object types and three material smoothness values. Our results show the quality benefits associated with currently used reflection methods, and we discuss the implications for rendering VR environments.