Reviewerly/exHarmony huggingface dataset:
Blockchain-Enabled Intelligent IoT

The past few years have witnessed an exponential growth of diverse Internet of Things (IoT) devices as well as compelling applications ranging from industrial production, intelligent transport, and warehouse logistics to medical care. Dramatic advances in IoT technology not only bring enormous economic opportunities but also challenges. Recently, with the appearance of blockchain technology, the integration of IoT and blockchain (BCoT) is considered a promising solution to address these issues. Blockchain provides a secure and scalable data management framework for IoT devices. However, the huge computation and energy cost of the consensus process in blockchain prevents it from being directly applied as a generic platform. To overcome this challenge, we first propose a cloud mining pool-aided BCoT architecture. Based on this architecture, we study the mining pool selection problem and analyze the colony behaviors of IoT devices with different pooling strategies. We propose a centralized evolutionary game-based pool selection algorithm for the sake of maximizing the system utility. Secondly, to overcome the power and computation constraints of the IoT devices in the blockchain platform, we introduce the cloud computing service to the blockchain platform for the sake of assisting to offload computational task from the IIoT network itself. Also, we study the resource management and pricing problem between the cloud provider and miners. And a multi-agent reinforcement learning algorithm is conceived for searching the near-optimal policy.
Designing Campaigns and Optimizing Campaign Performance: The Ingredients of Campaign Excellence

This chapter describes how to link the strategic sphere of business and campaign strategy to the operational sphere of specific marketing campaigns through an annual cycle. It highlights the key elements of a customer-centric campaign strategy and outlines how to translate an annual marketing plan, aligned with multiple marketing stakeholders, into individual marketing activities. The authors explain then how to navigate with these marketing activities effectively and efficiently customers within this framework through one of many predefined customer journeys by creating personalized content, focusing on a specific customer segment and buyer personas, and achieving predetermined goals. Other cornerstones of strategic campaign planning covered by the authors involve a solid marketing plan that orchestrates all marketing activities throughout the year and includes high-level objectives, as well as the mechanisms of allocating budgets and priorities for all stakeholders included.
Towards an Energy Complexity Model for Distributed Data Processing Algorithms

Modern data centers exist as infrastructure in the era of big data. Big data processing applications are the major computing workload of data centers. Electricity cost accounts for about 50% of data centers' operational costs. Therefore, the energy consumed for running distributed data processing algorithms on a data center is starting to attract both academia and industry. Most works study the energy consumption from the hardware perspective and only a few of them from the algorithm perspective. A general and hardware-independent energy evaluation model for the algorithms is in demand. With the model, algorithm designers can evaluate the energy consumption, compare energy consumption features and facilitate energy consumption optimization of distributed data processing algorithms. Inspired by the time complexity model, we propose an energy complexity model for describing the trends that an algorithm's energy consumption grows with the algorithm's input size. We argue that a good algorithm, especially for processing big data, should have a ‘small’ energy complexity. We define <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$E(n)$</tex-math></inline-formula> to represent the functional relationship that associates an algorithm's input size <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n$</tex-math></inline-formula> with its notional energy consumption <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$E$</tex-math></inline-formula> . Based on the well-known abstract Bulk Synchronous Parallel (BSP) computer and programming model, we present a complete <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$E(n)$</tex-math></inline-formula> solution, including abstraction, generalization, quantification, derivation, comparison, analysis, examples, verification, and applications. Comprehensive experimental analysis shows that the proposed energy complexity model is practical, interestingly, and not equivalent to time complexity.
Improving the Results in Credit Scoring by Increasing Diversity in Ensembles of Classifiers

The ensembles of classifiers are techniques that have obtained excellent results in the credit scoring domain. It is known that Decision Trees (DTs) are suitable for ensembles because they encourage diversity, the key point for the success of an ensemble scheme. Ensembles of DTs have obtained good performance in a wide range of areas, including credit scoring. Some works have highlighted that DTs that employ imprecise probability models, called Credal Decision Trees (CDTs), improve the results of ensembles in credit scoring. The performance of CDT is strongly influenced by a hyperparameter. In fact, it was shown that different values of the hyperparameter might yield different models. Hence, the diversity in ensemble schemes can be increased by randomly selecting the value of the hyperparameter in each CDT, instead of fixing one. In this work, it is shown that increasing the diversity of the ensembles that use CDT by varying the value of the hyperparameter in each base classifier improves the results in credit scoring. Thereby, the use of CDT randomly selecting the value of the hyperparameter would suppose notable economic benefits for banks and financial institutions. Few gains in accuracy might imply huge gains in economic benefits.
Density Peaks Clustering Based on Candidate Center and Multi Assignment Policies

Density peak clustering (DPC) the center selection ignores the environment of samples, which can easily result in the incorrect identification of cluster centers. Additionally, a single allocation approach can easily produce the joint effect of allocation mistakes. To address the above issues, a density peaks clustering based on candidate center and multi assignment policies (DPC-CM) is proposed. Firstly, to retain cluster centers with relatively small densities, DPC-CM introduces thresholds to delineate candidate center regions in the decision graph. It also incorporates shared nearest neighbor definitions to ensure proper cluster center selection. Then, the allocation results of the cluster backbone are corrected using shared nearest neighbor information, suppressing allocation cascading errors effectively. Finally, the allocation of cluster backbone points used as the basis for label propagation also improves the allocation efficiency while taking into account the surroundings in which the sample sites are situated. The fault tolerance of the algorithm has also been enhanced by the multi-step allocation strategy. Experimental results on synthetic and UCI datasets validate that the DPC-CM algorithm outperforms comparative algorithms for clustering on complex morphological and unevenly distributed datasets.
A Dataset for User Visual Behaviour with Multi-View Video Content

Immersive video applications impose unpractical bandwidth requirements for best-effort networks. With Multi-View (MV) streaming, these can be minimized by resorting to view prediction techniques. SmoothMV is a multi-view system that uses a non-intrusive head tracking mechanism to detect the viewer's interest and select appropriate views. By coupling Neural Networks (NNs) to anticipate the viewer's interest, a reduction of view-switching latency is likely to be obtained. The objective of this paper is twofold: 1) Present a solution for acquisition of gaze data from users when viewing MV content; 2) Describe a dataset, collected with a large-scale testbed, capable of being used to train NNs to predict the user's viewing interest. Tracking data from head movements was obtained from 45 participants using an Intel Realsense F200 camera, with 7 video playlists, each being viewed a minimum of 17 times. This dataset is publicly available to the research community and constitutes an important contribution to reducing the current scarcity of such data. Tools to obtain saliency/heat maps and generate complementary plots are also provided as an open-source software package.
QANS: Toward Quantized Neural Network Adversarial Noise Suppression

Neural network quantization techniques play an important role in efficiently deploying deep learning models on the hardware with limited computing and storage resources. Numerous applications of this technology, such as autopilot, necessitate not just efficiency but also robustness. Research on the robustness of quantized networks against adversarial attacks is becoming one of the major points of interest. In this work, we rethink the impact of quantization on adversarial attacks and explore the boundary of the robustness of quantized neural networks. This study reveals that activation quantization can be used as a defense to weaken adversarial noise, but the robustness of quantized models is still limited by the amplification effect of network errors, including quantization errors and adversarial noise. To address this problem, we propose the quantization adversarial noise suppression (QANS) method that employs a Gaussian kernel regularization constraint to stabilize the model by restricting the perturbation error within two levels of tolerance. Extensive experiments are conducted with Wide ResNet and VGG-16 models on CIFAR-10 and street view house number datasets under different attack methods, including several white-box and black-box attacks. Experimental results show the proposed method achieves superior robustness to prior works.
Adaptive Active Fusion of Camera and Single-Point LiDAR for Depth Estimation

Depth sensing is an important problem in many applications such as autonomous driving, robotics, and automation. This paper presents an adaptive active fusion method for scene depth estimation by using an RGB camera and a single-point LiDAR. An active scanning mechanism is proposed to guide laser scanning based on critical visual and saliency features, and the convolutional spatial propagation network (CSPN) is designed to generate and refine full depth map from the sparse depth scans. The active scanning mechanism generates a depth mask by using log-spectrum saliency detection, Canny edge detection, and uniform sampling, which indicate critical regions that require a high resolution of laser scanning. To reconstruct a full depth map, the designed CSPN network extracts affinity matrices from the sparse depth scans, while reserving global spatial information in the RGB images. The performance of proposed method was evaluated and compared with the state-of-the-art methods on the NYUv2 dataset, and the experiment demonstrated its outperformance in reconstruction accuracy and robustness to measurement noise. The proposed method was also evaluated in real-world scenarios.
Color-aware Deep Temporal Backdrop Duplex Matting System

Deep learning-based alpha matting showed tremendous improvements in recent years, yet, feature film production studios still rely on classical chroma keying including costly post-production steps. This perceived discrepancy can be explained by some missing links necessary for production which are currently not adequately addressed in the alpha matting community, in particular foreground color estimation or color spill compensation. We propose a neural network-based temporal multi-backdrop production system that combines beneficial features from chroma keying and alpha matting. Given two consecutive frames with different background colors, our one-encoder-dual-decoder network predicts foreground colors and alpha values using a patch-based overlap-blend approach. The system is able to handle imprecise backdrops, dynamic cameras, and dynamic foregrounds and has no restrictions on foreground colors. We compare our method to state-of-the-art algorithms using benchmark datasets and a video sequence captured by a demonstrator setup. We verify that a dual backdrop input is superior to the usually applied trimap-based approach. In addition, the proposed studio set is actor friendly, and produces high-quality, temporal consistent alpha and color estimations that include a superior color spill compensation.
Pseudo Features Guided Self-training for Domain Adaptive Semantic Segmentation of Aerial Images

Semantic segmentation of urban scenes is widely recognized as a fundamental and crucial task that is of great significance to the urban planning and urban management. When applying the semantic segmentation models to large-scale or unseen scenarios, a so-called "domain shift" generally exists between the source and the target domain, where we train and evaluate our models. To mitigate such a problem, domain adaptation leverages the unlabeled data in the target domain to align or fine tuned the model using the target statistics. In previous domain adaptation approaches based on self-training, only the output-level knowledge (e.g., pseudo labels) from the source model is used to guide the adaptation process. However, we notice that the "pseudo features" generated by the source model could be of greater potential for capturing the relationship between target domain samples. To this end, we seek to exploit feature-level relation among neighboring pixels in the target domain to structure the prediction of the domain adaptation model. Experiments on public ISPRS datasets verify that the proposed method can outperform previous unsupervised domain adaptation methods, and demonstrate the benefits of mining local feature relation.
Content-based Movie Recommendation System and Sentimental analysis using ML

The recommendation system is important in today’s world and is employed by many famous apps. A recommendations system is one that suggests books, movies, music, and other resources to users based on a data collection. Typically, movie recommendation algorithms anticipate what movies a user would enjoy based on the characteristics of previously loved movies. The algorithms can propose movies based on one or more criteria such as the genre of the film, the people in it, or even the director of the film. The method used is content-based filtering with genre correlation.
An Ensemble Learning-Based HPC Multi-Resource Demand Prediction Model for Hybrid Clusters

Growing workload demands and rapidly evolving HPC infrastructures drive the diversification of HPC cluster environments, and complex HPC applications are increasingly demanding for rational resource allocation. Based on the current development status, we propose a multivariate resource load forecasting model- MRDP (Multivariate Resource Demand Prediction)-based on the combination of deep learning and integrated learning, which can predict the overall job running time, CPU requirements and memory requirements of jobs in multiple HPC production environments. To provide the necessary prerequisites for making decisions on HPC job-aware scheduling schemes in hybrid clusters. We design experiments to evaluate this model against other models and multiple output regression algorithms in machine learning. The experimental results show that our MRDP model outperforms other models and algorithms in terms of prediction accuracy, model stability, and fitting effectiveness
Oriented aircraft object detector using Scaled YOLOv4 on very high resolution satellite and synthetic datasets

We propose a deep learning solution in the frame of the management of airport ground flows. The developed tool aims to automatically detect aircrafts on satellite imagery at very high resolution (<1m). It relies on the Scaled YOLOv4 deep learning model which has proven to be a good object detector. Trained on open-source annotated datasets, our model shows very good performance to detect common commercial airplanes on Pléiades images, but misses small planes or other types of aircrafts. We indicate several possible improvements, thanks to a large dataset of Pléiades images that we annotated ourselves and thanks to the production of synthetic images combining Pléiades data with artificial superimposed aircrafts.
Redefining Annual Rate Occurrence (ARO) in Risk Calculation with Probability and Regression

This study converts archaic Annual Rate of Occurrence(ARO) into a novel probabilistic model with the help of a public dataset while improving Annualized Loss Expectancy (ALE). It considers internal and external influence when calculating cybersecurity risk, where granularity is its key characteristic. In addition, the exact dataset is modelled by machine learning classifiers to identify types of breaches. Though the results were less than satisfactory, it provides a framework for future studies.
TDEC: Deep Embedded Image Clustering with Transformer and Distribution Information

Image clustering is a crucial but challenging task in multimedia machine learning. Recently the combination of clustering with deep learning has achieved promising performance against conventional methods on high-dimensional image data. Unfortunately, existing deep clustering methods (DC) often ignore the importance of information fusion with a global perception field among different image regions for clustering images, especially complex ones. Additionally, the learned features are usually not clustering-friendly in terms of dimensionality and are based only on simple distance information for the clustering. In this regard, we propose a deep embedded image clustering TDEC, which for the first time to our knowledge, jointly considers feature representation, dimensional preference, and robust assignment for image clustering. Specifically, we introduce the Transformer to form a novel module T-Encoder to learn discriminative features with global dependency while using the Dim-Reduction block to build a friendly low-dimensional space favoring clustering. Moreover, the distribution information of embedded features is considered in the clustering process to provide reliable supervised signals for joint training. Our method is robust and allows for more flexibility in data size, the number of clusters, and the context complexity. More importantly, the clustering performance of TDEC is much higher than that of recent competitors. Extensive experiments with state-of-the-art approaches on complex datasets demonstrate the superiority of TDEC.
AVForensics: Audio-driven Deepfake Video Detection with Masking Strategy in Self-supervision

Existing cross-dataset deepfake detection approaches exploit mouth-related mismatches between the auditory and visual modalities in fake videos to enhance generalisation to unseen forgeries. However, such methods inevitably suffer performance degradation with limited or unaltered mouth motions, we argue that face forgery detection consistently benefits from using high-level cues across the whole face region. In this paper, we propose a two-phase audio-driven multi-modal transformer-based framework, termed AVForensics, to perform deepfake video content detection from an audio-visual matching view related to full face. In the first pre-training phase, we apply the novel uniform masking strategy to model global facial features and learn temporally dense video representations in a self-supervised cross-modal manner, by capturing the natural correspondence between the visual and auditory modalities regardless of large-scaled labelled data and heavy memory usage. Then we use these learned representations to fine-tune for the down-stream deepfake detection task in the second phase, which encourages the model to offer accurate predictions based on captured global facial movement features. Extensive experiments and visualizations on various public datasets demonstrate the superiority of our self-supervised pre-trained method for achieving generalisable and robust deepfake video detection.
Federated Learning Optimization for Balancing Communication Cost and Model Accuracy under Data Heterogeneity

Federated learning is a new distributed machine learning paradigm. A key and common challenge is the heterogeneity of data. Heterogeneous data across user devices can severely degrade the performance of federated learning, and different degrees of data heterogeneity also affect the performance of models. We find that increasing communication frequency can improve model prediction accuracy under heterogeneous data conditions. However, when a single communication is costly, we cannot simply increasing the communication frequency to obtain a better model. In this paper, we define a score function to determine how to select parameters at different degrees of data heterogeneity and with different preference for communication costs and model accuracy. Based on these, we propose a new method for model training by inferring the degree of data heterogeneity based on KL divergence of global model and local model, then choose the best parameters according to different degree of data heterogeneity and preference for communication cost and model accuracy, so as to achieve a good balance between communication cost and model accuracy.
navigu.net: NAvigation in Visual Image Graphs gets User-friendly

Due to the size of today’s image collections it can be challenging to fully understand their content. Recent technological advances have enabled efficient visual search. These systems use joint visual and textual feature vectors to identify similar images based on image queries or text descriptions. Despite their effectiveness, high-dimensional feature vectors can lead to long search times for large collections. In this demonstration, we propose a solution that significantly reduces search times and increases the efficiency of the search system. By combining two separate image graphs, our method provides fast approximate nearest neighbor search and allows seamless visual exploration of the entire collection in real time through a standard web browser, using familiar navigation techniques such as zooming and dragging, common in systems like Google Maps.
MMSF: A Multimodal Sentiment-Fused Method to Recognize Video Speaking Style

As talking takes a large proportion of human lives, it is necessary to perform deeper understanding of human conversations. Speaking style recognition is aimed at recognizing the styles of conversations, which provides a fine-grained description about talking. Current works focus on adopting only visual clues to recognize speaking styles, which cannot accurately distinguish different speaking styles when they are visually similar. To recognize speaking styles more effectively, we propose a novel multimodal sentiment-fused method, MMSF, which extracts and integrates visual, audio and textual features of videos. In addition, as sentiment is one of the motivations of human behavior, we first introduce sentiment into our multimodal method with cross-attention mechanism, which enhance the video feature to recognize speaking styles. The proposed MMSF is evaluated on long-form video understanding benchmark, and the experiment results show that it is superior to the state-of-the-arts.