Reviewerly/exHarmony huggingface dataset:
Contrastive Multimodal Fusion with TupleInfoNCE

This paper proposes a method for representation learning of multimodal data using contrastive losses. A traditional approach is to contrast different modalities to learn the information shared among them. However, that approach could fail to learn the complementary synergies between modalities that might be useful for downstream tasks. Another approach is to concatenate all the modalities into a tuple and then contrast positive and negative tuple correspondences. However, that approach could consider only the stronger modalities while ignoring the weaker ones. To address these issues, we propose a novel contrastive learning objective, TupleInfoNCE. It contrasts tuples based not only on positive and negative correspondences, but also by composing new negative tuples using modalities describing different scenes. Training with these additional negatives encourages the learning model to examine the correspondences among modalities in the same tuple, ensuring that weak modalities are not ignored. We provide a theoretical justification based on mutual-information for why this approach works, and we propose a sample optimization algorithm to generate positive and negative samples to maximize training efficacy. We find that TupleInfoNCE significantly outperforms previous state of the arts on three different downstream tasks.
Towards Better Explanations of Class Activation Mapping

Increasing demands for understanding the internal behavior of convolutional neural networks (CNNs) have led to remarkable improvements in explanation methods. Particularly, several class activation mapping (CAM) based methods, which generate visual explanation maps by a linear combination of activation maps from CNNs, have been proposed. However, the majority of the methods lack a clear theoretical basis on how they assign the coefficients of the linear combination. In this paper, we revisit the intrinsic linearity of CAM with respect to the activation maps; we construct an explanation model of CNN as a linear function of binary variables that denote the existence of the corresponding activation maps. With this approach, the explanation model can be determined by additive feature attribution methods in an analytic manner. We then demonstrate the adequacy of SHAP values, which is a unique solution for the explanation model with a set of desirable properties, as the coefficients of CAM. Since the exact SHAP values are unattainable, we introduce an efficient approximation method, LIFT-CAM, based on DeepLIFT. Our proposed LIFT-CAM can estimate the SHAP values of the activation maps with high speed and accuracy. Furthermore, it greatly outperforms other previous CAM-based methods in both qualitative and quantitative aspects.
DMRA: Depth-Induced Multi-Scale Recurrent Attention Network for RGB-D Saliency Detection

In this work, we propose a novel depth-induced multi-scale recurrent attention network for RGB-D saliency detection, named as DMRA. It achieves dramatic performance especially in complex scenarios. There are four main contributions of our network that are experimentally demonstrated to have significant practical merits. First, we design an effective depth refinement block using residual connections to fully extract and fuse cross-modal complementary cues from RGB and depth streams. Second, depth cues with abundant spatial information are innovatively combined with multi-scale contextual features for accurately locating salient objects. Third, a novel recurrent attention module inspired by Internal Generative Mechanism of human brain is designed to generate more accurate saliency results via comprehensively learning the internal semantic relation of the fused feature and progressively optimizing local details with memory-oriented scene understanding. Finally, a cascaded hierarchical feature fusion strategy is designed to promote efficient information interaction of multi-level contextual features and further improve the contextual representability of model. In addition, we introduce a new real-life RGB-D saliency dataset containing a variety of complex scenarios that has been widely used as a benchmark dataset in recent RGB-D saliency detection research. Extensive empirical experiments demonstrate that our method can accurately identify salient objects and achieve appealing performance against 18 state-of-the-art RGB-D saliency models on nine benchmark datasets.
A Machine Learning and Explainable AI Approach for Predicting Secondary School Student Performance

An essential component of the educational activity is rigorous examination and assessment of students' results, with a potential substantial influence on student growth. This paper offers a predictional model for student's success in secondary education using five classification algorithms: Logistic Regression, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), XGBoost, and Naive Bayes, where the data is gathered from two Portuguese school reports and surveys. The two core disciplines (Mathematics subject and Portuguese language) of the dataset were modeled around binary/five-level classification tasks which is imbalanced. The imbalanced dataset is also balanced by using K-Means SMOTE (Synthetic Minority Oversampling Technique) before classification. The test results reveal that obtaining the most outstanding accuracy value is 96.89% of Support Vector Machine (SVM) is superior to Logistic Regression, KNN, XG-Boost, and Naive Bayes. Therefore, it is essential to consider whether a model makes a particular prediction. Thus, we then train an interpretable LIME (Local Interpretable Model-agnostic Explanations) model for all the classifiers and the construction of explainable models can have a major advantage: the model can be confident, the transparency of the model helps to understand the underlying processes for working.
Dynamic analysis and image encryption application of a sinusoidal-polynomial composite chaotic system

Due to complex properties of ergodicity, non-periodic ability and sensitivity to initial states, chaotic systems are widely used in cryptography. In this paper, we propose a sinusoidal-polynomial composite chaotic system (SPCCS), and prove that it satisfies Devaney’s definition of chaos: the sensitivity to initial conditions, topological transitivity and density of periodic points. The experimental results show that SPCCS has better unpredictability and more complex chaotic behavior than the classical chaotic maps. Furthermore, we provide a new image encryption algorithm combining pixel segmentation operation, block chaotic matrix confusing operation and pixel diffusion operation with the proposed SPCCS. The detailed simulation results verify superiority and effectiveness of the proposed image encryption algorithm.
ReCU: Reviving the Dead Weights in Binary Neural Networks

Binary neural networks (BNNs) have received increasing attention due to their superior reductions of computation and memory. Most existing works focus on either lessening the quantization error by minimizing the gap between the full-precision weights and their binarization or designing a gradient approximation to mitigate the gradient mismatch, while leaving the "dead weights" untouched. This leads to slow convergence when training BNNs. In this paper, for the first time, we explore the influence of "dead weights" which refer to a group of weights that are barely updated during the training of BNNs, and then introduce rectified clamp unit (ReCU) to revive the "dead weights" for updating. We prove that reviving the "dead weights" by ReCU can result in a smaller quantization error. Besides, we also take into account the information entropy of the weights, and then mathematically analyze why the weight standardization can benefit BNNs. We demonstrate the inherent contradiction between minimizing the quantization error and maximizing the information entropy, and then propose an adaptive exponential scheduler to identify the range of the "dead weights". By considering the "dead weights", our method offers not only faster BNN training, but also state-of-the-art performance on CIFAR-10 and ImageNet, compared with recent methods. Code can be available at https://github.com/z-hXu/ReCU.
Quantum Linear System Solver Based on Time-optimal Adiabatic Quantum Computing and Quantum Approximate Optimization Algorithm

We demonstrate that with an optimally tuned scheduling function, adiabatic quantum computing (AQC) can readily solve a quantum linear system problem (QLSP) with $\mathcal{O}(\kappa~\text{poly}(\log(\kappa/\epsilon)))$ runtime, where $\kappa$ is the condition number, and $\epsilon$ is the target accuracy. This is near optimal with respect to both $\kappa$ and $\epsilon$. Our method is applicable to general non-Hermitian matrices, and the cost as well as the number of qubits can be reduced when restricted to Hermitian matrices, and further to Hermitian positive definite matrices. The success of the time-optimal AQC implies that the quantum approximate optimization algorithm (QAOA) with an optimal control protocol can also achieve the same complexity in terms of the runtime. Numerical results indicate that QAOA can yield the lowest runtime compared to the time-optimal AQC, vanilla AQC, and the recently proposed randomization method.
A Perspective Review on Integrating VR/AR with Haptics into STEM Education for Multi-Sensory Learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.
Progress in regional PV power forecasting: A sensitivity analysis on the Italian case study

The increasing penetration of PV generation, driven by climate strategies and objectives, calls for accurate production forecasting to mitigate the negative effects associated with inherent variability, such as overgeneration, grid instability, supplementary reserve request. The regional PV power forecasting is crucial for Transmission and Distribution system operators for a better management of energy flows. In this work many aspects of regional PV power forecasting are investigated, by means of a comparison of six different forecasting models applied to predict the hourly production of the following days on six Italian bidding zones for one year. In particular, the work shows that the forecasting accuracy is mainly affected by the algorithm and its pre and post processing, with a range of 30% in performance accuracy, while it is less impacted by the forecasting horizon. It has been verified that the accuracy in the irradiation prediction, used in input to the power forecasting algorithm, has less impact compared to single plants. The work confirms the performance improvement which can be obtained by increasing the size of the area to which the prediction refers, through a comparison between the forecasting at bidding zone and national level. Finally, we show that the larger the controlled forecast area, the smaller the impact on the forecast accuracy due to the non-uniform spatial and capacity distribution of the PV fleet. This means that as the size of the region increases, the average irradiance progressively becomes the best PV power predictor. We refer to this phenomenon as: “input smoothing effect".
ABL-TC: A lightweight design for network traffic classification empowered by deep learning

Network traffic classification is an increasingly significant prerequisite for network management. An accurate traffic classifier can contribute to traffic engineering, traffic intrusion detection and user behavior analysis. Recently, deep learning (DL) has attracted considerable attention for traffic classification due to its excellent learning ability without the need for handcrafted feature engineering or privacy invasion. However, most DL-based traffic classification solutions mainly focus on improving the identification performance, with no regard for the computational and memory burden caused by a large model size or data redundancy. Motivated by the above gaps, we elaborate on the preprocessing of raw traffic data and investigate a lightweight traffic classifier. Specifically, we design a preprocessing approach to convert raw traffic data into available datasets for deep learning based traffic classifiers, which tailors raw traffic data as training datasets by resolving its structure and content and pruning redundant information. Thanks to its ability to focus on key information, attention mechanism is introduced to design an attention-based long short-term memory (LSTM) model for traffic classification, termed the ABL-TC. ABL-TC can effectively identify global dependencies between the input and the output with a limited number of important raw features, which can contribute to learning key information to identify various traffic types. Extensive experiments on real-world public datasets show that the ABL-TC outperforms state-of-the-art approaches in terms of various metrics for recognizing traffic categories while remaining competitive in terms of time and memory efficiency.
Trojan horse or useful helper? A relationship perspective on artificial intelligence assistants with humanlike features

Abstract Artificial intelligence assistants (AIAs) such as Alexa are prevalent in consumers’ homes. Owing to their powerful artificial intelligence, consumers may perceive that AIAs have a mind of their own, that is, they anthropomorphize them. Past marketing research points to beneficial effects of AIA anthropomorphism for consumers and companies, while potential harmful effects have not been empirically explored. In examining both beneficial and harmful effects, this paper adopts a relationship perspective. Indeed, consumers spend large amounts of time with their AIAs, potentially developing a relationship over time that builds on an exchange of benefits and (psychological) costs. A preliminary survey and user interviews, a field study and a field experiment with AIA users show that AIA anthropomorphism may threaten users’ identity, which disempowers them, creates data privacy concerns and ultimately undermines their well-being. These harmful effects particularly emerge in close, long relationships. The field experiment uncovers three empowering interventions which attenuate harmful effects of AIA anthropomorphism in relationships with consumers. With AI-powered technologies taking larger roles in our daily lives, our research highlights key future directions to investigate the permanent ongoing nature of the consumer–AI relationships.
An overview of edge and object contour detection

In computer vision, edge and object contour detection is essential for higher-level vision tasks, such as shape matching, visual salience, image segmentation, and object recognition. It has attracted much attention during the past several decades, and many excellent methods have been proposed. In this paper, we make a comprehensive introduction to representative edge and object contour detection methods in the past two decades. Based on the development of these methods, we mainly classify them into two categories: traditional methods and learning-based methods. We further divide traditional methods into local pattern methods, edge grouping methods, active contour models, and bio-inspired methods. Further, we divide learning-based methods into classical learning-based methods and deep learning-based methods. At the same time, we introduce the most popular benchmarks and evaluation measures and quantitatively compare the performances of these promising methods. Moreover, we discuss current challenges in edge and object contour detection and suggest some future trends to bridge gaps with human vision. We believe that this overview will benefit newcomers and promote the development of edge and object contour detection.
Quaternion-based weighted nuclear norm minimization for color image restoration

Color image restoration is one of the basic tasks in pattern recognition. Unlike grayscale image, each color image has three channels in the RGB color space. Due to the inner-relationship within the three channels, color image restoration is usually much more difficult than its grayscale counterpart. Indeed, new problems such as color artifacts could emerge when the grayscale image processing methods are extended to color images directly. Note that one of the most effective gray image restoration methods is the weighted nuclear norm minimization (WNNM) approach. However, when applied to color images, the results of WNNM are usually not as promising as that of grayscale images. In order to solve this problem, in this paper, we propose to restore color images with the quaternion-based WNNM method (QWNNM) since the structure of color channels can be well preserved with quaternion representation. The proposed model can be solved efficiently by the alternating direction method of multipliers (ADMM). The theoretical analysis of the optimal solution is also presented. Numerical experiments are carefully conducted with different kinds of degradation to illustrate the superior performance of our proposed QWNNM over the state-of-the-art methods, including a celebrated deep learning approach, in both visual quality and numerical results.
Survey of Image Edge Detection

Edge detection technology aims to identify and extract the boundary information of image pixel mutation, which is a research hotspot in the field of computer vision. This technology has been widely used in image segmentation, target detection, and other high-level image processing technologies. In recent years, considering the problems of thick image edge contour, inaccurate positioning, and poor detection accuracy, researchers have proposed a variety of edge detection algorithms based on deep learning, such as multi-scale feature fusion, codec, network reconstruction, and so on. This paper dedicates to making a comprehensive analysis and special research on the edge detection algorithms. Firstly, by classifying the multi-level structure of traditional edge detection algorithms, the theory and method of each algorithm are introduced. Secondly, through focusing on the edge detection algorithm based on deep learning, the technical difficulties, advantages of methods, and backbone network selection of each algorithm are analysed. Then, through the experiments on the BSDS500 and NYUD dataset, the performance of each algorithm is further evaluated. It can be seen that the performance of the current edge detection algorithms is close to or even beyond the human visual level. At present, there are a few comprehensive review articles on image edge detection. This paper dedicates to making a comprehensive analysis of edge detection technology and aims to offer reference and guidance for the relevant personnel to follow up easily the current developments of edge detection and to make further improvements and innovations.
IEHC: An efficient image encryption technique using hybrid chaotic map

Image encryption is essential for the transmission of the sensitive images through insecure channels used in wired or wireless communication nowadays. With the advent of cloud enabled surveillance applications, Internet of Things applications, social media communications, the requirement of image encryption is also increasing day by day. The conventional encryption schemes demand high resources in terms of processing power, memory and bandwidth resulting degradation in overall system response time. In this paper, first a novel shuffling technique has been introduced to shuffle the pixels of an input color image. Subsequently, this paper proposes an image cipher that uses Logistic map and Arnold's cat map for encryption of color images. The experimental results and various security analyses exhibit good security characteristics and also prove its robustness against various types of noises, data loss and cryptanalysis attacks, making it appropriate for implementing in real-time applications.
On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles

Trajectory prediction is a critical component for autonomous vehicles (AVs) to perform safe planning and navigation. However, few studies have analyzed the adversarial robustness of trajectory prediction or investigated whether the worst-case prediction can still lead to safe planning. To bridge this gap, we study the adversarial robustness of trajectory prediction models by proposing a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error. Our experiments on three models and three datasets show that the adversarial prediction increases the prediction error by more than 150%. Our case studies show that if an adversary drives a vehicle close to the target AV following the adversarial trajectory, the AV may make an inaccurate prediction and even make unsafe driving decisions. We also explore possible mitigation techniques via data augmentation and trajectory smoothing.
Blockchain‐based recycling and its impact on recycling performance: A network theory perspective

Abstract The purpose of this study is to explore the application of blockchain technology (BCT) in recycling. This research applies a multiple case study approach with six pioneer organisations, based on secondary data. We found that BCT is an effective approach to promote recycling performance: it can provide tokenisation, waste flow tracking and recycling chain integration. The benefits include ‘Eco‐friendly’, ‘Stimulate participation’, ‘Social inclusion’, ‘Transparent recycling chains’ and ‘Extended producer responsibility accountability’. However, the majority of existing BCT‐based initiatives are in the pilot stage and face cognitive, technology, internal and external barriers. Our research is one of the first studies on blockchain‐based recycling. We applied the network theory of ‘Reachability’, ‘Richness’ and ‘Receptivity’ and ‘network formation’ barriers to propose a conceptual framework of BCT‐based recycling, which serves as a practical reference for the recycling industry.