Reviewerly/exHarmony huggingface dataset:
Explanation vs. attention: A two-player game to obtain attention for VQA and visual dialog

In this paper, we aim to obtain improved attention for a visual question answering (VQA) task. It is challenging to provide supervision for attention. An observation we make is that visual explanations as obtained through class activation mappings (specifically Grad-CAM) that are meant to explain the performance of various networks could form a means of supervision. However, as the distributions of attention maps and that of Grad-CAMs differ, it would not be suitable to directly use these as a form of supervision. Rather, we propose the use of a discriminator that aims to distinguish samples of visual explanation and attention maps. The use of adversarial training of the attention regions as a two-player game between attention and explanation serves to bring the distributions of attention maps and visual explanations closer. Significantly, we observe that providing such a means of supervision also results in attention maps that are more closely related to human attention resulting in a substantial improvement over baseline stacked attention network (SAN) models. It also results in a good improvement in rank correlation metric on the VQA task. This method can also be combined with recent MCB based methods and results in consistent improvement. We also provide comparisons with other means for learning distributions such as based on Correlation Alignment (Coral), Maximum Mean Discrepancy (MMD) and Mean Square Error (MSE) losses and observe that the adversarial loss outperforms the other forms of learning the attention maps. A generalization of the work is also provided by extending our approach to the task of 'Visual Dialog' where the attention is more contextual. Thorough evaluation for this task is also provided. Visualization of the results confirms our hypothesis that attention maps improve using the proposed form of supervision.
Forward Secure Message Franking

Message franking is introduced by Facebook in end-to-end encrypted messaging services. It allows to produce verifiable reports of malicious messages by including cryptographic proofs generated by Facebook. Recently, Grubbs et al. (CRYPTO’17) proceeded with the formal study of message franking and introduced committing authenticated encryption with associated data (CAEAD) as a core primitive for obtaining message franking. In this work, we aim to enhance the security of message franking and propose forward security for message franking. It guarantees the security associated with the past keys even if the current keys are exposed. Firstly, we propose the notion of key-evolving message franking including additional key update algorithms. Then, we formalize forward security for five security requirements: confidentiality, ciphertext integrity, unforgeability, receiver binding, and sender binding. Finally, we show a construction of forward secure message franking based on CAEAD, forward secure pseudorandom generator, and forward secure message authentication code.
Designated-Verifier Linkable Ring Signatures

We introduce Designated-Verifier Linkable Ring Signatures (DVLRS), a novel cryptographic primitive which combines designated-verifier and linkable ring signatures. Our goal is to guarantee signer ambiguity and provide the capability to the designated verifier to add ‘noise’ using simulated signatures that are publicly verifiable, thus increasing overall privacy. More formally, we model unforgeability, anonymity, linkability and non-transferability for DVLRS and provide a secure construction in the Random Oracle model. Finally, we explore applications for our primitive, that revolve around the use case of an anonymous assessment system that also protects the subject of the evaluation, even if the private key is compromised.
ℓ -Norm Support Vector Data Description

• The support vector data description method optimises an-norm cost w.r.t. errors • We generalise this modelling formalism to an-norm () slack penalty function • The proposed method enables formulating a non-linear cost function w.r.t. errors • Through a dual-norm, it introduces a controlling mechanism over the relative sparsity • Experiments confirm the merits of the proposed method against other alternatives The support vector data description (SVDD) approach serves as a de facto standard for one-class classification where the learning task entails inferring the smallest hyper-sphere to enclose target objects while linearly penalising the errors/slacks via an ℓ 1 -norm penalty term. In this study, we generalise this modelling formalism to a general ℓ p -norm ( p ≥ 1 ) penalty function on slacks. By virtue of an ℓ p -norm function, in the primal space, the proposed approach enables formulating a non-linear cost for slacks. From a dual problem perspective, the proposed method introduces a dual norm into the objective function, thus, proving a controlling mechanism to tune into the intrinsic sparsity/uniformity of the problem for enhanced descriptive capability. A theoretical analysis based on Rademacher complexities characterises the generalisation performance of the proposed approach while the experimental results on several datasets confirm the merits of the proposed method compared to other alternatives.
Riemannian dynamic generalized space quantization learning

Many existing works represent signals by covariance matrices and then develop learning methods on the Riemannian symmetric positive-definite (SPD) manifold to deal with such data. However, they summarize each instance with a single covariance matrix, omitting some potential important information, such as the time evolution of the correlation in signals. In this paper, we represent each instance by a sequence of covariance matrices and develop a novel dynamic generalized learning Riemannian space quantization (DGLRSQ) method to deal with such data representations. The proposed DGLRSQ method incorporates short-term memory mechanism in generalized learning Riemannian space quantization (GLRSQ), which is an extension of Euclidean generalized learning vector quantization to deal with SPD matrix-valued data. The proposed method can capture the temporal evolution of the correlation in signals and thus provides better performance to its the counterpart – GLRSQ, which treats each instance as a signal covariance matrix. Empirical investigations on synthetic data and motor imagery EEG data show the superior performance of the proposed method.
Towards using visual, semantic and structural features to improve code readability classification

Code readability, which correlates strongly with software quality, plays a critical role in software maintenance and evolvement. Although existing deep learning-based code readability models have reached a rather high classification accuracy, only structural features are utilized which inevitably limits their model performance. To address this problem, we propose to extract readability-related features from visual, semantic, and structural aspects from source code in an attempt to further improve code readability classification. First, we convert a code snippet into a RGB matrix (for visual feature extraction), a token sequence (for semantic feature extraction) and a character matrix (for structural feature extraction). Then, we input them into a hybrid neural network that is composed of BERT, CNN, and BiLSTM for feature extraction. Finally, the extracted features are concatenated and input into a classifier to make a code readability classification. A series of experiments are conducted to evaluate our method. The results show that the average accuracy could reach 85.3%, which outperforms all existing models. As an innovative work of extracting readability-related features automatically from visual, semantic, and structural aspects, our method is proved to be effective for the task of code readability classification.
ViV-Ano: Anomaly Detection and Localization Combining Vision Transformer and Variational Autoencoder in the Manufacturing Process

The goal of image anomaly detection is to determine whether there is an abnormality in an image. Image anomaly detection is currently used in various fields such as medicine, intelligent information, military fields, and manufacturing. The encoder–decoder structure, which learns a normal-looking periodic normal pattern and shows good performance in judging anomaly scores through reconstruction errors showing the differences between the reconstructed images and the input image, is widely used in the field of anomaly detection. The existing image anomaly detection method extracts normal information through local features of the image, but the vision transformer base and the probability distribution are generated by learning the global relationship between image anomaly detection and an image patch that can locate anomalies to extract normal information. We propose Vision Transformer and VAE for Anomaly Detection (ViV-Ano), an anomaly detection model that combines a model variational autoencoder (VAE) with Vision Transformer (ViT). The proposed ViV-Ano model showed similar or better performance when compared to the existing model on a benchmark dataset. In addition, an MVTec anomaly detection dataset (MVTecAD), a dataset for industrial anomaly detection, showed similar or improved performance when compared to the existing model.
3D Space Layout Design of Holographic Command Cabin Information Display in Mixed Reality Environment Based on HoloLens 2

After the command and control information of the command and control cabin is displayed in the form of mixed reality, the large amount of real-time information and static information contained in it will form a dynamic situation that changes all the time. This brings a great burden to the system operator's cognition, decision-making and operation. In order to solve this problem, this paper studies the three-dimensional spatial layout of holographic command cabin information display in a mixed reality environment. A total of 15 people participated in the experiment, of which 10 were the subjects of the experiment and 5 were the staff of the auxiliary experiment. Ten subjects used the HoloLens 2 generation to conduct visual characteristics and cognitive load experiments and collected and analyzed the subjects' task completion time, error rate, eye movement and EEG and subjective evaluation data. Through the analysis of experimental data, the laws of visual and cognitive features of three-dimensional space in a mixed reality environment can be obtained. This paper systematically explores the effects of three key attributes: depth distance, information layer number and target relative position depth distance of information distribution in a 3D space, on visual search performance and on cognitive load. The experimental results showed that the optimal depth distance range for information display in the mixed reality environment is: the best depth distance for operation interactions (0.6 m~1.0 m), the best depth distance for accurate identification (2.4 m~2.8 m) and the overall situational awareness best-in-class depth distance (3.4 m~3.6 m). Under a certain angle of view, the number of information layers in the space is as small as possible, and the number of information layers should not exceed five at most. The relative position depth distance between the information layers in space ranges from 0.2 m to 0.35 m. Based on this theory, information layout in a 3D space can achieve a faster and more accurate visual search in a mixed reality environment and effectively reduce the cognitive load.
Robot Static Path Planning Method Based on Deterministic Annealing

Heuristic calculation is an essential method to solve optimisation problems. However, its vast computing requirements limit its real-time and online applications, especially in embedded systems with limited computing resources, such as mobile robots. This paper presents a robot path planning algorithm called DA-APF based on deterministic annealing. It is derived from the artificial potential field and can effectively solve the local minimum problem of the model established by the potential field method. The calculation performance of DA-APF is considerably improved by introducing temperature parameters to enhance the potential field function and by using annealing and tempering methods. Moreover, an optimal or near-optimal robot path planning scheme is given. A comprehensive case study is performed using heuristic methods, such as genetic algorithm and simulated annealing. Simulation results show that DA-APF performs well in various static path planning environments.
Why adversarial reprogramming works, when it fails, and how to tell the difference

Adversarial reprogramming allows repurposing a machine-learning model to perform a different task. For example, a model trained to recognize animals can be reprogrammed to recognize digits by embedding an adversarial program in the digit images provided as input. Recent work has shown that adversarial reprogramming may not only be used to abuse machine-learning models provided as a service, but also beneficially, to improve transfer learning when training data is scarce. However, the factors affecting its success are still largely unexplained. In this work, we develop a first-order linear model of adversarial reprogramming to show that its success inherently depends on the size of the average input gradient, which grows when input gradients are more aligned, and when inputs have higher dimensionality. The results of our experimental analysis, involving fourteen distinct reprogramming tasks, show that the above factors are correlated with the success and the failure of adversarial reprogramming.
Multi-Source Domain Adaptation for Object Detection

To reduce annotation labor associated with object detection, an increasing number of studies focus on transferring the learned knowledge from a labeled source domain to another unlabeled target domain. However, existing methods assume that the labeled data are sampled from a single source domain, which ignores a more generalized scenario, where labeled data are from multiple source domains. For the more challenging task, we propose a unified Faster R-CNN based framework, termed Divide-and-Merge Spindle Network (DMSN), which can simultaneously enhance domain invariance and preserve discriminative power. Specifically, the framework contains multiple source subnets and a pseudo target subnet. First, we propose a hierarchical feature alignment strategy to conduct strong and weak alignments for low- and high-level features, respectively, considering their different effects for object detection. Second, we develop a novel pseudo subnet learning algorithm to approximate optimal parameters of pseudo target subset by weighted combination of parameters in different source subnets. Finally, a consistency regularization for region proposal network is proposed to facilitate each subnet to learn more abstract invariances. Extensive experiments on different adaptation scenarios demonstrate the effectiveness of the proposed model.
1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB’s Convergence Speed

To train large machine learning models (like BERT and GPT-3) on hundreds of GPUs, communication has become a significant bottleneck, especially on commodity systems with limited-bandwidth TCP networks. On one side, large batch-size optimization such as the LAMB algorithm was proposed to reduce the frequency of communication. On the other side, communication compression algorithms such as 1-bit Adam help to reduce the volume of each communication. However, we find that simply using one of the techniques is not sufficient to solve the communication challenge, especially under low network bandwidth. Motivated by this we aim to combine the power of large-batch optimization and communication compression but we find that existing compression strategies cannot be directly applied to LAMB due to its unique adaptive layerwise learning rates. To this end, we design a new communication-efficient optimization algorithm, 1-bit LAMB, which introduces a novel way to support adaptive layerwise learning rates under compression. In addition to the algorithm and corresponding theoretical analysis, we propose three novel system implementations in order to achieve actual wall clock speedup: a momentum fusion mechanism to reduce the number of communications, a momentum scaling technique to reduce compression error, and a NCCL-based compressed communication backend to improve both usability and performance. For the BERT-Large pre-training task with batch sizes from 8K to 64K, our evaluations on up to 256 GPUs demonstrate that our optimized implementation of 1-bit LAMB is able to achieve up to 4.6x communication volume reduction, up to 2.8x end-to-end time-wise speedup, and the same sample-wise convergence speed (and same fine-tuning task accuracy) compared to uncompressed LAMB. Furthermore, 1-bit LAMB achieves the same accuracy as LAMB on computer vision tasks like ImageNet and CIFAR100.
Exploring Soil Pollution Patterns Using Self-Organizing Maps

The geochemical composition of bedrock is the key feature determining elemental concentrations in soil, followed by anthropogenic factors that have less impact. Concerning the latter, harmful effects on the trophic chain are increasingly affecting people living in and around urban areas. In the study area of the present survey, the municipalities of Cosenza and Rende (Calabria, southern Italy), topsoil were collected and analysed for 25 elements by inductively coupled plasma mass spectrometry (ICP-MS) in order to discriminate the different possible sources of elemental concentrations and define soil quality status. Statistical and geostatistical methods were applied to monitoring the concentrations of major oxides and minor elements, while the Self-Organizing Maps (SOM) algorithm was used for unsupervised grouping. Results show that seven clusters were identified—(I) Cr, Co, Fe, V, Ti, Al; (II) Ni, Na; (III) Y, Zr, Rb; (IV) Si, Mg, Ba; (V) Nb, Ce, La; (VI) Sr, P, Ca; (VII) As, Zn, Pb—according to soil elemental associations, which are controlled by chemical and mineralogical factors of the study area parent material and by soil-forming processes, but with some exceptions linked to anthropogenic input.
Multimodal Feature Fusion Method for Unbalanced Sample Data in Social Network Public Opinion

With the wide application of social media, public opinion analysis in social networks has been unable to be met through text alone because the existing public opinion information includes data information of various modalities, such as voice, text, and facial expressions. Therefore multi-modal emotion analysis is the current focus of public opinion analysis. In addition, multi-modal emotion recognition of speech is an important factor restricting the multi-modal emotion analysis. In this paper, the emotion feature retrieval method for speech is firstly explored and the processing method of sample disequilibrium data is then analyzed. By comparing and studying the different feature fusion methods of text and speech, respectively, the multi-modal feature fusion method for sample disequilibrium data is proposed to realize multi-modal emotion recognition. Experiments are performed using two publicly available datasets (IEMOCAP and MELD), which shows that processing multi-modality data through this method can obtain good fine-grained emotion recognition results, laying a foundation for subsequent social public opinion analysis.
Evolutionary Algorithm with Deep Auto Encoder Network Based Website Phishing Detection and Classification

Website phishing is a cyberattack that targets online users for stealing their sensitive data containing login credential and banking details. The phishing websites appear very similar to their equivalent legitimate websites for attracting a huge amount of Internet users. The attacker fools the user by offering the masked webpage as legitimate or reliable for retrieving its important information. Presently, anti-phishing approaches necessitate experts to extract phishing site features and utilize third-party services for phishing website detection. These techniques have some drawbacks, as the requirement of experts for extracting phishing features is time consuming. Many solutions for phishing websites attack have been presented, such as blacklist or whitelist, heuristics, and machine learning (ML) based approaches, which face difficulty in accomplishing effectual recognition performance due to the continual improvements of phishing technologies. Therefore, this study presents an optimal deep autoencoder network based website phishing detection and classification (ODAE-WPDC) model. The proposed ODAE-WPDC model applies input data pre-processing at the initial stage to get rid of missing values in the dataset. Then, feature extraction and artificial algae algorithm (AAA) based feature selection (FS) are utilized. The DAE model with the received features carried out the classification process, and the parameter tuning of the DAE technique was performed using the invasive weed optimization (IWO) algorithm to accomplish enhanced performance. The performance validation of the ODAE-WPDC technique was tested using the Phishing URL dataset from the Kaggle repository. The experimental findings confirm the better performance of the ODAE-WPDC model with maximum accuracy of 99.28%.
A 3D Predictive Method for Deep-Seated Gold Deposits in the Northwest Jiaodong Peninsula and Predicted Results of Main Metallogenic Belts

With the rapid depletion of mineral resources, deep prospecting is becoming a frontier field in international geological exploration. The prediction of deep mineral resources is the premise and foundation of deep prospecting. However, conventional metallogenic predictive methods, which are mainly based on surface geophysical, geochemical, and remote sensing data and geological information, are no longer suitable for deep metallogenic prediction due to the large burial depth of deep-seated deposits. Consequently, 3D metallogenic prediction becomes a critical method for delineating deep prospecting target areas. As a world-class giant gold metallogenic province, the Jiaodong Peninsula is at the forefront in China in terms of deep prospecting achievements and exploration depth. Therefore, it has unique conditions for 3D metallogenic prediction and plays an important exemplary role in promoting the development of global deep prospecting. This study briefly introduced the method, bases, and results of the 3D metallogenic prediction in the northwest Jiaodong Peninsula and then established 3D geological models of gold concentration areas in the northwest Jiaodong Peninsula using drilling combined with geophysics. Since gold deposits in the northwest Jiaodong Peninsula are often controlled by faulting in the 3D space, this study proposed a method for predicting deep prospecting target areas based on a stepped metallogenic model and a method for predicting the deep resource potential of gold deposits based on the shallow resources of ore-controlling faults. Multiple characteristic variables were extracted from the 3D geological models of the gold concentration areas, including the buffer zone and dip angle of faults, the changing rate of fault dip angle, and the equidistant distribution of orebodies. Using these characteristic variables, five deep prospecting target areas in the Jiaojia and Sanshandao faults were predicted. Moreover, based on the proven gold resources at an elevation of −2000 m and above, the total gold resources of the Sanshandao, Jiaojia, and Zhaoping ore-controlling faults at an elevation of −5000–−2000 m were predicted to be approximately 3377–6490 t of Au. Therefore, it is believed that the total gold resources in the Jiaodong Peninsula are expected to exceed 10,000 t. These new predicted results suggest that the northwest Jiaodong Peninsula has huge potential for the resources of deep gold deposits, laying the foundation for further deep prospecting.