Reviewerly/exHarmony huggingface dataset:
Online Multi-modal Hashing with Dynamic Query-adaption

Multi-modal hashing is an effective technique to support large-scale multimedia retrieval, due to its capability of encoding heterogeneous multi-modal features into compact and similarity-preserving binary codes. Although great progress has been achieved so far, existing methods still suffer from several problems, including: 1) All existing methods simply adopt fixed modality combination weights in online hashing process to generate the query hash codes. This strategy cannot adaptively capture the variations of different queries. 2) They either suffer from insufficient semantics (for unsupervised methods) or require high computation and storage cost (for the supervised methods, which rely on pair-wise semantic matrix). 3) They solve the hash codes with relaxed optimization strategy or bit-by-bit discrete optimization, which results in significant quantization loss or consumes considerable computation time. To address the above limitations, in this paper, we propose an Online Multi-modal Hashing with Dynamic Query-adaption (OMH-DQ) method in a novel fashion. Specifically, a self-weighted fusion strategy is designed to adaptively preserve the multi-modal feature information into hash codes by exploiting their complementarity. The hash codes are learned with the supervision of pair-wise semantic labels to enhance their discriminative capability, while avoiding the challenging symmetric similarity matrix factorization. Under such learning framework, the binary hash codes can be directly obtained with efficient operations and without quantization errors. Accordingly, our method can benefit from the semantic labels, and simultaneously, avoid the high computation complexity. Moreover, to accurately capture the query variations, at the online retrieval stage, we design a parameter-free online hashing module which can adaptively learn the query hash codes according to the dynamic query contents. Extensive experiments demonstrate the state-of-the-art performance of the proposed approach from various aspects.
Blockchain Enabled Industrial Internet of Things Technology

The emerging blockchain technology shows promising potential to enhance industrial systems and the Internet of things (IoT) by providing applications with redundancy, immutable storage, and encryption. In the past few years, many more applications in industrial IoT (IIoT) have emerged and the blockchain technologies have attracted huge amounts of attention from both industrial and academic researchers. In this paper, we address the integration of blockchain and IIoT from the industrial prospective. A blockchain-enabled IIoT framework is introduced and involved fundamental techniques are presented. Moreover, the main applications and key challenges are addressed. A comprehensive analysis for the most recent research trends and open issues is provided associated with the blockchain-enabled IIoT.
RegularFace: Deep Face Recognition via Exclusive Regularization

We consider the face recognition task where facial images of the same identity (person) is expected to be closer in the representation space, while different identities be far apart. Several recent studies encourage the intra-class compactness by developing loss functions that penalize the variance of representations of the same identity. In this paper, we propose the `exclusive regularization' that focuses on the other aspect of discriminability -- the inter-class separability, which is neglected in many recent approaches. The proposed method, named RegularFace, explicitly distances identities by penalizing the angle between an identity and its nearest neighbor, resulting in discriminative face representations. Our method has intuitive geometric interpretation and presents unique benefits that are absent in previous works. Quantitative comparisons against prior methods on several open benchmarks demonstrate the superiority of our method. In addition, our method is easy to implement and requires only a few lines of python code on modern deep learning frameworks.
Two-Stream Collaborative Learning With Spatial-Temporal Attention for Video Classification

Video classification is highly important and has widespread applications, such as video search and intelligent surveillance. Video naturally contains both static and motion information, which can be represented by frames and optical flow, respectively. Recently, researchers have generally adopted deep networks to capture the static and motion information separately, which has two main limitations. First, the coexistence relationship between spatial and temporal attention is ignored, although they should be jointly modeled as the spatial and temporal evolutions of video to learn discriminative video features. Second, the strong complementarity between static and motion information is ignored, although they should be collaboratively learned to enhance each other. To address the above two limitations, this paper proposes the two-stream collaborative learning with spatial-temporal attention (TCLSTA) approach, which consists of two models. First, for the spatial-temporal attention model, the spatial-level attention emphasizes the salient regions in a frame, and the temporal-level attention exploits the discriminative frames in a video. They are mutually enhanced to jointly learn the discriminative static and motion features for better classification performance. Second, for the static-motion collaborative model, it not only achieves mutual guidance between static and motion information to enhance the feature learning but also adaptively learns the fusion weights of static and motion streams, thus exploiting the strong complementarity between static and motion information to improve video classification. Experiments on four widely used data sets show that our TCLSTA approach achieves the best performance compared with more than 10 state-of-the-art methods.
Multi-Modal Biometric-Based Implicit Authentication of Wearable Device Users

The Internet of Things (IoT) is increasingly empowering people with an interconnected world of physical objects ranging from smart buildings to portable smart devices, such as wearables. With recent advances in mobile sensing, wearables have become a rich collection of portable sensors and are able to provide various types of services, including tracking of health and fitness, making financial transactions, and unlocking smart locks and vehicles. Most of these services are delivered based on users' confidential and personal data, which are stored on these wearables. Existing explicit authentication approaches (i.e., PINs or pattern locks) for wearables suffer from several limitations, including small or no displays, risk of shoulder surfing, and users' recall burden. Oftentimes, users completely disable security features out of convenience. Therefore, there is a need for a burden-free (implicit) authentication mechanism for wearable device users based on easily obtainable biometric data. In this paper, we present an implicit wearable device user authentication mechanism using combinations of three types of coarse-grain minute-level biometrics: behavioral (step counts), physiological (heart rate), and hybrid (calorie burn and metabolic equivalent of task). From our analysis of over 400 Fitbit users from a 17-month long health study, we are able to authenticate subjects with average accuracy values of around .93 (sedentary) and .90 (non-sedentary) with equal error rates of .05 using binary SVM classifiers. Our findings also show that the hybrid biometrics perform better than other biometrics and behavioral biometrics do not have a significant impact, even during non-sedentary periods.
Shopping in Virtual Reality Stores: The Influence of Immersion on System Adoption

Companies have the opportunity to better engage potential customers by presenting products to them in a highly immersive virtual reality (VR) shopping environment. However, a minimal amount is known about why and whether customers will adopt such fully immersive shopping environments. We therefore develop and experimentally validate a theoretical model, which explains how immersion affects adoption. The participants experienced the environment by using a head-mounted display (high immersion) or by viewing product models in 3D on a desktop (low immersion). We find that immersion does not affect the users’ intention to reuse the shopping environment, because two paths cancel each other out: Highly immersive shopping environments positively influence a hedonic path through telepresence, but surprisingly, they negatively influence a utilitarian path through product diagnosticity. We can explain this effect via low readability of product information in the VR environment and expect VR’s full potential to develop when the technology is further advanced. Our study contributes to literature on immersive systems and IS adoption research by introducing a research model for the adoption of VR shopping environments. A key practical implication of our study is that system designers need to pay special attention to the current state of technology when designing VR applications.
Scalable Zero Knowledge with No Trusted Setup

One of the approaches to constructing zero knowledge (ZK) arguments relies on “PCP techniques” that date back to influential works from the early 1990’s [Babai et al., Arora et al. 1991-2]. These techniques require only minimal cryptographic assumptions, namely, the existence of a family of collision-resistant hash functions [Kilian, STOC 1992], and achieve two remarkable properties: (i) all messages generated by the verifier are public random coins, and (ii) total verification time is merely poly-logarithmic in the time needed to naïvely execute the computation being verified [Babai et al., STOC 1991]. Those early constructions were never realized in code, mostly because proving time was too large. To address this, the model of interactive oracle proofs (IOPs), which generalizes the PCP model, was recently suggested. Proving time for ZK-IOPs was reduced to quasi-linear, even for problems that require nondeterministic exponential time to decide [Ben-Sasson et al., TCC 2016, ICALP 2017]. Despite these recent advances it was still not clear whether ZK-IOP systems can lead to concretely efficient succinct argument systems. Our main claim is that this is indeed the case. We present a new construction of an IOP of knowledge (which we call a zk-STIK) that improves, asymptotically, on the state of art: for log-space computations of length T it is the first to $$O(T \log T)$$ arithmetic prover complexity and $$O(\log T)$$ verifier arithmetic complexity. Prior IOPs had additional $$\mathsf{poly} \log T$$ factors in both prover and verifier. Additionally, we report a C++ realization of this system (which we call libSTARK). Compared to prevailing ZK realizations, it has the fastest proving and (total) verification time for sufficiently large sequential computations.
An Introductory Survey on Attention Mechanisms in NLP Problems

First derived from human intuition, later adapted to machine translation for automatic token alignment, attention mechanism, a simple method that can be used for encoding sequence data based on the importance score each element is assigned, has been widely applied to and attained significant improvement in various tasks in natural language processing, including sentiment classification, text summarization, question answering, dependency parsing, etc. In this paper, we survey through recent works and conduct an introductory summary of the attention mechanism in different NLP problems, aiming to provide our readers with basic knowledge on this widely used method, discuss its different variants for different tasks, explore its association with other techniques in machine learning, and examine methods for evaluating its performance.
Distributed Heavy-Ball: A Generalization and Acceleration of First-Order Methods With Gradient Tracking

We study distributed optimization to minimize a sum of smooth and strongly-convex functions. Recent work on this problem uses gradient tracking to achieve linear convergence to the exact global minimizer. However, a connection among different approaches has been unclear. In this paper, we first show that many of the existing first-order algorithms are related with a simple state transformation, at the heart of which lies a recently introduced algorithm known as AB. We then present distributed heavy-ball, denoted as ABm, that combines AB with a momentum term and uses nonidentical local step-sizes. By simultaneously implementing both rowand column-stochastic weights, ABm removes the conservatism in the related work due to doubly stochastic weights or eigenvector estimation. ABm thus naturally leads to optimization and average consensus over both undirected and directed graphs. We show that ABm has a global R-linear rate when the largest step-size and momentum parameter are positive and sufficiently small. We numerically show that ABm achieves acceleration, particularly when the objective functions are ill-conditioned.
NetSpectre: Read Arbitrary Memory over Network

All Spectre attacks so far required local code execution. We present the first fully remote Spectre attack. For this purpose, we demonstrate the first access-driven remote Evict+Reload cache attack over the network, leaking 15 bits per hour. We present a novel high-performance AVX-based covert channel that we use in our cache-free Spectre attack. We show that in particular remote Spectre attacks perform significantly better with the AVX-based covert channel, leaking 60 bits per hour from the target system. We demonstrate practical NetSpectre attacks on the Google cloud, remotely leaking data and remotely breaking ASLR.
Adversarial Defense by Restricting the Hidden Space of Deep Neural Networks

Deep neural networks are vulnerable to adversarial attacks which can fool them by adding minuscule perturbations to the input images. The robustness of existing defenses suffers greatly under white-box attack settings, where an adversary has full knowledge about the network and can iterate several times to find strong perturbations. We observe that the main reason for the existence of such perturbations is the close proximity of different class samples in the learned feature space. This allows model decisions to be totally changed by adding an imperceptible perturbation in the inputs. To counter this, we propose to class-wise disentangle the intermediate feature representations of deep networks. Specifically, we force the features for each class to lie inside a convex polytope that is maximally separated from the polytopes of other classes. In this manner, the network is forced to learn distinct and distant decision regions for each class. We observe that this simple constraint on the features greatly enhances the robustness of learned models, even against the strongest white-box attacks, without degrading the classification performance on clean images. We report extensive evaluations in both black-box and white-box attack scenarios and show significant gains in comparison to state-of-the art defenses.
MDFN: Multi-scale deep feature learning network for object detection

This paper proposes an innovative object detector by leveraging deep features learned in high-level layers. Compared with features produced in earlier layers, the deep features are better at expressing semantic and contextual information. The proposed deep feature learning scheme shifts the focus from concrete features with details to abstract ones with semantic information. It considers not only individual objects and local contexts but also their relationships by building a multi-scale deep feature learning network (MDFN). MDFN efficiently detects the objects by introducing information square and cubic inception modules into the high-level layers, which employs parameter-sharing to enhance the computational efficiency. MDFN provides a multi-scale object detector by integrating multi-box, multi-scale and multi-level technologies. Although MDFN employs a simple framework with a relatively small base network (VGG-16), it achieves better or competitive detection results than those with a macro hierarchical structure that is either very deep or very wide for stronger ability of feature extraction. The proposed technique is evaluated extensively on KITTI, PASCAL VOC, and COCO datasets, which achieves the best results on KITTI and leading performance on PASCAL VOC and COCO. This study reveals that deep features provide prominent semantic information and a variety of contextual contents, which contribute to its superior performance in detecting small or occluded objects. In addition, the MDFN model is computationally efficient, making a good trade-off between the accuracy and speed.
Deep Clustering With Sample-Assignment Invariance Prior

Most popular clustering methods map raw image data into a projection space in which the clustering assignment is obtained with the vanilla k-means approach. In this article, we discovered a novel prior, namely, there exists a common invariance when assigning an image sample to clusters using different metrics. In short, different distance metrics will lead to similar soft clustering assignments on the manifold. Based on such a novel prior, we propose a novel clustering method by minimizing the discrepancy between pairwise sample assignments for each data point. To the best of our knowledge, this could be the first work to reveal the sample-assignment invariance prior based on the idea of treating labels as ideal representations. Furthermore, the proposed method is one of the first end-to-end clustering approaches, which jointly learns clustering assignment and representation. Extensive experimental results show that the proposed method is remarkably superior to 16 state-of-the-art clustering methods on five image data sets in terms of four evaluation metrics.
Neural Cryptography Based on Complex-Valued Neural Network

Neural cryptography is a public key exchange algorithm based on the principle of neural network synchronization. By using the learning algorithm of a neural network, the two neural networks update their own weight through exchanging output from each other. Once the synchronization is completed, the weights of the two neural networks are the same. The weights of the neural network can be used for the secret key. However, all the existing works are based on the real-valued neural network model. There are seldom works studying the neural cryptography based on a complex-valued neural network model. In this technical note, a neural cryptography based on the complex-valued tree parity machine network (CVTPM) is proposed. The input, output, and weights of CVTPM are a complex value, which can be considered as an extension of TPM. There are two advantages of the CVTPM: 1) the security of CVTPM is higher than that of TPM with the same hidden units, input neurons, and synaptic depths and 2) the two parties with the CVTPM can exchange two group keys in one neural synchronization process. A series of numerical simulation experiments is provided to verify our results.
A crypto safe haven against Bitcoin

The design of Bitcoin is closely related to gold which has led to the idea that Bitcoin has gold-like features such as being a store of value and a safe haven. However, given Bitcoin’s extreme volatility investors may rather need a safe haven against Bitcoin. We hypothesize that stablecoins provide such a safe haven and analyze high-frequency price changes of the largest stablecoins. We find that they act as safe havens, and Tether showing the strongest effect. The results demonstrate that Bitcoin investors seek out stablecoins when Bitcoin experiences extreme negative prices changes. Since stablecoins react to such price changes they are not stable at all times.
Adaptive chaotic maps and their application to pseudo-random numbers generation

Chaos-based stream ciphers form a prospective class of data encryption techniques. Usually, in chaos-based encryption schemes, the pseudo-random generators based on chaotic maps are used as a source of randomness. Despite the variety of proposed algorithms, nearly all of them possess many shortcomings. While sequences generated from single-parameter chaotic maps can be easily compromised using the phase space reconstruction method, the employment of multi-parametric maps requires a thorough analysis of the parameter space to establish the areas of chaotic behavior. This complicates the determination of the possible keys for the encryption scheme. Another problem is the degradation of chaotic dynamics in the implementation of the digital chaos generator with finite precision. To avoid the appearance of quasi-chaotic regimes, additional perturbations are usually introduced into the chaotic maps, making the generation scheme more complex and influencing the oscillations regime. In this study, we propose a novel technique utilizing the chaotic maps with adaptive symmetry to create chaos-based encryption schemes with larger parameter space. We compare pseudo-random generators based on the traditional Zaslavsky map and the new adaptive Zaslavsky web map through multi-parametric bifurcation analysis and investigate the parameter spaces of the maps. We explicitly show that pseudo-random sequences generated by the adaptive Zaslavsky map are random, have a weak correlation and possess a larger parameter space. We also present the technique of increasing the period of the chaotic sequence based on the variability of the symmetry coefficient. The speed analysis shows that the proposed encryption algorithm possesses a high encryption speed, being compatible with the best solutions in a field. The obtained results can improve the chaos-based cryptography and inspire further studies of chaotic maps as well as the synthesis of novel discrete models with desirable statistical properties.
Augmented and Virtual Reality in Construction: Drivers and Limitations for Industry Adoption

Augmented and virtual reality have the potential to provide a step-change in productivity in the construction sector; however, the level of adoption is very low. This paper presents a systematic study of the factors that limit and drive adoption in a construction sector–specific context. A mixed research method was employed, combining qualitative and quantitative data collection and analysis. Eight focus groups with 54 experts and an online questionnaire were conducted. Forty-two limiting and driving factors were identified and ranked. Principal component analysis was conducted to group the identified factors into a smaller number of factors based on correlations. Four types of limiting factors and four types of driving factors were identified. The main limitation of adoption is that AR and VR technologies are regarded as expensive and immature technologies that are not suitable for engineering and construction. The main drivers are that AR and VR enable improvements in project delivery and provision of new and better services. This study provides valuable insights to stakeholders to devise actions that mitigate the limiting factors and that boost the driving factors. This is one of the first systematic studies to present a detailed analysis of the factors that limit and drive adoption of AR and VR in the construction industry. The main contribution of this study is that it grouped and characterized myriad limiting and driving factors into easily understandable categories, so that the limiting factors can be effectively mitigated and the driving factors potentiated. A roadmap with specific short-term and medium-term actions for improving adoption was outlined.