Reviewerly/exHarmony huggingface dataset:
Review of Highlight Suppression Methods for Structured Light 3D Measurement

Structured light 3D measurement technology undergoes continuous development and perfection since its appearance. Today, it has become a relatively mature technology widely used in various fields. However, there are still some problems to be solved in the development of structured light 3D measurement technology. In particular, the effect of highlight on the measurement results when measuring specularly reflective objects gets more and more noticeable. The specular reflection-induced highlight is one of the difficulties in 3D measurement of structured light, and the existing technology is unable to meet the requirements of increasingly complex and precise measurement work. This paper summarizes the domestic and foreign representative research results on highlight suppression methods based on structured light 3D measurement, which can be divided into three categories: software-based suppression method, hardware-based suppression method, and software-hardware combined suppression method.
OVQE: Omniscient Network for Compressed Video Quality Enhancement

How to use information from temporal, spatial, and frequency domain dimensions is crucial for the quality enhancement of compressed video. The state-of-the-art methods generally design powerful networks to fuse the spatiotemporal information of the videos. But the spatiotemporal information of the entire video is not fully utilized and effectively fused, resulting in the learned context information that is not closely related to the target frame. In addition, various compressed videos have varying degrees of frequency domain information loss. The previous methods ignored the non-uniform distortion of compressed video in different frequency domains and did not design unique algorithms for different frequency domains, so the real texture details of the video could not be restored. In this paper, we propose an omniscient network, which learns video spatiotemporal and omni-frequency information more effectively. The omniscient network consists of two novel components: a Spatio-Temporal Feature Fusion (STFF) module and an Omni-Frequency Adaptive Enhancement (OFAE) block. The former aims to capture spatiotemporal information in adjacent frames, while the latter aims to adaptively recover different frequency domains of compressed video. The information is designed to be bidirectionally propagated in a grid manner such that the omni-enhanced results can be applied. Extensive experiments show that our method outperforms the state-of-the-art method in terms of objective metrics, subjective visual effects, and model complexity.
Self-Supervised Vision Transformer Based Nearest Neighbor Classification for Multi-Source Open-Set Domain Adaptation

Domain adaptation alleviates the performance drop when models are deployed in a target domain. Models assuming a close-set world fail in realistic open-set scenarios where novel classes not present in the source domain exist. Moreover, there are likely to be multiple source domains sharing the same label set but having different data distributions. These real situations make multi-source open-set domain adaptation (MSOSDA) a practical problem but have not been fully explored. The difficulty of MSOSDA is learning a common discriminative feature space among all domains while maximizing the separation between source classes and target-private ones. In this work, we propose a self-supervised vision transformer (ViT) based nearest neighbor classifier for MSOSDA. Our critical insight is to exploit the strong nearest neighbor classification property of self-supervised ViT along with supervised contrastive learning. Straightforward strategies and an adaptive data-driven threshold are adopted to explicitly align among domains and recognize open-set classes in the target domain. Extensive experiments on three popular benchmarks demonstrate the effectiveness of our approach.
Knowledge Transfer for Label-Efficient Monocular Height Estimation

Estimating height from monocular remote sensing images is one of the most efficient ways for building large-scale 3D city models. However, existing deep learning based methods usu-ally require a large amount of training data, which could be cost-consuming or even not possible to obtain. Towards a label-efficient deep learning model, we propose a new task and dataset for weak-shot monocular height estimation. In this task, only the relative height labels between pairs of a small portion of points are given, which is cheaper and more friendly for humans to annotate. In addition, to enhance the model performance under the sparse and weak-shot super-vision, we propose a Transformer-based network for trans-ferring the learned knowledge from a large-scale synthetic dataset to real-world data. Experimental results have shown the effectiveness of the proposed method on a public dataset under the sparse and weak supervision.
A Novel Counterexample-Guided Inductive Synthesis Framework for Barrier Certificate Generation

Barrier certificate is a powerful and practical approach of safety verification for hybrid systems. In this paper, we propose a novel Counterexample-Guided Inductive Synthesis (CEGIS) procedure for synthesizing neural barrier certificates. The CEGIS procedure is structured as an inductive loop where a learner and a verifier interact to synthesize barrier certificates. The learner trains candidate barrier certificates expressed as feedforward neural networks with polynomial activations, and the verifier employs computer algebra techniques to either ensure the validity of the trained candidate barrier certificate or produce informative counterexamples, which can effectively reduce the number of CEGIS iterations. We implement the CEGIS tool and evaluate its performance over a set of benchmarks. The experimental results demonstrate the effectiveness and efficiency of our approach.
SoK: Limitations of Confidential Computing via TEEs for High-Performance Compute Systems

Trusted execution environments (TEEs) are primary enablers of confidential computing. This paper presents a systematization of the existing trusted execution environments in industry and academia. We highlight the common mechanisms these TEEs employ to provide different security guarantees and offer a detailed comparative analysis of different TEE proposals. TEEs are anticipated to be a promising solution for the security challenges in the high-performance computing (HPC) domain. However, this paper shows why the existing TEEs are unsuitable for high-performance computing systems. Finally, we present our call for action to work to evolve the TEE technologies with the evolving high-performance computing landscape.
Visual Analysis of Spatio-Temporal Earthquake Events

Earthquakes are one of the most destructive natural forces on Earth. Helping uncover and understand the hidden patterns behind the occurrence of Earthquakes is vital for preparation and emergency response. Unfortunately, it is difficult to do so as those patterns probably depend on too many variables to be easily perceived with common visualization techniques. In this paper we expand on a technique designed for eye movement data and apply it to spatio-temporal earthquake events. This visualization combines the spatial and temporal dimension into a line-based splatted diagram that provides an overview and serves as a starting point for further data explorations in space and time.
Image Derain Method for Generative Adversarial Network Based on Wavelet High Frequency Feature Fusion

Photos taken on rainy days can be affected by rain streaks that reduce the sharpness of the image. Due to insufficient attention on feature extraction of rain streaks area, the removal effect of noise area needs to be improved. Wavelet transform is used to separate the high frequency information of the image, convolution is used to extract the high frequency features of the image, and the features of the original image with rain streaks extracted by the network are superimposed and fused. High frequency information graph represents the location of image noise. By incorporating high frequency features, the network can further learn the features of the rain stripes region. Feature fusion is introduced into the generative network, and the generative network guided by the attention distribution map considers global information more on the precondition of paying attention to the rain stripes region, so as to improve the clarity of the image after removing the rain fringes. The comparison experiment of wavelet high frequency feature fusion generative adversarial network and other methods is completed. The evaluation metrics are peak signal-to-noise ratio (PSNR) and structure similarity (SSIM), which verify the superiority of the proposed method compared with other methods.
Lock-Based Proof of Authority: A Faster and Low-Forking PoA Fault Tolerance Protocol for Blockchain Systems

A key component of a blockchain system is the consensus protocol, which directly affects the reliability and performance of blockchain systems. There are lots of blockchain consensus protocols. Proof of Authority (PoA) is a popular consensus protocol, and it has been embedded in many blockchain systems. However, PoA protocol cannot satisfy high transaction throughput, low transaction latency, and low forking rate at the same time, and can only take parts of them. This contradiction limits the practical application of PoA and needs to be resolved. Facing the challenges, in this paper, we present LPoA (Lock-based Proof of Authority), a new improved PoA consensus protocol. The experimental results show that LPoA delivers around 3x throughput, 1/3 transaction latency, and almost 0 forking rate over PoA under the same transactions.
Graph Embedding: A Methodological Survey

Embedding a high dimensional combinatorial object like tokens in text or nodes in graphs into a lower dimensional Euclidean space is a form of (lossy) data compression. We will demonstrate a class of procedures to embed vertices of a (connected) graph into a low-dimensional Euclidean space. We explore two kinds of embedding, one node2vec, similar to word2vec, which deploys a shallow network and a recurrent network which remembers past moves and takes [sic] spatial correlations into an account. We also explore the extent in which graph embedding preserves information and the practicality of using the information stored in a compressed form to discern meaningful patterns. With growth in their popularity, we too make an extensive use of the neural networks computational frameworks; we propose the usage of various neural network architectures to implement an encoder-decoder scheme to learn ‘hidden’ features. Since training a network requires data, we describe various sampling techniques including novel methods to sample from a graph; one using a vertex cover and another is an Eulerian tour of a (possibly) modified graph.
Domain Adaptation with Optimal Transport for Extended Variable Space

Domain adaptation aims to transfer knowledge of labeled instances obtained from a source domain to a target domain to fill the gap between the domains. Most domain adaptation methods assume that the source and target domains have the same dimensionality. Methods that are applicable when the number of features for each sample is different in each domain have rarely been studied, especially when no label information is given for the test data obtained from the target domain. In this paper, it is assumed that common features exist in both domains and that extra (new additional) features are observed in the target domain; hence, the dimensionality of the target domain is higher than that of the source domain. To leverage the homogeneity of the common features, the adaptation between the source and target domains is formulated as an optimal transport (OT) problem. In addition, a learning bound in the target domain for the proposed OT-based method is derived. The experiments with simulated and real-world data show that our proposed algorithm is able to obtain better model for the target domain by considering the extra features given for the target domain.
Trainable Compound Activation Functions for Machine Learning

Activation functions (AF) are necessary components of neural networks that allow approximation of functions, but AFs in current use are usually simple monotonically increasing functions. In this paper, we propose trainable compound AF (TCA) composed of a sum of shifted and scaled simple AFs. TCAs increase the effectiveness of networks with fewer parameters compared to added layers. TCAs have a special interpretation in generative networks because they effectively estimate the marginal distributions of each dimension of the data using a mixture distribution, reducing modality and making linear dimension reduction more effective. When used in restricted Boltzmann machines (RBMs), they result in a novel type of RBM with mixture-based stochastic units. Improved performance is demonstrated in experiments using RBMs, deep belief networks (DBN), projected belief networks (PBN), and variational auto-encoders (VAE).
IMBR: Interactive Multi-relation Bundle Recommendation with Graph Neural Network

Traditional approaches focus on an individual item of most interest to users. However, in most realistic scenarios, the platform needs to recommend a group of items at one time for users' convenience, called bundle recommendation. e.g., a music playlist containing multiple songs. The existing bundle recommendations usually use manual methods to artificially build bundles for different items, ignoring the obtained bundles and the potential relationships among the items in the bundle, especially the relationships between bundles. Therefore, how integrating multiple complex interactions into bundles and obtaining high-quality bundle recommendation is an important problem. To solve the problem, we propose a novel model named IMBR (short for Interactive Multi-Relation Bundle Recommendation with Graph Neural Network). Firstly, we construct a multi-relation interaction graph to capture the interaction relation from the user view. At the same time, we get bundle subordination relation from the item view. They can obtain richer representations of users, bundles, and items. Secondly, we design a bundle frequent term constraint algorithm (BFTC) to constrain the composition of items in a bundle and pay attention to the similarity between bundles. Finally, we leverage a multi-task learning framework to capture user personalized preferences to improve the performance of bundle recommendation. Extensive experiments on two real-world datasets with different scales show that our method can significantly outperform various baseline approaches.
Evaluating Adversarial Attacks and Defences in Infrared Deep Learning Monitoring Systems

This paper studies adversarial attacks and defences against deep learning models trained on infrared data to classify the presence of humans and detect their bounding boxes, which differently from the standard RGB case is an open research problem with multiple consequences related to safety and secure artificial intelligence applications. The paper has two major contributions. Firstly, we study the effectiveness of the Projected Gradient Descent (PGD) adversarial attack against Convolutional Neural Networks (CNNs) trained exclusively on infrared data, and the effectiveness of adversarial training as a possible defense against the attack. Secondly, we study the response of an object detection model trained on infrared images under adversarial attacks. In particular, we propose and empirically evaluate two attacks: one classical attack from the literature on object detection, and a new hybrid attack which exploits a common CNN base architecture of the classifier and the object detector. We show for the first time that adversarial attacks weaken the performance of classification and detection models trained on infrared images only. We also prove that the defense adversarial training optimized for the infinity norm increases the robustness of different classification models trained on infrared data.
Heterogeneous Computing and Applications in Deep Learning: A Survey

With the rapid development of deep learning, a variety of neural network models emerge in endlessly, which leads to a huge demand for computing resources. For the intensive numerical computation of neural networks, various computing devices represented by GPUs are favored by researchers. Heterogeneous computing is a kind of technology that can integrate a variety of computing devices with different architectures, and it will be further developed. Therefore, this paper reviews research on some key technologies of heterogeneous computing, including the architecture of heterogeneous computing, the programming language of heterogeneous computing, and the scheduling algorithm for heterogeneous systems. Then, we focus on the research of heterogeneous computing in deep learning, including the parallel technology of neural networks and optimization technology based on heterogeneous systems. Finally, the present research situation is discussed and analyzed, and the future research direction is prospected, aiming to provide some basis for related research.
Near-Optimal Search Time in $$\delta $$-Optimal Space

Two recent lower bounds on the compressiblity of repetitive sequences, $$\delta \le \gamma $$ , have received much attention. It has been shown that a string S[1..n] can be represented within the optimal $$O(\delta \log \tfrac{n}{\delta })$$ space, and further, that within that space one can find all the occ occurrences in S of any pattern of length m in time $$O(m\log n + occ \log ^\epsilon n)$$ for any constant $$\epsilon >0$$ . Instead, the near-optimal search time $$O(m+({occ+1})\log ^\epsilon n)$$ was achieved only within $$O(\gamma \log \frac{n}{\gamma })$$ space. Both results are based on considerably different locally consistent parsing techniques. The question of whether the better search time could be obtained within the $$\delta $$ -optimal space was open. In this paper, we prove that both techniques can indeed be combined in order to obtain the best of both worlds, $$O(m+({occ+1})\log ^\epsilon n)$$ search time within $$O(\delta \log \tfrac{n}{\delta })$$ space.
Evaluating Situated Visualization in AR with Eye Tracking

Augmented reality (AR) technology provides means for embedding visualization in a real-world context. Such techniques allow situated analyses of live data in their spatial domain. However, as existing techniques have to be adapted for this context and new approaches will be developed, the evaluation thereof poses new challenges for researchers. Apart from established performance measures, eye tracking has proven to be a valuable means to assess visualizations qualitatively and quantitatively. We discuss the challenges and opportunities of eye tracking for the evaluation of situated visualizations. We envision that an extension of gaze-based evaluation methodology into this field will provide new insights on how people perceive and interact with visualizations in augmented reality.
Adversarial Eigen Attack on BlackBox Models

Black-box adversarial attack has aroused much research attention for its difficulty on nearly no available information of the attacked model and the additional constraint on the query budget. A common way to improve attack efficiency is to transfer the gradient information of a white-box substitute model trained on an extra dataset. In this paper, we deal with a more practical setting where a pre-trained white-box model with network parameters is provided without extra training data. To solve the model mismatch problem between the white-box and black-box models, we propose a novel algorithm EigenBA by systematically integrating gradient-based white-box method and zeroth-order optimization in black-box methods. We theoretically show the optimal directions of perturbations for each step are closely related to the right singular vectors of the Jacobian matrix of the pretrained white-box model. Extensive experiments on ImageNet, CIFAR-10 and WebVision show that EigenBA can consistently and significantly outperform state-of-the-art baselines in terms of success rate and attack efficiency.
Mutual information based objective model for assessment of visual quality

The humans are the ultimate assessor of image or video quality. Thus, evaluating the quality using human perception is a very crucial and interesting topic in the quality assessment research domain. Even though the subjective assessment is the most reliable one, it is time consuming and costly. On the other hand, the objective model provides the assessment result with much less time and labour, albeit with certain degree of tolerable error. In this piece of work, a new objective model, namely visual attentive information fidelity (VAIF) has been proposed in the spatial domain, instead of transformed domain, for assessment of visual quality in a scene or in a patch of the scene. VAIF relates mutual information (MI) and image fidelity using source and destination model. The proposed objective model metrics has been correlated with subjective measurement metrics like MOS, NAS.
Information Adversarial Disentanglement for Face Swapping

Face swapping can provide data support for face forgery detection, which is a very significant topic in forensics. It is the task of converting the source identity to the target face while preserving target attributes, thus disentangling identity and identity-unrelated (i.e., attribute) features is still a challenging task. In this work, we focus on intra-class (i.e. identities) and inter-class (i.e. identity and attribute) relationships to comprehensively decouple identity and attribute features in an adversarial way for face swapping. The whole network includes Identity-Attribute Adversary (IAA) module, Identity Reconstruction (IR) module and Re-Feeding module. Specifically, for the inter-class relationship, we first propose the IAA module to initially extract independent identity and attribute features. Besides, the Re-Feeding module re-disentangles the generated images and reconstructs original images to further confirm the complete disentanglement of the inter-class information. Finally, for the intra-class relationship, we adopt the IR module based on the same identity image pairs to learn the consistent identity feature without being influenced by attributes. Extensive experiments and comparisons to the existing state-of-the-art face swapping methods demonstrate the effectiveness of our framework.
A Conv-Attention Network for Detecting the Presence of ENF Signal in Short-Duration Audio

Detecting the presence of the electric network frequency (ENF) signal in audio recordings is a prerequisite of applying the ENF criterion that plays an essential role in numerous forensic applications. However, existing detection methods are powerless to handle short-duration audio recordings that have attracted considerable attention due to the popularity of voice messaging apps. This paper proposes a novel deep learning-based approach for ENF detection in short audio recordings, reducing the minimum operating range of audio duration to 1/10 of the state-of-the-art methods. Meanwhile, a convolutional attention network termed Conv-AttNet is proposed to improve the detection performance of convolutional neural networks (CNN) through the attention mechanism. Experiments on both synthetic and real-world audio recordings reveal that Conv-AttNet is able to detect the ENF signal buried in only 2 seconds of audio recordings, surpassing both matched filtering and typical CNN like ResNet50. In addition, the detection accuracy can be further increased by utilizing audio recordings of longer duration.