Reviewerly/exHarmony huggingface dataset:
Secure deduplication method based on tag clustering

Although cloud storage technology can provide users with convenient storage services, a large amount of duplicate data in the cloud brings a huge storage burden and the risk of privacy leakage. To improve the utilization of cloud storage resources and protect data confidentiality, random message lock encryption technology (R-MLE) can be used to delete redundant data in the cloud. But the theoretical basis of the deduplication scheme based on R-MLE is bilinear mapping, so the computational cost of finding duplicate fingerprint-tags is relatively large. To improve the deduplication efficiency, we proposed a secure deduplication scheme based on the autoencoder model in our previous research, using the model to generate the abstract-tags of the data, and using the similarity of the abstract-tags to quickly filter out the fingerprint-tags with high repeatability, which greatly reduces the number of fingerprint-tag comparisons. On this basis, this paper further proposes a secure deduplication method based on k-means clustering. First, the abstract-tags in cloud storage are clustered, and then the distance between the abstract-tags uploaded by users and the centroid is calculated. Then, the abstract-tags of the category with the closest distance are selected. Finally, duplicate data detection is performed only on the fingerprint-tags corresponding to these abstract-tags. In this way, the filtering speed of fingerprint-tags can be further accelerated. Experiments show that our method has higher performance than the secure deduplication method based on the autoencoder model.
Symbolic Semantic Memory in Transformer Language Models

This paper demonstrates how transformer language models can be improved by giving them access to relevant structured data extracted from a knowledge base. The methods for doing so include identifying entities in a text corpus, sorting the entities using a novel attention-based approach, linking entities to a knowledge base, then extracting and filtering the knowledge to create a knowledge-augmented dataset. We evaluate these methods with the WikiText-103 corpus using standard language modeling objectives. These results show that even simple additional knowledge augmentation leads to a reduction in validation perplexity by 81.04%. These methods also significantly outperform common ways of improving language models such as increasing the model size or adding more data.
An Efficient Metric-Based Approach for Static Use-After-Free Detection

Nowadays, attackers are increasingly using Use-After-Free(UAF) vulnerabilities to create threats against software security. Existing static approaches for UAF detection are capable of finding potential bugs in the large code base. In most cases, analysts perform manual inspections to verify whether the warnings detected by static analysis are real vulnerabilities. However, due to the complex constraints of constructing UAF vulnerability, it is very time and cost-intensive to screen all warnings. In fact, many warnings should be discarded before the manual inspection phase because they are almost impossible to get triggered in real-world, and it is often overlooked by current static analysis techniques. In this paper, we introduce a metric-based static analysis approach, named Mad, for efficiently identifying UAF vulner-abilities by removing redundant warnings. We design two sets of systematic metrics to drive Mad. First, we apply lightweight static analysis to locate potential UAF pairs and use Feature Metrics to gather their feature information into an evaluation pool. Then, we use Controllability Metrics to rank the evaluation pools and filter out the high ones as candidates for subsequent manual inspection. We have implemented Madand evaluated it using Juliet Test Suite and a set of eight open-source C programs. Madcan locate all UAF bugs in Juliet Test Suite within a recognizable range, showing effectiveness and scalability by detecting 5 known CVEs with 1,286 KLOC in just 1.6 hours. Furthermore, we can achieve an average 75% reduction rate for reported warnings and save about half the time in locating UAF vulnerabilities during manual inspection.
Transformer-Based Speech Synthesizer Attribution in an Open Set Scenario

Speech synthesis methods can create realistic-sounding speech, which may be used for fraud, spoofing, and mis-information campaigns. Forensic methods that detect synthesized speech are important for protection against such attacks. Forensic attribution methods provide even more information about the nature of synthesized speech signals because they identify the specific speech synthesis method (i.e., speech synthesizer) used to create a speech signal. Due to the increasing number of realistic-sounding speech synthesizers, we propose a speech attribution method that generalizes to new synthesizers not seen during training. To do so, we investigate speech synthesizer attribution in both a closed set scenario and an open set scenario. In other words, we consider some speech synthesizers to be "known" synthesizers (i.e., part of the closed set) and others to be "unknown" synthesizers (i.e., part of the open set). We represent speech signals as spectrograms and train our proposed method, known as compact attribution transformer (CAT), on the closed set for multi-class classification. Then, we extend our analysis to the open set to attribute synthesized speech signals to both known and unknown synthesizers. We utilize a t-distributed stochastic neighbor embedding (tSNE) on the latent space of the trained CAT to differentiate between each unknown synthesizer. Additionally, we explore poly-1 loss formulations to improve attribution results. Our proposed approach successfully attributes synthesized speech signals to their respective speech synthesizers in both closed and open set scenarios.
Indoor Intelligent IoT Datasets for Behavioral Learning and Autonomous Decision Making

The current indoor Internet of Things (IoT) system is mainly realized by presetting rules, voice interaction, or remote control, which fails to meet the need for systematic, global, and autonomous intelligence. The cost of high-quality large-scale data annotation is also too high, which hinders the effective application of supervised learning paradigm in such scenarios. Therefore, unsupervised or reinforcement learning is worthy of consideration for the future IoT system. In view of the large-scale organization and deployment, high dynamic coordination, and precise service requirement of the future IoT system, this study constructs a perceptual data acquisition platform to obtain the indoor intelligent IoT datasets suitable for unsupervised or reinforcement learning. First, various environment-aware and controllable electrical devices are deployed in a multi-person office area that is divided into multiple zones. The devices are connected to the edge gateway through long range radio (LoRa) wireless network, and an message queuing telemetry transport (MQTT) cloud server is constructed to realize the design of cloud edge collaborative architecture. Then, two kinds of information, namely, environment and controlled action, are collected in real time without human perception, which forms and publicizes complete datasets, including time, space, environmental parameters, and control actions. On the basis of the datasets, relevant data analysis is conducted and preliminary experimental results using unsupervised learning method are presented.
Balancing Similarity-Contrast in Unsupervised Representation Learning: Evaluation with Reinforcement Learning

In this paper, we provided an unsupervised contrastive representation learning method which uses contrastive views in which both spatial and temporal similarity-contrast is balanced. The balanced views are created by taking pixels from the anchor sample and any randomly selected negative sample and balancing the ratio of number of pixels taken from the anchor and the negative. Then these balanced views are paired with the anchor to create the positive contrastive views and all other samples paired with the anchor are taken as negative contrastive views. We made the evaluation using reinforcement learning tasks on Atari games and Deep Mind Control suites (DMControl). Our evaluations on 26 Atari games and six DMControl tasks show that the proposed method is superior in learning spatio-temporally evolving factors of the environment by capturing the relevant task controlling generative factors from the agents’ raw observations.
A Blockchain-Based Multi-Cloud Storage Data Consistency Verification Scheme

Users worldwide widely use cloud storage because of its efficiency, convenience, and high availability. Multi-cloud storage is usually selected to ensure the high availability of data. Unfortunately, when data is migrated and replicated between multi-cloud data centers, it is not easy to guarantee data con-sistency. This paper proposes an efficient, secure, and new data consistency verification scheme using blockchain technology. In order to reduce the computation and communication overhead in the verification process, our scheme uses encrypted tags to build Merkle hash tree to generate unique and lightweight verification proofs and does not use third-party auditors. The final theoretical and experimental analysis shows that our scheme has higher security and a faster verification process in multi-cloud storage.
WGeod: A General and Efficient FPGA Accelerator for Object Detection

With the development of high-performance embedded applications, object detection algorithms are starting to be deployed to embedded devices with fewer resources, and the classification and location information they provide is the basis for making further decisions in some high-performance embedded applications, so real-time and low-power consumption are especially needed. In this paper, we propose an FPGA-based neural network ac-celerator for object detection, which improves the speed, throughput and power efficiency of the inference with little loss of accuracy. The accelerator balances the speed difference between computation and memory access. The accelerator is scalable and can be configured for CNN of any size, and is general enough to support the acceleration of YOLOv2, YOLOv3 and their Tiny versions. We deploy the Tiny YOLOv3 network on FPGA-based accelerator, CPU and GPU platforms, and compare them in terms of speed, throughput and power efficiency. For the Tiny YOLOv3 network, the accelerator delivers an FPS of 1.61 and throughput of 8.95 GOP/s, which is 23.81x, 4.11x and 3.00x better than the Intel Core i5-10210U CPU with the 00, 01 and O3 compilation optimization options respectively. With 2.11 W of power, the accelerator delivers a power efficiency of 4.24 GOP/s/W, which is 141.33x, 28.27x and 21.20x better than the Intel Core i5-10210U CPU with the 00, O1 and O3 options respectively.
A Neural Model for Regular Grammar Induction

Grammatical inference is a classical problem in computational learning theory and a topic of wider influence in natural language processing. We treat grammars as a model of computation and propose a novel neural approach to induction of regular grammars from positive and negative examples. Our model is fully explainable, its intermediate results are directly interpretable as partial parses, and it can be used to learn arbitrary regular grammars when provided with sufficient data. We find that our method consistently attains high recall and precision scores across a range of tests of varying complexity.
SimCURL: Simple Contrastive User Representation Learning from Command Sequences

User modeling is crucial to understanding user behavior and essential for improving user experience and personalized recommendations. When users interact with software, vast amounts of command sequences are generated through logging and analytics systems. These command sequences contain clues to the users’ goals and intents. However, these data modalities are highly unstructured and unlabeled, making it difficult for standard predictive systems to learn from. We propose SimCURL, a simple yet effective contrastive self-supervised deep learning framework that learns user representation from unlabeled command sequences. Our method introduces a user-session network architecture, as well as session dropout as a novel way of data augmentation. We train and evaluate our method on a real-world command sequence dataset of more than half a billion commands. Our method shows significant improvement over existing methods when the learned representation is transferred to downstream tasks such as experience and expertise classification.
SVTON: Simplified Virtual Try-On

2D based Virtual Try-On (VTON) has been trending towards using human parsing to improve the quality of the try-on image. However, it remains a challenging problem for most existing VTON models to generate realistic images for situations with unpaired candidate-clothing images and body-part occlusions. We have developed a Simplified Virtual Try-On (SVTON) model to rectify the above problem. The SVTON uses refined input data to produce accurate labels and has fewer trainable parameters than existing methods. Also, it is designed with a simplified network architecture for segmentation and an efficient Affine Transform for warping to target clothing. Experiments on benchmark datasets show that the proposed model performs better than the state-of-the-art VTON models for unpaired and occlusion cases, while maintaining the similar overall performance level for normal cases.
Software package for regression algorithms based on Gaussian Conditional Random Fields

The Gaussian Conditional Random Fields (GCRF) algorithm and its extensions are used for machine learning regression problems in which the attributes of objects and the correlation between objects should be considered when making predictions. These algorithms can be applied in different domains where problems can be seen as graphs, but their implementation requires complex calculations and good programming skills. This paper presents an open source software package that includes a tool with graphical user interface (GCRFs tool) and Java library (GCRFs library). GCRFs tool is software that integrates various GCRF-based algorithms and supports training and testing of those algorithms on real-world datasets. The main goal of GCRFs tool is to provide a straightforward and user-friendly graphical user interface that will simplify the use of GCRF-based algorithms. GCRFs Java library contains basic classes for GCRF concepts and can be used by researchers who have experience in Java programming. Also, this paper presents the results of a pilot usability evaluation of the GCRFs tool, where the software was evaluated with expert and non-expert users. This evaluation gave us detailed insight into the experiences and opinions of the users and helped us outline priorities for future development.
Interpretability of ReLU for Inversion

Interpretability continues to be a focus of much research in deep neural network. In this work, we focus on the mathematical interpretability of fully-connected neural networks, especially those that use a rectified linear unit (ReLU) activation function. Our analysis elucidates the difficulty of approximating the reciprocal function. Notwithstanding, using the ReLU activation function halves the error compared with a linear model. In addition, one might have expected the errors to increase only towards the singular point x = 0, but both the linear and ReLU errors are fairly oscillatory and increase near both edge points.
VDGraph2Vec: Vulnerability Detection in Assembly Code using Message Passing Neural Networks

Software vulnerability detection is one of the most challenging tasks faced by reverse engineers. Recently, vulnerability detection has received a lot of attention due to a drastic increase in the volume and complexity of software. Reverse engineering is a time-consuming and labor-intensive process for detecting malware and software vulnerabilities. However, with the advent of deep learning and machine learning, it has become possible for researchers to automate the process of identifying potential security breaches in software by developing more intelligent technologies. In this research, we propose VDGraph2Vec, an automated deep learning method to generate representations of assembly code for the task of vulnerability detection. Previous approaches failed to attend to topological characteristics of assembly code while discovering the weakness in the software. VDGraph2Vec embeds the control flow and semantic information of assembly code effectively using the expressive capabilities of message passing neural networks and the RoBERTa model. Our model is able to learn the important features that help distinguish between vulnerable and non-vulnerable software. We carry out our experimental analysis for performance benchmark on three of the most common weaknesses and demonstrate that our model can identify vulnerabilities with high accuracy and outperforms the current state-of-the-art binary vulnerability detection models.
Ontology-Based Post-Hoc Explanations via Simultaneous Concept Extraction*

Ontology-based explanation techniques allow one to get explanation why a neural network arrived to some conclusion using human-understandable terms and their formal definitions. The paper proposes a method to build post-hoc ontology-based explanations by training a multi-label neural network mapping the activations of the specified "black box" network to ontology concepts. In order to simplify training of such network we employ semantic loss, taking into account relationships between concepts. The experiment with a synthetic dataset shows that the proposed method can generate accurate ontology-based explanations of a given network.
Towards Efficient Elastic Parallelism for Deep Learning Processor

Resources underutilization is the most troublesome problem faced by almost all deep learning processors. To address this challenge, researchers have proposed various methods, especially in exploiting the parallelism of deep neural network models. However, some parallelisms have mutual effects, for example, intra-operator parallelism has adverse effects on inter-operator parallelism, resulting in a waste of resources. In order to make full use of chip resources, this paper applies the idea of elastic scheduling to the driver level of deep learning processor to achieve elastic parallelism. That is, if a task consists of multiple independent subtasks, it can be executed as long as the minimum resource requirements of the sub tasks are met. In addition, based on greedy scheduling algorithm, some improved methods are proposed, including subtask awareness, NUMA awareness, and cluster alignment awareness. Our analysis and preliminary evaluation demonstrate that, by reducing intra-operator parallelism, elastic parallelism can help improve chip utilization, and our proposed scheduling optimizations can also further improve the overall system performance.
Bootstrap Advantage Estimation for Policy Optimization in Reinforcement Learning

This paper proposes an advantage estimation approach based on data augmentation for policy optimization. Unlike using data augmentation on the input to learn value and policy function as existing methods use, our method uses data augmentation to compute a bootstrap advantage estimation. This Bootstrap Advantage Estimation (BAE) is then used for learning and updating the gradient of policy and value function. To demonstrate the effectiveness of our approach, we conducted experiments on several environments. These environments are from three benchmarks: Procgen, Deepmind Control, and Pybullet, which include both image and vector-based observations; discrete and continuous action spaces. We observe that our method reduces the policy and the value loss better than the Generalized advantage estimation (GAE) method and eventually improves cumulative return. Furthermore, our method performs better than two recently proposed data augmentation techniques (RAD and DRAC). Overall, our method performs better empirically than baselines in sample efficiency and generalization, where the agent is tested in unseen environments.
Clustering image data with a fixed embedding

Clustering unlabeled image data using deep neural network (DNN) models is under active investigation. Most existing approaches transform the data through embedding operations and cluster the embedded data, and the embedding is learned to fit the data. In some applications, the embedding model is explicitly given due to the concerns of generalizability, transferability, privacy and security. Despite rapid progress in self-supervised learning, clustering data with a fixed embedding is rarely explored. We propose an Merge & Expand (ME) algorithm to cluster image data using a fixed embedding and a DNN classification model. ME achieves a comparable level of accuracy with some state-of-the-art algorithms. It further demarcates the "clean" and "unclean" images where their geometric relations in the embedded space are compatible and incompatible with their cluster structure respectively. Finally, we validate ME with three datasets and discuss its potential extension.
Your Voice is Not Yours? Black-Box Adversarial Attacks Against Speaker Recognition Systems

Speaker recognition (SR) systems play an essential part in many applications, such as speech control access systems and voice verification, where robustness is crucial for daily use. Therefore, the vulnerability of SR systems has become a hot study of interest. Notably, recent studies prove that SR systems are vulnerable to adversarial attacks. These kinds of attacks generate adversarial examples by adding a well-crafted inconspicuous perturbation to the original audio to fool the target model into making false predictions. However, dominant works conduct attacks in the white-box setting, which suffers from limited practices since the model parameters and architectures are usually unavailable in real-world scenarios. To this end, we propose a black-box based framework without requiring details of the target model. We leverage gradient estimation procedure based on natural evolution strategy to generate adversarial examples. The gradient estimation only needs confidence scores and decisions produced by SR systems. We also explore genetic algorithm to guide the direction of example generation, which accelerates model convergence. The experiments demonstrate that our approach can manipulate state-of-the-art SR systems at a high attack success rate of 97.5% with small distortions. Extensive investigations on benchmark datasets, VoxCeleb1, VoxCeleb2, and TIMIT, further verify the effectiveness and stealthiness of our attack method.