Reviewerly/exHarmony huggingface dataset:
Probabilistic Human Motion Prediction via A Bayesian Neural Network

Human motion prediction is an important and challenging topic that has promising prospects in efficient and safe human-robot-interaction systems. Currently, the majority of the human motion prediction algorithms are based on deterministic models, which may lead to risky decisions for robots. To solve this problem, we propose a probabilistic model for human motion prediction in this paper. The key idea of our approach is to extend the conventional deterministic motion prediction neural network to a Bayesian one. On one hand, our model could generate several future motions when given an observed motion sequence. On the other hand, by calculating the Epistemic Uncertainty and the Heteroscedastic Aleatoric Uncertainty, our model could tell the robot if the observation has been seen before and also give the optimal result among all possible predictions. We extensively validate our approach on a large scale benchmark dataset Human3.6m. The experiments show that our approach performs better than deterministic methods. We further evaluate our approach in a Human-Robot-Interaction (HRI) scenario. The experimental results show that our approach makes the interaction more efficient and safer.
BERT for Conversational Question Answering Systems Using Semantic Similarity Estimation

Most of the questions from users lack the context needed to thoroughly understand the problem at hand, thus making the questions impossible to answer. Semantic Similarity Estimation is based on relating user’s questions to the context from previous Conversational Search Systems (CSS) to provide answers without requesting the user's context. It imposes constraints on the time needed to produce an answer for the user. The proposed model enables the use of contextual data associated with previous Conversational Searches (CS). While receiving a question in a new conversational search, the model determines the question that refers to more past CS. The model then infers past contextual data related to the given question and predicts an answer based on the context inferred without engaging in multi-turn interactions or requesting additional data from the user for context. This model shows the ability to use the limited information in user queries for best context inferences based on Closed-Domain-based CS and Bidirectional Encoder Representations from Transformers for textual representations.
Differentially Private Synthetic Mixed-Type Data Generation For Unsupervised Learning

We introduce the DP-auto-GAN framework for synthetic data generation, which combines the low dimensional representation of autoencoders with the flexibility of Generative Adversarial Networks (GANs). This framework can be used to take in raw sensitive data and privately train a model for generating synthetic data that will satisfy similar statistical properties as the original data. This learned model can generate an arbitrary amount of synthetic data, which can then be freely shared due to the post-processing guarantee of differential privacy. Our framework is applicable to unlabeled mixed-type data, that may include binary, categorical, and real-valued data. We implement this framework on both binary data (MIMIC-III) and mixed-type data (ADULT), and compare its performance with existing private algorithms on metrics in unsupervised settings. We also introduce a new quantitative metric able to detect diversity, or lack thereof, of synthetic data.
A Contrastive Learning Approach for Compositional Zero-Shot Learning

An object can be in several states. For different states (attributes) the object could look dramatically different. Thus, the smart information retrieval systems of the future need to learn good state-object representations. Such a system should not only be able to recognize state-object compositions unseen during training but also be able to retrieve images based on multi-modal (image-text) query. In the literature, these tasks are treated separately. In this work, we propose a unified model, ContraNet, which leverages the rich semantics of the state-object to learn multimodal representation in a contrastive manner. We adopt a deep metric learning approach and learn a multimodal representation by pulling similar images and texts closer to each other and pushing apart different ones. Our autoencoder based model learns the text-aware representation of image which is suitable for both tasks. The reconstruction losses provide additional regularization for learning of the representation. Our approach outperforms the state-of-the-art (SOTA) methods on widely-used benchmarks. Specifically, on the task of state-object composition, ContraNet achieves 8.7% and 8.1% performance gain on UT-Zappos and MIT-States on best HM metric, respectively. For the image retrieval task, ContraNet surpasses the SOTA performance by 4% on MIT-States and 5.3% on Fashion200k.
Mandatory Access Control Method for Windows Embedded OS Security

The Windows Embedded operating system (OS) adopts a discretionary access control (DAC)-based policy, but underlying vulnerabilities exist because of external hacker attacks and other factors. In this study, we propose a system that improves the security of the Windows Embedded OS by applying a mandatory access control (MAC) policy in which the access rights of objects, such as files and folders, and subjects’ privileges, such as processes, are compared. We conducted access control tests to verify whether the proposed system could avoid the vulnerabilities of DAC-based systems. Our results indicate that the existing DAC-based security systems could be neutralized if a principal’s security policy is removed. However, in the proposed MAC-based Windows Embedded OS, even if the clearance and category values of a subject’s files are given the highest rating, all accesses are automatically denied. Therefore, the execution of all files that were not previously registered on the whitelist was denied, proving that security was improved relative to DAC-based systems.
Emerging Applications of Artificial Intelligence, Machine learning and Data Science

There are various intense forces causing customers to use evaluated data when using social media platforms and microblogging sites.Today, customers throughout the world share their points of view on all kinds of topics through these sources.The massive volume of data created by these customers makes it impossible to analyze such data manually.Therefore, an efficient and intelligent method for evaluating social media data and their divergence needs to be developed.Today, various types of equipment and techniques are available for automatically estimating the classification of sentiments.Sentiment analysis involves determining people's emotions using facial expressions.Sentiment analysis can be performed for any individual based on specific incidents.The present study describes the analysis of an image dataset using CNNs with PCA intended to detect people's sentiments (specifically, whether a person is happy or sad).This process is optimized using a genetic algorithm to get better results.Further, a comparative analysis has been conducted between the different models generated by changing the mutation factor, performing batch normalization, and applying feature reduction using PCA.These steps are carried out across five experiments using the Kaggle dataset.The maximum accuracy obtained is 96.984%, which is associated with the Happy and Sad sentiments.
Shared Realities: Avatar Identification and Privacy Concerns in Reconstructed Experiences

Recent advances in 3D reconstruction technology allow people to capture and share their experiences in 3D. However, little is known about people's sharing preferences and privacy concerns for these reconstructed experiences. To fill this gap, we first present ReliveReality, an experience-sharing method utilizing deep learning-based computer vision techniques to reconstruct clothed humans and 3D environments and estimate 3D pose with only a RGB camera. ReliveReality can be integrated into social virtual environments, allowing others to socially relive a shared experience by moving around the experience from different perspectives, on desktop or in VR. We conducted a 44-participant within-subject study to compare ReliveReality to viewing recorded videos, and to a ReliveReality version with blurring obfuscation. Our results shed light on how people identify with reconstructed avatars, how obfuscation affects reliving experiences, and sharing preferences and privacy concerns for reconstructed experiences. We propose design implications for addressing these issues.
Multi-granularity Mobile Encrypted Traffic Classification Based on Fusion Features

The prosperity and development of mobile network makes mobile applications inseparable from people’s life. Although the encryption of mobile communication traffic protects the privacy of users to a certain extent, it also brings great challenges to network management and supervision. At present, many researches use machine learning or deep learning to classify encrypted traffic, but most of them only focus on single granularity classification task. Moreover, the method of multi granularity classification is more artificial, which cannot fully mine the effective information of multi granularity encrypted traffic classification. In this paper, we propose fusion feature based model, an end-to-end framework, which can automatically generate distinguishing fingerprints at three different granularities: app, in-app activity, and app-activity. Specifically, we use 1D-CNN to extract the spatial characteristics of the first packet payloads, and bidirectional LSTM to learn the timing characteristics of the packet length sequences and the packet direction sequences. Extensive experiments based on real-world encrypted mobile traffic show that, the proposed model achieves the best results in the multi-granularity classification task of mobile encrypted traffic, compared with the four state-of-the-art methods. Our work can provide an effective solution for the hierarchical and refined management of mobile networks.
Attention-Driven Cross-Modal Remote Sensing Image Retrieval

In this work, we address a cross-modal retrieval problem in remote sensing (RS) data. A cross-modal retrieval problem is more challenging than the conventional uni-modal data retrieval frameworks as it requires learning of two completely different data representations to map onto a shared feature space. For this purpose, we chose a photo-sketch RS database. We exploit the data modality comprising more spatial information (sketch) to extract the other modality features (photo) with cross-attention networks. This sketch-attended photo features are more robust and yield better retrieval results. We validate our proposal by performing experiments on the benchmarked Earth on Canvas dataset. We show a boost in the overall performance in comparison to the existing literature. Besides, we also display the Grad-CAM visualizations of the trained model's weights to highlight the framework's efficacy.
HPC AI500 V2.0: The Methodology, Tools, and Metrics for Benchmarking HPC AI Systems

Recent years witness a trend of applying large-scale distributed deep learning algorithms (HPC AI) in both business and scientific computing areas, whose goal is to speed up the training time to achieve a state-of-the-art quality. The HPC AI benchmarks accelerate the process. Unfortunately, benchmarking HPC AI systems at scale raises serious challenges. This paper presents a comprehensive HPC AI benchmarking methodology that achieves equivalence, representativeness, repeatability, and affordability. Among the nineteen AI workloads of AIBench Training–by far the most comprehensive AI benchmarks suite, we choose two representative and repeatable AI workloads in terms of both AI model and micro-architectural characteristics. The selected HPC AI benchmarks include both business and scientific computing: Image Classification and Extreme Weather Analytics. Finally, we propose three high levels of benchmarking and the corresponding rules to assure equivalence. To rank the performance of HPC AI systems, we present a new metric named Valid FLOPS, emphasizing both throughput performance and target quality. The evaluations show our methodology, benchmarks, and metrics can measure and rank the HPC AI systems in a simple, affordable and repeatable way. The specification, source code, datasets, and HPC AI500 ranking numbers are publicly available from https://www.benchcouncil.org/aibench/hpcai500/index.html.
Semi-supervised Multi-task Learning for Semantics and Depth

Multi-Task Learning (MTL) aims to enhance the model generalization by sharing representations between related tasks for better performance. Typical MTL methods are jointly trained with the complete multitude of ground-truths for all tasks simultaneously. However, one single dataset may not contain the annotations for each task of interest. To address this issue, we propose the Semi-supervised Multi-Task Learning (SemiMTL) method to leverage the available supervisory signals from different datasets, particularly for semantic segmentation and depth estimation tasks. To this end, we design an adversarial learning scheme in our semi-supervised training by leveraging unlabeled data to optimize all the task branches simultaneously and accomplish all tasks across datasets with partial annotations. We further present a domain-aware discriminator structure with various alignment formulations to mitigate the domain discrepancy issue among datasets. Finally, we demonstrate the effectiveness of the proposed method to learn across different datasets on challenging street view and remote sensing benchmarks.
Security Analysis of DGM and GM Group Signature Schemes Instantiated with XMSS-T

Group Merkle (GM) (PQCrypto 2018) and Dynamic Group Merkle (DGM) (ESORICS 2019) are recent proposals for post-quantum hash-based group signature schemes. They are designed as generic constructions that employ any stateful Merkle hash-based signature scheme. XMSS-T (PKC 2016, RFC8391) is the latest stateful Markle hash-based signature scheme where (almost) optimal parameters are provided. In this paper, we show that the setup phase of both GM and DGM does not enable drop-in instantiation by XMSS-T which limits both designs in employing earlier XMSS versions with sub-optimal parameters which negatively affects the performance of both schemes. Thus, we provide a tweak to the setup phase of GM and DGM to overcome this limitation and enable the adoption of XMSS-T. Moreover, we analyze the bit security of DGM when instantiated with XMSS-T and show that it is susceptible to multi-target attacks because of the parallel Signing Merkle Trees (SMT) approach. More precisely, when DGM is used to sign \(2^{64}\) messages, its bit security is 44 bits less than that of XMSS-T. Finally, we provide a DGM variant that mitigates multi-target attacks and show that it attains the same bit security as XMSS-T.
An Autonomous Cyber-Physical Anomaly Detection System Based on Unsupervised Disentangled Representation Learning

Cyber-Physical Systems (CPS) in heavy industry are a combination of closely integrated physical processes, networking, and scientific computing. The physical production process is monitored and controlled by the CPS in question, through advanced real-time networking systems, where high-precision feedback loops can be changed when the overgrid of cooperative computing and communication components that make up the industrial process is required. These CPS operate independently but integrate interaction capabilities as well as with the external environment, creating the connection of the physical with the digital world. The outline is that the most effective modeling and development of high-reliability CPS are directly related to the maximization of the production process, extroversion, and industrial competition. In this paper, considering the high importance of the operational status of CPS for heavy industry, an innovative autonomous anomaly detection system based on unsupervised disentangled representation learning is presented. It is a temporal disentangled variational autoencoder (TDVA) which, mimicking the process of rapid human intuition, using high- or low-dimensional reasoning, finds and models the useful information independently, regardless of the given problem. Specifically, taking samples from the real data distribution representation space, separating them appropriately, and encoding them as separate disentangling dimensions create new examples that the system has not yet dealt with. In this way, first, it utilizes information from potentially inconsistent sources to learn the right representations that can then be broken down into subspace subcategories for easier and simpler categorization, and second, utilizing the latent representation of the model, it performs high-precision estimates of how similar or dissimilar the inputs are to each other, thus recognizing, with great precision and in a fully automated way, the system anomalies.
How the Word Adjacency Network (WAN) works

Abstract The Word Adjacency Network (WAN) method is a newly invented method for attributing the authorship of texts based on internal evidence about the proximities of high-frequency words in those texts. The method has been the subject of mathematically rigorous explanations in scientific journals and here is offered instead a relatively non-technical account for humanist readers unfamiliar with advanced mathematical nomenclature. The description will refer to the operation of the algorithm as it is implemented in a newly completed open-source, open-access version that the authors have made available for free unlimited download on the WorldWide Web.
Blockchain and Self-Sovereign Identity Empowered Cyber Threat Information Sharing Platform

Cyber threat information (CTI) sharing involves processes of the collection, analysis and sharing of cyber threat information among multiple organizations. CTI is highly sensitive and inadvertent access can harm an organisation’s reputation. Moreover, CTI sharing may also inadvertently advertise a vulnerability that may be present in the organisation’s infrastructure. Therefore, preserving the privacy and anonymity of the CTI participants is critical. This paper proposes "Siddhi", a blockchain and Self-Sovereign Identity(SSI) enabled CTI platform that will realize traceability, anonymization and data provenance in a scalable fashion. Siddhi is equipped with SSI-enabled mobile wallet to ensure anonymous reporting of threat information and supports TAXII and STIX standards for exchanging the threat information between participants in the blockchain network.
Image Quality Caption with Attentive and Recurrent Semantic Attractor Network

In this paper, a novel quality caption model is inventively developed to assess the image quality with hierarchical semantics. Existing image quality assessment (IQA) methods usually represent image quality with a quantitative value, resulting in inconsistency with human cognition. Generally, human beings are good at perceiving image quality in terms of semantic description rather than quantitative value. Moreover, cognition is a needs-oriented task where hierarchical semantics are extracted. The mediocre quality value fails to reflect degradations on hierarchical semantics. Therefore, a new IQA framework is proposed to describe the quality for needs-oriented cognition. A novel quality caption procedure is firstly introduced, in which the quality is represented as patterns of activation distributed across the diverse degradations on hierarchical semantics. Then, an attentive and recurrent semantic attractor network (ARSANet) is designed to activate the distributed patterns for image quality description. Experiments demonstrate that our method achieves superior performance and is highly compliant with human cognition.
Comparative Evaluation of Semi-Supervised Anomaly Detection Algorithms on High-Integrity Digital Systems

Anomaly detection algorithms solve the problem of identifying unexpected values in data sets. Such algorithms have been classically used for cleaning unlabelled data sets from potentially unwanted values. However, the ability to detect outlying values in data sets can also be used to detect anomalies in systems. Semi-supervised anomaly detection algorithms learn from data for known correct behavior. Such algorithms have been used in various fields, e.g., system security, fault detection, medical applications.In this paper, we use the Area Under the Receiver Operating Characteristic (AUROC) score to evaluate algorithms for semi-supervised anomaly detection when applied to high-integrity distributed digital systems. We identify the relevant parameter for each algorithm and observe how the parameter influences the score and the runtime.
Deep Correlation Multimodal Neural Style Transfer

Style transfer is a well-known approach used to transfer the art style of a style image to an input content image, and the core method of the style transfer is to use the Gram matrix for representing the style features of images. In this paper, we investigate the advantages and disadvantages of using the Gram matrix and introduce several alternatives. In addition, we propose an end-to-end multimodal style transfer network, called deep correlation multimodal style transfer (DeCorMST), which automatically generates multiple images from a single pair of content and style images at once. We introduce deep correlation loss that integrates style losses from different correlation methods, allowing the proposed network to transfer the style of the source to the input content image in different manners. We qualitatively and quantitatively experiment with and compare DeCorMST outputs, we prove that the Gram matrix generated images is more efficient in balancing performance content preservation and style adaptation compared to the other correlations. Source code is available at https://github.com/ichirokira/CorrelationNeuralStyleTransfer.
CDP

Neural network pruning has shown promising performance in reducing computational complexity and facilitate the deployment of deep neural networks on resource-limited edge devices. Most existing pruning methods focus on the indicators of the filter's weight, gradient, or feature map and regard the weak or similar filters as network redundancy. In contrast, the representation of discriminative power is also a fundamental attribute that analog neural networks to have extraordinary performance in various tasks. However, such representation is neglected in existing works. Alternatively, we propose a novel filter pruning strategy via class-wise discriminative power (CDP). Unlike the previous methods, CDP treats the filters that always yield large or small activation values as redundant and reserves the filters that show different magnitudes in activations as they yield high discriminative power. We further propose to obtain such discriminative power by employing the widely-used Term Frequency-Inverse Document Frequency (TF-IDF) on feature representations across classes. Specifically, the output of a filter is considered as a word, and the whole feature map is considered as a document. Then, TF-IDF is used to generate the relevant score between words and all documents. If a filter has low TF-IDF scores is less discriminate and can be pruned. Thus, the filters with high TF-IDF scores are reserved. To our best knowledge, this is the first work that prunes neural networks through class-wise discriminative power and measures such power by introducing TF-IDF in feature representation among different classes. Without any iterative process, CDP achieves better compression trade-offs comparing to the state-of-the-art compression algorithms. For instance, in VGG-16, we achieve a 68.05%-FLOPs reduction, with a 94.86% Top-1 accuracy on CIFAR-10. Specifically, we compress a 90.12%-FLOPs reduction VGG-16, even retains 93.30% Top-1 accuracy on CIFAR-10. The code is available at https://github.com/Tianshuo-Xu/CDP-Towards-Optimal-Filter-Pruning-via-Class-wise-Discriminative-Power.git