Reviewerly/exHarmony huggingface dataset:
A Cognitive Memory-Augmented Network for Visual Anomaly Detection

With the rapid development of automated visual analysis, visual analysis systems have become a popular research topic in the field of computer vision and automated analysis. Visual analysis systems can assist humans to detect anomalous events (e.g., fighting, walking alone on the grass, etc). In general, the existing methods for visual anomaly detection are usually based on an autoencoder architecture, i.e., reconstructing the current frame or predicting the future frame. Then, the reconstruction error is adopted as the evaluation metric to identify whether an input is abnormal or not. The flaws of the existing methods are that abnormal samples can also be reconstructed well. In this paper, inspired by the human memory ability, we propose a novel deep neural network (DNN) based model termed cognitive memory-augmented network (CMAN) for the visual anomaly detection problem. The proposed CMAN model assumes that the visual analysis system imitates humans to remember normal samples and then distinguishes abnormal events from the collected videos. Specifically, in the proposed CMAN model, we introduce a memory module that is able to simulate the memory capacity of humans and a density estimation network that can learn the data distribution. The reconstruction errors and the novelty scores are used to distinguish abnormal events from videos. In addition, we develop a two-step scheme to train the proposed model so that the proposed memory module and the density estimation network can cooperate to improve performance. Comprehensive experiments evaluated on various popular benchmarks show the superiority and effectiveness of the proposed CMAN model for visual anomaly detection comparing with the state-of-the-arts methods. The implementation code of our CMAN method can be accessed at https://github.com/CMAN-code/CMAN_pytorch.
Crowd Counting Using End-to-End Semantic Image Segmentation

Crowd counting is an active research area within scene analysis. Over the last 20 years, researchers proposed various algorithms for crowd counting in real-time scenarios due to many applications in disaster management systems, public events, safety monitoring, and so on. In our paper, we proposed an end-to-end semantic segmentation framework for crowd counting in a dense crowded image. Our proposed framework was based on semantic scene segmentation using an optimized convolutional neural network. The framework successfully highlighted the foreground and suppressed the background part. The framework encoded the high-density maps through a guided attention mechanism system. We obtained crowd counting through integrating the density maps. Our proposed algorithm classified the crowd counting in each image into groups to adapt the variations occurring in crowd counting. Our algorithm overcame the scale variations of a crowded image through multi-scale features extracted from the images. We conducted experiments with four standard crowd-counting datasets, reporting better results as compared to previous results.
Voice Spoofing Countermeasure for Synthetic Speech Detection

In the last few years, we have witnessed an exponential growth in voice spoofing attacks. The intruders employ different types of attacks such as speech synthesis where they use the machine generated speech against any target person to fool the automatic speaker verification (ASV) systems for various tasks i.e. home control, bank account access, etc. The availability of modern-day advanced tools has made it convenient to launch such types of voice spoofing attacks. To overcome the challenges associated with bypassing the security of ASV systems using the synthetic speech, we propose an effective synthetic speech detector using a fusion of spectral features. More specifically, we propose a fused feature vector consisting of MFCC, GTCC, Spectral Flux, and Spectral Centroid for audio signal representation. This fused feature set is capable of capturing the traits of speech variation attributes of genuine signal and algorithmic artifacts of synthetic signals. These features are further used to train the bilstm to classify the signal as genuine or spoof. The proposed framework is capable of detecting both the voice conversion and synthetic speech attacks on ASV systems. Performance of our framework is evaluated on ASVspoof 2019 LA dataset. Our experimental results illustrate the effectiveness of the proposed framework for logical access attacks (voice conversion and cloned/synthetic voice) detection.
Unsupervised Feature Selection via Orthogonal Basis Clustering and Local Structure Preserving

Due to the "curse of dimensionality" issue, how to discard redundant features and select informative features in high-dimensional data has become a critical problem, hence there are many research studies dedicated to solving this problem. Unsupervised feature selection technique, which does not require any prior category information to conduct with, has gained a prominent place in preprocessing high-dimensional data among all feature selection techniques, and it has been applied to many neural networks and learning systems related applications, e.g., pattern classification. In this article, we propose an efficient method for unsupervised feature selection via orthogonal basis clustering and reliable local structure preserving, which is referred to as OCLSP briefly. Our OCLSP method consists of an orthogonal basis clustering together with an adaptive graph regularization, which realizes the functionality of simultaneously achieving excellent cluster separation and preserving the local information of data. Besides, we exploit an efficient alternative optimization algorithm to solve the challenging optimization problem of our proposed OCLSP method, and we perform a theoretical analysis of its computational complexity and convergence. Eventually, we conduct comprehensive experiments on nine real-world datasets to test the validity of our proposed OCLSP method, and the experimental results demonstrate that our proposed OCLSP method outperforms many state-of-the-art unsupervised feature selection methods in terms of clustering accuracy and normalized mutual information, which indicates that our proposed OCLSP method has a strong ability in identifying more important features.
Hierarchical-aware relation rotational knowledge graph embedding for link prediction

Knowledge graph embedding, as the upstream task of link prediction which aims to predict new links between entities under the premise of known relations, its reliability greatly affects the performance of link prediction. However, previous distance-based models focus on modeling complicated relation patterns while ignoring the semantic hierarchy of knowledge graph, from TransE to RotatE. In this setting, all entities are regarded as the same type, and the fact that different entities belong to different levels is neglected. Therefore, we propose the general form of RotatE, the hierarchical-aware relation rotational knowledge graph embedding (HA-RotatE), to model the hierarchical-aware knowledge graph. HA-RotatE represents entities and relations as complex vectors and uses different moduli of entity embeddings to indicate the different hierarchical levels they belong to. The transformation of modulus and rotation from head entity to tail entity depends on different relations. Some relations are used to link entities of the same level, and others are used to link entities of different levels. We adopt the shared modulus transformation parameter method for avoiding overfitting. As the general form of RotatE, HA-RotatE also has the ability to model and infer various relation modes, i.e., symmetry/antisymmetric, inversion and composition. On benchmark datasets WN18RR and FB15k-237, the experiments on link prediction tasks show that: (1) HA-RotatE can effectively model the semantic hierarchy of the knowledge graph; (2) Compared with competitive benchmarks, our model substantially outperforms them in most metrics.
Enhanced Deep Discrete Hashing with semantic-visual similarity for image retrieval

Hashing has been shown to be successful in a number of Approximate Nearest Neighbor (ANN) domains, ranging from medicine, computer vision to information retrieval. However, current deep hashing methods either ignore both rich information of labels and visual linkages of image pairs, or leverage relaxation-based algorithms to address discrete problems, resulting in a large information loss. To address the aforementioned problems, in this paper, we propose an Enhanced Deep Discrete Hashing (EDDH) method to leverage both label embedding and semantic-visual similarity to learn the compact hash codes. In EDDH, the discriminative capability of hash codes is enhanced by a distribution-based continuous semantic-visual similarity matrix, where not only the margin between the positive pairs and negative pairs is expanded, but also the visual linkages between image pairs is considered. Specifically, the semantic-visual continuous similarity matrix is constructed by analyzing the asymmetric generalized Gaussian distribution of the visual linkages between pairs with label consideration. Besides, in order to achieve an efficient hash learning framework, EDDH employs an asymmetric real-valued learning structure to learn the compact hash codes. In addition, we develop a fast discrete optimization algorithm, which can directly generate discrete binary codes in single step, and introduce an intermediate term before iterations to avoid the problems caused by directly the use of large semantic-visual similarity matrix, which results in a significant reduction in the computational overhead. Finally, we conducted extensive experiments on three datasets to show that EDDH has a significantly enhanced performance compared to the compared state-of-the-art baselines.
Integrating Multi-Source Transfer Learning, Active Learning and Metric Learning paradigms for Time Series Prediction

Traditional Time Series Prediction (TSP) algorithms assume that the training and testing data follow the same distribution and a large amount of data can be obtained. However, the time-varying nature of time series data is a very difficult problem, which will lead to the inconsistent distribution between new data and old data. In TSP problem scenario, how to transfer knowledge in a relatively long-time span effectively is a serious challenge. To address this issue, in this work, a novel Multi-Source Active Metric Transfer Learning (MS-AMTL) algorithm, integrating Multi-Source Transfer Learning, Active Learning, and Metric Learning paradigms, is designed to make the most of, instead of throw away directly, the long-ago data, with effect, along with its precise mathematic derivation. The setting of multi-source domains can make the dependability of domains and the similarity of sources be fully exerted, reducing the impact of negative transfer. The Active Learning (AL) module in MS-AMTL is designed to select the most representative instances, so that dependability of the domains can be ensured and information redundancy can be reduced. The Metric Learning (ML) module in MS-AMTL is developed to measure the similarity between instances and then the similarity of sources, more effectively. Extensive experiments on one artificial dataset, three real-world datasets and four financial datasets proved the superiority of our proposed algorithm in transfer learning and prediction performance in comparison with several other state-of-the-art algorithms. • This work proposes MS-AMTL algorithm, integrating MSTL, AL, and ML paradigms. • MS-AMTL transfers knowledge in a long-time span effectively. • MS-AMTL fully considers subtle changes in data distributions. • A density weighted uncertain sample selection scheme is designed in MS-AMTL. • MS-AMTL employs ML to take advantage of internal geometric information.
Generic Perceptual Loss for Modeling Structured Output Dependencies

The perceptual loss has been widely used as an effective loss term in image synthesis tasks including image super-resolution [16], and style transfer [14]. It was believed that the success lies in the high-level perceptual feature representations extracted from CNNs pretrained with a large set of images. Here we reveal that, what matters is the network structure instead of the trained weights. Without any learning, the structure of a deep network is sufficient to capture the dependencies between multiple levels of variable statistics using multiple layers of CNNs. This insight removes the requirements of pre-training and a particular network structure (commonly, VGG) that are previously assumed for the perceptual loss, thus enabling a significantly wider range of applications. To this end, we demonstrate that a randomly-weighted deep CNN can be used to model the structured dependencies of outputs. On a few dense per-pixel prediction tasks such as semantic segmentation, depth estimation and instance segmentation, we show improved results of using the extended randomized perceptual loss, compared to the baselines using pixel-wise loss alone. We hope that this simple, extended perceptual loss may serve as a generic structured-output loss that is applicable to most structured output learning tasks.
Mutual information-based recommender system using autoencoder

Nowadays, most of the websites like Amazon, YouTube and Netflix use collaborative filtering methods to recommend various types of items to users. There are two principal categories of collaborative filtering; memory-based and model-based. The memory-based methods use the users' similarity measures and have several advantages over the model-based techniques, including being easily explained and easy modeling updates with new ratings and items. However, the memory-based methods' performance reduces when the data is sparse, and unlike the model-based methods, memory-based methods are not scalable. In this paper, we propose a method that exploit the benefits of both similarity-based and model-based approaches. We address both the reliability and the online updating problems based on a novel user-similarity based method. To calculate the new similarity metric we use the predicted user rating vectors in the autoencoder's output and apply mutual information to the predicted vectors in order to find similar users. We depict a similarity graph according to the mutual information rate, which is calculated for each pair of users. We implement the proposed method on the Netflix movie recommendation dataset. According to our experiments, the proposed approach has a significant advantage over the other methods, such as the standard autoencoder, the matrix factorization, and the similarity-based methods.
Micro-Ring Resonator Based Photonic Reservoir Computing for PAM Equalization

Photonic reservoir computing (RC) enables us to introduce artificial neural networks (ANNs) as a deployable hardware-based processing unit beyond the von Neumann architecture. Its unique approach to consider only the output layer weights as adaptive, reduces training complexity and enables the use of ANNs in time-sensitive areas like optical communications. In this paper, we numerically investigate and evaluate the requirements of a photonic RC using an active silicon micro-ring resonator as a photonic integrated circuit (PIC) based reservoir system to compensate for transmission impairments such as chromatic dispersion and nonlinear effects occurring in optical transmission systems. We compare the performance of this RC with results from digital signal processing using Kramers-Kronig reception of distorted PAM-4 symbols from a 56 GBaud WDM transmission experiment over 100 km uncompensated SSMF.
Proposing a novel Cascade Ensemble Super Resolution Generative Adversarial Network (CESR-GAN) method for the reconstruction of super-resolution skin lesion images

Skin cancer is one of the most malignant cancers worldwide. Its early detection plays a prominent role in the patients' treatment. The quality of skin lesion images to ease the diagnosis of skin cancer is highly regarded. One of the most common technologies to take the skin lesion images is through a dermoscopy device. However, it is not accessible to all people. Capturing the images via other technologies such as mobile devices, is available everywhere, although they suffer from poor quality. In this paper, a novel Cascade Ensemble Super Resolution Generative Adversarial Network (CESR-GAN) method is proposed to reconstruct super-resolution skin lesion images using low-resolution counterparts. Specifically, a novel feature-based measurement loss function is designed to obtain more details as much as possible and generate higher quality images. Experimental results from quantitative and qualitative comparisons between our CESR-GAN model and other state-of-the-art methods show that our proposed method outperforms the compared methods on ISIC, and PH2 datasets, respectively. The CESR-GANs method can be used to generate super resolution skin images of skin lesions with highly notable performances.
Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression

Uncertainty is the only certainty there is. Modeling data uncertainty is essential for regression, especially in unconstrained settings. Traditionally the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. On the other hand, classification based regression and ranking based solutions are more popular in practice while the direct regression methods suffer from the limited performance. How to model the uncertainty within the present-day technologies for regression remains an open issue. In this paper, we propose to learn probabilistic ordinal embeddings which represent each data as a multivariate Gaussian distribution rather than a deterministic point in the latent space. An ordinal distribution constraint is proposed to exploit the ordinal nature of regression. Our probabilistic ordinal embeddings can be integrated into popular regression approaches and empower them with the ability of uncertainty estimation. Experimental results show that our approach achieves competitive performance. Code is available at https://github.com/Li-Wanhua/POEs.
Semantic Scene Completion via Integrating Instances and Scene in-the-Loop

Semantic Scene Completion aims at reconstructing a complete 3D scene with precise voxel-wise semantics from a single-view depth or RGBD image. It is a crucial but challenging problem for indoor scene understanding. In this work, we present a novel framework named Scene-Instance-Scene Network (SISNet), which takes advantages of both in-stance and scene level semantic information. Our method is capable of inferring fine-grained shape details as well as nearby objects whose semantic categories are easily mixed-up. The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the over-all scene. SISNet conducts iterative scene-to-instance (SI) and instance-to-scene (IS) semantic completion. Specifically, the SI is able to encode objects’ surrounding context for effectively decoupling instances from the scene and each instance could be voxelized into higher resolution to capture finer details. With IS, fine-grained instance information can be integrated back into the 3D scene and thus leads to more accurate semantic scene completion. Utilizing such an iterative mechanism, the scene and instance completion benefits each other to achieve higher completion accuracy. Extensively experiments show that our proposed method consistently outperforms state-of-the-art methods on both real NYU, NYUCAD and synthetic SUNCG-RGBD datasets. The code and the supplementary material will be available at https://github.com/yjcaimeow/SISNet.
Privacy-Preserving Task Matching With Threshold Similarity Search via Vehicular Crowdsourcing

In vehicular crowdsourcing, task requesters rely on a server to distribute spatial crowdsourcing tasks to on-road vehicular workers based on interests and locations. To protect the privacy of the interests and locations, both requesters and workers prefer to encrypt the information before uploading them to the server. However, such an encryption-before-outsourcing paradigm makes it a challenging issue to conduct the task matching. In this paper, we propose a Privacy-Preserving Task Matching (PPTM) with threshold similarity search via vehicular crowdsourcing. We first propose an interest-based PPTM by transforming vehicular workers' interests into binary vectors. By using Symmetric-key Threshold Predicate Encryption (STPE) and proxy re-encryption, PPTM achieves privacy-preserving multi-keyword task matching with Jaccard similarity search in multi-worker multi-requester setting. Furthermore, by comparing the Euclidean distances between workers and requesters against a pre-defined threshold, PPTM preserves the location privacy of workers and requesters that only reveals the comparison results to the crowdsourcing server. The security analysis and extensive experiments demonstrate that PPTM protects the confidentiality of locations and interests of requesters and workers while achieving the efficient task matching.
Immersive <i>Spring Morning in the Han Palac</i>e: Learning Traditional Chinese Art Via Virtual Reality and Multi-Touch Tabletop

We examine the user experience, learning effectiveness, learning behaviors, and learning motivation of 54 young adults in pairs using an immersive virtual reality (IVR) environment and a multi-touch tabletop (MTT) to gain a cultural-historical understanding of the traditional Chinese paintings, Spring Morning in the Han Palace and The Night Revels of Han Xizai respectively. We used mixed methods of knowledge assessments, questionnaires, observation, and interviews to collect and analyze data. The results reveal the IVR environment significantly increased the learning effectiveness and motivation compared to the MTT system, particularly in tasks related to recall of details or spatiality. The IVR design incorporates five learning scenes, and as part of this aspect, the qualitative results indicate users in the IVR condition (1) developed a variety of exploratory and embodied learning strategies; (2) quickly switched their attention to the central learning content, and (3) had equitable interactions in collaborative learning. This research contributes to the potential of IVR learning effectiveness and associated design considerations to learn traditional cultural heritage.
Auxiliary Diagnosis of COVID-19 Based on 5G-Enabled Federated Learning

The development of 5G and artificial intelligence technologies have brought novel ideas to the prevention, control, and diagnosis of disease. Due to the limitation of the privacy protection of medical big data, releasing the data of patients is not allowed. However, as COVID-19 spreads globally, it is urgent to develop a robust diagnostic model to serve as many institutions as possible. Therefore, we propose a 5G-enabled architecture of auxiliary diagnosis based on federated learning for multiple institutions and central cloud collaboration to realize the sharing of diagnosis models with high generalization performance. In order to exchange model and parameters between the central and distributed nodes, a framework of diagnosis model cognition is constructed for sharing and updating the model adaptively. The severity classification experiments of COVID-19 were carried out on the central cloud and three edge cloud servers to verify the effectiveness of the proposed architecture and model cognition strategy. At the same time, the aggregated model shows good performance with accuracy rates of 95.3, 79.4, and 97.7 percent on distributed nodes, and the recognition results can assist doctors in executing auxiliary diagnosis. Finally, the open issues of model fusion of multimodal data in the 5G network architecture are discussed.