Reviewerly/exHarmony huggingface dataset:
Predicting the onset of breast cancer using mammogram imaging data with irregular boundary

Summary With mammography being the primary breast cancer screening strategy, it is essential to make full use of the mammogram imaging data to better identify women who are at higher and lower than average risk. Our primary goal in this study is to extract mammogram-based features that augment the well-established breast cancer risk factors to improve prediction accuracy. In this article, we propose a supervised functional principal component analysis (sFPCA) over triangulations method for extracting features that are ordered by the magnitude of association with the failure time outcome. The proposed method accommodates the irregular boundary issue posed by the breast area within the mammogram imaging data with flexible bivariate splines over triangulations. We also provide an eigenvalue decomposition algorithm that is computationally efficient. Compared to the conventional unsupervised FPCA method, the proposed method results in a lower Brier Score and higher area under the ROC curve (AUC) in simulation studies. We apply our method to data from the Joanne Knight Breast Health Cohort at Siteman Cancer Center. Our approach not only obtains the best prediction performance comparing to unsupervised FPCA and benchmark models but also reveals important risk patterns within the mammogram images. This demonstrates the importance of utilizing additional supervised image-based features to clarify breast cancer risk.
A Two-Stage Adaptation Network (TSAN) for Remote Sensing Scene Classification in Single-Source-Mixed-Multiple-Target Domain Adaptation (<i>S</i>²<i>M</i>²<i>T</i> DA) Scenarios

Over the past decade, domain adaptation (DA) algorithms have been proposed to address domain gap problems as they do not need any interpretation in the target domain. However, most existing efforts focus on scenarios with only one source domain and one target domain. In this article, we explore the scenario with one source domain and mixed multiple target domains for remote sensing applications and propose a new algorithm, named the two-stage adaptation network (TSAN). First, we utilize the adversarial learning approach to confuse the classifier to discriminate between the source domain and the whole mixed-multiple-target domain. Second, we adopt self-supervised learning to divide the mixed-multiple-target domain with automated generation of “pseudo”-domain labels, which guides our network to learn intrinsic features of multiple target domains. Finally, these two steps are combined as an iterative procedure. We integrate a test dataset that includes five remote sensing datasets and ten classes. Our method achieves an average accuracy of 63.25% and 73.68% with two typical backbones, considerably outperforming other DA methods with an average accuracy improvement of 4.84%–20.19% and 9.06%–17.04%, respectively. Furthermore, we identify the negative transfer effect in existing mainstream DA methods in remote sensing image classification with multiple different domains.
The benchmark framework and exploratory study to investigate the feasibility of 360-degree video-based virtual reality to induce a full body illusion

The feeling of ownership of a virtual body has been a topic of interest in recent years. In order to observe the mechanisms involved in the perception of the body illusion and its manipulation, the use of virtual reality (VR) has been essential. Various technical VR set-ups have been adopted by different authors to induce the sense of embodiment. Recently, 360-degree technology camera emerged as an innovative instrument to generate an immersive experience, with positive results in terms of involvement with the scenario. The current study aims to test the feasibility of the 360-degree video-based VR to induce a full body illusion. To do this, we evaluated two different groups receiving different levels of immersion: a 3D immersive video and a 2D non-immersive video. Self-reported sense of embodiment and heart rate variability (HRV) measures were analyzed. The results of the embodiment questionnaire showed that the immersive condition can trigger a full body illusion, with significant differences between the 3D and 2D conditions (ownership: p = .003, agency: p = .000, location: p = .013, haptic sensation: p = .027). No difference was found on the Root Mean Square of the Successive Differences (RMSSD) index—the beat-to-beat variance of the heart rate—of the HRV measure (first 90 s: p = .168, last 90 s: p = .401). Based on these results, future studies are needed to investigate the 360-degree video-based VR technology as a medium to generate the sense of embodiment.
Real Time Face Recognition using Effective Supervised Machine Learning Algorithms

Abstract In security authentication systems, face recognition has become an emerging new trend. Modern FR processes can detect whether the individual is real (live) or not through face recognition, thus preventing the systems from being compromised by displaying an actual person’s image. These new systems of face recognition are the result of recent advances in the field of computer vision and efficient algorithms for machine learning. This thesis describes in depth how the identification of a real time face can be order to create a safety alert framework for the workplace, the machine learning algorithm Haarcascade classifier was used to build four distinct classes for identification of security equipment and eventually identify the faces in both images and video using python open CV. Initially, face detection technique is performed to scan the face identity marks through.
Exploring Long Tail Visual Relationship Recognition with Large Vocabulary

Several approaches have been proposed in recent literature to alleviate the long-tail problem, mainly in object classification tasks. In this paper, we make the first largescale study concerning the task of Long-Tail Visual Relationship Recognition (LTVRR). LTVRR aims at improving the learning of structured visual relationships that come from the long-tail (e.g., "rabbit grazing on grass"). In this setup, the subject, relation, and object classes each follow a long-tail distribution. To begin our study and make a future benchmark for the community, we introduce two LTVRR-related benchmarks, dubbed VG8K-LT and GQA-LT, built upon the widely used Visual Genome and GQA datasets. We use these benchmarks to study the performance of several state-of-the-art long-tail models on the LTVRR setup. Lastly, we propose a visiolinguistic hubless (VilHub) loss and a Mixup augmentation technique adapted to LTVRR setup, dubbed as RelMix. Both VilHub and RelMix can be easily integrated on top of existing models and despite being simple, our results show that they can remarkably improve the performance, especially on tail classes. Benchmarks, code, and models have been made available at: https://github.com/Vision-CAIR/LTVRR.
Practical skills training in enclosure fires: An experimental study with cadets and firefighters using CAVE and HMD-based virtual training simulators

Training in enclosure fires is essential because it is one of the most dangerous tasks for firefighters. The training simulators should consider the specificity of the tasks performed to achieve the best training outcomes. Additionally, the end-users may not accept even the best technical solutions. Due to the availability of many different interfaces and methods (e.g., for image presentation), end-users' opinions are essential in choosing the best possible technologies. In this research, the aim is to create and compare two realistic virtual training systems for fighting enclosure fires, CAVE (Cave Automated Virtual Environment) with projectors and VR (Virtual Reality) with HMD (Head Mounted Display). Both focused on nozzle person training to fill the gap between theoretical training and real-life training with fires (e.g., drills with class A fuels). Efforts were made to minimize the disadvantages of training simulators while maintaining high simulation realism and low simulator cost. The study was conducted with the participation of 67 cadets and seven active firefighters. The method of reducing the limitations of VR training simulators while maintaining relatively low costs is discussed. The simulator enables multisensory experience with passive haptic and multi-user interaction considering the sense of touch. The VR simulator was rated better than the CAVE simulator, but both were accepted and rated as very useful. The paper also discusses the issues of simulation sickness symptoms, simulation realism (spatial presence), usability, acceptance of technology, and stress associated with using a training simulator.
Synthetic and Private Smart Health Care Data Generation using GANs

With the rapid advancements in machine learning, the health care paradigm is shifting from treatment towards prevention. The smart health care industry relies on the availability of large-scale health datasets in order to benefit from machine learning-based services. As a consequence, preserving the individuals’ privacy becomes vital for sharing sensitive personal information. Synthetic datasets with generative models are considered to be one of the most promising solutions for privacy-preserving data sharing. Among the generative models, generative adversarial networks (GANs) have emerged as the most impressive models for synthetic data generation in recent times. However, smart health care data is attributed with unique challenges such as volume, velocity, and various data types and distributions. We propose a GAN coupled with differential privacy mechanisms for generating a realistic and private smart health care dataset. The proposed approach is not only able to generate realistic synthetic data samples but also the differentially private data samples under different settings: learning from a noisy distribution or noising the learned distribution. We tested and evaluated our proposed approach using a real-world Fitbit dataset. Our results indicate that our proposed approach is able to generate quality synthetic and differentially private dataset that preserves the statistical properties of the original dataset.
FastAudio: A Learnable Audio Front-End For Spoof Speech Detection

Spoof speech can be used to try and fool speaker verification systems that determine the identity of the speaker based on voice characteristics. This paper compares popular learnable front-ends on this task. We categorize the front-ends by defining two generic architectures and then analyze the filtering stages of both types in terms of learning constraints. We pro-pose replacing fixed filterbanks with a learnable layer that can better adapt to anti-spoofing tasks. The proposed FastAudio front-end is then tested with two popular back-ends to measure the performance on the Logical Access track of the ASVspoof 2019 dataset. The FastAudio front-end achieves a relative improvement of 29.7% when compared with fixed front-ends, outperforming all other learnable front-ends on this task.
Zero-Shot Chinese Text Recognition via Matching Class Embedding

This paper studies the challenging problem of zero-shot Chinese text recognition, which requires the model to train on text line images containing only the seen characters, and then recognize the unseen characters from new text line images. Most of the previous methods only consider the zero-shot Chinese character recognition problem. They attempt to decompose the Chinese characters into radical representations and then recognize them at the radical level. Some methods developed recently have extended the radical-based recognition model from recognizing characters to recognizing text lines. However, the disadvantages of these methods include the requirement of long training time and a complicated decoding process. In addition, these methods are unsuitable for long text sequences. In this paper, we have proposed a novel zero-shot Chinese text recognition network (ZCTRN) by matching the class embeddings with the visual features. Specifically, our proposed model consists of three components: a text line encoder that extracts the visual features from the text line images, a class embedding module that encodes the character classes into class embeddings, and a bidirectional embedding transfer module that can map the class embeddings into the visual space and preserve the information of the original class embeddings. In addition, we use a distance-based CTC decoder to match the visual features with the class embeddings and output the recognition results. Experimental obtained by applying our proposed network to the MTHv2 dataset and the ICDAR-2013 handwriting competition dataset show that our method not only preserves high accuracy in recognizing text line images containing seen characters, but also outperforms the existing state-of-the-art models in recognizing text line images containing unseen characters.
A new VAE-GAN model to synthesize arterial spin labeling images from structural MRI

Arterial spin labeling (ASL) is a relatively new MRI technique that can measure cerebral blood flow, which is of great importance for the diagnosis of dementia diseases. Besides, this valuable imaging modality does not need exogenous tracers and has no radiation, which makes it favorable for elder patients. However, ASL data does lack in many contemporary image-based dementia diseases datasets, which include popular ADNI-1/GO/2/3 datasets. In order to supplement the valuable ASL data, a new Generative adversarial network (GAN)-based model is proposed to synthesize ASL images in this study. This new model is unique, as the popular variational auto-encoder (VAE) has been utilized as the generator of the GAN-based model. Hence, a new VAE-GAN architecture is introduced in this study. In order to demonstrate its superiority, dozens of experiments have been conducted. Experimental results demonstrate that, this new VAE-GAN model is superior to other state-of-the-art ASL image synthesis methods, and the accuracy improvement after incorporating synthesized ASL images from the new model can be as high as 42.41% in dementia diagnosis tasks.
Monotonic relation-constrained Takagi-Sugeno-Kang fuzzy system

The Takagi-Sugeno-Kang fuzzy system has wide applications across different areas, e.g., regression, classification and decision making, attributed to its high precision and interpretability. However, the existing Takagi-Sugeno-Kang fuzzy system is not an ideal solution to some special scenarios, particularly for those that are constrained monotonically. To this end, a monotonic relation-constrained Takagi-Sugeno-Kang fuzzy system classifier is proposed in this paper. The proposed method introduces a monotonic relation between the inputs and the outputs, where the objective function is expressed in a monotonically constrained form and a strategy for generating monotonicity constraint pairs is developed. Furthermore, to address the convexity loss caused by the increasing monotonicity constraints, the proposed method introduces the Tikhonov regularization strategy to ensure the uniqueness and boundedness of the solution. The results from extensive experiments show that the proposed method exhibits better classification performance than the original Takagi-Sugeno-Kang fuzzy system and state-of-the-art monotonic classification methods in handling monotonic datasets.
FILDNE: A Framework for Incremental Learning of Dynamic Networks Embeddings

Representation learning on graphs has emerged as a powerful mechanism to automate feature vector generation for downstream machine learning tasks. The advances in representation on graphs have centered on both homogeneous and heterogeneous graphs, where the latter presenting the challenges associated with multi-typed nodes and/or edges. In this paper, we consider the additional challenge of evolving graphs. We ask the question of whether the advances in representation learning for static graphs can be leveraged for dynamic graphs and how? It is important to be able to incorporate those advances to maximize the utility and generalization of methods. To that end, we propose the Framework for Incremental Learning of Dynamic Networks Embedding (FILDNE), which can utilize any existing static representation learning method for learning node embeddings while keeping the computational costs low. FILDNE integrates the feature vectors computed using the standard methods over different timesteps into a single representation by developing a convex combination function and alignment mechanism. Experimental results on several downstream tasks, over seven real-world datasets, show that FILDNE is able to reduce memory (up to 6x) and computational time (up to 50x) costs while providing competitive quality measure gains (e.g., improvements up to 19 pp AUC on link prediction and up to 33 pp mAP on graph reconstruction) with respect to the contemporary methods for representation learning on dynamic graphs.
Effective and efficient pixel-level detection for diverse video copy-move forgery types

Video copy-move forgery detection (VCMFD) is a significant and greatly challenging task due to a variety of difficulties, including a huge amount of video information, diverse forgery types, rich forgery objects, and homogenous forgery sources. These difficulties raise four unresolved key challenges in VCMFD: i) ineffective detection in some popular forgery cases; ii) inefficient matching in processing numerous video pixels with hundred-dimensional features under dozens of matching iterations; iii) high false positive (FP) in detecting forgery videos; iv) low trade-off of efficiency and effectiveness in filling forgery region, and even failing in indicating forgeries at the pixel level. In this paper, a novel VCMFD method is proposed to address these issues: i) an innovatively improved SIFT structure that can address the thorough feature extraction in all video copy-move forgery cases; ii) a novel fast keypoint-label matching (FKLM) algorithm is proposed that creates some keypoint-label groups so that every high-dimensional feature is assigned into one of these groups. As a result, matching of video pixels can be directly done on a small number of keypoint-label groups only, leading to a nearly 500% raise in matching efficiency; iii) a new coarse-to-fine filtering relying on intrinsic attributes of exact keypoint-matches is designed to more effectively reduce the false keypoint-matches; iv) the adaptive block filling relying on true keypoint-matches contributes to the accurate and efficient suspicious region filling, even at the pixel level. Finally, the suspicious region locations with the forgery vision persistence concept indicate forgery videos. Compared to the state-of-art methods, the experiments show that our proposed method achieves the best detection accuracy, lowest FP, and improved at least 16% and 8% of F1 scores on the GRIP 2.0 dataset and a combination of SULFA 2.0 & REWIND datasets. Furthermore, the proposed method is with low computational time (4.45 s/Mpixels), which is about 1/2-1/3 times of the latest DFMI-BM (8.02 s/Mpixels) and PM-2D (13.1 s/Mpixels) methods.
Knowledge extraction using semantic similarity of concepts from Web of Things knowledge bases

The Internet of Things (IoT) is one of the rapidly growing technologies with the aim of establishing communication among objects, people, and processes. This rapidly growing technology faces a lot of challenges that hinder its wider adoption, specifically in developing applications that involve heterogeneous domains. Currently, developing such interoperable applications require substantial efforts by the developers to hard code the requirements to ensure the correctness of transferring knowledge. The efforts can be significantly reduced by developing an interoperable platform that ensures seamless communication between heterogeneous IoT devices. W3C Web of Things (WoT) is a significant step towards enabling interoperability between IoT devices by integrating the existing Web ecosystem with “Things”. WoT provides a unified interface over a suitable network protocol facilitating interactions between different IoT protocols. WoT Thing Descriptions (TD) enrich interoperability providing both human and machine readable metadata about a Thing. However, the WoT still falls short in providing semantic interoperability due to insufficient standard vocabularies which can describe different IoT application domains. In this paper, we propose a semantic similarity-based approach to automatically identify and extract the most common concepts from sixteen popular ontologies belonging to smart home and smart building domains. The proposed method helps the developers and researchers to develop a domain ontology with reduced effort. The extracted concepts are evaluated by the domain experts and are found to be sufficient in describing the smart home and smart building domains.
Transform, Warp, and Dress: A New Transformation-guided Model for Virtual Try-on

Virtual try-on has recently emerged in computer vision and multimedia communities with the development of architectures that can generate realistic images of a target person wearing a custom garment. This research interest is motivated by the large role played by e-commerce and online shopping in our society. Indeed, the virtual try-on task can offer many opportunities to improve the efficiency of preparing fashion catalogs and to enhance the online user experience. The problem is far to be solved: current architectures do not reach sufficient accuracy with respect to manually generated images and can only be trained on image pairs with a limited variety. Existing virtual try-on datasets have two main limits: they contain only female models, and all the images are available only in low resolution. This not only affects the generalization capabilities of the trained architectures but makes the deployment to real applications impractical. To overcome these issues, we present Dress Code , a new dataset for virtual try-on that contains high-resolution images of a large variety of upper-body clothes and both male and female models. Leveraging this enriched dataset, we propose a new model for virtual try-on capable of generating high-quality and photo-realistic images using a three-stage pipeline. The first two stages perform two different geometric transformations to warp the desired garment and make it fit into the target person’s body pose and shape. Then, we generate the new image of that same person wearing the try-on garment using a generative network. We test the proposed solution on the most widely used dataset for this task as well as on our newly collected dataset and demonstrate its effectiveness when compared to current state-of-the-art methods. Through extensive analyses on our Dress Code dataset, we show the adaptability of our model, which can generate try-on images even with a higher resolution.