Reviewerly/exHarmony huggingface dataset:
Automatic CNN-Based Enhancement of 360° Video Experience With Multisensorial Effects

High-resolution audio-visual virtual reality (VR) technologies currently offer satisfying experiences for both sight and hearing senses in the world of multimedia.However, the delivery of truly immersive experiences requires the incorporation of other senses such as touch and smell.Multisensorial effects are usually manually synchronized with videos and data is stored in companion files, which contain timestamps for these effects.This manual task becomes very complex for 360°videos, as the scenes triggering effects can occur in different viewpoints.The solution proposed in this paper aims to automatically add extra sensory information to immersive 360°videos.A novel scent prediction scheme using Convolutional Neural Networks (CNN) is proposed to perform scene predictions on 360°videos represented in the Equi-Angular Cubemap format in order to add scents relevant to the detected content.Digital signal processing is used to detect loud sounds in the video with a Root Mean Squared (RMS) function, which are then associated with haptic feedback.A prototype was developed, which outputs multisensorial stimuli by using an olfaction dispenser and a haptic mouse.The proposed solution has been tested and it achieved excellent results in terms of accuracy of scene detection, olfaction latency and correct execution of the relevant effects.Different CNN architectures, including AlexNet, ResNet18 and ResNet50, were also assessed comparatively, achieving a labeling accuracy of up to 72.67% for olfaction-enhanced media.
High-Capacity Image Steganography Algorithm Based on Image Style Transfer

Steganography is a technique for publicly transmitting secret information through a cover. Most of the existing steganography algorithms are based on modifying the cover image, generating a stego image that is very similar to the cover image but has different pixel values, or establishing a mapping relationship between the stego image and the secret message. Attackers will discover the existence of secret communications from these modifications or differences. In order to solve this problem, we propose a steganography algorithm ISTNet based on image style transfer, which can convert a cover image into another stego image with a completely different style. We have improved the decoder so that the secret image features can be fused with style features in a variety of sizes to improve the accuracy of secret image extraction. The algorithm has the functions of image steganography and image style transfer at the same time, and the images it generates are both stego images and stylized images. Attackers will pay more attention to the style transfer side of the algorithm, but it is difficult to find the steganography side. Experiments show that our algorithm effectively increases the steganography capacity from 0.06 bpp to 8 bpp, and the generated stylized images are not significantly different from the stylized images on the Internet.
5G and Beyond for Contact Tracing

The COVID-19 pandemic has suddenly raised the need for technological solutions able to trace contacts of people and provide location-based analytics. Several countries have adopted proximity-based (short-range) technologies, such as Bluetooth, which, however, appear to be hindered by deployment issues, security leakage, lack of reliability, and data governance concerns. This article posits that 5G and beyond can play a primary role in contact tracing and group movement monitoring. Contact tracing based on 5G location-based analytics benefits from the pervasive deployment of cellular networks, the several years of effort to design cellular standards for localization and analytics, and the best practices of cellular operators to handle location data.
Orientation and Channel-Independent RF Fingerprinting for 5G IEEE 802.11ad Devices

Physical-layer fingerprinting is a promising technique to identify Internet of Things (IoT) devices. In this article, we investigate a new radio-frequency (RF) fingerprinting based on the distinctive signal-to-noise-ratio (SNR) trace in the sector-level sweep (SLS) procedure of 5G IEEE 802.11ad devices. This SLS SNR trace-based fingerprinting can directly apply to off-the-shelf devices without any extra hardware requirements and be independent of the wireless channel and environment. To tackle the impact of orientation on the RF fingerprinting, we propose a novel fingerprinting framework, involving correlation analysis, surface fitting, curve pursuing, and binary classification, named the CSCB framework. Using this framework, the proposed SLS SNR trace-based fingerprinting can achieve device authentication at any orientation with one receiver under line-of-sight (LOS) or non-LOS (NLOS) scenarios. We conduct proof-of-concept experiments using off-the-shelf IEEE 802.11ad devices (Talon AD7200 and MG360 WiGig) to evaluate the performance of the proposed fingerprinting schemes. Experimental results show the effectiveness of the proposed fingerprinting schemes where the verification accuracy of the proposed scheme can reach 99% with only 200 training samples.
H&amp;E-adversarial network: a convolutional neural network to learn stain-invariant features through Hematoxylin &amp; Eosin regression

Computational pathology is a domain that aims to develop algorithms to automatically analyze large digitized histopathology images, called whole slide images (WSI).WSIs are produced scanning thin tissue samples that are stained to make specific structures visible.They show stain colour heterogeneity due to different preparation and scanning settings applied across medical centers.Stain colour heterogeneity is a problem to train convolutional neural networks (CNN), the state-of-the-art algorithms for most computational pathology tasks, since CNNs usually underperform when tested on images including different stain variations than those within data used to train the CNN.Despite several methods that were developed, stain colour heterogeneity is still an unsolved challenge that limits the development of CNNs that can generalize on data from several medical centers.This paper aims to present a novel method to train CNNs that better generalize on data including several colour variations.The method, called H&E-adversarial CNN, exploits H&E matrix information to learn staininvariant features during the training.The method is evaluated on the classification of colon and prostate histopathology images, involving eleven heterogeneous datasets, and compared with five other techniques used to handle stain colour heterogeneity.H&E-adversarial CNNs show an improvement in performance compared to the other algorithms, demonstrating that it can help to better deal with stain colour heterogeneous images.
HARNAS: Human Activity Recognition Based on Automatic Neural Architecture Search Using Evolutionary Algorithms

Human activity recognition (HAR) based on wearable sensors is a promising research direction. The resources of handheld terminals and wearable devices limit the performance of recognition and require lightweight architectures. With the development of deep learning, the neural architecture search (NAS) has emerged in an attempt to minimize human intervention. We propose an approach for using NAS to search for models suitable for HAR tasks, namely, HARNAS. The multi-objective search algorithm NSGA-II is used as the search strategy of HARNAS. To make a trade-off between the performance and computation speed of a model, the F1 score and the number of floating-point operations (FLOPs) are selected, resulting in a bi-objective problem. However, the computation speed of a model not only depends on the complexity, but is also related to the memory access cost (MAC). Therefore, we expand the bi-objective search to a tri-objective strategy. We use the Opportunity dataset as the basis for most experiments and also evaluate the portability of the model on the UniMiB-SHAR dataset. The experimental results show that HARNAS designed without manual adjustments can achieve better performance than the best model tweaked by humans. HARNAS obtained an F1 score of 92.16% and parameters of 0.32 MB on the Opportunity dataset.
Entity-aware Collaborative Relation Network with Knowledge Graph for Recommendation

As the source of side information, knowledge graph (KG) plays a critical role in recommender systems. Recently, graph neural networks (GNN) have shown their technical advancements at boosting recommendation performances. Existing GNN-based models mainly focus on aggregation technique and regularization allocation, ignoring the rich entity-aware information hidden in the relation network of KG. In this paper, we explore the relational semantics at the granularity of entities behind a user-item interaction by leveraging knowledge graph, named Entity-aware Collaborative Relation Network (ECRN). Technically, we construct multiple meta-paths from users to entities based on the user-item interaction and item-entity connectivity to obtain user representation, while designing a relation-aware self-attention mechanism to aggregate collaborative signals of items. Empirical results on three benchmarks show that ECRN significantly outperforms state-of-the-art baselines.
Power Modeling for Effective Datacenter Planning and Compute Management

Over the past decade, there has been a global growth in datacenter capacity, power consumption and the associated costs. Accurate mapping of datacenter resource usage (CPU, RAM, etc.) and hardware configurations (servers, accelerators, etc.) to its power consumption is necessary for efficient long-term infrastructure planning and real-time compute load management. This paper presents two types of statistical power models that relate CPU usage of Google&#x2019;s Power Distribution Units (PDUs, commonly referred to as power domains) to their power consumption. The models are deployed in production and are used for cost- and carbon-aware load management, power provisioning and infrastructure rightsizing. They are simple, interpretable and exhibit uniformly high prediction accuracy in modeling power domains with large diversity of hardware configurations and workload types across Google fleet. A multi-year validation of the deployed models demonstrate that they can predict power with less than 5&#x0025; Mean Absolute Percent Error (MAPE) for more than 95&#x0025; diverse PDUs across Google fleet. This performance matches the best reported accuracies coming from studies that focus on specific workload types, hardware platforms and, typically, more complex statistical models.
Memory-Augmented Transformer for Remote Sensing Image Semantic Segmentation

The semantic segmentation of remote sensing images requires distinguishing local regions of different classes and exploiting a uniform global representation of the same-class instances. Such requirements make it necessary for the segmentation methods to extract discriminative local features between different classes and to explore representative features for all instances of a given class. While common deep convolutional neural networks (DCNNs) can effectively focus on local features, they are limited by their receptive field to obtain consistent global information. In this paper, we propose a memory-augmented transformer (MAT) to effectively model both the local and global information. The feature extraction pipeline of the MAT is split into a memory-based global relationship guidance module and a local feature extraction module. The local feature extraction module mainly consists of a transformer, which is used to extract features from the input images. The global relationship guidance module maintains a memory bank for the consistent encoding of the global information. Global guidance is performed by memory interaction. Bidirectional information flow between the global and local branches is conducted by a memory-query module, as well as a memory-update module, respectively. Experiment results on the ISPRS Potsdam and ISPRS Vaihingen datasets demonstrated that our method can perform competitively with state-of-the-art methods.
An Approach to Detect Anomaly in Video Using Deep Generative Network

Anomaly detection in the video has recently gained attention due to its importance in the intelligent surveillance system. Even though the performance of the state-of-art methods has been competitive in the benchmark dataset, the trade-off between the computational resource and the accuracy of the anomaly detection should be considered. In this paper, we present a framework to detect anomalies in video. We proposed a “multi-scale U-Net” network architecture, the unsupervised learning for anomaly detection in video based on generative adversarial network (GAN) structure. Shortcut Inception Modules (SIMs) and residual skip connection are employed to the generator network to increase the ability of the training and testing of the neural network. An asymmetric convolution has been applied instead of traditional convolution layers to decrease the number of training parameters without performance penalty in terms of detection accuracy. In the training phase, the generator network was trained to generate the normal events and attempt to make the generated image and the ground truth to be similar. A multi-scale U-Net kept useful features of an image that were lost during training caused by the convolution operator. The generator network is trained by minimizing the reconstruction error on the normal data and then using the reconstruction error as an indicator of anomalies in the testing phase. Our proposed framework has been evaluated on three benchmark datasets, including UCSD pedestrian, CHUK Avenue, and ShanghaiTech. As a result, the proposed framework surpasses the state-of-the-art learning-based methods on all these datasets, which achieved 95.7%, 86.9%, and 73.0% in terms of AUC. Moreover, the numbers of training and testing parameters in our framework are reduced compared to the baseline network architecture, while the detection accuracy is still improved.
Analysis of Machine Learning Models Predicting Quality of Life for Cancer Patients

Quality of life (QoL) is one of the major issues for cancer patients. With the advent of medical databases containing large amounts of relevant QoL information it becomes possible to train predictive QoL models by machine learning (ML) techniques. However, the training of predictive QoL models poses several challenges mostly due to data privacy concerns and missing values in patient data. In this paper, we analyze several classification and regression ML models predicting QoL indicators for breast and prostate cancer patients. Two different approaches are employed for imputing missing values. The examined ML models are trained on datasets formed from two databases containing a large number of anonymized medical records of cancer patients from Sweden. Two learning scenarios are considered: centralized and federated learning. In the centralized learning scenario all patient data coming from different data sources is collected at a central location prior to model training. On the other hand, federated learning enables collective training of machine learning models without data sharing. The results of our experimental evaluation show that the predictive power of federated models is comparable to that of centrally trained models for short-term QoL predictions, whereas for long-term periods centralized models provide more accurate QoL predictions.
Authorship Attribution of Social Media Messages

The world is facing a new era in which social media communication plays a fundamental role in people’s lives. Along with proven benefits, several collateral drawbacks have risen, one being the widespread of false information with malicious intents, oftentimes using anonymous or false identities. Fighting this problem is challenging, especially when considering the nature of text messages involved on social media platforms: a sea of small messages and a myriad of users. Attributing the authorship of such messages is an ambitious endeavor; nevertheless, it is a way to fight this undesired disinformation scenario. In this work, we tackle the problem of authorship attribution of tiny messages, but, different from what has been done with longer texts, we rely upon data-driven approaches, avoiding handcraft features and harnessing recent advances of deep neural networks in the field of pattern recognition. By modeling small texts employed in social media as unidimensional signals, we propose a deep learning model to project these messages onto a manifold suitable for the task of authorship attribution. We provide two state-of-the-art solutions tailored for different setups and strategies for the scenario of authorship verification. These advances were possible, thanks to three additional contributions: an updated dataset based on the Twitter® platform, new sanitization techniques to improve the quality of the training data, and novel visual analytics techniques to help the development of authorship attribution solutions.
Parsing AUC Result-Figures in Machine Learning Specific Scholarly Documents for Semantically-enriched Summarization

Machine learning specific scholarly full-text documents contain a number of result-figures expressing valuable data, including experimental results, evaluations, and cross-model comparisons. The scholarly search system often overlooks this vital information while indexing important terms using conventional text-based content extraction approaches. In this paper, we propose creating semantically enriched document summaries by extracting meaningful data from the results-figures specific to the evaluation metric of the area under the curve (AUC) and their associated captions from full-text documents. At first, classify the extracted figures and analyze them by parsing the figure text, legends, and data plots – using a convolutional neural network classification model with a pre-trained ResNet-50 on 1.2 million Images from ImageNet. Next, we extract information from the result figures specific to AUC by approximating the region under the function’s graph as a trapezoid and calculating its area, i.e., the trapezoidal rule. Using over 12,000 figures extracted from 1000 scholarly documents, we show that figure specialized summaries contain more enriched terms about figure semantics. Furthermore, we empirically show that the trapezoidal rule can calculate the area under the curve by dividing the curve into multiple intervals. Finally, we measure the quality of specialized summaries using ROUGE, Edit distance, and Jaccard Similarity metrics. Overall, we observed that figure specialized summaries are more comprehensive and semantically enriched. The applications of our research are enormous, including improved document searching, figure searching, and figure focused plagiarism. The data and code used in this paper can be accessed at the following URL: https://github.com/slab-itu/fig-ir/.
Recommending Research Articles: A Multi-Level Chronological Learning-Based Approach Using Unsupervised Keyphrase Extraction and Lexical Similarity Calculation

A research article recommendation approach aims to recommend appropriate research articles to analogous researchers to help them better grasp a new topic in a particular research area. Due to the accessibility of research articles on the web, it is tedious to recommend a relevant article to a researcher who strives to understand a particular article. Most of the existing approaches for recommending research articles are metadata-based, citation-based, bibliographic coupling-based, content-based, and collaborative filtering-based. They require a large amount of data and do not recommend reference articles to the researcher who wants to understand a particular article going through the reference articles of that particular article. Therefore, an approach that can recommend reference articles for a given article is needed. In this paper, a new multi-level chronological learning-based approach is proposed for recommending research articles to understand the topics/concepts of an article in detail. The proposed method utilizes the TeKET keyphrase extraction technique, among other unsupervised techniques, which performs better in extracting keyphrases from the articles. Cosine and Jaccard similarity measures are employed to calculate the similarity between the parent article and its reference articles using the extracted keyphrases. The cosine similarity measure outperforms the Jaccard similarity measure for finding and recommending relevant articles to understand a particular article. The performance of the recommendation approach seems satisfactory, with an NDCG value of 0.87. The proposed approach can play an essential role alongside other existing approaches to recommend research articles.
Eternally non-Markovian dynamics of a qubit interacting with a single-photon wavepacket

An evolution of a two-level system (qubit) interacting with a single-photon wave packet is analyzed. It is shown that a hierarchy of master equations gives rise to phase covariant qubit evolution. The temporal correlations in the input field induce nontrivial memory effects for the evolution of a qubit. It is shown that in the resonant case whenever time-local generator is regular (does not display singularities) the qubit evolution never displays information backflow. However, in general the generator might be highly singular leading to intricate non-Markovian effects. A detailed analysis of the exponential profile is provided which allows to illustrate all characteristic feature of the qubit evolution.
Smart Textiles for Improved Quality of Life and Cognitive Assessment

Smart textiles can be used as innovative solutions to amuse, meaningfully engage, comfort, entertain, stimulate, and to overall improve the quality of life for people living in care homes with dementia or its precursor mild cognitive impairment (MCI). This concept paper presents a smart textile prototype to both entertain and monitor/assess the behavior of the relevant clients. The prototype includes physical computing components for music playing and simple interaction, but additionally games and data logging systems, to determine baselines of activity and interaction. Using microelectronics, light-emitting diodes (LEDs) and capacitive touch sensors woven into a fabric, the study demonstrates the kinds of augmentations possible over the normal manipulation of the traditional non-smart activity apron by incorporating light and sound effects as feedback when patients interact with different regions of the textile. A data logging system will record the patient's behavioral patterns. This would include the location, frequency, and time of the patient's activities within the different textile areas. The textile will be placed across the laps of the resident, which they then play with, permitting the development of a behavioral profile through the gamification of cognitive tests. This concept paper outlines the development of a prototype sensor system and highlights the challenges related to its use in a care home setting. The research implements a wide range of functionality through a novel architecture involving loosely coupling and concentrating artifacts on the top layer and technology on the bottom layer. Components in a loosely coupled system can be replaced with alternative implementations that provide the same services, and so this gives the solution the best flexibility. The literature shows that existing architectures that are strongly coupled result in difficulties modeling different individuals without incurring significant costs.