Reviewerly/exHarmony huggingface dataset:
Real-time human detection and behavior recognition using low-cost hardware

Cameras are becoming more pervasive and ubiquitous. The daily activities of individuals are being captured by millions of cameras in public spaces, while individuals are obtaining massive amounts of egocentric videos by employing wearable cameras intended for life-logging. However, recording devices are inexpensive, highly computational, and inconvenient for privacy. We used a low-resolution infrared sensor to detect human activity, including sitting, standing, and lying down, and to locate humans. We acquired the data from a low-cost infrared device and preprocessed them to train the YOLO-v5 network. We developed and tested an infrared technology-based system consisting of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$32 \times 24$</tex> thermal input. Our proposed model is trained on 3,864 low-resolution images and made publicly available. The trained YOLO-v5 achieved 96.34% mean Average Precision (mAP) using our designed lightweight and low-cost activity recognition device. We proposed Artificial Intelligence of Things (A-IoT) system can be used either as a stand-alone data collection such as an IoT device or as a data processing and analysis sub-center. Our system consists of a low-power edge computing device and a cost-effective low-resolution infrared module. Our proposed dataset is now available at https://github.com/InfoLab-SKKU/Thermal-Human-Detection
A Study on Partially Homomorphic Encryption

Recently, data experts can obtain a large amount of data with the development of the Internet. When computing such data, cloud services that do not use the personal device's memory are becoming popular. However, storing sensitive data as a source in the cloud carries the risk of hijacking. To compensate for this, homomorphic encryption, which encrypts and stores sensitive data, and can safely operate in an encrypted state, is being studied. In this paper, we analyze four methods of partially homomorphic encryption among homomorphic encryption methods. We compare and analyze the key size and ciphertext size of four partially homomorphic encryptions, Paillier, ElGamal, ASHE, and Symmetria.
Machine Learning Model to Evaluate the Appropriateness of Layout for Automatic Generation of Graphic Design Works

Generative networks (GANs) are commonly used for image generation to generate images with noise as input to the trained generator. However, it is difficult to generate appropriate graphic designs with GANs when the user specifies the material images and text to be used as input, as in the case of graphic design. In this study, we focus on the graphic design layout and propose a method for automatically generating graphic designs using adversarial GANs. We trained the generator and discriminator using GANs by converting the training data of graphic designs into images that represent their visual importance. The trained discriminators were then used to evaluate the automatically generated layouts using specified materials. The results of experiments with subjective evaluations using subjects indicates that the proposed method was effective.
A deep learning based classifier framework for automated nuclear atypia scoring of breast carcinoma

Nuclear atypia scoring is an essential procedure in the grading of breast carcinoma. Manual procedure of nuclear atypia scoring is error-prone, and marked by pathologists’ disagreement and low reproducibility. Automated methods are actively attempted by researchers to solve the problems of manual scoring. In this work, we propose a novel deep learning-based framework for automated nuclear atypia scoring of breast cancer from histopathology slide images. The framework consists of three major phases namely preprocessing, deep learning, and postprocessing. The original three-class problem of atypia scoring at slide level is not suitable for direct application of deep learning algorithms. This is due to the large dimensions and structural complexity of slide images, compounded by the small sample size of the available dataset. Redesign of this problem into a six-class nuclei classification problem through a set of preprocessing steps to facilitate effective use of deep learning algorithms, and the flexibility of the proposed three-phase framework to use different algorithms in each phase are the novel aspects of the proposed work. We used the publicly available slide image dataset MITOS-ATYPIA that contains 600 slide images of high spatial dimension for the experiments. A five-fold cross validation with the train-test sample ratio 80:20 in each fold is used for the performance evaluation. The performance of the method based on this framework exceeds the state-of-the-art with the results 0.8766, 0.8760, and 0.8745 for the metrics precision, recall, and F1 score respectively.
Lambertian-based adversarial attacks on deep-learning-based underwater side-scan sonar image classification

Deep convolutional neural networks (CNNs) are extensively applied to the classification tasks for Side-scan sonar (SSS) images. However, state-of-the-art neural networks are prone to be confused by adversarial attacks that generate a tiny modification of the images, threatening the security of SSS classification. The robustness of CNN to adversarial attacks can be improved by introducing adversarial examples through adversarial training. Practical adversarial examples are often generated from elaborate adversarial attackers. For the underwater scenario of sonar, a specially designed adversarial attack method to weaken SSS image classification can make the research community better understand the weakness of CNN in this scenario and improve the security measures in a well-directed way. Thus, exploring adversarial attack methods for SSS image classification is essential. Nevertheless, the existing adversarial attack methods are designed for optical images, reflecting no physical characteristics of sonar images. To fill this gap and investigate the adversarial attack related to real-world conditions, in this paper, we propose an adversarial attack method named Lambertian Adversarial Sonar Attack (LASA). It initially leverages the Lambertian model to simulate the formation of the SSS image, factoring the image to three parameters, then updates the parameters on the direction of gradients by the chain rule. Finally, the parameters regenerate the adversarial example to fool the classifier. To validate the performance of LASA, we constructed a diversified SSS image dataset containing three categories. On our dataset, LASA reduces the Top-1 accuracy of a well-trained ResNet-101 to 7.31%±0.21 (one-shot version) and 0.00% (iterative version), the success rate of targeted attack reaches 97.03±2.24, far beyond the performance of the existing state-of-the-art adversarial attack methods. Meanwhile, we show that the adversarial training using examples generated from LASA makes the classifier more robust. We expect that our methods can be applied as a benchmark of adversarial attacks on SSS images, motivating future research to design novel neural networks or defensive methods to resist real-world adversarial attacks on SSS images.
Non-fungible token potentials for hotel bookings

• We discuss an NFT solution for booking and reselling hotel reservations. • A secondary NFT market for the hospitality industry is a prominent example. • Hotel NFTs are vehicles to eliminate intermediaries. • NFTs are even more revolutionary than cryptocurrencies.
Generative adversarial network-based deep learning technique for smart grid data security

The smart grid (SG) is an intelligent electricity grid that implements evolving information and communication infrastructure for controlling power generation, transmission, distribution, and consumers. In contrast to traditional electricity grids, SG facilitates a two-way dataflow between the consumer and electrical utilities. This facility enables real-time electricity generation based on consumers’ energy demands. Hence, consumer data secrecy becomes imperative while energy usage data collection. There are multiple inevitable security challenges that need to be addressed at the levels of consumer, communication, and energy provider. SG data security facets such as privacy, validation, approval, etc., require in-depth investigation to implement data security. In this method, data are encrypted first and then the encrypted data are embedded using a generative adversarial network (GAN). The encrypted data will be hidden in a cover image produced by GAN. Hence, confidential information is encrypted. Thus, the proposed technique can provide higher security.
Forecasting off-grid solar power generation using case-based reasoning algorithm for a small-scale system

The solar power generation is in the spotlight since it is eco-friendly in nature and offers economic benefits. The prediction of the generated energy has become a part of the solar power generation system. All the countries are nowadays encouraging research, development, and contributing toward this nonpolluting system. There are still some challenges associated with solar photovoltaics. Its output is highly ambiguous and varied as a result of dependence on the environmental factors such as atmospheric aerosol level, speed of the wind, irradiance, expanse of cloud cover, temperature, humidity, and settlement of dust particles. Power prediction can lift some of these problems of solar photovoltaics by implementing an improved version of power prediction algorithms.
Emergence of blockchain technology in the healthcare and insurance industries

Innovation in healthcare sector has resulted in a disparity of opinions among clients. A report by HealthTech Magazine indicated that there were nearly 32 million breaches of health records in the year 2019 alone, which is among the highest ever recorded in this sector. As healthcare facilities have shifted to the utilization of digital means in sharing information, cybercriminals have found loopholes of intercepting the data and demanding ransom for the demographic data acquired in the process. It has become more common than ever to have cases where people are using others’ identities for money laundering. To some, such actions have significantly increased the cost of healthcare, since people are unable to trust the process of sharing medical records within digital circles. The research community contest that technological solutions represent the best way to confront these. However, despite the current technological systems within the industry, privacy and security concerns are turning out to be an ever bigger challenge. The popularity of blockchain technology as one of the biggest developments of the fourth industrial revolution is attracting attention from researchers as the best technological fit for security and privacy concerns within the industry. This study evaluates the current state of technological systems within the industry and the concept of blockchain technology, and provides analysis to determine the feasibility of this technological concept in the healthcare and insurance industries. The outcomes suggest greater research support for blockchain’s advantages and feasibility within the industry.
Blockchain embedded security and privacy preserving in healthcare systems

The Internet of Things (IoT) is being heavily leveraged in the field of healthcare nowadays. The field of healthcare deals with highly personal data, such as an individual’s history of disease, blood pressure, etc. Due to this, the technologies that are used in this domain become more prone to attackers, given their sensitivity. Thus, it is imperative that the data is protected. To ensure this, the use of blockchain has proved to be highly beneficial. Embedding blockchain in IoT-based devices has become very beneficial due to the many technological advances in the field. This chapter aims to provide an overview of the features that blockchain provides that make it so helpful in a domain that deals with such sensitive data and helps protect the security as well as privacy of the data. Additionally, this chapter also describes, in detail, the many security challenges the field deals with and how blockchain can help overcome them.
Hierarchical Instance Mixing Across Domains in Aerial Segmentation

We investigate the task of unsupervised domain adaptation in aerial semantic segmentation observing that there are some shortcomings in the class mixing strategies used by the recent state-of-the-art methods that tackle this task: (i) they do not account for the large disparity in the extension of the semantic categories that is common in the aerial setting, which causes a domain imbalance in the mixed image; (ii) they do not consider that aerial scenes have a weaker structural consistency in comparison to the driving scenes for which the mixing technique was originally proposed, which causes the mixed images to have elements placed out of their natural context; (iii) the source model used to generate the pseudo-labels may be susceptible to perturbations across domains, which causes inconsistent predictions on the target images and can jeopardize the mixing strategy. We address these shortcomings with a novel aerial semantic segmentation framework for UDA, named HIUDA, which is composed of two main technical novelties: (i) a new mixing strategy for aerial segmentation across domains called Hierarchical Instance Mixing (HIMix), which extracts a set of connected components from each semantic mask and mixes them according to a semantic hierarchy and, (ii) a twin-head architecture in which two separate segmentation heads are fed with variations of the same images in a contrastive fashion to produce finer segmentation maps. We conduct extensive experiments on the LoveDA benchmark, where our solution outperforms the current state-of-the-art.
OBBStacking: An Ensemble Method for Remote Sensing Object Detection

Ensemble methods are a reliable way to combine several models to achieve superior performance. However, research on the application of ensemble methods in the remote sensing object detection scenario is mostly overlooked. Two problems arise. First, one unique characteristic of remote sensing object detection is the Oriented Bounding Boxes (OBB) of the objects and the fusion of multiple OBBs requires further research attention. Second, the widely used deep learning object detectors provide a score for each detected object as an indicator of confidence, but how to use these indicators effectively in an ensemble method remains a problem. Trying to address these problems, this paper proposes OBBStacking, an ensemble method that is compatible with OBBs and combines the detection results in a learned fashion. This ensemble method helps take 1st place in the Challenge Track <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fine-grained Object Recognition in High-Resolution Optical Images</i> , which was featured in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2021 Gaofen Challenge on Automated High-Resolution Earth Observation Image Interpretation</i> . The experiments on DOTA dataset and FAIR1M dataset demonstrate the improved performance of OBBStacking and the features of OBBStacking are analyzed. Code will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Haoning724/obbstacking</uri>
The Extreme Learning Machine for the Extraction of Mono-Exponential Decay Time

The mono-exponential decay has been used to describe various physical phenomena such as cavity ring-down signal, fluorescence decay, etc. In this paper, a neural network method of extreme learning machine (ELM) is adopted to efficiently extract decay time. The theoretical extraction precision, accuracy, and computation cost are all preliminarily analyzed and quantitatively compared with the traditional Levenberg-Marquardt (LM) algorithm. The training set and the testing set for the ELM are built based on our experimental system parameters. After dataset training, the ELM model for mono-exponential decay extraction is obtained. In the dataset testing, this model gives almost the same results with the LM algorithm. The relative deviation of precision is only about ±1 nanosecond. This ELM model can also be directly used in experimental cavity ring-down system. Comparing with the LM algorithm, the relative deviations are less than ±2 nanoseconds when the decay time is in the range of 0.98 μs∼2.20 μs. The ELM method for mono-exponential decay extraction has high efficiency and fine robustness. It has the potential for the applications in cavity ring-down spectroscopy, fluorescence decay analysis, and nuclear radioactive technique, etc.
Qubit-oscillator relationships in the open quantum Rabi model: the role of dissipation

Using a dissipative quantum Rabi model, we study the dynamics of a slow qubit coupled to a fast quantum harmonic oscillator interacting with a bosonic bath from weak to strong and ultra-strong coupling regimes. Solving the quantum Heisenberg equations of motion, perturbative in the internal coupling between qubit and oscillator, we derive functional relationships directly linking the qubit coordinates in the Bloch sphere to oscillator observables. We then perform accurate time-dependent Matrix Product State simulations, and compare our results both with the analytical solutions of the Heisenberg equations of motion, and with numerical solutions of a Lindblad master equation, perturbative in the external coupling between oscillator and environment. Indeed, we show that, up to the strong coupling regime, the qubit state accurately fulfills the derived functional relationships. We analyse in detail the case of a qubit starting with generic coordinates on the Bloch sphere of which we evaluate the three components of the Bloch vector through the averages of oscillator observables. Interestingly, a weak to intermediate oscillator coupling to the bath is able to simplify the Bloch vector evaluation since qubit-oscillator relationships are more immediate. Moreover, by monitoring the qubit fidelity with respect to free limit, we find the parameter regime where the combined effect of internal and external couplings is able to hinder the reliable evaluation of the qubit Bloch vector. Finally, in the ultra-strong coupling regime, non-Markovian effects become robust and the dynamics of qubit and oscillator are inextricably entangled making the qubit Bloch vector evaluation difficult.
TsrNet: A two-stage unsupervised approach for clothing region-specific textures style transfer

Style transfer has been widely applied in the fashion industry for design assistance. This paper focuses on object-level image style transfer for specific texture regions on fashion images. The current methods mostly utilize extra conditional inputs to obtain regions of interest in fashion images, which take extensive time and cost to prepare these data. To address this issue, we propose a two-stage unsupervised approach named TsrNet for clothing texture style transfer between images. Firstly, we construct a network named Mask-GAN to unsupervisedly split out the clothing texture region. Secondly, we improve CycleGAN to interchange the texture on specific target regions of the two images. Specifically, considering the diversity of fashion texture, we construct a multiscale module (Drf-module) with dynamic perceptual fields and design an image gradient-based texture structure loss (GTS) to perform texture transfer. Qualitative and quantitative experimental results show the effectiveness of our algorithm for clothing-specific region style transfer.
Backdoor Poisoning of Encrypted Traffic Classifiers

Significant recent research has focused on applying deep neural network models to the problem of network traffic classification. At the same time, much has been written about the vulnerability of deep neural networks to adversarial inputs, both during training and inference. In this work, we consider launching backdoor poisoning attacks against an encrypted network traffic classifier. We consider attacks based on padding network packets, which has the benefit of preserving the functionality of the network traffic. In particular, we consider a handcrafted attack, as well as an optimized attack leveraging universal adversarial perturbations. We find that poisoning attacks can be extremely successful if the adversary has the ability to modify both the labels and the data (dirty label attacks) and somewhat successful, depending on the attack strength and the target class, if the adversary perturbs only the data (clean label attacks).
Component-Wise Natural Gradient Descent - An Efficient Neural Network Optimization

Natural Gradient Descent (NGD) is a second-order neural network training that preconditions the gradient descent with the inverse of the Fisher Information Matrix (FIM). Although NGD provides an efficient preconditioner, it is not practicable due to the expensive computation required when inverting the FIM. This paper proposes a new NGD variant algorithm named Component-Wise Natural Gradient Descent (CW-NGD). CW-NGD is composed of 2 steps. Similar to several existing works, the first step is to consider the FIM matrix as a block-diagonal matrix whose diagonal blocks correspond to the FIM of each layer's weights. In the second step, unique to CW-NGD, we analyze the layer's structure and further decompose the layer's FIM into smaller segments whose derivatives are approximately independent. As a result, individual layers' FIMs are approximated in a block-diagonal form that trivially supports the inversion. The segment decomposition strategy is varied by layer structure. Specifically, we analyze the dense and convolutional layers and design their decomposition strategies appropriately. In an experiment of training a network containing these 2 types of layers, we empirically prove that CW-NGD requires fewer iterations to converge compared to the state-of-the-art first-order and second-order methods.
COVID-19 and Haters — A User Model Perspective

In this study, we present an in-depth analysis of users’ propensity toward negative and hateful behavior during the COVID-19 pandemic. We analyze a large dataset extracted from Twitter from the months of January 2020 up until June 2020. The dataset includes 2,470,888 tweets from 3,269 users who are active over a period of six months. We model users’ propensity toward hateful content over time by leveraging Random Forest regressor model and Long Short-Term Memory (LSTM) based many-to-one and Sequence2Sequence models for both short and long-term predictions. Our models leverage a set of features for each user, including the user’s psychological traits. We also study the impact of external triggers, such as COVID-related news concurrent with the users’ activities. To encode popular news, we propose using encoder states of a Sequence2Sequence model as features for a Tree-based regressor. The regressor, when combined with the vectorized news, results in an accurate prediction of tweeter’s hateful behavior in the short (decoder size of four weeks) and long term (decoder size of 10 weeks) with a total training data of 15 weeks x 3269 users. We also show that our model accurately profiles selected groups of users, as they are defined by specific psychological traits.