Reviewerly/exHarmony huggingface dataset:
Toward a non-binary sense of mobility: insights from self-presentation in Instagram photography during COVID-19 pandemic

In this paper, I investigate how Instagrammers re-construct themselves as agentive and digitally mobile despite their physical immobility during the early COVID-19 pandemic. By examining four types of post that emerged or became popular during this period, I delineate how Instagrammers take subject positions to negotiate their (im)mobile selfhood in the online-offline nexus. I contend that as the Instagrammers negotiate their own invisibility and sociality during the lockdown, their physical bodies are reconfigured digitally as part of their multimodal self-presentation, troubling the traditional (im)mobile divide. While this study illuminates the democratic potential of Instagram, it also contemplates the extent of an individual’s agency on a global scale.
On the connection between microscopic description and memory effects in open quantum system dynamics

The exchange of information between an open quantum system and its environment allows us to discriminate among different kinds of dynamics, in particular detecting memory effects to characterize non-Markovianity. Here, we investigate the role played by the system-environment correlations and the environmental evolution in the flow of information. First, we derive general conditions ensuring that two generalized dephasing microscopic models of the global system-environment evolution result exactly in the same open-system dynamics, for any initial state of the system. Then, we use the trace distance to quantify the distinct contributions to the information inside and outside the open system in the two models. Our analysis clarifies how the interplay between system-environment correlations and environmental-state distinguishability can lead to the same information flow from and toward the open system, despite significant qualitative and quantitative differences at the level of the global evolution.
Statistical learning guarantees for compressive clustering and compressive mixture modeling

We provide statistical learning guarantees for two unsupervised learning tasks in the context of compressive statistical learning, a general framework for resource-efficient large-scale learning that we introduced in a companion paper.The principle of compressive statistical learning is to compress a training collection, in one pass, into a low-dimensional sketch (a vector of random empirical generalized moments) that captures the information relevant to the considered learning task. We explicitly describe and analyze random feature functions which empirical averages preserve the needed information for compressive clustering and compressive Gaussian mixture modeling with fixed known variance, and establish sufficient sketch sizes given the problem dimensions.
Attention-based clinical note summarization

In recent years, the trend of deploying digital systems in numerous industries has hiked. The health sector has observed an extensive adoption of digital systems and services that generate significant medical records. Electronic health records contain valuable information for prospective and retrospective analysis that is often not entirely exploited because of the complicated dense information storage. The crude purpose of condensing health records is to select the information that holds most characteristics of the original documents based on a reported disease. These summaries may boost diagnosis and save a doctor's time during a saturated workload situation like the COVID-19 pandemic. In this paper, we are applying a multi-head attention-based mechanism to perform extractive summarization of meaningful phrases on clinical notes. Our method finds major sentences for a summary by correlating tokens, segments, and positional embeddings of sentences in a clinical note. The model outputs attention scores that are statistically transformed to extract critical phrases for visualization on the heat-mapping tool and for human use.
Gradient-Based Feature Extraction From Raw Bayer Pattern Images

In this paper, the impact of demosaicing on gradient extraction is studied and a gradient-based feature extraction pipeline based on raw Bayer pattern images is proposed. It is shown both theoretically and experimentally that the Bayer pattern images are applicable to the central difference gradient-based feature extraction algorithms with negligible performance degradation, as long as the arrangement of color filter array (CFA) patterns matches the gradient operators. The color difference constancy assumption, which is widely used in various demosaicing algorithms, is applied in the proposed Bayer pattern image-based gradient extraction pipeline. Experimental results show that the gradients extracted from Bayer pattern images are robust enough to be used in histogram of oriented gradients (HOG)-based pedestrian detection algorithms and shift-invariant feature transform (SIFT)-based matching algorithms. By skipping most of the steps in the image signal processing (ISP) pipeline, the computational complexity and power consumption of a computer vision system can be reduced significantly.
Minority oversampling for imbalanced time series classification

Many vital real-world applications involve time-series data with skewed distribution. Compared to traditional imbalanced learning problems, the classification of imbalanced time-series data is more challenging due to the high dimensionality and high inter-variable correlation. This paper proposes a structure-preserving Oversampling method to resolve the High-dimensional Imbalanced Time-series classification (OHIT). OHIT leverages a density-ratio-based shared nearest neighbor clustering algorithm to capture the modes of minority class in high-dimensional space. It for each mode applies the shrinkage technique of large-dimensional covariance matrix to obtain an accurate and reliable covariance structure. The structure-preserving synthetic samples are eventually generated based on the multivariate Gaussian distribution with the estimated covariance matrix. In addition, to further promote the performance of classifying imbalanced time-series data, we integrate OHIT into boosting framework to obtain a new ensemble algorithm OHITBoost. Extensive experiments on several publicly available time-series datasets (including unimodal and multimodal) demonstrate the effectiveness of OHIT and OHITBoost in terms of F1, G-mean, and AUC.
A semantic similarity computation method for virtual resources in cloud manufacturing environment based on information content

To ensure the effective and reasonable classification of virtual resources and provide support for the search and matching of required resources that accomplish tasks in cloud manufacturing (CMfg) environments, the semantic similarity computation of virtual resources needs to be promoted for that it is difficult to ensure that a large number of effective features participate in the calculation of semantic similarity and it is difficult to practically apply to efficient classification. However, to promote semantic similarity calculations still have problems such as difficulty in unifying the virtual resource pool and lagging in updating semantic depth calculations. In this paper, we propose a creative approach that can compute the semantic similarity of virtual resources described by video semantic description (VSD) based on information content (IC). In this approach, first, a multi-level progression semantics description framework for virtual resources is proposed with the help of VSD that can provide a new solution because of its explicit description and dynamic update to obtain real and reliable data; second, an event cloud (EC) that serves as the computing scope for semantics similarity calculation of virtual resources is constructed to settle the problem of substantial data growth caused by real-time resource updates on top of the resource manufacturing processes; third, based on an effective solution of corpus dependency and data sparseness, an extended semantic computing model is developed to calculate quantificationally the semantic similarities of virtual resources based on semantic relationships. The results show that compared with classifying virtual resources qualitatively, the semantic similarity computing method that starts from the manufacturing result and considers the comprehensive relationship is more reliable and effective.
Nudge me: tailoring text messages for prescription adherence through N-of-1 interviews

Almost 50% of patients with cardiovascular diseases face challenges in taking medications and increased morbidity and mortality. Text messaging may impact medication refill behavior and can be delivered at scale to patients by texting mobile phones. To obtain feedback from persons with chronic conditions on the design of interactive text messages and determine language of message for making messages that can motivate patients to refill medications on time. We purposively sampled 35 English and Spanish speaking patients with at least one chronic condition from three large healthcare delivery systems to participate in N-of-1 video-based synchronous interviews. Research assistants shared ideas for theory-informed text messages with content intended to persuade patients to refill their medication. We transcribed recorded interviews and conducted a content analysis to identify strategies to employ generating a dynamic interactive text message library intended to increase medication refill. Those interviewed were of diverse age and race/ethnicity and typical of persons with multiple chronic conditions. Several participants emphasized that personally tailored and positively framed messages would be more persuasive than generic and/or negative messages. Some patients appreciated humor and messages that could evoke a sense of social support from their providers and rejected the use of emojis. Messages to remind patients to refill medications may facilitate improvements in adherence, which in turn can improve chronic care. Designing messages that are persuasive and can prompt action is feasible and should be considered given the ease with which such messages can be delivered automatically at scale.
Transformations of multilevel coherent states under coherence-preserving operations

Quantum coherence, emerging from the “superposition” of quantum states, is widely used in various information processing tasks. Recently, the resource theory of multilevel quantum coherence is attracting substantial attention. In this paper, we mainly study the transformations of resource pure states via free operations in the theoretical framework for multilevel coherence. We prove that any two multilevel coherent resource pure states can be interconverted with a nonzero probability via a completely positive and trace non-increasing k-coherence-preserving map. Meanwhile, we present the condition of the interconversions of two multilevel coherent resource pure states under k-coherence-preserving operations. In addition, we obtain that in the resource-theoretic framework of multilevel coherence, no resource state is isolated, that is, given a multilevel coherent pure state ∣ψ〉, there exists another multilevel coherent pure state ∣ϕ〉 and a k-coherence-preserving operation Λk, such that Λk(∣ϕ〉) = ∣ψ〉.
Disentangling Semantic-to-Visual Confusion for Zero-Shot Learning

Using generative models to synthesize visual features from semantic distribution is one of the most popular solutions to ZSL image classification in recent years. The triplet loss (TL) is popularly used to generate realistic visual distributions from semantics by automatically searching discriminative representations. However, the traditional TL cannot search reliable unseen disentangled representations due to the unavailability of unseen classes in ZSL. To alleviate this drawback, we propose in this work a multi-modal triplet loss (MMTL) which utilizes multi-modal information to search a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">disentangled</i> representation space. As such, all classes can interplay which can benefit learning disentangled class representations in the searched space. Furthermore, we develop a novel model called Disentangling Class Representation Generative Adversarial Network (DCR-GAN) focusing on exploiting the disentangled representations in training, feature synthesis, and final recognition stages. Benefiting from the disentangled representations, DCR-GAN could fit a more realistic distribution over both seen and unseen features. Extensive experiments show that our proposed model can lead to superior performance to the state-of-the-arts on four benchmark datasets.
hyper-sinh: An accurate and reliable function from shallow to deep learning in TensorFlow and Keras

This paper presents the ‘hyper-sinh’, a variation of the m-arcsinh activation function suit-able for Deep Learning (DL)-based algorithms for supervised learning, including Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), such as the Long Short-Term Memory (LSTM). hyper-sinh, developed in the open-source Python libraries TensorFlow and Keras, is thus described and validated as an accurate and reliable activation function for shallow and deep neural networks. Improvements in accuracy and reliability in image and text classification tasks on six (N=6) medium-to-large open-source benchmark datasets are discussed. Experimental results demonstrate that the overall competitive classification performance of the novel hyper-sinh function on shallow and deep neural networks yielded the highest performance. Furthermore, this activation is evaluated against other gold standard activation functions, demonstrating its overall competitive accuracy and reliability for both image and text classification tasks.
A Two-Level Speaker Identification System via Fusion of Heterogeneous Classifiers and Complementary Feature Cooperation

We present a new architecture to address the challenges of speaker identification that arise in interaction of humans with social robots. Though deep learning systems have led to impressive performance in many speech applications, limited speech data at training stage and short utterances with background noise at test stage present challenges and are still open problems as no optimum solution has been reported to date. The proposed design employs a generative model namely the Gaussian mixture model (GMM) and a discriminative model—support vector machine (SVM) classifiers as well as prosodic features and short-term spectral features to concurrently classify a speaker’s gender and his/her identity. The proposed architecture works in a semi-sequential manner consisting of two stages: the first classifier exploits the prosodic features to determine the speaker’s gender which in turn is used with the short-term spectral features as inputs to the second classifier system in order to identify the speaker. The second classifier system employs two types of short-term spectral features; namely mel-frequency cepstral coefficients (MFCC) and gammatone frequency cepstral coefficients (GFCC) as well as gender information as inputs to two different classifiers (GMM and GMM supervector-based SVM) which in total leads to construction of four classifiers. The outputs from the second stage classifiers; namely GMM-MFCC maximum likelihood classifier (MLC), GMM-GFCC MLC, GMM-MFCC supervector SVM, and GMM-GFCC supervector SVM are fused at score level by the weighted Borda count approach. The weight factors are computed on the fly via Mamdani fuzzy inference system that its inputs are the signal to noise ratio and the length of utterance. Experimental evaluations suggest that the proposed architecture and the fusion framework are promising and can improve the recognition performance of the system in challenging environments where the signal-to-noise ratio is low, and the length of utterance is short; such scenarios often arise in social robot interactions with humans.
Robust hand gesture recognition using multiple shape-oriented visual cues

Abstract Robust vision-based hand pose estimation is highly sought but still remains a challenging task, due to its inherent difficulty partially caused by self-occlusion among hand fingers. In this paper, an innovative framework for real-time static hand gesture recognition is introduced, based on an optimized shape representation build from multiple shape cues. The framework incorporates a specific module for hand pose estimation based on depth map data, where the hand silhouette is first extracted from the extremely detailed and accurate depth map captured by a time-of-flight (ToF) depth sensor. A hybrid multi-modal descriptor that integrates multiple affine-invariant boundary-based and region-based features is created from the hand silhouette to obtain a reliable and representative description of individual gestures. Finally, an ensemble of one-vs.-all support vector machines (SVMs) is independently trained on each of these learned feature representations to perform gesture classification. When evaluated on a publicly available dataset incorporating a relatively large and diverse collection of egocentric hand gestures, the approach yields encouraging results that agree very favorably with those reported in the literature, while maintaining real-time operation.
The Efficient Preparation of Normal Distributions in Quantum Registers

The efficient preparation of input distributions is an important problem in obtaining quantum advantage in a wide range of domains. We propose a novel quantum algorithm for the efficient preparation of arbitrary normal distributions in quantum registers. To the best of our knowledge, our work is the first to leverage the power of Mid-Circuit Measurement and Reuse (MCMR), in a way that is broadly applicable to a range of state-preparation problems. Specifically, our algorithm employs a repeat-until-success scheme, and only requires a constant-bounded number of repetitions in expectation. In the experiments presented, the use of MCMR enables up to a 862.6x reduction in required qubits. Furthermore, the algorithm is provably resistant to both phase-flip and bit-flip errors, leading to a first-of-its-kind empirical demonstration on real quantum hardware, the MCMR-enabled Honeywell System Models H0 and H1-2.
Spotting Deepfakes and Face Manipulations by Fusing Features from Multi-Stream CNNs Models

Deepfake is one of the applications that is deemed harmful. Deepfakes are a sort of image or video manipulation in which a person’s image is changed or swapped with that of another person’s face using artificial neural networks. Deepfake manipulations may be done with a variety of techniques and applications. A quintessential countermeasure against deepfake or face manipulation is deepfake detection method. Most of the existing detection methods perform well under symmetric data distributions, but are still not robust to asymmetric datasets variations and novel deepfake/manipulation types. In this paper, for the identification of fake faces in videos, a new multistream deep learning algorithm is developed, where three streams are merged at the feature level using the fusion layer. After the fusion layer, the fully connected, Softmax, and classification layers are used to classify the data. The pre-trained VGG16 model is adopted for transferred CNN1stream. In transfer learning, the weights of the pre-trained CNN model are further used for training the new classification problem. In the second stream (transferred CNN2), the pre-trained VGG19 model is used. Whereas, in the third stream, the pre-trained ResNet18 model is considered. In this paper, a new large-scale dataset (i.e., World Politicians Deepfake Dataset (WPDD)) is introduced to improve deepfake detection systems. The dataset was created by downloading videos of 20 different politicians from YouTube. Over 320,000 frames were retrieved after dividing the downloaded movie into little sections and extracting the frames. Finally, various manipulations were performed to these frames, resulting in seven separate manipulation classes for men and women. In the experiments, three fake face detection scenarios are investigated. First, fake and real face discrimination is studied. Second, seven face manipulations are performed, including age, beard, face swap, glasses, hair color, hairstyle, smiling, and genuine face discrimination. Third, performance of deepfake detection system under novel type of face manipulation is analyzed. The proposed strategy outperforms the prior existing methods. The calculated performance metrics are over 99%.
MCTSteg: A Monte Carlo Tree Search-Based Reinforcement Learning Framework for Universal Non-Additive Steganography

Recent research has shown that non-additive image steganographic frameworks effectively improve security performance through adjusting distortion distribution. However, as far as we know, all of the existing non-additive proposals are based on handcrafted policies, and can only be applied to a specific image domain, which heavily prevent non-additive steganography from releasing its full potentiality. In this paper, we propose an automatic non-additive steganographic distortion learning framework called MCTSteg to remove the above restrictions. Guided by the reinforcement learning paradigm, we combine Monte Carlo Tree Search (MCTS) and steganalyzer-based environmental model to build MCTSteg. MCTS makes sequential decisions to adjust distortion distribution without human intervention. Our proposed environmental model is used to obtain feedbacks from each decision. Due to its self-learning characteristic and domain-independent reward function, MCTSteg has become the first reported universal non-additive steganographic framework which can work in both spatial and JPEG domains. Extensive experimental results show that MCTSteg can effectively withstand the detection of both hand-crafted feature-based and deep-learning-based steganalyzers. In both spatial and JPEG domains, the security performance of MCTSteg steadily outperforms the state of the art by a clear margin under different scenarios.
LCS graph kernel based on Wasserstein distance in longest common subsequence metric space

For graph learning tasks, many existing methods utilize a message-passing mechanism where vertex features are updated iteratively by aggregation of neighbor information. This strategy provides an efficient means for graph features extraction, but obtained features after many iterations might contain too much information from other vertices, and tend to be similar to each other. This makes their representations less expressive. Learning graphs using paths, on the other hand, can be less adversely affected by this problem because it does not involve all vertex neighbors. However, most of them can only compare paths with the same length, which might engender information loss. To resolve this difficulty, we propose a new Graph Kernel based on a Longest Common Subsequence (LCS) similarity. Moreover, we found that the widely-used R-convolution framework is unsuitable for path-based Graph Kernel because a huge number of comparisons between dissimilar paths might deteriorate graph distances calculation. Therefore, we propose a novel metric space by exploiting the proposed LCS-based similarity, and compute a new Wasserstein-based graph distance in this metric space, which emphasizes more the comparison between similar paths. Furthermore, to reduce the computational cost, we propose an adjacent point merging operation to sparsify point clouds in the metric space.