Reviewerly/exHarmony huggingface dataset:
Across Scales and Across Dimensions: Temporal Super-Resolution Using Deep Internal Learning

When a very fast dynamic event is recorded with a low-framerate camera, the resulting video suffers from severe motion blur (due to exposure time) and motion aliasing (due to low sampling rate in time). True Temporal Super-Resolution (TSR) is more than just Temporal-Interpolation (increasing framerate). It can also recover new high temporal frequencies beyond the temporal Nyquist limit of the input video, thus resolving both motion-blur and motion-aliasing – effects that temporal frame interpolation (as sophisticated as it may be) cannot undo. In this paper we propose a “Deep Internal Learning” approach for true TSR. We train a video-specific CNN on examples extracted directly from the low-framerate input video. Our method exploits the strong recurrence of small space-time patches inside a single video sequence, both within and across different spatio-temporal scales of the video. We further observe (for the first time) that small space-time patches recur also across-dimensions of the video sequence – i.e., by swapping the spatial and temporal dimensions. In particular, the higher spatial resolution of video frames provides strong examples as to how to increase the temporal resolution of that video. Such internal video-specific examples give rise to strong self-supervision, requiring no data but the input video itself. This results in Zero-Shot Temporal-SR of complex videos, which removes both motion blur and motion aliasing, outperforming previous supervised methods trained on external video datasets.
Recent advances in multimedia security and information hiding

I confirm that my Data Availability Statement (pasted below) complies with the Expects Data Policy
The Psychometrics of Cybersickness in Augmented Reality

Augmented reality (AR) is rapidly being adopted by industry leaders and militaries around the globe. With the Defense Health Agency pushing AR as a solution to the distributed learning problem, along with AR applications being explored within primary care and operational medical settings, it is crucial for these immersive platforms to have a standardized, scientifically based paradigm on which they are designed and used. One area of particular concern is the potential for physiological maladaptation following prolonged AR exposure, which is expected to vary from that associated with virtual reality exposure. Such maladaptation is potentially driven by limitations that exist with regard to the types and extent of perceptual issues characteristic of AR head-worn displays (e.g., mismatches between visually displayed information and other senses, restricted field of view, mismatched interpupillary distance). Associated perceptual limitations can reduce training effectiveness or impose patient and/or trainee safety concerns. Thus, while AR technology has the potential to advance simulation training, there is a need to approach AR-based research—particularly that which relates to long-exposure-duration scenarios—from a bottom-up perspective, where its physiological impact is more fully understood. In the hopes of assisting this process, this study presents a comparison of cybersickness between two common forms of AR displays. Specifically, by comparing the Microsoft HoloLens, a head-worn display that has seen rapid adoption by the scientific community, with an AR Tablet–based platform within the context of long-duration AR training exposure, it will be possible to determine what differences, if any, exist between the two display platforms in terms of their physiological impact as measured via cybersickness severity and symptom profile. Results from this psychometric assessment will be used to evaluate the physiological impact of AR exposure and develop usage protocols to ensure AR is safe and effective to use for military medical training.
Adversarial batch image steganography against CNN-based pooled steganalysis

The application of adversarial embedding in single image steganography exhibits its advantage in resisting convolutional neural network (CNN)-based steganalysis. As an important technique to move the steganography from the laboratory to the real world, batch steganography is developed based on the single image steganography, which uses a series of images as carriers. Furthermore, existing pooled steganalysis also applied CNN architecture for feature extraction, which aims to detect batch steganography. Therefore, it is reasonable and meaningful to introduce adversarial embedding in batch steganography to resist pooled steganalysis. However, as far as we know, there is no work about adversarial batch steganography. Adversarial batch image steganography should be able to resist pooled steganalysis which takes a group of images as a unit, therefore the loss function of the single image steganalyzer can not be directly used for adversarial embedding. In addition, adversarial embedding should be combined with batch strategy. In this paper, we propose a general framework of adversarial embedding for batch steganography, in which a new loss function is designed and the batch strategy is combined with adversarial embedding. By this framework, we can adapt most adversarial embedding algorithms for single image steganography to batch steganography. To verify the efficiency of the proposed framework, we design an algorithm called ADVersarial Image Merging Steganography (ADV-IMS) based on ADVersarial EMBedding (ADV-EMB), and carry out a series corresponding experiments. Experimental results show the proposed method significantly improves the security performance of batch steganography against pooled steganalysis and keeps a high-security level against single image steganalysis.
Special Collection on Open Collaboration Across Geosciences

Join open writing teams to collaborate on commentaries for a special collection describing approaches that embody synthesis, cross-disciplinary integration, and open science across geosciences.
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases

Interpretability or explainability is an emerging research field in NLP.From a user-centric point of view, the goal is to build models that provide proper justification for their decisions, similar to those of humans, by requiring the models to satisfy additional constraints.To this end, we introduce a new application on legal text where, contrary to mainstream literature targeting word-level rationales, we conceive rationales as selected paragraphs in multi-paragraph structured court cases.We also release a new dataset comprising European Court of Human Rights cases, including annotations for paragraph-level rationales.We use this dataset to study the effect of already proposed rationale constraints, i.e., sparsity, continuity, and comprehensiveness, formulated as regularizers.Our findings indicate that some of these constraints are not beneficial in paragraph-level rationale extraction, while others need re-formulation to better handle the multi-label nature of the task we consider.We also introduce a new constraint, singularity, which further improves the quality of rationales, even compared with noisy rationale supervision.Experimental results indicate that the newly introduced task is very challenging and there is a large scope for further research.
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases

Abstract Text-to-SQL is the problem of converting a user question into an SQL query, when the question and database are given. In this article, we present a neural network approach called RYANSQL (Recursively Yielding Annotation Network for SQL) to solve complex Text-to-SQL tasks for cross-domain databases. Statement Position Code (SPC) is defined to transform a nested SQL query into a set of non-nested SELECT statements; a sketch-based slot filling approach is proposed to synthesize each SELECT statement for its corresponding SPC. Additionally, two input manipulation methods are presented to improve generation performance further. RYANSQL achieved competitive result of 58.2% accuracy on the challenging Spider benchmark. At the time of submission (April 2020), RYANSQL v2, a variant of original RYANSQL, is positioned at 3rd place among all systems and 1st place among the systems not using database content with 60.6% exact matching accuracy. The source code is available at https://github.com/kakaoenterprise/RYANSQL.
A Comparative Study of Machine Learning-Based Methods for Global Horizontal Irradiance Forecasting

The proliferation of photovoltaic (PV) power generation in power distribution grids induces increasing safety and service quality concerns for grid operators. The inherent variability, essentially due to meteorological conditions, of PV power generation affects the power grid reliability. In order to develop efficient monitoring and control schemes for distribution grids, reliable forecasting of the solar resource at several time horizons that are related to regulation, scheduling, dispatching, and unit commitment, is necessary. PV power generation forecasting can result from forecasting global horizontal irradiance (GHI), which is the total amount of shortwave radiation received from above by a surface horizontal to the ground. A comparative study of machine learning methods is given in this paper, with a focus on the most widely used: Gaussian process regression (GPR), support vector regression (SVR), and artificial neural networks (ANN). Two years of GHI data with a time step of 10 min are used to train the models and forecast GHI at varying time horizons, ranging from 10 min to 4 h. Persistence on the clear-sky index, also known as scaled persistence model, is included in this paper as a reference model. Three criteria are used for in-depth performance estimation: normalized root mean square error (nRMSE), dynamic mean absolute error (DMAE) and coverage width-based criterion (CWC). Results confirm that machine learning-based methods outperform the scaled persistence model. The best-performing machine learning-based methods included in this comparative study are the long short-term memory (LSTM) neural network and the GPR model using a rational quadratic kernel with automatic relevance determination.
LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching

Chinese short text matching is a fundamental task in natural language processing. Existing approaches usually take Chinese characters or words as input tokens. They have two limitations: 1) Some Chinese words are polysemous, and semantic information is not fully utilized. 2) Some models suffer potential issues caused by word segmentation. Here we introduce HowNet as an external knowledge base and propose a Linguistic knowledge Enhanced graph Transformer (LET) to deal with word ambiguity. Additionally, we adopt the word lattice graph as input to maintain multi-granularity information. Our model is also complementary to pre-trained language models. Experimental results on two Chinese datasets show that our models outperform various typical text matching approaches. Ablation study also indicates that both semantic information and multi-granularity information are important for text matching modeling.
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation

Existing work in multilingual pretraining has demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages.However, much of this work only relies on the shared vocabulary and bilingual contexts to encourage the correlation across languages, which is loose and implicit for aligning the contextual representations between languages.In this paper, we plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages.It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language.More importantly, when fine-tuning on downstream tasks, the cross-attention module can be plugged in or out on-demand, thus naturally benefiting a wider range of cross-lingual tasks, from language understanding to generation.As a result, the proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark, covering text classification, sequence labeling, question answering, and sentence retrieval.For cross-lingual generation tasks, it also outperforms all existing cross-lingual models and state-of-theart Transformer variants on WMT14 Englishto-German and English-to-French translation datasets, with gains of up to 1∼2 BLEU. 1
A Deep Learning Approach for Recognizing Activity of Daily Living (ADL) for Senior Care: Exploiting Interaction Dependency and Temporal Patterns

Ensuring the health and safety of senior citizens who live alone is a growing societal concern. The Activity of Daily Living (ADL) approach is a common means to monitor disease progression and the ability of these individuals to care for themselves. However, the prevailing sensor-based ADL monitoring systems primarily rely on wearable motion sensors, capture insufficient information for accurate ADL recognition, and do not provide a comprehensive understanding of ADLs at different granularities. Current healthcare IS and mobile analytics research focuses on studying the system, device, and provided services, and is in need of an end-to-end solution to comprehensively recognize ADLs based on mobile sensor data. This study adopts the design science paradigm and employs advanced deep learning algorithms to develop a novel hierarchical, multiphase ADL recognition framework to model ADLs at different granularities. We propose a novel 2D interaction kernel for convolutional neural networks to leverage interactions between human and object motion sensors. We rigorously evaluate each proposed module and the entire framework against state-of-the-art benchmarks (e.g., support vector machines, DeepConvLSTM, hidden Markov models, and topic-modeling-based ADLR) on two real-life motion sensor datasets that consist of ADLs at varying granularities: Opportunity and INTER. Results and a case study demonstrate that our framework can recognize ADLs at different levels more accurately. We discuss how stakeholders can further benefit from our proposed framework. Beyond demonstrating practical utility, we discuss contributions to the IS knowledge base for future design science-based cybersecurity, healthcare, and mobile analytics applications.
Efficient parallelization of tensor network contraction for simulating quantum computation

We develop an algorithmic framework for contracting tensor networks and demonstrate its power by classically simulating quantum computation of sizes previously deemed out of reach. Our main contribution, index slicing, is a method that efficiently parallelizes the contraction by breaking it down into much smaller and identically structured subtasks, which can then be executed in parallel without dependencies. We benchmark our algorithm on a class of random quantum circuits, achieving greater than 105 times acceleration over the original estimate of the simulation cost. We then demonstrate applications of the simulation framework for aiding the development of quantum algorithms and quantum error correction. As tensor networks are widely used in computational science, our simulation framework may find further applications.
Learning With Privileged Multimodal Knowledge for Unimodal Segmentation

Multimodal learning usually requires a complete set of modalities during inference to maintain performance. Although training data can be well-prepared with high-quality multiple modalities, in many cases of clinical practice, only one modality can be acquired and important clinical evaluations have to be made based on the limited single modality information. In this work, we propose a privileged knowledge learning framework with the ‘Teacher-Student’ architecture, in which the complete multimodal knowledge that is only available in the training data (called privileged information) is transferred from a multimodal teacher network to a unimodal student network, via both a pixel-level and an image-level distillation scheme. Specifically, for the pixel-level distillation, we introduce a regularized knowledge distillation loss which encourages the student to mimic the teacher’s softened outputs in a pixel-wise manner and incorporates a regularization factor to reduce the effect of incorrect predictions from the teacher. For the image-level distillation, we propose a contrastive knowledge distillation loss which encodes image-level structured information to enrich the knowledge encoding in combination with the pixel-level distillation. We extensively evaluate our method on two different multi-class segmentation tasks, i.e., cardiac substructure segmentation and brain tumor segmentation. Experimental results on both tasks demonstrate that our privileged knowledge learning is effective in improving unimodal segmentation and outperforms previous methods.
Hybrid quantum classical graph neural networks for particle track reconstruction

Abstract The Large Hadron Collider (LHC) at the European Organisation for Nuclear Research (CERN) will be upgraded to further increase the instantaneous rate of particle collisions (luminosity) and become the High Luminosity LHC (HL-LHC). This increase in luminosity will significantly increase the number of particles interacting with the detector. The interaction of particles with a detector is referred to as “hit”. The HL-LHC will yield many more detector hits, which will pose a combinatorial challenge by using reconstruction algorithms to determine particle trajectories from those hits. This work explores the possibility of converting a novel graph neural network model, that can optimally take into account the sparse nature of the tracking detector data and their complex geometry, to a hybrid quantum-classical graph neural network that benefits from using variational quantum layers. We show that this hybrid model can perform similar to the classical approach. Also, we explore parametrized quantum circuits (PQC) with different expressibility and entangling capacities, and compare their training performance in order to quantify the expected benefits. These results can be used to build a future road map to further develop circuit-based hybrid quantum-classical graph neural networks.
A quantization assisted U-Net study with ICA and deep features fusion for breast cancer identification using ultrasonic data

Breast cancer is one of the leading causes of death in women worldwide-the rapid increase in breast cancer has brought about more accessible diagnosis resources. The ultrasonic breast cancer modality for diagnosis is relatively cost-effective and valuable. Lesion isolation in ultrasonic images is a challenging task due to its robustness and intensity similarity. Accurate detection of breast lesions using ultrasonic breast cancer images can reduce death rates. In this research, a quantization-assisted U-Net approach for segmentation of breast lesions is proposed. It contains two step for segmentation: (1) U-Net and (2) quantization. The quantization assists to U-Net-based segmentation in order to isolate exact lesion areas from sonography images. The Independent Component Analysis (ICA) method then uses the isolated lesions to extract features and are then fused with deep automatic features. Public ultrasonic-modality-based datasets such as the Breast Ultrasound Images Dataset (BUSI) and the Open Access Database of Raw Ultrasonic Signals (OASBUD) are used for evaluation comparison. The OASBUD data extracted the same features. However, classification was done after feature regularization using the lasso method. The obtained results allow us to propose a computer-aided design (CAD) system for breast cancer identification using ultrasonic modalities.
Virtual reality: a powerful technology to provide novel insight into treatment mechanisms of addiction

Due to its high ecological validity, virtual reality (VR) technology has emerged as a powerful tool for mental health research. Despite the wide use of VR simulations in research on mental illnesses, the study of addictive processes through the use of VR environments is still at its dawn. In a systematic literature search, we identified 38 reports of research projects using highly immersive head-mounted displays, goggles, or CAVE technologies to provide insight into treatment mechanisms of addictive behaviors. So far, VR research has mainly addressed the roles of craving, psychophysiology, affective states, cognition, and brain activity in addiction. The computer-generated VR environments offer very realistic, dynamic, interactive, and complex real-life simulations requesting active participation. They create a high sense of immersion in users by combining stereoscopic three-dimensional visual, auditory, olfactory, and tactile perceptions, tracking systems responding to user movements, and social interactions. VR is an emerging tool to study how proximal multi-sensorial cues, contextual environmental cues, as well as their interaction (complex cues) modulate addictive behaviors. VR allows for experimental designs under highly standardized, strictly controlled, predictable, and repeatable conditions. Moreover, VR simulations can be personalized. They are currently refined for psychotherapeutic interventions. Embodiment, eye-tracking, and neurobiological factors represent novel future directions. The progress of VR applications has bred auspicious ways to advance the understanding of treatment mechanisms underlying addictions, which researchers have only recently begun to exploit. VR methods promise to yield significant achievements to the addiction field. These are necessary to develop more efficacious and efficient preventive and therapeutic strategies.
Generating De-identification facial images based on the attention models and adversarial examples

In response to the problem that facial features are likely to cause privacy leakage and identity theft of the user, this paper presents a method for generating De-identification facial images based on attention models and adversarial examples. The method has two training stages. In Stage 1, target classification networks are used as an attention module to extract Refined Features of a facial image to perform feature fusion and obtain the face feature fusion matrix. In Stage 2, the face feature fusion matrix is used as an initial data distribution of the generator in the adversarial generation network and adds pixel-level constraint loss functions and loss functions for cutting adversarial example perturbations. The purpose of this is for ensuring the generated private image of the face is usable and add attention modules to discriminator of generative adversarial network. Next, the method extracts the feature matrix of the de-identity face image and trains it to mimic the face feature fusion matrix to increase the migration of the adversarial response in the de-identity face image. Experimental results have shown that the performance of the proposed method is superior to or approach state-of-the-art methods in terms of image quality and robustness.