Reviewerly/exHarmony huggingface dataset:
Virtual Museums as a New Type of Cyber-Physical-Social System

Museums are institutions that primarily care for cultural heritage exhibition, preservation and conservation of historical artifacts. However, simply displaying artifacts and provide complex information to describe them is simply not sufficient to effectively engage museum visitors. To improve visitors engagement and their overall museum experience, the use of technology utilized by museums, introducing the concept of Virtual Museums. This paper discusses the use of Virtual Reality through the use of smart phone devices as a mean of a Cyber-Physical-Social system to support, improve and enhance the visitors’ experience. The RoboSHU prototype, its current development stage and future work are presented, together with the future research directions of the research team.
Image denoising using 2-D wavelet algorithm for Gaussian-corrupted confocal microscopy images

Confocal laser scanning microscopy (CLSM) imaging is a non-invasive optical imaging technique for the examination of the living tissues. CLSM inherently enables in-depth sectioning (z-slices) of the focused specimen. Z-slices of the targeted tissue are gathered by adjusting the focal point on the z-axis into the tissue. Unfortunately, these images can get corrupted with noise of different levels caused by out-of-focus light originating from above and below the focal plane. This study proposes a reliable method to indicate and eliminate the additive white Gaussian noise (AWGN) present in real CLSM images of skin tissue. In this work, a denoising algorithm using discrete wavelet transform (DWT) is developed in order to remove the noise by preserving the energy conservation. The effect and performance of different wavelet thresholding algorithms are compared and studied along with different tuning parameters. The selection of components employed in the algorithm affects the noise reduction performance therefore, a systematic approach is presented to obtain and utilize the best combination of these parameter values. Analysis of variance (ANOVA) is exploited to inspect the main and the interaction effects of treated parameters. Computational results show the effectiveness of the methodical tuning approach to CLSM image denoising.
Stochastic

Hareesh Bahuleyan, Lili Mou, Hao Zhou, Olga Vechtomova. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Progressive Representation Adaptation for Weakly Supervised Object Localization

We address the problem of weakly supervised object localization where only image-level annotations are available for training object detectors. Numerous methods have been proposed to tackle this problem through mining object proposals. However, a substantial amount of noise in object proposals causes ambiguities for learning discriminative object models. Such approaches are sensitive to model initialization and often converge to undesirable local minimum solutions. In this paper, we propose to overcome these drawbacks by progressive representation adaptation with two main steps: 1) classification adaptation and 2) detection adaptation. In classification adaptation, we transfer a pre-trained network to a multi-label classification task for recognizing the presence of a certain object in an image. Through the classification adaptation step, the network learns discriminative representations that are specific to object categories of interest. In detection adaptation, we mine class-specific object proposals by exploiting two scoring strategies based on the adapted classification network. Class-specific proposal mining helps remove substantial noise from the background clutter and potential confusion from similar objects. We further refine these proposals using multiple instance learning and segmentation cues. Using these refined object bounding boxes, we fine-tune all the layer of the classification network and obtain a fully adapted detection network. We present detailed experimental validation on the PASCAL VOC and ILSVRC datasets. Experimental results demonstrate that our progressive representation adaptation algorithm performs favorably against the state-of-the-art methods.
Evaluating Text

Guy Tevet, Gavriel Habib, Vered Shwartz, Jonathan Berant. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
A tractable multi-partitions clustering

In the framework of model-based clustering, a model allowing several latent class variables is proposed. This model assumes that the distribution of the observed data can be factorized into several independent blocks of variables. Each block is assumed to follow a latent class model (i.e., mixture with conditional independence assumption). The proposed model includes variable selection, as a special case, and is able to cope with the mixed-data setting. The simplicity of the model allows to estimate the repartition of the variables into blocks and the mixture parameters simultaneously, thus avoiding to run EM algorithms for each possible repartition of variables into blocks. For the proposed method, a model is defined by the number of blocks, the number of clusters inside each block and the repartition of variables into block. Model selection can be done with two information criteria, the BIC and the MICL, for which an efficient optimization is proposed. The performances of the model are investigated on simulated and real data. It is shown that the proposed method gives a rich interpretation of the data set at hand (i.e., analysis of the repartition of the variables into blocks and analysis of the clusters produced by each block of variables).
Open Information Extraction from Question-Answer Pairs

Nikita Bhutani, Yoshihiko Suhara, Wang-Chiew Tan, Alon Halevy, H. V. Jagadish. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Entity Recognition at First Sight:

Nora Hollenstein, Ce Zhang. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Cross-lingual Annotation Projection Is Effective for Neural Part-of-Speech Tagging

We tackle the important task of part-of-speech tagging using a neural model in the zero-resource scenario, where we have no access to gold-standard POS training data. We compare this scenario with the low-resource scenario, where we have access to a small amount of gold-standard POS training data. Our experiments focus on Ukrainian as a representative of under-resourced languages. Russian is highly related to Ukrainian, so we exploit gold-standard Russian POS tags. We consider four techniques to perform Ukrainian POS tagging: zero-shot tagging and cross-lingual annotation projection (for the zero-resource scenario), and compare these with self-training and multilingual learning (for the low-resource scenario). We find that cross-lingual annotation projection works particularly well in the zero-resource scenario.
On the limits of automatic speaker verification: Explaining degraded recognizer scores through acoustic changes resulting from voice disguise

In speaker verification research, objective performance benchmarking of listeners and automatic speaker verification (ASV) systems are of key importance in understanding the limits of speaker recognition. While the adoption of common data and metrics has been instrumental to progress in ASV, there are two major shortcomings. First, the utterances lack intentional voice changes imposed by the speaker. Second, the standard evaluation metrics focus on average performance across all speakers and trials. As a result, a knowledge gap remains in how the acoustic changes impact recognition performance at the level of individual speakers. This paper addresses the limits of speaker recognition in ASV systems under voice disguise using a linear mixed effects model to analyze the impact of change in long-term statistics of selected features (formants F1–F4, the bandwidths B1–B4, F0, and speaking rate) to ASV log-likelihood ratio (LLR) score. The correlations between the proposed predictive model and the LLR scores are 0.72 for females and 0.81 for male speakers. As a whole, the difference in long-term F0 between enrollment and test utterances was found to be the individually most detrimental factor, even if the ASV system uses only spectral, rather than prosodic, features.
Deep and Sparse features For Anomaly Detection and Localization in video

Detection of abnormality in video crowded scenes is highly challenging in computer vision over the past decades. We propose a method for the detection of abnormal events in surveillance video sequences. In this method, we combine the spatial-temporal convolution neural network (CNN) with handcrafted feature sets such as Histograms of Optical Flow (HOF) and Histogram of Oriented Gradients (HOG) for anomaly detection in contiguous video frames. Handcrafted features learned sparse by using our novel method which we named it Iterative Weighted nonNegative Matrix Factorization (IW-NMF) is based on sparse NMF. These feature extracted from active volume cells that included moving pixels to reduce computational costs. The architecture of the CNN model allows us to extract spatial-temporal features and using handcrafted features to increase the accuracy of detection and to ensure robustness against local noise. we evaluate our framework on popular datasets that contain abnormal activites. our method results are better than most of the other method. and achieves a very competitive detection performance compared to state-of-the-art methods.
Developing an Accessible Evaluation Method of VR Cybersickness

Despite its growing popularity, Virtual Reality (VR) has yet to make a significant impact in conventional education due to its high cost, unconvincing learning data, complexity of the technologies and, persistently, cybersickness. To alleviate this dilemma, it is necessary to develop a straightforward and reliable measurement of cybersickness for VR application developers and mainstream educators. The Empatica E4 wearable device and its eco-system were utilized to record Heart Rate Variability (HRV) and Electrodermal Activity (EDA) during customized computer-based and VR tasks with 16 participants. The metrics of S, NNMean, SDNN, RMSSD, and Poincaré Plot in HRV data and SCR width in EDA data were found to be potential indicators of cybersickness. Further research aims to determine a specific cybersickness index.
VR-based Batter Training System with Motion Sensing and Performance Visualization

This paper aims to establish a novel VR system for evaluating the performance of baseball batters. Existing VR systems for sports have been utilized as a tool for image training. In order to move such VR systems to the next stage, we introduce functions that sense the users' reaction to the VR stimulus. Our VR system has three features; (a) it synthesizes highly realistic VR video from the data captured in actual games, (b) it estimates the reaction of the user to the VR stimulus by capturing the 3D positions of full body parts, and (c) it consists of off-the-shelf devices and is easy to use. Our demonstration provides users with a chance to experience our VR system and give them some quick feedback by visualizing the estimated 3D positions of their body parts.
Shadowless Projector: Suppressing Shadows in Projection Mapping with Micro Mirror Array Plate

Shadowless Projector is projection mapping system in which a shadow (more specifically, umbra) does not suffer the projected result. A typical shadow removal technique used a multiple overlapping projection system. In this paper, we propose a shadow-less projection method with single projector. Inspired by a surgical light system that does not cast shadows on patients' bodies in clinical practice, we apply a special optical system that consists of methodically positioned vertical mirrors. This optical system works as a large aperture lens, it is impossible to block all projected ray by a small object such as a hand. Consequently, only penumbra is caused, which leads to a shadow-less projection.
Multi-resolution Multi-task Gaussian Processes

We consider evidence integration from potentially dependent observation processes under varying spatio-temporal sampling resolutions and noise levels. We offer a multi-resolution multi-task (MRGP) framework that allows for both inter-task and intra-task multi-resolution and multi-fidelity. We develop shallow Gaussian Process (GP) mixtures that approximate the difficult to estimate joint likelihood with a composite one and deep GP constructions that naturally handle biases. In doing so, we generalize existing approaches and offer information-theoretic corrections and efficient variational approximations. We demonstrate the competitiveness of MRGPs on synthetic settings and on the challenging problem of hyper-local estimation of air pollution levels across London from multiple sensing modalities operating at disparate spatio-temporal resolutions.
Multiview Aggregation for Learning Category-Specific Shape Reconstruction

We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances. Most approaches for multiview shape reconstruction operate on sparse shape representations, or assume a fixed number of views. We present a method that can estimate dense 3D shape, and aggregate shape across multiple and varying number of input views. Given a single input view of an object instance, we propose a representation that encodes the dense shape of the visible object surface as well as the surface behind line of sight occluded by the visible surface. When multiple input views are available, the shape representation is designed to be aggregated into a single 3D shape using an inexpensive union operation. We train a 2D CNN to learn to predict this representation from a variable number of views (1 or more). We further aggregate multiview information by using permutation equivariant layers that promote order-agnostic view information exchange at the feature level. Experiments show that our approach is able to produce dense 3D reconstructions of objects that improve in quality as more views are added.
Combining Safety and Security in Autonomous Cars Using Blockchain Technologies

Modern cars increasingly deploy complex software systems consisting of millions of lines of code that may be subject to cyber attacks. An infamous example is the Jeep hack which allowed an attacker to remotely control the car engine by just exploiting a software bug in the infotainment system. The digitalization and connectivity of modern cars demands a rethinking of car safety as security breaches now affect the driver’s safety. To address the new threat landscape, we develop a novel concept that simultaneously addresses both car safety and security based on the arising blockchain technology, which we mainly exploit to ensure integrity. Previous related work exploited the blockchain for the purpose of forensics, where vehicle data is stored on an externally shared ledger that is accessible by authorized third parties. However, those approaches cannot ensure integrity of information used by the vehicle’s components. In contrast, we propose a blockchain-based architecture based on a shared ledger inside the car, where each ECU can act as a miner and shares its information with other ECUs. The architecture does not only improve the integrity of information for forensics. Some algorithms, e.g. the recognition of dangerous situations, are adaptive and can be improved using for example sensor data. Using our architecture, we ensure that those algorithms only take verified and correct information as input.
Self-Routing Capsule Networks

Capsule networks have recently gained a great deal of interest as a new architecture of neural networks that can be more robust to input perturbations than similar-sized CNNs. Capsule networks have two major distinctions from the conventional CNNs: (i) each layer consists of a set of capsules that specialize in disjoint regions of the feature space and (ii) the routing-by-agreement coordinates connections between adjacent capsule layers. Although the routing-by-agreement is capable of filtering out noisy predictions of capsules by dynamically adjusting their influences, its unsupervised clustering nature causes two weaknesses: (i) high computational complexity and (ii) cluster assumption that may not hold in presence of heavy input noise. In this work, we propose a novel and surprisingly simple routing strategy called self-routing where each capsule is routed independently by its subordinate routing network. Therefore, the agreement between capsules is not required anymore but both poses and activations of upper-level capsules are obtained in a way similar to Mixture-of-Experts. Our experiments on CIFAR-10, SVHN and SmallNORB show that the self-routing performs more robustly against white-box adversarial attacks and affine transformations, requiring less computation.
Motion Planning for Collision Mitigation via FEM–Based Crash Severity Maps

This paper presents a novel collision mitigation method for traffic scenarios in which accidents are inevitable. When a predicted accident is imminent, the suggested software agent changes the trajectory of the ego vehicle to optimize the predicted crash configuration. The target function results from the degree of crash severity, based on the deformation of the safety cage in the occupant area. We have performed numerous crash simulations with detailed FEM models to determine the crash severity for possible crash configurations. Since inevitable crash scenarios are inherently time-critical, the focus is on fast optimization. The proposed heuristic search algorithm computes a collision trajectory from a given initial state by searching for the maneuver leading to the lowest crash severity in the reachable crash configuration set. We rely on a previously developed approach to trajectory modeling that guarantees a minimum execution time for trajectory generation and collision detection and respects the limits of vehicle dynamics. Simulations prove the applicability of the proposed algorithm.
Robust Attribution Regularization

An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions. We take a step towards solving this problem through the lens of axiomatic attribution of neural networks. Our theory is grounded in the recent work, Integrated Gradients (IG) [STY17], in axiomatically attributing a neural network’s output change to its input change. We propose training objectives in classic robust optimization models to achieve robust IG attributions. Our objectives give principled generalizations of previous objectives designed for robust predictions, and they naturally degenerate to classic soft-margin training for one-layer neural networks. We also generalize previous theory and prove that the objectives for different robust optimization models are closely related. Experiments demonstrate the effectiveness of our method, and also point to intriguing problems which hint at the need for better optimization techniques or better neural network architectures for robust attribution training.
Beating SGD Saturation with Tail-Averaging and Minibatching

While stochastic gradient descent (SGD) is one of the major workhorses in machine learning, the learning properties of many practically used variants are still poorly understood. In this paper, we consider least squares learning in a nonparametric setting and contribute to filling this gap by focusing on the effect and interplay of multiple passes, mini-batching and averaging, in particular tail averaging. Our results show how these different variants of SGD can be combined to achieve optimal learning rates, also providing practical insights. A novel key result is that tail averaging allows faster convergence rates than uniform averaging in the nonparametric setting. Further, we show that a combination of tail-averaging and minibatching allows more aggressive step-size choices than using any one of said components.
Learning to Learn By Self-Critique

In few-shot learning, a machine learning system is required to learn from a small set of labelled examples of a specific task, such that it can achieve strong generalization on new unlabelled examples of the same task. Given the limited availability of labelled examples in such tasks, we need to make use of all the information we can. For this reason we propose the use of transductive meta-learning for few shot settings to obtain state-of-the-art few-shot learning. Usually a model learns task-specific information from a small training-set (the \emph{support-set}) and subsequently produces predictions on a small unlabelled validation set (\emph{target-set}). The target-set contains additional task-specific information which is not utilized by existing few-shot learning methods. This is a challenge requiring approaches beyond the current methods as at inference time, the target-set contains only input data-points, and so discriminative-based learning cannot be used. In this paper, we propose a framework called \emph{Self-Critique and Adapt} or SCA. This approach learns to learn a label-free loss function, parameterized as a neural network, which leverages target-set information. A base-model learns on a support-set using existing methods (e.g. stochastic gradient descent combined with the cross-entropy loss), and then is updated for the incoming target-task using a new learned loss function (i.e. the meta-learned label-free loss). This unsupervised loss function is optimized such that the learnt model achieves higher generalization performance. Experiments demonstrate that SCA offers substantially higher and state-of-the-art generalization performance compared to baselines which only adapt on the support-set.
LSTM-MA: A LSTM Method with Multi-Modality and Adjacency Constraint for Brain Image Segmentation

MR brain tissue segmentation is a significant problem in biomedical image processing. Inhomogeneous intensity and image noise influence the segmentation accuracy. In this paper, we propose a LSTM method with multi-modality and adjacency constraint for brain image segmentation, named LSTM-MA. Two feature sequence generation ways in our method are used, i.e., features with pixel-wise and superpixel-wise adjacency constraint. The LSTM model classifies the generated features into semantic labels to form the segmentation result. The evaluation experiments on BrainWeb and MRBrainS demonstrate that the proposed LSTM-MA with pixel-wise adjacency constraint achieves promising segmentation results, while LSTM-MA with superpixel-wise adjacency constraint shows its computational efficiency as well as robustness to noise.
Learning about an exponential amount of conditional distributions

We introduce the Neural Conditioner (NC), a self-supervised machine able to learn about all the conditional distributions of a random vector X. The NC is a function NC(x⋅a,a,r) that leverages adversarial training to match each conditional distribution P(Xr|Xa=xa). After training, the NC generalizes to sample from conditional distributions never seen, including the joint distribution. The NC is also able to auto-encode examples, providing data representations useful for downstream classification tasks. In sum, the NC integrates different self-supervised tasks (each being the estimation of a conditional distribution) and levels of supervision (partially observed data) seamlessly into a single learning experience.