Reviewerly/exHarmony huggingface dataset:
Cascaded and Recursive ConvNets (CRCNN): An effective and flexible approach for image denoising

Recently, discriminative learning methods have gained substantial interest in solving inverse imaging problems due to their decent performance and fast inferencing capability. Those methods need separate models for specific noise levels, which in turn require multiple models to be trained to denoise an image. However, images exhibit spatial variant noise which limits the applicability of such methods. In addition, the discriminative learning methods introduce artifacts such as blurring, deblocking, and so forth while denoising an image. To address these issues, we propose a cascaded and recursive convolutional neural network (CRCNN) framework which can cope with spatial variant noise and blur artifacts in a single denoising framework. The CRCNN takes into account down-sampled sub-images for fast inferencing along with the noise level map. We adopt the hybrid orthogonal projection and estimation method on the convolutional layers to improve the generalization capability of the network in terms of non-uniform and spatial variant noise levels. In contrast to the existing methods, the CRCNN framework allows both denoising and deblurring of images using a single framework which preserves the fine details in a denoised image. Extensive experiments have been conducted to validate the effectiveness and flexibility of the CRCNN framework on real as well as synthetic noisy images in comparison to the state-of-the-art denoising methods. The results show that the CRCNN performs effectively on both synthetic as well as spatial variant noise-induced images, thus, proving the practicability of the framework.
Edge-Preserving Filtering-Based Dehazing for Remote Sensing Images

Haze in remote sensing images severely degrades image visibility, making it hard to identify different land covers. This work proposes an edge-preserving filtering-based image dehazing method for remote sensing images, mainly consisting of following several steps. First, the original image contaminated with haze is decomposed by a multiscale guided filtering into base layers that contain haze components and detail layers that reflect the spatial details of input. Then, an optimized atmospheric scattering model is performed on the base layers to eliminate haze. Next, adaptive nonlinear mapping is exploited to enhance image details. Finally, the resulting image is reconstructed by combining the dehazed base layers and the refined detail layers. Experiments on several images demonstrate that the proposed method has an outstanding haze removal capability and yields better dehazing performance compared to other dehazing approaches in terms of objective indexes and subjective results.
Enhancing the Scalability of Multi-FPGA Stencil Computations via Highly Optimized HDL Components

Stencil-based algorithms are a relevant class of computational kernels in high-performance systems, as they appear in a plethora of fields, from image processing to seismic simulations, from numerical methods to physical modeling. Among the various incarnations of stencil-based computations, Iterative Stencil Loops (ISLs) and Convolutional Neural Networks (CNNs) represent two well-known examples of kernels belonging to the stencil class. Indeed, ISLs apply the same stencil several times until convergence, while CNN layers leverage stencils to extract features from an image. The computationally intensive essence of ISLs, CNNs, and in general stencil-based workloads, requires solutions able to produce efficient implementations in terms of throughput and power efficiency. In this context, FPGAs are ideal candidates for such workloads, as they allow design architectures tailored to the stencil regular computational pattern. Moreover, the ever-growing need for performance enhancement leads FPGA-based architectures to scale to multiple devices to benefit from a distributed acceleration. For this reason, we propose a library of HDL components to effectively compute ISLs and CNNs inference on FPGA, along with a scalable multi-FPGA architecture, based on custom PCB interconnects. Our solution eases the design flow and guarantees both scalability and performance competitive with state-of-the-art works.
Assessing the Role of Cyberbiosecurity in Agriculture: A Case Study

Agriculture has adopted the use of smart technology to help meet growing food demands. This increased automation and associated connectivity increases the risk of farms being targeted by cyber-attacks. Increasing frequency of cybersecurity breaches in many industries illustrates the need for securing our food supply chain. The uniqueness of biological data, the complexity of integration across the food and agricultural system, and the importance of this system to the U.S. bioeconomy and public welfare suggests an urgency as well as unique challenges that are not common across all industries. To identify and address the gaps in awareness and knowledge as well as encourage collaborations, Virginia Tech hosted a virtual workshop consisting of professionals from agriculture, cybersecurity, government, and academia. During the workshop, thought leaders and influencers discussed 1) common food and agricultural system challenges, scenarios, outcomes and risks to various sectors of the system; 2) cyberbiosecurity strategies for the system, gaps in workforce and training, and research and policy needs. The meeting sessions were transcribed and analyzed using qualitative methodology. The most common themes that emerged were challenges, solutions, viewpoints, common vocabulary. From the results of the analysis, it is evident that none of the participating groups had available cybersecurity training and resources. Participants were uncertain about future pathways for training, implementation, and outreach related to cyberbiosecurity. Recommendations include creating training and education, continued interdisciplinary collaboration, and recruiting government involvement to speed up better security practices related to cyberbiosecurity.
Zero-shot Learning via the fusion of generation and embedding for image recognition

Traditional image recognition methods require abundant labelled images to learn robust recognition models, and only the images from the classes that appear in the training set (also called seen classes) can be recognized. Thus, determining how to recognize an image from a novel class (also called unseen class) is very challenging. Zero-Shot Learning (ZSL) is proposed to address the above issue. In this paper, we present a novel ZSL model called ZSL based on class prototype and dual latent subspace learning with reconstruction (ZSL-CPLSR). Aiming at the problems of domain shift and information loss, ZSL-CPLSR integrates generation and embedding into a unified framework in the inductive setting, which takes full advantage of the semantic information of unseen classes to alleviate the domain shift and learns more discriminative information with less effective information loss. We conduct comprehensive experiments on four benchmark datasets widely adopted in ZSL. The experimental results show the superiority of ZSL-CPLSR to the state-of-the-art ZSL methods, validating the effectiveness of ZSL-CPLSR.
A Graph Neural Network For Multiple-Image Super-Resolution

Super-resolution consists in reconstructing a high-resolution image from single or multiple low-resolution observations. Deep learning has been reported extremely successful for single-image super-resolution, but its applications to the multiple-image scenarios are limited due to the challenges that arise from feeding a network with a stack of images with sub-pixel translations. In this paper, we introduce Magnet—a new graph neural network that benefits from representing the input low-resolution images as a graph. This enables us to exploit the sub-pixel shifts among the input images while preserving the original low-resolution pixel values for feature extraction and information fusion. Despite a relatively simple architecture, Magnet outperforms the state-of-the-art methods for multiple-image super-resolution, and due to the flexible graph representation, it allows for using a variable number of low-resolution images for reconstruction.
Machine Learning Approach to Estimate Video QoE of Encrypted DASH Traffic in 5G Networks

5G communication technologies promise reduced latency and increased throughput, among other features. The so-called enhanced Mobile Broadband (eMBB) type of services will contribute to further adoption of video streaming services. In this work, we use a realistic emulation environment based on 5G traces to investigate how Dynamic Adaptive Streaming over HTTP (DASH) video content using three state-of-art Adaptive Bitrate Streaming (ABS) algorithms is impacted in static and mobility scenarios. Given the wide adoption of end-to-end encryption, we use machine learning (ML) models to estimate multiple key video Quality of Experience (QoE) indicators (KQIs) taking network-level Quality of Service (QoS) metrics as input features. The proposed feature extraction method does not require chunk-detection, significantly reducing the complexity of the monitoring approach and providing new means for QoE evaluation of HAS protocols. We show that our ML classifiers achieve a QoE prediction accuracy above 91%.
Adopting Diffractive Reading to Advance HCI Research: A Case Study on Technology for Aging

Researchers in Human-Computer Interaction (HCI) have long developed technologies for older adults. Recently, researchers are engaging in critical reflections of these approaches. IoT for aging in place is one area around which these conflicting discourses have converged, likely in part driven by government and industry interest. This article introduces diffractive analysis as an approach that examines difference to yield new empirical understandings about our methods and the topics we study. We constructed three analyses of a dataset collected at an IoT design workshop and then conducted a diffractive analysis. We present themes from this analysis regarding the ways that participants are inscribed in our research, considerations related to transferability and novelty between work centered on older adults and other work, and insights about methodologies. Our discussion contributes implications for researchers to form teams and account for their roles in research, as well as recommendations how diffractive analysis can support other research agendas.
Learning Latent Interactions for Event Classification via Graph Neural Networks and PMU Data

Phasor measurement units (PMUs) are being widely installed on power systems, providing a unique opportunity to enhance wide-area situational awareness. One essential application is the use of PMU data for real-time event identification. However, how to take full advantage of all PMU data in event identification is still an open problem. Thus, we propose a novel method that performs event identification by mining interaction graphs among different PMUs. The proposed interaction graph inference method follows an entirely data-driven manner without knowing the physical topology. Moreover, unlike previous works that treat interactive learning and event identification as two different stages, our method learns interactions jointly with the identification task, thereby improving the accuracy of graph learning and ensuring seamless integration between the two stages. Moreover, to capture multi-scale event patterns, a dilated inception-based method is investigated to perform feature extraction of PMU data. To test the proposed data-driven approach, a large real-world dataset from tens of PMU sources and the corresponding event logs have been utilized in this work. Numerical results validate that our method has higher classification accuracy compared to previous methods.
Bayesian compression for dynamically expandable networks

This paper develops Bayesian Compression for Dynamically Expandable Network (BCDEN), which can learn a compact model structure with preserving the accuracy in a continual learning scenarios. Dynamically Expandable Network (DEN) is efficiently trained by performing selective retraining, dynamically expands network capacity with only the necessary number of units, and effectively prevents semantic drift by duplicating and timestamping units in an online manner. Overcoming conventional DEN only giving point estimates, we providing the Bayesian inference under the principle framework. We validate our BCDEN on multiple public datasets under continual learning setting, on which it can outperform existing continual learning methods on a variety of tasks, and with the state-of-the-art compression results, while still maintaining comparable performance.
Two-Pathway Transformer Network for Video Action Recognition

Traditional two-stream neural networks have shown that both appearance and motion information are important for video action recognition. However, their way of naively averaging two streams’ scores at the end of the framework neglects the underlying relationship between these two kinds of information. In this paper, we propose a two-pathway transformer network that uses memory-based attention to explore such relationship, which further improves the classification performance. Specifically, a transformer-based decoder takes one pathway’s features as the query while the other’s as the key and value. Then based on the similarity matrix estimated by the query and key, relevant information from the value can be selected to enhance the query for the final classification task. Experiments demonstrate that our proposed method outperforms existing fusion strategies at the end of the two-stream methods.
PGNet: Panoptic parsing guided deep stereo matching

As an intensive research area of computer vision, stereovision has a wide range of applications. With the development of deep learning, the performance of stereo matching has been greatly improved. Despite of great progress, it is still very challenging to obtain accurate disparities in the complex scenes with low texture, occlusion or large illumination changes. Panoptic parsing, also known as the combination of semantic and instance segmentation, has the probability to provide valuable high-level scene clues to tackle these challenges. In this paper, we propose a panoptic parsing guided deep network (PGNet) to fulfill the stereo matching task. Upon the PSMNet backbone, three novel modules are designed to embed the panoptic guidance. The confidence module generates the confidence value of cost volume to adjust the probability of candidate disparities. The residual module refines the initial disparity results according to their semantic categories and instance layout. Lastly, the loss module supervises the training of network by inducing extra boundary and smooth constraints from semantic and instance groundtruth. The proposed network has been evaluated on various public datasets like KITTI 2015, KITTI 2012, Virtual KITTI and DrivingStereo, and achieves the state-of-the-art performance.
Heterotic String Model Building with Monad Bundles and Reinforcement Learning

We use reinforcement learning as a means of constructing string compactifications with prescribed properties. Specifically, we study heterotic SO(10) GUT models on Calabi-Yau three-folds with monad bundles, in search of phenomenologically promising examples. Due to the vast number of bundles and the sparseness of viable choices, methods based on systematic scanning are not suitable for this class of models. By focusing on two specific manifolds with Picard numbers two and three, we show that reinforcement learning can be used successfully to explore monad bundles. Training can be accomplished with minimal computing resources and leads to highly efficient policy networks. They produce phenomenologically promising states for nearly 100% of episodes and within a small number of steps. In this way, hundreds of new candidate standard models are found.
Towards Understanding the Generative Capability of Adversarially Robust Classifiers

Recently, some works found an interesting phenomenon that adversarially robust classifiers can generate good images comparable to generative models. We investigate this phenomenon from an energy perspective and provide a novel explanation. We reformulate adversarial example generation, adversarial training, and image generation in terms of an energy function. We find that adversarial training contributes to obtaining an energy function that is flat and has low energy around the real data, which is the key for generative capability. Based on our new understanding, we further propose a better adversarial training method, Joint Energy Adversarial Training (JEAT), which can generate high-quality images and achieve new state-of-the-art robustness under a wide range of attacks. The Inception Score of the images (CIFAR-10) generated by JEAT is 8.80, much better than original robust classifiers (7.50). In particular, we find that the robustness of JEAT is better than other hybrid models.
Enhanced Attention Tracking With Multi-Branch Network for Egocentric Activity Recognition

The emergence of wearable devices has opened up new potentials for egocentric activity recognition. Although some methods integrate attention mechanisms into deep neural networks to capture fine-grained human-object interactions in a weak-supervision manner, they either ignore exploiting the temporal consistency or generate attention based on considering appearance cues only. To address these limitations, in this paper, we propose an enhanced attention-tracking method, combined with multi-branch network (EAT-MBNet), for egocentric activity recognition. Specifically, we propose class-aware attention maps (CAAMs) by employing a self-attention-based module to refine the class activation maps (CAMs). Our proposed method can enhance the semantic dependency between the activity categories and the feature maps. To highlight the discriminative features from the regions of interest across frames, we propose a flow-guided attention-tracking (F-AT) module, by simultaneously leveraging historical attention and motion patterns. Furthermore, we propose a cross-modality modeling branch based on an interactive GRU module, which captures the time-synchronized long-term relationships between the appearance and motion branches. Experimental results on four egocentric activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance.
Study of Interaction Methods in Virtual Electrician Training

Virtual, Augmented and Mixed Reality applications can be used as tools to improve the competence of technical service employees.The paper describes results of the first stage of developing a VR training system dedicated to electricians working for one of Poland's major electricity providers.Three methods of interaction with the virtual environment and three navigation methods have been implemented in the application.To choose the best method of interaction and navigation in the training application, 30 electrical technicians tested different combinations of methods in two training scenarios.Each study participant went through the scenario twice, using two different navigation modes for the same interaction mode and scenario.The time of each pass was measured and the participants filled in a questionnaire to give feedback on their subjective impressions of using a given mode.Based on the gathered data analysis, the best interaction and navigation methods have been selected.These initial results will improve interaction and navigation at the next stage of the application development and implementation in the enterprise.
Hamiltonian coordination primitives for decentralized multiagent navigation

We focus on decentralized navigation among multiple non-communicating agents in continuous domains without explicit traffic rules, such as sidewalks, hallways, or squares. Following collision-free motion in such domains requires effective mechanisms of multiagent behavior prediction. Although this prediction problem can be shown to be NP-hard, humans are often capable of solving it efficiently by leveraging sophisticated mechanisms of implicit coordination. Inspired by the human paradigm, we propose a novel topological formalism that explicitly models multiagent coordination. Our formalism features both geometric and algebraic descriptions enabling the use of standard gradient-based optimization techniques for trajectory generation but also symbolic inference over coordination strategies. In this article, we contribute (a) HCP (Hamiltonian Coordination Primitives), a novel multiagent trajectory-generation pipeline that accommodates spatiotemporal constraints formulated as symbolic topological specifications corresponding to a desired coordination strategy; (b) HCPnav, an online planning framework for decentralized collision avoidance that generates motion by following multiagent trajectory primitives corresponding to high-likelihood, low-cost coordination strategies. Through a series of challenging trajectory-generation experiments, we show that HCP outperforms a trajectory-optimization baseline in generating trajectories of desired topological specifications in terms of success rate and computational efficiency. Finally, through a variety of navigation experiments, we illustrate the efficacy of HCPnav in handling challenging multiagent navigation scenarios under homogeneous or heterogeneous agents across a series of environments of different geometry.
Predicting intent behind selections in scatterplot visualizations

Predicting and capturing an analyst’s intent behind a selection in a data visualization is valuable in two scenarios: First, a successful prediction of a pattern an analyst intended to select can be used to auto-complete a partial selection which, in turn, can improve the correctness of the selection. Second, knowing the intent behind a selection can be used to improve recall and reproducibility. In this paper, we introduce methods to infer analyst’s intents behind selections in data visualizations, such as scatterplots. We describe intents based on patterns in the data, and identify algorithms that can capture these patterns. Upon an interactive selection, we compare the selected items with the results of a large set of computed patterns, and use various ranking approaches to identify the best pattern for an analyst’s selection. We store annotations and the metadata to reconstruct a selection, such as the type of algorithm and its parameterization, in a provenance graph. We present a prototype system that implements these methods for tabular data and scatterplots. Analysts can select a prediction to auto-complete partial selections and to seamlessly log their intents. We discuss implications of our approach for reproducibility and reuse of analysis workflows. We evaluate our approach in a crowd-sourced study, where we show that auto-completing selection improves accuracy, and that we can accurately capture pattern-based intent.