Reviewerly/exHarmony huggingface dataset:
Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders

Generating conversational gestures from speech audio is challenging due to the inherent one-to-many mapping be-tween audio and body motions. Conventional CNNs/RNNs assume one-to-one mapping, and thus tend to predict the average of all possible target motions, resulting in plain/boring motions during inference. In order to over-come this problem, we propose a novel conditional variational autoencoder (VAE) that explicitly models one-to-many audio-to-motion mapping by splitting the cross-modal latent code into shared code and motion-specific code. The shared code mainly models the strong correlation between audio and motion (such as the synchronized audio and motion beats), while the motion-specific code captures diverse motion information independent of the audio. However, splitting the latent code into two parts poses training difficulties for the VAE model. A mapping network facilitating random sampling along with other techniques including relaxed motion loss, bicycle constraint, and diversity loss are designed to better train the VAE. Experiments on both 3D and 2D motion datasets verify that our method generates more realistic and diverse motions than state-of-the-art methods, quantitatively and qualitatively. Finally, we demonstrate that our method can be readily used to generate motion sequences with user-specified motion clips on the timeline. Code and more results are at https://jingli513.github.io/audio2gestures.
Spatio-Temporal Dynamic Inference Network for Group Activity Recognition

Group activity recognition aims to understand the activity performed by a group of people. In order to solve it, modeling complex spatiotemporal interactions is the key. Previous methods are limited in reasoning on a predefined graph, which ignores the inherent person-specific interaction context. Moreover, they adopt inference schemes that are computationally expensive and easily result in the over-smoothing problem. In this paper, we manage to achieve spatio-temporal person-specific inferences by proposing Dynamic Inference Network (DIN), which composes of Dynamic Relation (DR) module and Dynamic Walk (DW) module. We firstly propose to initialize interaction fields on a primary spatio-temporal graph. Within each interaction field, we apply DR to predict the relation matrix and DW to predict the dynamic walk offsets in a joint-processing manner, thus forming a person-specific interaction graph. By updating features on the specific graph, a person can possess a global-level interaction field with a local initialization. Experiments indicate both modules’ effectiveness. Moreover, DIN <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> achieves significant improvement compared to previous state-of-the-art methods on two popular datasets under the same setting, while costing much less computation overhead of the reasoning module.
Structure-Preserving Deraining with Residue Channel Prior Guidance

Single image deraining is important for many high-level computer vision tasks since the rain streaks can severely degrade the visibility of images, thereby affecting the recognition and analysis of the image. Recently, many CNN-based methods have been proposed for rain removal. Although these methods can remove part of the rain streaks, it is difficult for them to adapt to real-world scenarios and restore high-quality rain-free images with clear and accurate structures. To solve this problem, we propose a Structure-Preserving Deraining Network (SPDNet) with RCP guidance. SPDNet directly generates high-quality rain-free images with clear and accurate structures under the guidance of RCP but does not rely on any rain-generating assumptions. Specifically, we found that the RCP of images contains more accurate structural information than rainy images. Therefore, we introduced it to our deraining network to protect structure information of the rain-free image. Meanwhile, a Wavelet-based Multi-Level Module (WMLM) is proposed as the backbone for learning the background information of rainy images and an Interactive Fusion Module (IFM) is designed to make full use of RCP information. In addition, an iterative guidance strategy is proposed to gradually improve the accuracy of RCP, refining the result in a progressive path. Extensive experimental results on both synthetic and real-world datasets demonstrate that the proposed model achieves new state-of-the-art results. Code: https://github.com/Joyies/SPDNet
Lonely together? Identifying the determinants of collective corrective action against uncivil comments

Journalists, scholars, and citizens have raised concerns regarding the high share of incivility in comment sections of news outlets. The current study surveyed members of the social movement #ichbinhier, which aims at collectively countering uncivil comments to cultivate a civil discussion atmosphere in comment sections. We root the activities of #ichbinhier as corrective action and identify the determinants of the members’ engagement by integrating research on bystander behavior and collective action. The findings of our survey show that factors pertaining to individual skills, perceived responsibility, and expected benefits relate to the members’ likelihood to engage against uncivil online comments. Regarding factors derived from collective action research, group efficacy and knowledge of the rules and structures of the movement account for higher levels of engagement. These results shed light on the factors that motivate and inhibit #ichbinhier members—and, potentially, Facebook users in general—to engage against uncivil comments.
Time-Optimal Two- and Three-Qubit Gates for Rydberg Atoms

We identify time-optimal laser pulses to implement the controlled-Z gate and its three qubit generalization, the C$_2$Z gate, for Rydberg atoms in the blockade regime. Pulses are optimized using a combination of numerical and semi-analytical quantum optimal control techniques that result in smooth Ans\"atze with just a few variational parameters. For the CZ gate, the time-optimal implementation corresponds to a global laser pulse that does not require single site addressability of the atoms, simplifying experimental implementation of the gate. We employ quantum optimal control techniques to mitigate errors arising due to the finite lifetime of Rydberg states and finite blockade strengths, while several other types of errors affecting the gates are directly mitigated by the short gate duration. For the considered error sources, we achieve theoretical gate fidelities compatible with error correction using reasonable experimental parameters for CZ and C$_2$Z gates.
A novel long term solar photovoltaic power forecasting approach using LSTM with Nadam optimizer: A case study of India

Abstract Solar photovoltaic (PV) power is emerging as one of the most viable renewable energy sources. The recent enhancements in the integration of renewable energy sources into the power grid create a dire need for reliable solar power forecasting techniques. In this paper, a new long‐term solar PV power forecasting approach using long short‐term memory (LSTM) model with Nadam optimizer is presented. The LSTM model performs better with the time‐series data as it persists information of more time steps. The experimental models are realized on a 250.25 kW installed capacity solar PV power system located at MANIT Bhopal, Madhya Pradesh, India. The proposed model is compared with two time‐series models and eight neural network models using LSTM with different optimizers. The obtained results using LSTM with Nadam optimizer present a significant improvement in the forecasting accuracy of 30.56% over autoregressive integrated moving average, 47.48% over seasonal autoregressive integrated moving average, and 1.35%, 1.43%, 3.51%, 4.88%, 11.84%, 50.69%, and 58.29% over models using RMSprop, Adam, Adamax, SGD, Adagrad, Adadelta, and Ftrl optimizer, respectively. The experimental results prove that the proposed methodology is more conclusive for solar PV power forecasting and can be employed for enhanced system planning and management.
An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization

We consider maximum likelihood estimation for Gaussian Mixture Models (Gmm s). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM grounded in the Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop Riemannian batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods.
Quantum gate identification: Error analysis, numerical results and optical experiment

The identification of an unknown quantum gate is a significant issue in quantum technology. In this paper, we propose a quantum gate identification method within the framework of quantum process tomography. In this method, a series of pure states are inputted to the gate and then a fast state tomography on the output states is performed and the data are used to reconstruct the quantum gate. Our algorithm has computational complexity $O(d^3)$ with the system dimension $d$. The algorithm is compared with maximum likelihood estimation method for the running time, which shows the efficiency advantage of our method. An error upper bound is established for the identification algorithm and the robustness of the algorithm against the purity of input states is also tested. We perform quantum optical experiment on single-qubit Hadamard gate to verify the effectiveness of the identification algorithm.
A New Secure and Lightweight Searchable Encryption Scheme over Encrypted Cloud Data

Searchable Encryption is an emerging cryptographic technique that enables searching capabilities over encrypted data on the cloud. In this paper, a novel searchable encryption scheme for the client-server architecture has been presented. The scheme exploits the properties of the modular inverse to generate a probabilistic trapdoor which facilitates the search over the secure inverted index table. We propose indistinguishability that is achieved by using the property of a probabilistic trapdoor. We design and implement a proof of concept prototype and test our scheme with a real dataset of files. We analyze the performance of our scheme against our claim of the scheme being light weight. The security analysis yields that our scheme assures a higher level of security as compared to other existing schemes.
SIoT: Securing Internet of Things through distributed systems analysis

The Internet of Things (IoT) is increasingly more relevant. This growing importance calls for tools able to provide users with correct, reliable and secure systems. In this paper, we claim that traditional approaches to analyze distributed systems are not expressive enough to address this challenge. As a solution to this problem, we present SIoT, a tool to analyze security aspects of distributed IoT programs and thus protect them against buffer overflow attacks. Our key insight is to look at a distributed system as a single body, and not as separate programs that exchange messages. We then can crosscheck information inferred from different nodes. To construct this global view of a distributed system, we introduce a novel algorithm that discovers inter-program links efficiently. Such links let us build an inter-program view, a knowledge that we can thus forward to a traditional buffer overflow static analysis tool. We prove that our algorithm always terminates and it correctly models the semantics of a distributed system. We have implemented our solution on top of the LLVM compiler, and have used it to secure five ContikiOS applications against buffer overflow attacks. Our solution produces code as safe as the code secured by more traditional analyses; however, applications instrumented by our solution have less than 6% of runtime and program size overhead on average.
Color Homography: Theory and Applications

Images of co-planar points in 3-dimensional space taken from different camera positions are a homography apart. Homographies are at the heart of geometric methods in computer vision and are used in geometric camera calibration, 3D reconstruction, stereo vision and image mosaicking among other tasks. In this paper we show the surprising result that homographies are the apposite tool for relating image colors of the same scene when the capture conditions-illumination color, shading and device-change. Three applications of color homographies are investigated. First, we show that color calibration is correctly formulated as a homography problem. Second, we compare the chromaticity distributions of an image of colorful objects to a database of object chromaticity distributions using homography matching. In the color transfer problem, the colors in one image are mapped so that the resulting image color style matches that of a target image. We show that natural image color transfer can be re-interpreted as a color homography mapping. Experiments demonstrate that solving the color homography problem leads to more accurate calibration, improved color-based object recognition, and we present a new direction for developing natural color transfer algorithms.
Large-scale gesture recognition with a fusion of RGB-D data based on optical flow and the C3D model

Gesture recognition has attracted great attention owing to its applications in many fields such as Human Computer Interaction. However, in video-based gesture recognition, some gesture-irrelevant factors like the background handicap the improvement of recognition rate. In this paper, we propose an effective 3D Convolutional Neural Network based method for large-scale gesture recognition using RGB-D video data. To obtain compact but with sufficient motion path information data for the network, the inputs are unified into 32-frame videos first. Then the optical flow images are constructed from the RGB videos frame by frame, to help with eliminating the disturbing background inside them. After that, the spatiotemporal features of de-background RGB and depth data are extracted with the C3D model (a 3D CNN model) respectively and blended together in the next stage according to the discriminant correlation analysis to boost the performance. Finally the classes are predicted with a linear SVM classifier. Our proposed method achieves 54.50% accuracy on the validation subset and 60.93% on the testing subset of the Chalearn LAP IsoGD dataset, both of which outperform our results (ranked 1st place) in the Chalearn LAP Large-scale Gesture Recognition Challenge.
Quantitative Fault-Tolerance for Reliable Workflows on Heterogeneous IaaS Clouds

Reliability requirement is one of the most important quality of services (QoS) and should be satisfied for a reliable workflow in cloud computing. Primary-backup replication is an important software fault-tolerant technique used to satisfy reliability requirement. Recent works studied quantitative fault-tolerant scheduling to reduce execution cost by minimizing the number of replicas while satisfying the reliability requirement of a workflow on heterogeneous infrastructure as a service (IaaS) clouds. However, a minimum number of replicas does not necessarily lead to the minimum execution cost and shortest schedule length in a heterogeneous IaaS cloud. In this study, we propose the quantitative fault-tolerant scheduling algorithms QFEC and QFEC+ with minimum execution costs and QFSL and QFSL+ with shortest schedule lengths while satisfing the reliability requirements of workflows. Extensive experimental results show that (1) compared with the state-of-the-art algorithms, the proposed algorithms achieve less execution cost and shorter schedule length, although the number of replicas are not minimum; (2) QFEC and QFEC+ are designed to reduce execution cost, and QFEC+ is better than QFEC for all low-parallelism and high-parallelism workflows; and (3) QFSL and QFSL+ are designed to decrease schedule length, and QFSL+ is better than QFSL for all low-parallelism and high-parallelism workflows.
Automatic Identification and Classification of Software Development Video Tutorial Fragments

Software development video tutorials have seen a steep increase in popularity in recent years.Their main advantage is that they thoroughly illustrate how certain technologies, programming languages, etc. are to be used.However, they come with a caveat: there is currently little support for searching and browsing their content.This makes it difficult to quickly find the useful parts in a longer video, as the only options are watching the entire video, leading to wasted time, or fast-forwarding through it, leading to missed information.We present an approach to mine video tutorials found on the web and enable developers to query their contents as opposed to just their metadata.The video tutorials are processed and split into coherent fragments, such that only relevant fragments are returned in response to a query.Moreover, fragments are automatically classified according to their purpose, such as introducing theoretical concepts, explaining code implementation steps, or dealing with errors.This allows developers to set filters in their search to target a specific type of video fragment they are interested in.In addition, the video fragments in CODETUBE are complemented with information from other sources, such as Stack Overflow discussions, giving more context and useful information for understanding the concepts.
Adversarial classification: An adversarial risk analysis approach

Classification problems in security settings are usually contemplated as confrontations in which one or more adversaries try to fool a classifier to obtain a benefit. Most approaches to such adversarial classification problems have focused on game theoretical ideas with strong underlying common knowledge assumptions, which are actually not realistic in security domains. We provide an alternative framework to such problem based on adversarial risk analysis, which we illustrate with several examples. Computational and implementation issues are discussed.
An Adaptive Method for Image Dynamic Range Adjustment

In this paper, we relate the operation of image dynamic range adjustment to the following two tasks: 1) for a high dynamic range (HDR) image, its dynamic range will be mapped to the available dynamic range of display devices and 2) for a low dynamic range (LDR) image, its distribution of intensity will be extended to adequately utilize the full dynamic range of display devices. The common goal of both tasks is to preserve or even enhance the details and improve the visibility of scenes when being matched to the available dynamic range of a display device. In this paper, we propose an efficient method for image dynamic range adjustment with three adaptive steps. First, according to the histogram of the luminance map separated from the given RGB image, two suitable Gamma functions are adaptively selected to separately adjust the luminance of the dark and bright components. Second, an adaptive fusion strategy is proposed to combine the two adjusted luminance maps in order to balance the enhancement of the details in different regions. Third, an adaptive luminance-dependent color restoration method is designed to combine the fused luminance map with the original color components to obtain more consistent color saturation between the images before and after dynamic range adjustment. Extensive experiments show that the proposed method can efficiently compress the dynamic range of HDR scenes with good contrast, clear details, and high structural fidelity of the original image appearance. In addition, the proposed method can also obtain promising performance when being used to enhance LDR nighttime images and greatly facilitates the object (car) detection in nighttime traffic scenes.
How Developers Diagnose Potential Security Vulnerabilities with a Static Analysis Tool

While using security tools to resolve security defects, software developers must apply considerable effort. Success depends on a developer's ability to interact with tools, ask the right questions, and make strategic decisions. To build better security tools and subsequently help developers resolve defects more accurately and efficiently, we studied the defect resolution process-from the questions developers ask to their strategies for answering them. In this paper, we report on an exploratory study with novice and experienced software developers. We equipped them with Find Security Bugs, a security-oriented static analysis tool, and observed their interactions with security vulnerabilities in an open-source system that they had previously contributed to. We found that they asked questions not only about security vulnerabilities, associated attacks, and fixes, but also questions about the software itself, the social ecosystem that built the software, and related resources and tools. We describe the strategic successes and failures we observed and how future tools can leverage our findings to encourage better strategies.
A Directed Graph Fourier Transform With Spread Frequency Components

We study the problem of constructing a graph Fourier transform (GFT) for directed graphs (digraphs), which decomposes graph signals into different modes of variation with respect to the underlying network. Accordingly, to capture low, medium and high frequencies we seek a digraph (D)GFT such that the orthonormal frequency components are as spread as possible in the graph spectral domain. To that end, we advocate a two-step design whereby we: (i) find the maximum directed variation (i.e., a novel notion of frequency on a digraph) a candidate basis vector can attain; and (ii) minimize a smooth spectral dispersion function over the achievable frequency range to obtain the desired spread DGFT basis. Both steps involve non-convex, orthonormality-constrained optimization problems, which are efficiently tackled via a provably convergent, feasible optimization method on the Stiefel manifold. We also propose a heuristic to construct the DGFT basis from Laplacian eigen-vectors of an undirected version of the digraph. We show that the spectral-dispersion minimization problem can be cast as supermodular optimization over the set of candidate frequency components, whose orthonormality can be enforced via a matroid basis constraint. This motivates adopting a scalable greedy algorithm to obtain an approximate solution with quantifiable worst-case spectral dispersion. We illustrate the effectiveness of our DGFT algorithms through numerical tests on synthetic and real-world networks. We also carry out a graph-signal denoising task, whereby the DGFT basis is used to decompose and then low-pass filter temperatures recorded across the United States.