Reviewerly/exHarmony huggingface dataset:
Closure Statistics in Interferometric Data

Interferometric visibilities, reflecting the complex correlations between signals recorded at antennas in an interferometric array, carry information about the angular structure of a distant source. While unknown antenna gains in both amplitude and phase can prevent direct interpretation of these measurements, certain combinations of visibilities called closure phases and closure amplitudes are independent of antenna gains and provide a convenient set of robust observables. However, these closure quantities have subtle noise properties and are generally both linearly and statistically dependent. These complications have obstructed the proper use of closure quantities in interferometric analysis, and they have obscured the relationship between analysis with closure quantities and other analysis techniques such as self calibration. We review the statistics of closure quantities, noting common pitfalls that arise when approaching low signal-to-noise due to the nonlinear propagation of statistical errors. We then develop a strategy for isolating and fitting to the independent degrees of freedom captured by the closure quantities through explicit construction of linearly independent sets of quantities along with their noise covariance in the Gaussian limit, valid for moderate signal-to-noise, and we demonstrate that model fits have biased posteriors when this covariance is ignored. Finally, we introduce a unified procedure for fitting to both closure information and partially calibrated visibilities, and we demonstrate both analytically and numerically the direct equivalence of inference based on closure quantities to that based on self calibration of complex visibilities with unconstrained antenna gains.
The ubiquity of large graphs and surprising challenges of graph processing: extended survey

Graph processing is becoming increasingly prevalent across many application domains. In spite of this prevalence, there is little research about how graphs are actually used in practice. We performed an extensive study that consisted of an online survey of 89 users, a review of the mailing lists, source repositories, and whitepapers of a large suite of graph software products, and in-person interviews with 6 users and 2 developers of these products. Our online survey aimed at understanding: (i) the types of graphs users have; (ii) the graph computations users run; (iii) the types of graph software users use; and (iv) the major challenges users face when processing their graphs. We describe the participants' responses to our questions highlighting common patterns and challenges. Based on our interviews and survey of the rest of our sources, we were able to answer some new questions that were raised by participants' responses to our online survey and understand the specific applications that use graph data and software. Our study revealed surprising facts about graph processing in practice. In particular, real-world graphs represent a very diverse range of entities and are often very large, scalability and visualization are undeniably the most pressing challenges faced by participants, and data integration, recommendations, and fraud detection are very popular applications supported by existing graph software. We hope these findings can guide future research.
A conceptual and practical comparison of PSO-style optimization algorithms

Optimization algorithms are widely employed for finding optimal solutions in many applications. Stochastic optimization algorithms including nature-inspired optimization algorithms are simple and easy to implement, and this is the reason why there is a growing interest in this research area. Recently, many nature-inspired optimization algorithms have been proposed for solving many optimization problems. Moreover, with the aim of improving the performance of optimization algorithms, some modifications were applied such as combining different algorithms and employing some sampling techniques for replacing critical parameters in the optimization algorithms. This paper compares five different widely used PSO-style optimization algorithms to investigate if there is a significant difference between them or not. Theoretically, we explain different PSO-style algorithms and discuss the similarities and differences between them. Practically, a number of experiments were conducted to compare these algorithms. Theoretical analysis and practical results prove that there is not any significant difference between the PSO-style algorithms regarding their performance.
Feature-Aware Adaptation and Density Alignment for Crowd Counting in Video Surveillance

With the development of deep neural networks, the performance of crowd counting and pixel-wise density estimation is continually being refreshed. Despite this, there are still two challenging problems in this field: 1) current supervised learning needs a large amount of training data, but collecting and annotating them is difficult and 2) existing methods cannot generalize well to the unseen domain. A recently released synthetic crowd dataset alleviates these two problems. However, the domain gap between the real-world data and synthetic images decreases the models' performance. To reduce the gap, in this article, we propose a domain-adaptation-style crowd counting method, which can effectively adapt the model from synthetic data to the specific real-world scenes. It consists of multilevel feature-aware adaptation (MFA) and structured density map alignment (SDA). To be specific, MFA boosts the model to extract domain-invariant features from multiple layers. SDA guarantees the network outputs fine density maps with a reasonable distribution on the real domain. Finally, we evaluate the proposed method on four mainstream surveillance crowd datasets, Shanghai Tech Part B, WorldExpo'10, Mall, and UCSD. Extensive experiments are evidence that our approach outperforms the state-of-the-art methods for the same cross-domain counting problem.
Rewriting a Deep Generative Model

A deep generative model such as a GAN learns to model a rich set of semantic and physical rules about the target distribution, but up to now, it has been obscure how such rules are encoded in the network, or how a rule could be changed. In this paper, we introduce a new problem setting: manipulation of specific rules encoded by a deep generative model. To address the problem, we propose a formulation in which the desired rule is changed by manipulating a layer of a deep network as a linear associative memory. We derive an algorithm for modifying one entry of the associative memory, and we demonstrate that several interesting structural rules can be located and modified within the layers of state-of-the-art generative models. We present a user interface to enable users to interactively change the rules of a generative model to achieve desired effects, and we show several proof-of-concept applications. Finally, results on multiple datasets demonstrate the advantage of our method against standard fine-tuning methods and edit transfer algorithms.
Human Body Model Fitting by Learned Gradient Descent

We propose a novel algorithm for the fitting of 3D human shape to images. Combining the accuracy and refinement capabilities of iterative gradient-based optimization techniques with the robustness of deep neural networks, we propose a gradient descent algorithm that leverages a neural network to predict the parameter update rule for each iteration. This per-parameter and state-aware update guides the optimizer towards a good solution in very few steps, converging in typically few steps. During training our approach only requires MoCap data of human poses, parametrized via SMPL. From this data the network learns a subspace of valid poses and shapes in which optimization is performed much more efficiently. The approach does not require any hard to acquire image-to-3D correspondences. At test time we only optimize the 2D joint re-projection error without the need for any further priors or regularization terms. We show empirically that this algorithm is fast (avg. 120ms convergence), robust to initialization and dataset, and achieves state-of-the-art results on public evaluation datasets including the challenging 3DPW in-the-wild benchmark (improvement over SMPLify ( $$45\%$$ ) and also approaches using image-to-3D correspondences).
Weighing Counts: Sequential Crowd Counting by Reinforcement Learning

We formulate counting as a sequential decision problem and present a novel crowd counting model solvable by deep reinforcement learning. In contrast to existing counting models that directly output count values, we divide one-step estimation into a sequence of much easier and more tractable sub-decision problems. Such sequential decision nature corresponds exactly to a physical process in reality—scale weighing. Inspired by scale weighing, we propose a novel ‘counting scale’ termed LibraNet where the count value is analogized by weight. By virtually placing a crowd image on one side of a scale, LibraNet (agent) sequentially learns to place appropriate weights on the other side to match the crowd count. At each step, LibraNet chooses one weight (action) from the weight box (the pre-defined action pool) according to the current crowd image features and weights placed on the scale pan (state). LibraNet is required to learn to balance the scale according to the feedback of the needle (Q values). We show that LibraNet exactly implements scale weighing by visualizing the decision process how LibraNet chooses actions. Extensive experiments demonstrate the effectiveness of our design choices and report state-of-the-art results on a few crowd counting benchmarks, including ShanghaiTech, UCF_CC_50 and UCF-QNRF. We also demonstrate good cross-dataset generalization of LibraNet. Code and models are made available at https://git.io/libranet .
Table Structure Recognition Using Top-Down and Bottom-Up Cues

Tables are information-rich structured objects in document images. While significant work has been done in localizing tables as graphic objects in document images, only limited attempts exist on table structure recognition. Most existing literature on structure recognition depends on extraction of meta-features from the pdf document or on the optical character recognition (ocr) models to extract low-level layout features from the image. However, these methods fail to generalize well because of the absence of meta-features or errors made by the ocr when there is a significant variance in table layouts and text organization. In our work, we focus on tables that have complex structures, dense content, and varying layouts with no dependency on meta-features and/or ocr. We present an approach for table structure recognition that combines cell detection and interaction modules to localize the cells and predict their row and column associations with other detected cells. We incorporate structural constraints as additional differential components to the loss function for cell detection. We empirically validate our method on the publicly available real-world datasets - icdar-2013, icdar-2019 (ctdar) archival, unlv, scitsr, scitsr-comp, tablebank, and pubtabnet. Our attempt opens up a new direction for table structure recognition by combining top-down (table cells detection) and bottom-up (structure recognition) cues in visually understanding the tables.
Deep-Learning-Based Computer-Aided Systems for Breast Cancer Imaging: A Critical Review

This paper provides a critical review of the literature on deep learning applications in breast tumor diagnosis using ultrasound and mammography images. It also summarizes recent advances in computer-aided diagnosis/detection (CAD) systems, which make use of new deep learning methods to automatically recognize breast images and improve the accuracy of diagnoses made by radiologists. This review is based upon published literature in the past decade (January 2010–January 2020), where we obtained around 250 research articles, and after an eligibility process, 59 articles were presented in more detail. The main findings in the classification process revealed that new DL-CAD methods are useful and effective screening tools for breast cancer, thus reducing the need for manual feature extraction. The breast tumor research community can utilize this survey as a basis for their current and future studies.
The potential of 360-degree virtual reality videos to teach water-safety skills to children

This study investigated the potential of Virtual Reality (VR), using 360-degree (360°) videos, as a tool to teach children about water-safety skills. Children (n = 182) aged 10–12 years were randomly assigned to learn about water safety using one of three instructional mediums: 360° VR videos, traditional video, or poster. The training was designed to address specific themes that contributed to drowning in coastal environments, allowing children to learn about dangers of rip currents in a relatively safe environment. Overall, 95% showed improved knowledge of water safety after participating in the water-safety workshop and retained their knowledge during the subsequent tests both one and eight weeks later. No difference was found in the learning outcomes obtained across the three mediums; however, participants in the 360° VR medium reported higher levels of interest and enjoyment than participants using the other two mediums (91% of students in the 360° VR video group found the activity engaging versus 61% for traditional video and 51% for the poster). Teacher feedback also indicated that 360° VR videos are a useful, engaging, and effective method of learning. Overall, we found support for the hypothesis that 360° VR videos are a useful tool to teach targeted skills, and that using 360° VR videos is more motivating and engaging than using traditional learning methods. We also found evidence to support the hypothesis that 360° VR videos are an effective tool for self-directed learning.
Optimal Transport Driven CycleGAN for Unsupervised Learning in Inverse Problems

To improve the performance of classical generative adversarial networks (GANs), Wasserstein generative adversarial networks (WGANs) were developed as a Kantorovich dual formulation of the optimal transport (OT) problem using Wasserstein-1 distance. However, it was not clear how CycleGAN-type generative models can be derived from the OT theory. Here we show that a novel CycleGAN architecture can be derived as a Kantorovich dual OT formulation if a penalized least squares (PLS) cost with deep learning--based inverse path penalty is used as a transportation cost. One of the most important advantages of this formulation is that depending on the knowledge of the forward problem, distinct variations of CycleGAN architecture can be derived: for example, one with two pairs of generators and discriminators, and the other with only a single pair of generator and discriminator. Even for the two generator cases, we show that the structural knowledge of the forward operator can lead to a simpler generator architecture which significantly simplifies the neural network training. The new CycleGAN formulation, which we call the OT-CycleGAN, has been applied for various biomedical imaging problems, such as accelerated magnetic resonance imaging (MRI), super-resolution microscopy, and low-dose X-ray computed tomography (CT). Experimental results confirm the efficacy and flexibility of the theory.
Neural style-preserving visual dubbing

Dubbing is a technique for translating video content from one language to another. However, state-of-the-art visual dubbing techniques directly copy facial expressions from source to target actors without considering identity-specific idiosyncrasies such as a unique type of smile. We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages. At the heart of our approach is the concept of motion style, in particular for facial expressions, i.e., the person-specific expression change that is yet another essential factor beyond visual accuracy in face editing applications. Our method is based on a recurrent generative adversarial network that captures the spatiotemporal co-activation of facial expressions, and enables generating and modifying the facial expressions of the target actor while preserving their style. We train our model with unsynchronized source and target videos in an unsupervised manner using cycle-consistency and mouth expression losses, and synthesize photorealistic video frames using a layered neural face renderer. Our approach generates temporally coherent results, and handles dynamic backgrounds. Our results show that our dubbing approach maintains the idiosyncratic style of the target actor better than previous approaches, even for widely differing source and target actors.
Managing noncooperative behaviors in large-scale group decision-making with linguistic preference orderings: The application in Internet Venture Capital

In Internet venture capital (VC), it is very important to fully analyze and evaluate the influential factors. Because this activity usually involves amounts of experts, it makes sense to incorporate it into large-scale group decision-making (LSGDM). In this process, how to deal with the noncooperative behaviors and express the preference information are two important issues that need to be addressed. Given this, this paper is committed to evaluating the influential factors of Internet VC by managing noncooperative behaviors in LSGDM. Firstly, the preference information can be expressed by linguistic preference orderings (LPOs) which can be used to express unbalanced relationship between any two adjacent alternatives. Then, three kinds of noncooperative behaviors are taken into consideration, and we can identify the group who belongs to one of the three noncooperative behaviors, and then develop methods to manage them respectively. Furthermore, a consensus reaching model is established to manage these noncooperative behaviors. Moreover, we apply the proposed model to solve a practical LSGDM problem involving the evaluations of the influential factors in Internet VC. Finally, comparative analyses between the proposed model and some existing methods are made to show the validity and applicability of the proposed consensus reaching model.
Multilevel projections with adaptive neighbor graph for unsupervised multi-view feature selection

Multi-view feature selection aims at obtaining a subset of informative features from heterogeneous feature domains. Recent graph based approaches mostly learn view-specific feature selection matrices by virtue of prepared single-view graphs, and weight the view-wise objectives to discriminate them. However, the majority of them encounter that (i) the dimensions of vectors for evaluating features in different views are inconsistent and (ii) the similarities attained by using features in the original high-dimensional space are inferior. As a result, the joint evaluation of heterogeneous features using view-specific selection matrices are inaccurate and immensely depend on ill-defined similarity relations. To overcome them, we propose the Multilevel projections with Adaptive neighbor graph model for unsupervised Multi-view Feature Selection (MAMFS). Our formulation learns the collaborative graph adapting to the adaptive k-nearest neighbors in subspaces, wherein the multilevel projections are involved including the weighted view-specific projections that discriminate different views and the joint projection that collaborates different views. Benefited from this, both of view-within information and the view-across information are jointly explored and made use of. Extensive simulations are conducted to validate the superiority of the proposed method compared to the state-of-the-art competitors.
PMC

Deep learning models have been deployed to a wide range of edge devices. Since the data distribution on edge devices may differ from the cloud where the model was trained, it is typically desirable to customize the model for each edge device to improve accuracy. However, such customization is hard because collecting data from edge devices is usually prohibited due to privacy concerns. In this paper, we propose PMC, a privacy-preserving model customization framework to effectively customize a CNN model from the cloud to edge devices without collecting raw data. Instead, we introduce a method to extract statistical information from the edge, which contains adequate domain-related knowledge for model customization. PMC uses Gaussian distribution parameters to describe the edge data distribution, reweights the cloud data based on the parameters, and uses the reweighted data to train a specialized model for the edge device. During this process, differential privacy can be enforced by adding computed noises to the Gaussian parameters. Experiments on public datasets show that PMC can improve model accuracy by a large margin through customization. Finally, a study on user-generated data demonstrates the effectiveness of PMC in real-world settings.
Social NCE: Contrastive Learning of Socially-aware Motion Representations

Learning socially-aware motion representations is at the core of recent advances in multi-agent problems, such as human motion forecasting and robot navigation in crowds. Despite promising progress, existing representations learned with neural networks still struggle to generalize in closed-loop predictions (e.g., output colliding trajectories). This issue largely arises from the non-i.i.d. nature of sequential prediction in conjunction with ill-distributed training data. Intuitively, if the training data only comes from human behaviors in safe spaces, i.e., from "positive" examples, it is difficult for learning algorithms to capture the notion of "negative" examples like collisions. In this work, we aim to address this issue by explicitly modeling negative examples through self-supervision: (i) we intro-duce a social contrastive loss that regularizes the extracted motion representation by discerning the ground-truth positive events from synthetic negative ones; (ii) we construct informative negative samples based on our prior knowledge of rare but dangerous circumstances. Our method substantially reduces the collision rates of recent trajectory forecasting, behavioral cloning and reinforcement learning algorithms, outperforming state-of-the-art methods on several benchmarks. Our code is available at https://github.com/vita-epfl/social-nce.
FeatMatch: Feature-Based Augmentation for Semi-supervised Learning

Recent state-of-the-art semi-supervised learning (SSL) methods use a combination of image-based transformations and consistency regularization as core components. Such methods, however, are limited to simple transformations such as traditional data augmentation or convex combinations of two images. In this paper, we propose a novel learned feature-based refinement and augmentation method that produces a varied set of complex transformations. Importantly, these transformations also use information from both within-class and across-class prototypical representations that we extract through clustering. We use features already computed across iterations by storing them in a memory bank, obviating the need for significant extra computation. These transformations, combined with traditional image-based augmentation, are then used as part of the consistency-based regularization loss. We demonstrate that our method is comparable to current state of art for smaller datasets (CIFAR-10 and SVHN) while being able to scale up to larger datasets such as CIFAR-100 and mini-Imagenet where we achieve significant gains over the state of art (e.g., absolute 17.44% gain on mini-ImageNet). We further test our method on DomainNet, demonstrating better robustness to out-of-domain unlabeled data, and perform rigorous ablations and analysis to validate the method. Code is available here: https://sites.google.com/view/chiawen-kuo/home/featmatch .