Reviewerly/exHarmony huggingface dataset:
Collaboration by Competition: Self-coordinated Knowledge Amalgamation for Multi-talent Student Learning

A vast number of well-trained deep networks have been released by developers online for plug-and-play use. These networks specialize in different tasks and in many cases, the data and annotations used to train them are not publicly available. In this paper, we study how to reuse such heterogeneous pre-trained models as teachers, and build a versatile and compact student model, without accessing human annotations. To this end, we propose a self-coordinate knowledge amalgamation network (SOKA-Net) for learning the multi-talent student model. This is achieved via a dual-step adaptive competitive-cooperation training approach, where the knowledge of the heterogeneous teachers are in the first step amalgamated to guide the shared parameter learning of the student network, and followed by a gradient-based competition-balancing strategy to learn the multi-head prediction network as well as the loss weightings of the distinct tasks in the second step. The two steps, which we term as the collaboration and competition step respectively, are performed alternatively until the balance of the competition is reached for the ultimate collaboration. Experimental results demonstrate that, the learned student not only comes with a smaller size but achieves performances on par with or even superior to those of the teachers.
Learning Visual Context by Comparison

Finding diseases from an X-ray image is an important yet highly challenging task. Current methods for solving this task exploit various characteristics of the chest X-ray image, but one of the most important characteristics is still missing: the necessity of comparison between related regions in an image. In this paper, we present Attend-and-Compare Module (ACM) for capturing the difference between an object of interest and its corresponding context. We show that explicit difference modeling can be very helpful in tasks that require direct comparison between locations from afar. This module can be plugged into existing deep learning models. For evaluation, we apply our module to three chest X-ray recognition tasks and COCO object detection & segmentation tasks and observe consistent improvements across tasks. The code is available at https://github.com/mk-minchul/attend-and-compare .
DiPCo — Dinner Party Corpus

We present a speech data corpus that simulates a "dinner party" scenario taking place in an everyday home environment.The corpus was created by recording multiple groups of four Amazon employee volunteers having a natural conversation in English around a dining table.The participants were recorded by a single-channel close-talk microphone and by five far-field 7-microphone array devices positioned at different locations in the recording room.The dataset contains the audio recordings and human labeled transcripts of a total of 10 sessions with a duration between 15 and 45 minutes.The corpus was created to advance in the field of noise robust and distant speech processing and is intended to serve as a public research and benchmarking data set.
Intelligent Force-Measurement System Use in Shock Tunnel

The inertial vibration of the force measurement system (FMS) has a large influence on the force measuring result of aircraft, especially on some tests carried out in high-enthalpy impulse facilities, such as in a shock tunnel. When force tests are conducted in a shock tunnel, the low-frequency vibrations of the FMS and its motion cannot be addressed through digital filtering because of the inertial forces, which are caused by the impact flow during the starting process of the shock tunnel. Therefore, this paper focuses on the dynamic characteristics of the performance of the FMS. A new method—i.e., deep-learning-based single-vector dynamic self-calibration (DL-based SV-DSC) of an impulse FMS, is proposed to increase the accuracy of aerodynamic force measurements in a shock tunnel. A deep-learning technique is used to train the dynamic model of the FMS in this study. Convolutional neural networks with a simple structure are applied to describe the dynamic modeling so that the low-frequency vibration signals are eliminated from the test results of the shock tunnel. By validation of the force test results measured in a shock tunnel, the current trained model can realize intelligent processing of the balance signals of the FMS. Based on this new method of dynamic calibration, the reliability and accuracy of force data processing are well verified.
WITHDRAWN: English corpus translation system based on FPGA and machine learning

This article has been withdrawn: please see Elsevier Policy on Article Withdrawal (https://www.elsevier.com/about/our-business/policies/article-withdrawal). This article has been withdrawn at the request of the Editor in Chief. Subsequent to acceptance of this special issue paper by the responsible Guest Editor Sundhararajan Mahalingam, the integrity and rigor of the peer-review process was investigated and confirmed to fall beneath the high standards expected by Microprocessors & Microsystems. There are also indications that much of the Special Issue includes unoriginal and heavily paraphrased content. Due to a configuration error in the editorial system, unfortunately the Editor in Chief did not receive these papers for approval as per the journal's standard workflow.
DisARM: An Antithetic Gradient Estimator for Binary Latent Variables

Training models with discrete latent variables is challenging due to the difficulty of estimating the gradients accurately. Much of the recent progress has been achieved by taking advantage of continuous relaxations of the system, which are not always available or even possible. The Augment-REINFORCE-Merge (ARM) estimator provides an alternative that, instead of relaxation, uses continuous augmentation. Applying antithetic sampling over the augmenting variables yields a relatively low-variance and unbiased estimator applicable to any model with binary latent variables. However, while antithetic sampling reduces variance, the augmentation process increases variance. We show that ARM can be improved by analytically integrating out the randomness introduced by the augmentation process, guaranteeing substantial variance reduction. Our estimator, \emph{DisARM}, is simple to implement and has the same computational cost as ARM. We evaluate DisARM on several generative modeling benchmarks and show that it consistently outperforms ARM and a strong independent sample baseline in terms of both variance and log-likelihood. Furthermore, we propose a local version of DisARM designed for optimizing the multi-sample variational bound, and show that it outperforms VIMCO, the current state-of-the-art method.
On the Modularity of Hypernetworks

In the context of learning to map an input $I$ to a function $h_I:\mathcal{X}\to \mathbb{R}$, two alternative methods are compared: (i) an embedding-based method, which learns a fixed function in which $I$ is encoded as a conditioning signal $e(I)$ and the learned function takes the form $h_I(x) = q(x,e(I))$, and (ii) hypernetworks, in which the weights $\theta_I$ of the function $h_I(x) = g(x;\theta_I)$ are given by a hypernetwork $f$ as $\theta_I=f(I)$. In this paper, we define the property of modularity as the ability to effectively learn a different function for each input instance $I$. For this purpose, we adopt an expressivity perspective of this property and extend the theory of Devore et al. 1996 and provide a lower bound on the complexity (number of trainable parameters) of neural networks as function approximators, by eliminating the requirements for the approximation method to be robust. Our results are then used to compare the complexities of $q$ and $g$, showing that under certain conditions and when letting the functions $e$ and $f$ be as large as we wish, $g$ can be smaller than $q$ by orders of magnitude. This sheds light on the modularity of hypernetworks in comparison with the embedding-based method. Besides, we show that for a structured target function, the overall number of trainable parameters in a hypernetwork is smaller by orders of magnitude than the number of trainable parameters of a standard neural network and an embedding method.
Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation

Existing 3D human pose estimation models suffer performance drop when applying to new scenarios with unseen poses due to their limited generalizability. In this work, we propose a novel framework, Inference Stage Optimization (ISO), for improving the generalizability of 3D pose models when source and target data come from different pose distributions. Our main insight is that the target data, even though not labeled, carry valuable priors about their underlying distribution. To exploit such information, the proposed ISO performs geometry-aware self-supervised learning (SSL) on each single target instance and updates the 3D pose model before making prediction. In this way, the model can mine distributional knowledge about the target scenario and quickly adapt to it with enhanced generalization performance. In addition, to handle sequential target data, we propose an online mode for implementing our ISO framework via streaming the SSL, which substantially enhances its effectiveness. We systematically analyze why and how our ISO framework works on diverse benchmarks under cross-scenario setup. Remarkably, it yields new state-of-the-art of 83.6% 3D PCK on MPI-INF-3DHP, improving upon the previous best result by 9.7%. Code will be released.
If You Build Your Own NER Scorer, Non-replicable Results Will Come

We attempt to replicate a named entity recognition (NER) model implemented in a popular toolkit and discover that a critical barrier to doing so is the inconsistent evaluation of improper label sequences. We define these sequences and examine how two scorers differ in their handling of them, finding that one approach produces F1 scores approximately 0.5 points higher on the CoNLL 2003 English development and test sets. We propose best practices to increase the replicability of NER evaluations by increasing transparency regarding the handling of improper label sequences.
An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch.

We examine the problem of transferring a policy learned in a source environment to a target environment with different dynamics, particularly in the case where it is critical to reduce the amount of interaction with the target environment during learning. This problem is particularly important in sim-to-real transfer because simulators inevitably model real-world dynamics imperfectly. In this paper, we show that one existing solution to this transfer problem - grounded action transformation - is closely related to the problem of imitation from observation (IfO): learning behaviors that mimic the observations of behavior demonstrations. After establishing this relationship, we hypothesize that recent state-of-the-art approaches from the IfO literature can be effectively repurposed for grounded transfer this http URL validate our hypothesis we derive a new algorithm - generative adversarial reinforced action transformation (GARAT) - based on adversarial imitation from observation techniques. We run experiments in several domains with mismatched dynamics, and find that agents trained with GARAT achieve higher returns in the target environment compared to existing black-box transfer methods
Locally consistent parsing for text indexing in small space

We consider two closely related problems of text indexing in a sub-linear working space. The first problem is the Sparse Suffix Tree (SST) construction, where a text S is given in read-only memory, along with a set of suffixes B, and the goal is to construct the compressed trie of all these suffixes ordered lexicographically, using only O(|B|) words of space. The second problem is the Longest Common Extension (LCE) problem, where again a text S of length n is given in read-only memory with some parameter 1 ≤ τ ≤ n, and the goal is to construct a data structure that uses [MATH HERE] words of space and can compute for any pair of suffixes their longest common prefix length. We show how to use ideas based on the Locally Consistent Parsing technique, that were introduced by Sahinalp and Vishkin [44], in some non-trivial ways in order to improve the known results for the above problems. We introduce new Las-Vegas and deterministic algorithms for both problems. For the randomized algorithms, we introduce the first Las-Vegas SST construction algorithm that takes O(n) time. This is an improvement over the last result of Gawrychowski and Kociumaka [22] who obtained O(n) time for Monte Carlo algorithm, and [MATH HERE] time with hight probability for Las-Vegas algorithm. In addition, we introduce a randomized Las-Vegas construction for a data structure that uses [MATH HERE] words of space, can be constructed in linear time with high probability and answers LCE queries in O(τ) time. For the deterministic algorithms, we introduce an SST construction algorithm that takes [MATH HERE] time (for |B| = Ω(log n)). This is the first almost linear time, O(n · polylog n), deterministic SST construction algorithm, where all previous algorithms take at least [MATH HERE] time. For the LCE problem, we introduce a data structure that uses [MATH HERE] words of space and answers LCE queries in [MATH HERE] time, with O(n log τ) construction time [MATH HERE]. This data structure improves both query time and construction time upon the results of Tanimura et al. [47].
Synthetic Data -- A Privacy Mirage

Synthetic datasets produced by generative models are advertised as a silver-bullet solution to privacy-preserving data sharing. Claims about the privacy benefits of synthetic data, however, have not been supported by a rigorous privacy analysis. In this paper, we introduce an evaluation framework that enables data holders to (I) quantify the privacy gain of publishing a synthetic dataset instead of the raw data, and (II) compare the privacy properties of generative model training algorithms. We illustrate the utility of the framework and quantify privacy gain with respect to two concerns, the risk of re-identification via linkage and the risk of attribute disclosure, on synthetic data produced by a range of generative models, from simple independent histograms to differentially private GANs. We find that, across the board, synthetic data provides little privacy gain even under a black-box adversary with access to a single synthetic dataset only. Moreover, we observe that some target records receive substantially less protection than others and that the more complex the generative model, the more difficult it is to predict which targets will remain vulnerable to privacy attacks. Our findings highlight the need to re-consider whether synthetic data is an appropriate strategy to privacy-preserving data publishing.
Strictly Batch Imitation Learning by Energy-based Distribution Matching

Consider learning a policy purely on the basis of demonstrated behavior -- that is, with no access to reinforcement signals, no knowledge of transition dynamics, and no further interaction with the environment. This *strictly batch imitation learning* problem arises wherever live experimentation is costly, such as in healthcare. One solution is simply to retrofit existing algorithms for apprenticeship learning to work in the offline setting. But such an approach leans heavily on off-policy evaluation or offline model estimation, and can be indirect and inefficient. We argue that a good solution should be able to explicitly parameterize a policy (i.e. respecting action conditionals), implicitly learn from rollout dynamics (i.e. leveraging state marginals), and -- crucially -- operate in an entirely offline fashion. To address this challenge, we propose a novel technique by *energy-based distribution matching* (EDM): By identifying parameterizations of the (discriminative) model of a policy with the (generative) energy function for state distributions, EDM yields a simple but effective solution that equivalently minimizes a divergence between the occupancy measure for the demonstrator and a model thereof for the imitator. Through experiments with application to control and healthcare settings, we illustrate consistent performance gains over existing algorithms for strictly batch imitation learning.
Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping

In this paper, we propose a new accelerated stochastic first-order method called clipped-SSTM for smooth convex stochastic optimization with heavy-tailed distributed noise in stochastic gradients and derive the first high-probability complexity bounds for this method closing the gap in the theory of stochastic optimization with heavy-tailed noise. Our method is based on a special variant of accelerated Stochastic Gradient Descent (SGD) and clipping of stochastic gradients. We extend our method to the strongly convex case and prove new complexity bounds that outperform state-of-the-art results in this case. Finally, we extend our proof technique and derive the first non-trivial high-probability complexity bounds for SGD with clipping without light-tails assumption on the noise.
Deep Hashing with Active Pairwise Supervision

In this paper, we propose a Deep Hashing method with Active Pairwise Supervision (DH-APS). Conventional methods with passive pairwise supervision obtain labeled data for training and require large amount of annotations to reach their full potential, which are not feasible in realistic retrieval tasks. On the contrary, we actively select a small quantity of informative samples for annotation to provide effective pairwise supervision so that discriminative hash codes can be obtained with limited annotation budget. Specifically, we generalize the structural risk minimization principle and obtain three criteria for the pairwise supervision acquisition: uncertainty, representativeness and diversity. Accordingly, samples involved in the following training pairs should be labeled: pairs with most uncertain similarity, pairs that minimize the discrepancy between labeled and unlabeled data, and pairs which are most different from the annotated data, so that the discriminality and generalization ability of the learned hash codes are significantly strengthened. Moreover, our DH-APS can also be employed as a plug-and-play module for semi-supervised hashing methods to further enhance the performance. Experiments demonstrate that the presented DH-APS achieves the accuracy of supervised hashing methods with only $$30\%$$ labeled training samples and improves the semi-supervised binary codes by a sizable margin.
Effective Dimension Adaptive Sketching Methods for Faster Regularized Least-Squares Optimization

We propose a new randomized algorithm for solving L2-regularized least-squares problems based on sketching. We consider two of the most popular random embeddings, namely, Gaussian embeddings and the Subsampled Randomized Hadamard Transform (SRHT). While current randomized solvers for least-squares optimization prescribe an embedding dimension at least greater than the data dimension, we show that the embedding dimension can be reduced to the effective dimension of the optimization problem, and still preserve high-probability convergence guarantees. In this regard, we derive sharp matrix deviation inequalities over ellipsoids for both Gaussian and SRHT embeddings. Specifically, we improve on the constant of a classical Gaussian concentration bound whereas, for SRHT embeddings, our deviation inequality involves a novel technical approach. Leveraging these bounds, we are able to design a practical and adaptive algorithm which does not require to know the effective dimension beforehand. Our method starts with an initial embedding dimension equal to 1 and, over iterations, increases the embedding dimension up to the effective one at most. Hence, our algorithm improves the state-of-the-art computational complexity for solving regularized least-squares problems. Further, we show numerically that it outperforms standard iterative solvers such as the conjugate gradient method and its pre-conditioned version on several standard machine learning datasets.
Deep Structural Causal Models for Tractable Counterfactual Inference

We formulate a general framework for building structural causal models (SCMs) with deep learning components. The proposed approach employs normalising flows and variational inference to enable tractable inference of exogenous noise variables - a crucial step for counterfactual inference that is missing from existing deep causal learning methods. Our framework is validated on a synthetic dataset built on MNIST as well as on a real-world medical dataset of brain MRI scans. Our experimental results indicate that we can successfully train deep SCMs that are capable of all three levels of Pearl's ladder of causation: association, intervention, and counterfactuals, giving rise to a powerful new approach for answering causal questions in imaging applications and beyond. The code for all our experiments is available at this https URL.
Content strategy in LibGuides: An exploratory study

Since its inception in 2007, Springshare's LibGuides has become a popular content management system (CMS) with academic libraries. The challenges of managing LibGuides content through a cohesive content strategy have not been well documented. The researchers conducted a survey of academic libraries that were using the LibGuides platform in Fall 2019 to determine how institutions are managing content challenges. Of 120 respondents, only 53% of the respondents indicated that their institutions had content guidelines that LibGuides authors were expected to follow. The top three most observed elements of these content guidelines were page design (80%), content reuse policies (77%), naming conventions (73%), and navigation (3%). 15% of respondents' institutions did not review guides once they were published. Of those that reviewed guides post-publication, authors (84%) most frequently reviewed their own guides. These findings imply that LibGuides are largely cared for on an honour system. Guidelines tend towards quantifiable elements and omit more abstract concepts.
Immersive Virtual Reality Model Preparation Framework: Towards a Fair and Standardized Performance Comparison

Immersive virtual reality (IVR) has begun to provide business value with sufficiently mature equipment and related software. The construction industry has shown interest in IVR due to its high-level visualization needs. However, the lack of fair performance comparisons among IVR and other cutting-edge competitors hampered the industry implementation of IVR. This paper filled the knowledge gap by developing a standardized model preparation framework for IVR performance research. The framework emphasized essential operations to realize the full potential of IVR. Researchers performed human-involved experiments in a design error detection task with thirteen participants. Researchers validated the framework by comparing user performance data before and after utilizing the proposed framework in the IVR environment. Results showed that the framework substantially mitigated simulator sickness, enhanced user performance by 87.21%, and decreased 58.17% invalid operations per minute when utilizing supporting functions. This paper provided a fair IVR environment for performance comparison with other state-of-the-art visualization technologies, and accordingly, the model preparation framework can contribute to a broad range of IVR performance research.