Miracle-dz/UniFAR-FTUs huggingface dataset:
Prevention is Better than Cure: Handling Basis Collapse and Transparency in Dense Networks

Dense nets are an integral part of any classification and regression problem. Recently, these networks have found a new application as solvers for known representations in various domains. However, one crucial issue with dense nets is it's feature interpretation and lack of reproducibility over multiple training runs. In this work, we identify a basis collapse issue as a primary cause and propose a modified loss function that circumvents this problem. We also provide a few general guidelines relating the choice of activations to loss surface roughness and appropriate scaling for designing low-weight dense nets. We demonstrate through carefully chosen numerical experiments that the basis collapse issue leads to the design of massively redundant networks. Our approach results in substantially concise nets, having $100 \times$ fewer parameters, while achieving a much lower $(10\times)$ MSE loss at scale than reported in prior works. Further, we show that the width of a dense net is acutely dependent on the feature complexity. This is in contrast to the dimension dependent width choice reported in prior theoretical works. To the best of our knowledge, this is the first time these issues and contradictions have been reported and experimentally verified. With our design guidelines we render transparency in terms of a low-weight network design. We share our codes for full reproducibility available at this https URL.
One-dimensional twisted and tubular structures of zinc oxide by semiconductor-catalyzed vapor–liquid–solid synthesis

The exploration of unconventional catalysts for the vapor–liquid–solid synthesis of one-dimensional materials promises to yield new morphologies and functionality. Here, we show, for the model ZnO system, that unusual nanostructures can be produced via a semiconductor (Ge) catalyst. As well as the usual straight nanowires, we describe two other distinct morphologies: twisted nanowires and twisted nanotubes. The twisted nanotubes show large hollow cores and surprisingly high twisting rates, up to 9°/μm, that cannot be easily explained through the Eshelby twist model. A combination of ex situ and in situ transmission electron microscopy measurements suggest that the hollow core results from a competition between growth and etching at the Ge–ZnO interface during synthesis. The twisting rate is consistent with a softening of elastic rigidity. These results indicate that the use of unconventional, nonmetallic catalysts provides opportunities to synthesize unusual oxide nanostructures with potentially useful properties.
Global-Local Enhancement Network for NMF-Aware Sign Language Recognition

Sign language recognition (SLR) is a challenging problem, involving complex manual features (i.e., hand gestures) and fine-grained non-manual features (NMFs) (i.e., facial expression, mouth shapes, etc.). Although manual features are dominant, non-manual features also play an important role in the expression of a sign word. Specifically, many sign words convey different meanings due to non-manual features, even though they share the same hand gestures. This ambiguity introduces great challenges in the recognition of sign words. To tackle the above issue, we propose a simple yet effective architecture called Global-Local Enhancement Network (GLE-Net), including two mutually promoted streams toward different crucial aspects of SLR. Of the two streams, one captures the global contextual relationship, while the other stream captures the discriminative fine-grained cues. Moreover, due to the lack of datasets explicitly focusing on this kind of feature, we introduce the first non-manual-feature-aware isolated Chinese sign language dataset (NMFs-CSL) with a total vocabulary size of 1,067 sign words in daily life. Extensive experiments on NMFs-CSL and SLR500 datasets demonstrate the effectiveness of our method.
NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search

The most significant barrier to the advancement of Neural Architecture Search (NAS) is its demand for large computational resources, which hinders scientifically sound empirical evaluations. As a remedy, several tabular NAS benchmarks were proposed to simulate runs of NAS methods in seconds. However, all existing tabular NAS benchmarks are limited to extremely small architectural spaces since they rely on exhaustive evaluations of the space. This leads to unrealistic results that do not transfer to larger search spaces. To overcome this fundamental limitation, we propose NAS-Bench-301, the first surrogate NAS benchmark, using a search space containing $10^{18}$ architectures, many orders of magnitude larger than any previous tabular NAS benchmark. After motivating the benefits of a surrogate benchmark over a tabular one, we fit various regression models on our dataset, which consists of $\sim$60k architecture evaluations, and build surrogates via deep ensembles to also model uncertainty. We benchmark a wide range of NAS algorithms using NAS-Bench-301 and obtain comparable results to the true benchmark at a fraction of the real cost. Finally, we show how NAS-Bench-301 can be used to generate new scientific insights.
syrapropa at SemEval-2020 Task 11: BERT-based Models Design for Propagandistic Technique and Span Detection

This paper describes the BERT-based models proposed for two subtasks in SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. We first build the model for Span Identification (SI) based on SpanBERT, and facilitate the detection by a deeper model and a sentence-level representation. We then develop a hybrid model for the Technique Classification (TC). The hybrid model is composed of three submodels including two BERT models with different training methods, and a feature-based Logistic Regression model. We endeavor to deal with imbalanced dataset by adjusting cost function. We are in the seventh place in SI subtask (0.4711 of F1-measure), and in the third place in TC subtask (0.6783 of F1-measure) on the development set.
Toward Quantifying Ambiguities in Artistic Images

It has long been hypothesized that perceptual ambiguities play an important role in aesthetic experience: A work with some ambiguity engages a viewer more than one that does not. However, current frameworks for testing this theory are limited by the availability of stimuli and data collection methods. This article presents an approach to measuring the perceptual ambiguity of a collection of images. Crowdworkers are asked to describe image content, after different viewing durations. Experiments are performed using images created with Generative Adversarial Networks, using the Artbreeder website. We show that text processing of viewer responses can provide a fine-grained way to measure and describe image ambiguities.
3D for Free: Crossmodal Transfer Learning using HD Maps

3D object detection is a core perceptual challenge for robotics and autonomous driving. However, the class-taxonomies in modern autonomous driving datasets are significantly smaller than many influential 2D detection datasets. In this work, we address the long-tail problem by leveraging both the large class-taxonomies of modern 2D datasets and the robustness of state-of-the-art 2D detection methods. We proceed to mine a large, unlabeled dataset of images and LiDAR, and estimate 3D object bounding cuboids, seeded from an off-the-shelf 2D instance segmentation model. Critically, we constrain this ill-posed 2D-to-3D mapping by using high-definition maps and object size priors. The result of the mining process is 3D cuboids with varying confidence. This mining process is itself a 3D object detector, although not especially accurate when evaluated as such. However, we then train a 3D object detection model on these cuboids, consistent with other recent observations in the deep learning literature, we find that the resulting model is fairly robust to the noisy supervision that our mining process provides. We mine a collection of 1151 unlabeled, multimodal driving logs from an autonomous vehicle and use the discovered objects to train a LiDAR-based object detector. We show that detector performance increases as we mine more unlabeled data. With our full, unlabeled dataset, our method performs competitively with fully supervised methods, even exceeding the performance for certain object categories, without any human 3D annotations.
Much faster heat/mass than momentum transport in rotating Couette flows

Abstract Heat and mass transport are generally closely correlated to momentum transport in shear flows. This so-called Reynolds analogy between advective heat or mass transport and momentum transport hinders efficiency improvements in engineering heat and mass transfer applications. I show through direct numerical simulations that in plane Couette and Taylor–Couette flow, rotation can strongly influence wall-to-wall passive tracer transport and make it much faster than momentum transport, clearly in violation of the Reynolds analogy. This difference between passive tracer transport, representative of heat/mass transport, and momentum transport is observed in steady flows with large counter-rotating vortices at low Reynolds numbers as well as in fully turbulent flows at higher Reynolds numbers. It is especially large near the neutral (Rayleigh's) stability limit. The rotation-induced Coriolis force strongly damps the streamwise/azimuthal velocity fluctuations when this limit is approached, while tracer fluctuations are much less affected. Accordingly, momentum transport is much more reduced than tracer transport, showing that the Coriolis force breaks the Reynolds analogy. At higher Reynolds numbers, this strong advective transport dissimilarity is accompanied by approximate limit cycle dynamics with intense low-frequency bursts of turbulence when approaching the neutral stability limit. The study demonstrates that simple body forces can cause clear dissimilarities between heat/mass and momentum transport in shear flows.
DwarvesGraph: A High-Performance Graph Mining System with Pattern Decomposition

Graph mining tasks, which focus on extracting structural information from subgraphs, are gaining increasing attention recently. While the performance of recent systems---Arabesque, RStream, AutoMine, GraphZero, Pangolin and Peregrine---have been improved significantly over time, e.g., Peregrine can be up to 1317x and 2016x faster than Arabesque and RStream, all current systems treat each pattern to be mined as a whole. Since the computation cost could increase rapidly as the pattern size grows, the key obstacle is the size of patterns a system can handle, e.g., Automine fails to mine all 6-motif patterns within one week on a small graph with 7k vertices. To overcome the challenge, this paper takes a drastically different approach focusing on reducing pattern size. We present DwarvesGraph, the first graph mining system based on pattern decomposition, which first decomposes the target pattern into several sub-patterns, and then computes the count of each. The results of the target pattern can be calculated using the sub-pattern counts with very low additional cost. Although decomposition-based algorithms have been studied for years, this paper addresses several key system challenges. Specifically, we propose: 1) an efficient search method to jointly determine the decomposition of all concrete patterns of an application, considering the computation cost and cross-pattern computation reuse; 2) the partial symmetry breaking technique to eliminate redundant enumeration for each sub-pattern while preserving equivalence of computation semantics; and 3) a decomposition-centric programming model with efficient supports for pattern existence query and advanced graph mining applications such as FSM. Our extensive experiments show that DwarvesGraph is orders of magnitude faster than all existing state-of-the-art systems and capable of mining large patterns that none of them can handle.
Localization precision in chromatic multifocal imaging

Multifocal microscopy affords fast acquisition of microscopic 3D images. This is made possible using a multifocal grating optic, however this induces chromatic dispersion effects into the point spread function impacting image quality and single-molecule localization precision. To minimize this effect, researchers use narrow-band emission filters. However, the choice of optimal emission filter bandwidth in such systems is, thus far, unclear. This work presents a theoretical framework to investigate how the localization precision of a point emitter is affected by the emission filter bandwidth. We calculate the Cram\'er-Rao lower bound for the 3D position of a single emitter imaged using a chromatic multifocal microscope. Results show that the localization precision improves with broader emission filter bandwidth due to increased photon throughput, despite a larger chromatic dispersion. This study provides a framework for optimally designing chromatic multifocal optics and serves as a theoretical foundation for interpretting results.
Collaborative Filtering under Model Uncertainty

In their work, Dean, Rich, and Recht create a model to research recourse and availability of items in a recommender system. We used the definition of predictive multiplicity by Marx, Pin Calmon, and Ustun to examine different variations of this model, using different values for two model parameters. Pairwise comparison of their models show, that most of these models produce very similar results in terms of discrepancy and ambiguity for the availability and only in some cases the availability sets differ significantly.
UTMN at SemEval-2020 Task 11: A Kitchen Solution to Automatic Propaganda Detection

The article describes a fast solution to propaganda detection at SemEval-2020 Task 11, based on feature adjustment. We use per-token vectorization of features and a simple Logistic Regression classifier to quickly test different hypotheses about our data. We come up with what seems to us the best solution, however, we are unable to align it with the result of the metric suggested by the organizers of the task. We test how our system handles class and feature imbalance by varying the number of samples of two classes (Propaganda and None) in the training set, the size of a context window in which a token is vectorized and combination of vectorization means. The result of our system at SemEval2020 Task 11 is F-score=0.37.
Differential responses to high temperature during maturation in heat-stress-tolerant cultivars of Japonica rice

Abstract High-temperature stress during the grain-filling stage reduces grain quality of rice, and this is a serious problem in Japan, especially in the Kyushu region. To solve this problem, various heat-tolerant cultivars have been bred, such as ‘Nikomaru’, ‘Kumasannochikara’, ‘Genkitsukushi’, ‘Sagabiyori’, and ‘Otentosodachi’. When cultivated under high temperature after flowering, these heat-tolerant cultivars had lower percentages of chalky grains than in the heat-sensitive cultivar ‘Hinohikari’. All the heat-tolerant cultivars markedly decreased the nonstructural carbohydrate content in the stem under the high temperature compared to control condition during early grain-filling stage, which is considered to be a common trait of heat tolerance. Notably, ‘Sagabiyori’, ‘Genkitsukushi’, and ‘Nikomaru’ maintained a nucellar epidermis at 17 days after flowering (DAF) under high temperature, whereas the nucellar epidermis disappeared in ‘Hinohikari’. In addition, the expression of AGPS2b, thought to be a rate-limiting enzyme in starch synthesis, in ‘Kumasannochikara’, ‘Otentosodachi’, and ‘Nikomaru’ did not decrease under high temperature, whereas ‘Hinohikari’, ‘Sagabiyori’, and ‘Genkitsukushi’ could not maintain expression of the gene at 17 DAF. Moreover, the expression of Amy3E, a starch-degradation-related gene considered to induce grain chalkiness, in ‘Kumasannochikara’ at 17 DAF was not increased by high temperature. These results suggest that the heat-stress-tolerant cultivars have various mechanisms for dealing with high-temperature stress.
Two Stages Approach for Tweet Engagement Prediction

This paper describes the approach proposed by the D2KLab team for the 2020 RecSys Challenge on the task of predicting user engagement facing tweets. This approach relies on two distinct stages. First, relevant features are learned from the challenge dataset. These features are heterogeneous and are the results of different learning modules such as handcrafted features, knowledge graph embeddings, sentiment analysis features and BERT word embeddings. Second, these features are provided in input to an ensemble system based on XGBoost. This approach, only trained on a subset of the entire challenge dataset, ranked 22 in the final leaderboard.
A Dynamical Central Limit Theorem for Shallow Neural Networks

Recent theoretical work has characterized the dynamics of wide shallow neural networks trained via gradient descent in an asymptotic regime called the mean-field limit as the number of parameters tends towards infinity. At initialization, the randomly sampled parameters lead to a deviation from the mean-field limit that is dictated by the classical Central Limit Theorem (CLT). However, the dynamics of training introduces correlations among the parameters, raising the question of how the fluctuations evolve during training. Here, we analyze the mean-field dynamics as a Wasserstein gradient flow and prove that the deviations from the mean-field limit scaled by the width, in the width-asymptotic limit, remain bounded throughout training. In particular, they eventually vanish in the CLT scaling if the mean-field dynamics converges to a measure that interpolates the training data. This observation has implications for both the approximation rate and the generalization: the upper bound we obtain is given by a Monte-Carlo type resampling error, which does not depend explicitly on the dimension. This bound motivates a regularizaton term on the 2-norm of the underlying measure, which is also connected to generalization via the variation-norm function spaces.
Extratidal Stars and Chemical Abundance Properties of Two Metal-poor Globular Clusters M53 (NGC 5024) and NGC 5053

We search for extratidal stars around two metal-poor Galactic globular clusters, M53 and NGC 5053, using the near-infrared APOGEE spectra. Applying the t-distributed stochastic neighbor embedding (t-SNE) algorithm on the chemical abundances and radial velocities results in identification of two isolated stellar groups composed of cluster member stars in the t-SNE projection plane. With additional selection criteria of radial velocity, location in the color–magnitude diagram, and abundances from a manual chemical analysis, we find a total of 73 cluster member candidates; seven extratidal stars are found beyond the tidal radii of the two clusters. The extratidal stars around the clusters tend to be located along the leading direction of the cluster proper motion, and the individual proper motion of these stars also seems to be compatible to those of clusters. Interestingly, we find that one extratidal star of NGC 5053 is located on the southern outskirts of M53, which is part of common stellar envelope by the tidal interaction between two clusters. We discuss the nature of this star in the context of the tidal interaction between two clusters. We find apparent Mg–Al anticorrelations with a clear gap and spread (∼0.9 dex) in Al abundances for both clusters, and a light Si abundance spread (∼0.3 dex) for NGC 5053. Since all extratidal stars have Mg-enhanced and Al-depleted features, they could be first-generation stars of two globular clusters. Our results support that M53 and NGC 5053 originated in dwarf galaxies and are surrounded by extended stellar substructures of more numerous populations of clusters.
Automatic LiDAR Extrinsic Calibration System using Photodetector and Planar Board for Large-scale Applications

This paper presents a novel automatic calibration system to estimate the extrinsic parameters of LiDAR mounted on a mobile platform for sensor misalignment inspection in the large-scale production of highly automated vehicles. To obtain subdegree and subcentimeter accuracy levels of extrinsic calibration, this study proposed a new concept of a target board with embedded photodetector arrays, named the PD-target system, to find the precise position of the correspondence laser beams on the target surface. Furthermore, the proposed system requires only the simple design of the target board at the fixed pose in a close range to be readily applicable in the automobile manufacturing environment. The experimental evaluation of the proposed system on low-resolution LiDAR showed that the LiDAR offset pose can be estimated within 0.1 degree and 3 mm levels of precision. The high accuracy and simplicity of the proposed calibration system make it practical for large-scale applications for the reliability and safety of autonomous systems.
Nonperturbative leakage elimination for a logical qubit encoded in a mechanical oscillator

Continuous-variable (CV) systems are attracting increasing attention in the realization of universal quantum computation. Several recent experiments have shown the feasibility of using CV systems to, e.g., encode a qubit into a trapped-ion mechanical oscillator and perform logic gates [Nature 566, 513-517 (2019)]. The essential next step is to protect the encoded qubit from quantum decoherence, e.g., the motional decoherence due to the interaction between a mechanical oscillator and its environment. Here we propose a scheme to suppress quantum decoherence of a single-mode harmonic oscillator used to encode qubits by introducing a nonperturbative leakage elimination operator (LEO) specifically designed for this purpose. Remarkably, our nonperturbative LEO can be used to analytically derive exact equations of motion without approximations. It also allows us to prove that the effectiveness of these LEOs only depends on the integral of the pulse sequence in the time domain, while details of the pulse shape does not make a significant difference when the time period is chosen appropriately. This control method can be applied to a system at an arbitrary temperature and arbitrary system-bath coupling strength which makes it extremely useful for general open quantum systems.
Geometry-Guided Dense Perspective Network for Speech-Driven Facial Animation

Realistic speech-driven 3D facial animation is a challenging problem due to the complex relationship between speech and face. In this paper, we propose a deep architecture, called Geometry-guided Dense Perspective Network (GDPnet), to achieve speaker-independent realistic 3D facial animation. The encoder is designed with dense connections to strengthen feature propagation and encourage the re-use of audio features, and the decoder is integrated with an attention mechanism to adaptively recalibrate point-wise feature responses by explicitly modeling interdependencies between different neuron units. We also introduce a non-linear face reconstruction representation as a guidance of latent space to obtain more accurate deformation, which helps solve the geometry-related deformation and is good for generalization across subjects. Huber and HSIC (Hilbert-Schmidt Independence Criterion) constraints are adopted to promote the robustness of our model and to better exploit the non-linear and high-order correlations. Experimental results on the public dataset and real scanned dataset validate the superiority of our proposed GDPnet compared with state-of-the-art model. The code is available for research purposes at http://cic.tju. edu.cn/faculty/likun/projects/GDPnet.