Miracle-dz/UniFAR-FTUs huggingface dataset:
Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation

In this paper, we propose a novel Pattern-Affinitive Propagation (PAP) framework to jointly predict depth, surface normal and semantic segmentation. The motivation behind it comes from the statistic observation that pattern-affinitive pairs recur much frequently across different tasks as well as within a task. Thus, we can conduct two types of propagations, cross-task propagation and task-specific propagation, to adaptively diffuse those similar patterns. The former integrates cross-task affinity patterns to adapt to each task therein through the calculation on non-local relationships. Next the latter performs an iterative diffusion in the feature space so that the cross-task affinity patterns can be widely-spread within the task. Accordingly, the learning of each task can be regularized and boosted by the complementary task-level affinities. Extensive experiments demonstrate the effectiveness and the superiority of our method on the joint three tasks. Meanwhile, we achieve the state-of-the-art or competitive results on the three related datasets, NYUD-v2, SUN-RGBD and KITTI.
Reducing the variance in online optimization by transporting past gradients

Most stochastic optimization methods use gradients once before discarding them. While variance reduction methods have shown that reusing past gradients can be beneficial when there is a finite number of datapoints, they do not easily extend to the online setting. One issue is the staleness due to using past gradients. We propose to correct this staleness using the idea of implicit gradient transport (IGT) which transforms gradients computed at previous iterates into gradients evaluated at the current iterate without using the Hessian explicitly. In addition to reducing the variance and bias of our updates over time, IGT can be used as a drop-in replacement for the gradient estimate in a number of well-understood methods such as heavy ball or Adam. We show experimentally that it achieves state-of-the-art results on a wide range of architectures and benchmarks. Additionally, the IGT gradient estimator yields the optimal asymptotic convergence rate for online stochastic optimization in the restricted setting where the Hessians of all component functions are equal.
Learning Temporal Pose Estimation from Sparsely-Labeled Videos

Modern approaches for multi-person pose estimation in video require large amounts of dense annotations. However, labeling every frame in a video is costly and labor intensive. To reduce the need for dense annotations, we propose a PoseWarper network that leverages training videos with sparse annotations (every k frames) to learn to perform dense temporal pose propagation and estimation. Given a pair of video frames---a labeled Frame A and an unlabeled Frame B---we train our model to predict human pose in Frame A using the features from Frame B by means of deformable convolutions to implicitly learn the pose warping between A and B. We demonstrate that we can leverage our trained PoseWarper for several applications. First, at inference time we can reverse the application direction of our network in order to propagate pose information from manually annotated frames to unlabeled frames. This makes it possible to generate pose annotations for the entire video given only a few manually-labeled frames. Compared to modern label propagation methods based on optical flow, our warping mechanism is much more compact (6M vs 39M parameters), and also more accurate (88.7% mAP vs 83.8% mAP). We also show that we can improve the accuracy of a pose estimator by training it on an augmented dataset obtained by adding our propagated poses to the original manual labels. Lastly, we can use our PoseWarper to aggregate temporal pose information from neighboring frames during inference. This allows our system to achieve state-of-the-art pose detection results on the PoseTrack2017 and PoseTrack2018 datasets. Code has been made available at: this https URL.
Quantum random number generation by coherent detection of phase noise

We demonstrate a quantum random number generator based on the random nature of the phase difference between two independent laser sources. The speed of random bit generation is determined by the photodetector bandwidth and the linewidth of the lasers used. The system implemented is robust and generates a probability distribution of quantum origin which is intrinsically uniform and thus in principle needs no randomness extraction. The phase is measured with telecom equipment routinely used for high capacity coherent optical communications, which allows to keep track of the phase drift of the lasers and is readily available in the telecommunication industry.
Towards Practical Lipschitz Stochastic Bandits.

Stochastic Lipschitz bandit algorithms are methods that govern exploration-exploitation tradeoffs, and have been used for a variety of important task domains, including zeroth order optimization. While beautiful theory has been developed for the stochastic Lipschitz bandit problem, the methods arising from these theories are not practical, and accordingly, the development of practical well-performing Lipschitz bandit algorithms has stalled in recent years. To remedy this, we present a framework for Lipschitz bandit methods that adaptively learns partitions of context- and arm-space. Due to this flexibility, the algorithm is able to efficiently optimize rewards and minimize regret, by focusing on the portions of the space that are most relevant. Our experiments show that (1) using adaptively-learned partitioning, our method can surpass existing stochastic Lipschitz bandit algorithms, and (2) our algorithms can achieve state-of-the-art performance in challenging real-world tasks such as neural network hyperparameter tuning.
E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles

It has been recently shown that the hidden variables of convolutional neural networks make for an efficient perceptual similarity metric that accurately predicts human judgment on relative image similarity assessment. First, we show that such learned perceptual similarity metrics (LPIPS) are susceptible to adversarial attacks that dramatically contradict human visual similarity judgment. While this is not surprising in light of neural networks' well-known weakness to adversarial perturbations, we proceed to show that self-ensembling with an infinite family of random transformations of the input --- a technique known not to render classification networks robust --- is enough to turn the metric robust against attack, while retaining predictive power on human judgments. Finally, we study the geometry imposed by our our novel self-ensembled metric (E-LPIPS) on the space of natural images. We find evidence of "perceptual convexity" by showing that convex combinations of similar-looking images retain appearance, and that discrete geodesics yield meaningful frame interpolation and texture morphing, all without explicit correspondences.
Estimation Rates for Sparse Linear Cyclic Causal Models

Causal models are important tools to understand complex phenomena and predict the outcome of controlled experiments, also known as interventions. In this work, we present statistical rates of estimation for linear cyclic causal models under the assumption of homoscedastic Gaussian noise by analyzing both the LLC estimator introduced by Hyttinen, Eberhardt and Hoyer and a novel two-step penalized maximum likelihood estimator. We establish asymptotic near minimax optimality for the maximum likelihood estimator over a class of sparse causal graphs in the case of near-optimally chosen interventions. Moreover, we find evidence for practical advantages of this estimator compared to LLC in synthetic numerical experiments.
Linear Optimization of Polynomials and Rational Functions over Boxes

In this paper, we investigate the problem of finding tight linear lower bounding functions for multivariate polynomials over boxes. These functions are obtained by the expansion of polynomials into Bernstein form and using the linear least squares function. Convergence properties of the given polynomials to their lower bounds are shown with respect to raising the degree, width of the box and subdivision. Subsequently, we provide a new method for constructing an affine lower bounding function for a multivariate rational function based on the Bernstein control points, the convex hull of a non-positive polynomial $s$ and degree elevation. Numerical comparisons with the well known Bernstein constant lower bounding function are finally given.
Extension of a Diophantine triple with the property $D(4)$

In this paper we give an upper bound on the number of extensions of a triple to a quadruple for the Diophantine m-tuples with the property D(4) and confirm the conjecture of uniqueness of such extension in some special cases. 2010 Mathematics Subject Classification: 11D09, 11D45, 11J86
On the Secrecy Performance of NOMA Systems with both External and Internal Eavesdroppers

Sharing resource blocks in NOMA systems provides more opportunity to the internal users to overhear the messages of the other users. Therefore, some sort of secrecy against the internal users in addition to the external eavesdroppers must be provided. In this paper, we investigate the secrecy performance of a two-user NOMA system in existence of the external and internal passive eavesdroppers, where the far user acts as an internal eavesdropper and tries to overhear the message of the near user. Our system consists of a single antenna base station, two legitimate users and an external passive eavesdropper. We present the closed-forms for the ergodic secrecy rates of the users. Moreover, to derive the secrecy outage probability (SOP) of the system, we use Gaussian-Chebyshev quadrature method, which gives an approximation for the SOP. Numerical results show that this approximation is very close to the exact value of the SOP of the system. Finally, we eliminate the external eavesdropper and present the closed-forms for the ergodic rate of the far user, the ergodic secrecy rate of the near user and also the SOP of the system.
Soft-Ranking Label Encoding for Robust Facial Age Estimation

Automatic facial age estimation can be used in a wide range of real-world applications. However, this process is challenging due to the randomness and slowness of the aging process. Accordingly, in this paper, we propose a novel method aimed at overcoming the challenges associated with facial age estimation. First, we propose a novel age encoding method, referred to as ‘Soft-ranking’, which encodes two important properties of facial age, <inline-formula> <tex-math notation="LaTeX">${i.e.}$ </tex-math></inline-formula>, the ordinal property and the correlation between adjacent ages. Therefore, Soft-ranking provides a richer supervision signal for training deep models. Moreover, we carefully analyze existing evaluation protocols for age estimation, finding that the overlap in identity between the training and testing sets affects the relative performance of different age encoding methods. Moreover, we achieve state-of-the-art performance on four most popular age databases, <inline-formula> <tex-math notation="LaTeX">${i.e.}$ </tex-math></inline-formula>, Morph II, AgeDB, CLAP2015, and CLAP2016.
LSTM Networks Can Perform Dynamic Counting

In this paper, we systematically assess the ability of standard recurrent networks to perform dynamic counting and to encode hierarchical representations. All the neural models in our experiments are designed to be small-sized networks both to prevent them from memorizing the training sets and to visualize and interpret their behaviour at test time. Our results demonstrate that the Long Short-Term Memory (LSTM) networks can learn to recognize the well-balanced parenthesis language (Dyck-1) and the shuffles of multiple Dyck-1 languages, each defined over different parenthesis-pairs, by emulating simple real-time k-counter machines. To the best of our knowledge, this work is the first study to introduce the shuffle languages to analyze the computational power of neural networks. We also show that a single-layer LSTM with only one hidden unit is practically sufficient for recognizing the Dyck-1 language. However, none of our recurrent networks was able to yield a good performance on the Dyck-2 language learning task, which requires a model to have a stack-like mechanism for recognition.
Deep Music Analogy Via Latent Representation Disentanglement

Analogy-making is a key method for computer algorithms to generate both natural and creative music pieces. In general, an analogy is made by partially transferring the music abstractions, i.e., high-level representations and their relationships, from one piece to another; however, this procedure requires disentangling music representations, which usually takes little effort for musicians but is non-trivial for computers. Three sub-problems arise: extracting latent representations from the observation, disentangling the representations so that each part has a unique semantic interpretation, and mapping the latent representations back to actual music. In this paper, we contribute an explicitly-constrained variational autoencoder (EC$^2$-VAE) as a unified solution to all three sub-problems. We focus on disentangling the pitch and rhythm representations of 8-beat music clips conditioned on chords. In producing music analogies, this model helps us to realize the imaginary situation of "what if" a piece is composed using a different pitch contour, rhythm pattern, or chord progression by borrowing the representations from other pieces. Finally, we validate the proposed disentanglement method using objective measurements and evaluate the analogy examples by a subjective study.
Fidelity as a probe for a deconfined quantum critical point

Deconfined quantum critical point was proposed as a second-order quantum phase transition between two broken symmetry phases beyond the Landau-Ginzburg-Wilson paradigm. However, numerical studies cannot completely rule out a weakly first-order transition because of strong violations of finite-size scaling. We demonstrate that the fidelity is a simple probe to study deconfined quantum critical point. We study the ground-state fidelity susceptibility close to the deconfined quantum critical point in a spin chain using the large-scale finite-size density matrix renormalization group method. We find that the finite-size scaling of the fidelity susceptibility obeys the conventional scaling behavior for continuous phase transitions, supporting the deconfined quantum phase transition is continuous. We numerically determine the deconfined quantum critical point and the associated correlation length critical exponent from the finite-size scaling theory of the fidelity susceptibility. Our results are consistent with recent results obtained directly from the matrix product states for infinite-size lattices using others observables. Our work provides a useful probe to study critical behaviors at deconfined quantum critical point from the concept of quantum information.
The Generalization-Stability Tradeoff in Neural Network Pruning

Pruning neural network parameters is often viewed as a means to compress models, but pruning has also been motivated by the desire to prevent overfitting. This motivation is particularly relevant given the perhaps surprising observation that a wide variety of pruning approaches increase test accuracy despite sometimes massive reductions in parameter counts. To better understand this phenomenon, we analyze the behavior of pruning over the course of training, finding that pruning's benefit to generalization increases with pruning's instability (defined as the drop in test accuracy immediately following pruning). We demonstrate that this "generalization-stability tradeoff" is present across a wide variety of pruning settings and propose a mechanism for its cause: pruning regularizes similarly to noise injection. Supporting this, we find less pruning stability leads to more model flatness and the benefits of pruning do not depend on permanent parameter removal. These results explain the compatibility of pruning-based generalization improvements and the high generalization recently observed in overparameterized networks.
ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies

Empirical adversarial risk minimization (EARM) is a widely used mathematical framework to robustly train deep neural nets (DNNs) that are resistant to adversarial attacks. However, both natural and robust accuracies, in classifying clean and adversarial images, respectively, of the trained robust models are far from satisfactory. In this work, we unify the theory of optimal control of transport equations with the practice of training and testing of ResNets. Based on this unified viewpoint, we propose a simple yet effective ResNets ensemble algorithm to boost the accuracy of the robustly trained model on both clean and adversarial images. The proposed algorithm consists of two components: First, we modify the base ResNets by injecting a variance specified Gaussian noise to the output of each residual mapping. Second, we average over the production of multiple jointly trained modified ResNets to get the final prediction. These two steps give an approximation to the Feynman-Kac formula for representing the solution of a transport equation with viscosity, or a convection-diffusion equation. For the CIFAR10 benchmark, this simple algorithm leads to a robust model with a natural accuracy of {\bf 85.62}\% on clean images and a robust accuracy of ${\bf 57.94 \%}$ under the 20 iterations of the IFGSM attack, which outperforms the current state-of-the-art in defending against IFGSM attack on the CIFAR10. Both natural and robust accuracies of the proposed ResNets ensemble can be improved dynamically as the building block ResNet advances. The code is available at: \url{this https URL}.
Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks

Sequential labeling-based NER approaches restrict each word belonging to at most one entity mention, which will face a serious problem when recognizing nested entity mentions. In this paper, we propose to resolve this problem by modeling and leveraging the head-driven phrase structures of entity mentions, i.e., although a mention can nest other mentions, they will not share the same head word. Specifically, we propose Anchor-Region Networks (ARNs), a sequence-to-nuggets architecture for nested mention detection. ARNs first identify anchor words (i.e., possible head words) of all mentions, and then recognize the mention boundaries for each anchor word by exploiting regular phrase structures. Furthermore, we also design Bag Loss, an objective function which can train ARNs in an end-to-end manner without using any anchor word annotation. Experiments show that ARNs achieve the state-of-the-art performance on three standard nested entity mention detection benchmarks.
Understanding overfitting peaks in generalization error: Analytical risk curves for l2 and l1 penalized interpolation

Traditionally in regression one minimizes the number of fitting parameters or uses smoothing/regularization to trade training (TE) and generalization error (GE). Driving TE to zero by increasing fitting degrees of freedom (dof) is expected to increase GE. However modern big-data approaches, including deep nets, seem to over-parametrize and send TE to zero (data interpolation) without impacting GE. Overparametrization has the benefit that global minima of the empirical loss function proliferate and become easier to find. These phenomena have drawn theoretical attention. Regression and classification algorithms have been shown that interpolate data but also generalize optimally. An interesting related phenomenon has been noted: the existence of non-monotonic risk curves, with a peak in GE with increasing dof. It was suggested that this peak separates a classical regime from a modern regime where over-parametrization improves performance. Similar over-fitting peaks were reported previously (statistical physics approach to learning) and attributed to increased fitting model flexibility. We introduce a generative and fitting model pair ("Misparametrized Sparse Regression" or MiSpaR) and show that the overfitting peak can be dissociated from the point at which the fitting function gains enough dof's to match the data generative model and thus provides good generalization. This complicates the interpretation of overfitting peaks as separating a "classical" from a "modern" regime. Data interpolation itself cannot guarantee good generalization: we need to study the interpolation with different penalty terms. We present analytical formulae for GE curves for MiSpaR with $l_2$ and $l_1$ penalties, in the interpolating limit $\lambda\rightarrow 0$. These risk curves exhibit important differences and help elucidate the underlying phenomena.
Fast Hierarchical Neural Network for Feature Learning on Point Cloud

The analyses relying on 3D point clouds are an utterly complex task, often involving million of points, but also requiring computationally efficient algorithms because of many real-time applications; e.g. autonomous vehicle. However, point clouds are intrinsically irregular and the points are sparsely distributed in a non-Euclidean space, which normally requires point-wise processing to achieve high performances. Although shared filter matrices and pooling layers in convolutional neural networks (CNNs) are capable of reducing the dimensionality of the problem and extracting high-level information simultaneously, grids and highly regular data format are required as input. In order to balance model performance and complexity, we introduce a novel neural network architecture exploiting local features from a manually subsampled point set. In our network, a recursive farthest point sampling method is firstly applied to efficiently cover the entire point set. Successively, we employ the k-nearest neighbours (knn) algorithm to gather local neighbourhood for each group of the subsampled points. Finally, a multiple layer perceptron (MLP) is applied on the subsampled points and edges that connect corresponding point and neighbours to extract local features. The architecture has been tested for both shape classification and segmentation using the ModelNet40 and ShapeNet part datasets, in order to show that the network achieves the best trade-off in terms of competitive performance when compared to other state-of-the-art algorithms.
A splitting theorem for ordered hypergraphs

An {\em ordered $r$-graph} is an $r$-uniform hypergraph whose vertex set is linearly ordered; it is $r$-{\em interval-partite} if there are $r$ consecutive intervals such that each edge has one point in each interval. The basic observation of Erdős and Kleitman that every $r$-graph contains an $r$-partite subgraph with at least a constant (depending on $r$) proportion of edges cannot be extended to the setting of $r$-interval-partite subgraphs in ordered hypergraphs. However, we prove that one can obtain a relatively dense subgraph that has the following similar structure. An ordered $r$-graph is a {\em split} hypergraph if there exist $r-1$ consecutive intervals in the ordering with one of them specified, such that every edge has two vertices in the specified interval and one vertex in every other interval. Our main result is a ``splitting theorem'' that for each $\alpha > \lfloor r/2 \rfloor$, every $n$-vertex ordered $r$-graph with $d \,n^{\alpha}$ edges has for some $m\leq n$ an $m$-vertex split subgraph with $\Omega(d\, m^{\alpha})$ edges. The restriction $\alpha > \lfloor r/2 \rfloor$ is sharp. The splitting theorem has interesting applications to several extremal problems for ordered hypergraphs.