Miracle-dz/UniFAR-FTUs huggingface dataset:
Regularized Identification with Internal Positivity Side-Information

In this paper, we present an impulse response identification scheme that incorporates the internal positivity side-information of the system. The realization theory of positive systems establishes specific criteria for the existence of a positive realization for a given transfer function. These transfer function criteria are translated to a set of suitable conditions on the shape and structure of the impulse responses of positive systems. Utilizing these conditions, the impulse response estimation problem is formulated as a constrained optimization in a reproducing kernel Hilbert space equipped with a stable kernel, and suitable constraints are imposed to encode the internal positivity side-information. The optimization problem is infinite-dimensional with an infinite number of constraints. An equivalent finite-dimensional convex optimization in the form of a convex quadratic program is derived. The resulting equivalent reformulation makes the proposed approach suitable for numerical simulation and practical implementation. A Monte Carlo numerical experiment evaluates the impact of incorporating the internal positivity side-information in the proposed identification scheme. The effectiveness of the proposed method is demonstrated using data from a heating system experiment.
A variance principle explains why dropout finds flatter minima

Although dropout has achieved great success in deep learning, little is known about how it helps the training ﬁnd a good generalization solution in the high-dimensional parameter space. In this work, we show that the training with dropout ﬁnds the neural network with a ﬂatter minimum compared with standard gradient descent training. We further study the underlying mechanism of why dropout ﬁnds ﬂatter minima through experiments. We propose a Variance Principle that the variance of a noise is larger at the sharper direction of the loss landscape. Existing works show that SGD satisﬁes the variance principle, which leads the training to ﬂatter minima. Our work show that the noise induced by the dropout also satisﬁes the variance principle that explains why dropout ﬁnds ﬂatter minima. In general, our work points out that the variance principle is an important similarity between dropout and SGD that lead the training to ﬁnd ﬂatter minima and obtain good generalization.
Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach

Current SOTA adversarially robust models are mostly based on adversarial training (AT) and differ only by some regularizers either at inner maximization or outer minimization steps. Being repetitive in nature during the inner maximization step, they take a huge time to train. We propose a non-iterative method that enforces the following ideas during training. Attribution maps are more aligned to the actual object in the image for adversarially robust models compared to naturally trained models. Also, the allowed set of pixels to perturb an image (that changes model decision) should be restricted to the object pixels only, which reduces the attack strength by limiting the attack space. Our method achieves significant performance gains with a little extra effort (10-20%) over existing AT models and outperforms all other methods in terms of adversarial as well as natural accuracy. We have performed extensive experimentation with CIFAR-10, CIFAR-100, and TinyImageNet datasets and reported results against many popular strong adversarial attacks to prove the effectiveness of our method.
Radial Oscillations of Quark Stars Admixed with Dark Matter

We investigated compact stars consisting of cold quark matter and fermionic dark matter treated as two admixed fluids. We computed the stellar structures and fundamental radial oscillation frequencies of different masses of the dark fermion in the cases of weak and strong self-interacting dark matter. We found that the fundamental frequency can be dramatically modified and, in some cases, stable dark strange planets and dark strangelets with very low masses and radii can be formed.
Testing semiparametric model-equivalence hypotheses based on the characteristic function

We propose three test criteria each of which is appropriate for testing, respectively, the equivalence hypotheses of symmetry, of homogeneity, and of independence, with multivariate data. All quantities have the common feature of involving weighted--type distances between characteristic functions and are convenient from the computational point of view if the weight function is properly chosen. The asymptotic behavior of the tests under the null hypothesis is investigated, and numerical studies are conducted in order to examine the performance of the criteria in finite samples.
Hybrid Method for Simulation of a Fractional COVID-19 Model with Real Case Application

In this research, we provide a mathematical analysis for the novel coronavirus responsible for COVID-19, which continues to be a big source of threat for humanity. Our fractional-order analysis is carried out using a non-singular kernel type operator known as the Atangana-Baleanu-Caputo (ABC) derivative. We parametrize the model adopting available information of the disease from Pakistan in the period 9 April to 2 June 2020. We obtain the required solution with the help of a hybrid method, which is a combination of the decomposition method and the Laplace transform. Furthermore, a sensitivity analysis is carried out to evaluate the parameters that are more sensitive to the basic reproduction number of the model. Our results are compared with the real data of Pakistan and numerical plots are presented at various fractional orders.
Projected GANs Converge Faster

Generative Adversarial Networks (GANs) produce high-quality images but are challenging to train. They need careful regularization, vast amounts of compute, and expensive hyper-parameter sweeps. We make significant headway on these issues by projecting generated and real samples into a fixed, pretrained feature space. Motivated by the finding that the discriminator cannot fully exploit features from deeper layers of the pretrained model, we propose a more effective strategy that mixes features across channels and resolutions. Our Projected GAN improves image quality, sample efficiency, and convergence speed. It is further compatible with resolutions of up to one Megapixel and advances the state-of-the-art Fr\'echet Inception Distance (FID) on twenty-two benchmark datasets. Importantly, Projected GANs match the previously lowest FIDs up to 40 times faster, cutting the wall-clock time from 5 days to less than 3 hours given the same computational resources.
Knowledge-driven Site Selection via Urban Knowledge Graph

Site selection determines optimal locations for new stores, which is of crucial importance to business success. Especially, the wide application of artificial intelligence with multi-source urban data makes intelligent site selection promising. However, existing data-driven methods heavily rely on feature engineering, facing the issues of business generalization and complex relationship modeling. To get rid of the dilemma, in this work, we borrow ideas from knowledge graph (KG), and propose a knowledge-driven model for site selection, short for KnowSite. Specifically, motivated by distilled knowledge and rich semantics in KG, we firstly construct an urban KG (UrbanKG) with cities' key elements and semantic relationships captured. Based on UrbanKG, we employ pre-training techniques for semantic representations, which are fed into an encoder-decoder structure for site decisions. With multi-relational message passing and relation path-based attention mechanism developed, KnowSite successfully reveals the relationship between various businesses and site selection criteria. Extensive experiments on two datasets demonstrate that KnowSite outperforms representative baselines with both effectiveness and explainability achieved.
Robot Learning From Randomized Simulations: A Review

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the “reality gap. ” We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named “domain randomization” which is a method for learning from randomized simulations.
Cross-Modality Fusion Transformer for Multispectral Object Detection

Multispectral image pairs can provide the combined information, making object detection applications more reliable and robust in the open world. To fully exploit the different modalities, we present a simple yet effective cross-modality feature fusion approach, named Cross-Modality Fusion Transformer (CFT) in this paper. Unlike prior CNNs-based works, guided by the transformer scheme, our network learns long-range dependencies and integrates global contextual information in the feature extraction stage. More importantly, by leveraging the self attention of the transformer, the network can naturally carry out simultaneous intra-modality and inter-modality fusion, and robustly capture the latent interactions between RGB and Thermal domains, thereby significantly improving the performance of multispectral object detection. Extensive experiments and ablation studies on multiple datasets demonstrate that our approach is effective and achieves state-of-the-art detection performance. Our code and models are available at https://github.com/DocF/multispectral-object-detection.
Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling

Contrastive learning approaches have achieved great success in learning visual representations with few labels of the target classes. That implies a tantalizing possibility of scaling them up beyond a curated"seed"benchmark, to incorporating more unlabeled images from the internet-scale external sources to enhance its performance. However, in practice, larger amount of unlabeled data will require more computing resources due to the bigger model size and longer training needed. Moreover, open-world unlabeled data usually follows an implicit long-tail class or attribute distribution, many of which also do not belong to the target classes. Blindly leveraging all unlabeled data hence can lead to the data imbalance as well as distraction issues. This motivates us to seek a principled approach to strategically select unlabeled data from an external source, in order to learn generalizable, balanced and diverse representations for relevant classes. In this work, we present an open-world unlabeled data sampling framework called Model-Aware K-center (MAK), which follows three simple principles: (1) tailness, which encourages sampling of examples from tail classes, by sorting the empirical contrastive loss expectation (ECLE) of samples over random data augmentations; (2) proximity, which rejects the out-of-distribution outliers that may distract training; and (3) diversity, which ensures diversity in the set of sampled examples. Empirically, using ImageNet-100-LT (without labels) as the seed dataset and two"noisy"external data sources, we demonstrate that MAK can consistently improve both the overall representation quality and the class balancedness of the learned features, as evaluated via linear classifier evaluation on full-shot and few-shot settings. The code is available at: https://github.com/VITA-Group/MAK
Combining Public and Private Data

Differential privacy is widely adopted to provide provable privacy guarantees in data analysis. We consider the problem of combining public and private data (and, more generally, data with heterogeneous privacy needs) for estimating aggregate statistics. We introduce a mixed estimator of the mean optimized to minimize the variance. We argue that our mechanism is preferable to techniques that preserve the privacy of individuals by subsampling data proportionally to the privacy needs of users. Similarly, we present a mixed median estimator based on the exponential mechanism. We compare our mechanisms to the methods proposed in Jorgensen et al. [2015]. Our experiments provide empirical evidence that our mechanisms often outperform the baseline methods.
Strict $g$-convexity for generated Jacobian equations with applications to global regularity

This article has two purposes. The first is to prove solutions of the second boundary value problem for generated Jacobian equations are strictly $g$-convex. The second is to prove the global $C^3$ regularity of Aleksandrov solutions to the same problem under stronger hypothesis. These are related because the strict $g$-convexity is essential for the proof of the global regularity. The assumptions for the strict $g$-convexity are the natural extension of those used by Chen and Wang in the optimal transport case. They improve the existing domain conditions though at the expense of requiring a $C^3$ generating function. We prove the global regularity under the hypothesis that Jiang and Trudinger recently used to obtain the existence of a globally smooth solution and an additional condition on the height of solutions. Our proof of global regularity is by modifying Jiang and Trudinger's existence result to construct a globally $C^3$ solution intersecting the Aleksandrov solution. Then the strict convexity yields the interior regularity to apply the author's uniqueness results.
Cutting force prediction based on a curved uncut chip thickness model

The curved uncut chip thickness model is presented to predict the cutting forces for general uncut chip geometries. The cutting force is assumed to be distributed along a curved path on the rake face of the cutting tool, which makes the solution computable for inserts with nose radius and more complex cutting edge geometries. The curved paths originate from a basic mechanical model (a compressed plate model), which is used to mimic the motion of the chip on the rake face of the tool without performing real cutting simulations. Consequently, actual cutting forces are predicted using orthogonal cutting data and the orthogonal-to-oblique transformations. The solution satisfies the classical observations and assumptions made on the chip formation process, it is mathematically unique, free of inconsistency and computationally effective. Case studies are presented on real cutting tests. The results highlight that cutting force components can be sensitive to modeling assumptions in case of extreme machining parameters.
Pivotal Test Statistic for Nonparametric Cointegrating Regression Functions

This article focuses on cointegrating regression models in which covariate processes exhibit long range or semi-long range memory behaviors, and may involve endogeneity in which covariate and response error terms are not independent. We assume semilong range memory is produced in the covariate process by tempering of random shock coefficients. The fundamental properties of long memory processes are thus retained in the covariate process. We modify a test statistic proposed for the long memory case by Wang and Phillips (2016) to be suitable in the semi-long range memory setting. The limiting distribution is derived for this modified statistic and shown to depend only on the local memory process of standard Brownian motion. Because, unlike the original statistic of Wang and Phillips (2016), the limit distribution is independent of the differencing parameter of fractional Brownian motion, it is pivotal. Through simulation we investigate properties of nonparametric function estimation for semilong range memory cointegrating models, and consider behavior of both the modified test statistic under semi-long range memory and the original statistic under long range memory. We also provide a brief empirical example.
Pseudo-Spherical Contrastive Divergence

Energy-based models (EBMs) offer flexible distribution parametrization. However, due to the intractable partition function, they are typically trained via contrastive divergence for maximum likelihood estimation. In this paper, we propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum likelihood learning of EBMs. PS-CD is derived from the maximization of a family of strictly proper homogeneous scoring rules, which avoids the computation of the intractable partition function and provides a generalized family of learning objectives that include contrastive divergence as a special case. Moreover, PS-CD allows us to flexibly choose various learning objectives to train EBMs without additional computational cost or variational minimax optimization. Theoretical analysis on the proposed method and extensive experiments on both synthetic data and commonly used image datasets demonstrate the effectiveness and modeling flexibility of PS-CD, as well as its robustness to data contamination, thus showing its superiority over maximum likelihood and $f$-EBMs.
Decentralized Cooperative Reinforcement Learning with Hierarchical Information Structure

Multi-agent reinforcement learning (MARL) problems are challenging due to information asymmetry. To overcome this challenge, existing methods often require high level of coordination or communication between the agents. We consider two-agent multi-armed bandits (MABs) and Markov decision processes (MDPs) with a hierarchical information structure arising in applications, which we exploit to propose simpler and more efficient algorithms that require no coordination or communication. In the structure, in each step the ``leader"chooses her action first, and then the ``follower"decides his action after observing the leader's action. The two agents observe the same reward (and the same state transition in the MDP setting) that depends on their joint action. For the bandit setting, we propose a hierarchical bandit algorithm that achieves a near-optimal gap-independent regret of $\widetilde{\mathcal{O}}(\sqrt{ABT})$ and a near-optimal gap-dependent regret of $\mathcal{O}(\log(T))$, where $A$ and $B$ are the numbers of actions of the leader and the follower, respectively, and $T$ is the number of steps. We further extend to the case of multiple followers and the case with a deep hierarchy, where we both obtain near-optimal regret bounds. For the MDP setting, we obtain $\widetilde{\mathcal{O}}(\sqrt{H^7S^2ABT})$ regret, where $H$ is the number of steps per episode, $S$ is the number of states, $T$ is the number of episodes. This matches the existing lower bound in terms of $A, B$, and $T$.
An Empirical Analysis of HTTPS Configuration Security

It is notoriously difficult to securely configure HTTPS, and poor server configurations have contributed to several attacks including the FREAK, Logjam, and POODLE attacks. In this work, we empirically evaluate the TLS security posture of popular websites and endeavor to understand the configuration decisions that operators make. We correlate several sources of influence on sites' security postures, including software defaults, cloud providers, and online recommendations. We find a fragmented web ecosystem: while most websites have secure configurations, this is largely due to major cloud providers that offer secure defaults. Individually configured servers are more often insecure than not. This may be in part because common resources available to individual operators -- server software defaults and online configuration guides -- are frequently insecure. Our findings highlight the importance of considering SaaS services separately from individually-configured sites in measurement studies, and the need for server software to ship with secure defaults.
Testing generalized scalar-tensor theories of gravity with clusters of galaxies

We test the generalized scalar-tensor theory in static systems, namely galaxy clusters. The Degenerate higher-order scalar-tensor (DHOST) theory modifies the Newtonian potential through effective Newtonian constant and $\Xi_1$ parameter in the small scale, which modifies the hydrostatic equilibrium. We utilize the well-compiled X-COP catalog consisting of 12 clusters with Intra Cluster Medium (ICM) pressure profile by Sunyaev-Zeldovich effect data and temperature profile by X-ray data for each cluster. We perform a fully Bayesian analysis modeling Navarro-Frenk-White (NFW) for the mass profile, and the simplified Vikhlinin model for the electron density. Carefully selecting suitable clusters to present our results, we find a mild to moderate, i.e, $\sim 2\sigma$ significance for a deviation from the standard scenario in 4 of the clusters. However, in terms of Bayesian evidence, we find either equivalent or mild preference for GR. We estimate a joint constraint of $\Xi_1 = -0.030 \pm 0.043$ using 8 clusters, for a modification from a $\Lambda$CDM scenario. This limit is in very good agreement with theoretical ones and an order of magnitude more stringent than the previous constraint obtained using clusters. We also quote a more conservative limit of $\Xi_1 = -0.061 \pm 0.074$. Finally, we comment on the tentative redshift dependence ($\Xi_1(z)$), finding a mild preference ($ \lesssim 2\sigma$) for the same.
Two Heads are Better than One: Geometric-Latent Attention for Point Cloud Classification and Segmentation

We present an innovative two-headed attention layer that combines geometric and latent features to segment a 3D scene into semantically meaningful subsets. Each head combines local and global information, using either the geometric or latent features, of a neighborhood of points and uses this information to learn better local relationships. This Geometric-Latent attention layer (Ge-Latto) is combined with a sub-sampling strategy to capture global features. Our method is invariant to permutation thanks to the use of shared-MLP layers, and it can also be used with point clouds with varying densities because the local attention layer does not depend on the neighbor order. Our proposal is simple yet robust, which allows it to achieve competitive results in the ShapeNetPart and ModelNet40 datasets, and the state-of-the-art when segmenting the complex dataset S3DIS, with 69.2% IoU on Area 5, and 89.7% overall accuracy using K-fold cross-validation on the 6 areas.
Dynamic Differential-Privacy Preserving SGD

The vanilla Differentially-Private Stochastic Gradient Descent (DP-SGD), including DP-Adam and other variants, ensures the privacy of training data by uniformly distributing privacy costs across training steps. The equivalent privacy costs controlled by maintaining the same gradient clipping thresholds and noise powers in each step result in unstable updates and a lower model accuracy when compared to the non-DP counterpart. In this paper, we propose the dynamic DP-SGD (along with dynamic DP-Adam, and others) to reduce the performance loss gap while maintaining privacy by dynamically adjusting clipping thresholds and noise powers while adhering to a total privacy budget constraint. Extensive experiments on a variety of deep learning tasks, including image classification, natural language processing, and federated learning, demonstrate that the proposed dynamic DP-SGD algorithm stabilizes updates and, as a result, significantly improves model accuracy in the strong privacy protection region when compared to the vanilla DP-SGD. We also conduct theoretical analysis to better understand the privacy-utility trade-off with dynamic DP-SGD, as well as to learn why Dynamic DP-SGD can outperform vanilla DP-SGD.