Miracle-dz/UniFAR-FTUs huggingface dataset:
Generalized Adversarially Learned Inference

Allowing effective inference of latent vectors while training GANs can greatly increase their applicability in various downstream tasks. Recent approaches, such as ALI and BiGAN frameworks, develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs. We generalize these approaches to incorporate multiple layers of feedback on reconstructions, self-supervision, and other forms of supervision based on prior or learned knowledge about the desired solutions. We achieve this by modifying the discriminator's objective to correctly identify more than two joint distributions of tuples of an arbitrary number of random variables consisting of images, latent vectors, and other variables generated through auxiliary tasks, such as reconstruction and inpainting or as outputs of suitable pre-trained models. We design a non-saturating maximization objective for the generator-encoder pair and prove that the resulting adversarial game corresponds to a global optimum that simultaneously matches all the distributions. Within our proposed framework, we introduce a novel set of techniques for providing self-supervised feedback to the model based on properties, such as patch-level correspondence and cycle consistency of reconstructions. Through comprehensive experiments, we demonstrate the efficacy, scalability, and flexibility of the proposed approach for a variety of tasks. The appendix of the paper can be found at the following link: https://drive.google. com/file/d/1i99e682CqYWMEDXlnqkqrctGLVA9viiz/view? usp=sharing
PatchUp: A Regularization Technique for Convolutional Neural Networks

Large capacity deep learning models are often prone to a high generalization gap when trained with a limited amount of labeled training data. A recent class of methods to address this problem uses various ways to construct a new training sample by mixing a pair (or more) of training samples. We propose PatchUp, a hidden state block-level regularization technique for Convolutional Neural Networks (CNNs), that is applied on selected contiguous blocks of feature maps from a random pair of samples. Our approach improves the robustness of CNN models against the manifold intrusion problem that may occur in other state-of-the-art mixing approaches like Mixup and CutMix. Moreover, since we are mixing the contiguous block of features in the hidden space, which has more dimensions than the input space, we obtain more diverse samples for training towards different dimensions. Our experiments on CIFAR-10, CIFAR-100, and SVHN datasets with PreactResnet18, PreactResnet34, and WideResnet-28-10 models show that PatchUp improves upon, or equals, the performance of current state-of-the-art regularizers for CNNs. We also show that PatchUp can provide better generalization to affine transformations of samples and is more robust against adversarial attacks.
Weakly-supervised Any-shot Object Detection

Methods for object detection and segmentation rely on large scale instance-level annotations for training, which are difficult and time-consuming to collect. Efforts to alleviate this look at varying degrees and quality of supervision. Weakly-supervised approaches draw on image-level labels to build detectors/segmentors, while zero/few-shot methods assume abundant instance-level data for a set of base classes, and none to a few examples for novel classes. This taxonomy has largely siloed algorithmic designs. In this work, we aim to bridge this divide by proposing an intuitive weakly-supervised model that is applicable to a range of supervision: from zero to a few instance-level samples per novel class. For base classes, our model learns a mapping from weakly-supervised to fully-supervised detectors/segmentors. By learning and leveraging visual and lingual similarities between the novel and base classes, we transfer those mappings to obtain detectors/segmentors for novel classes; refining them with a few novel class instance-level annotated samples, if available. The overall model is end-to-end trainable and highly flexible. Through extensive experiments on MS-COCO and Pascal VOC benchmark datasets we show improved performance in a variety of settings.
Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity

Artificial life originated and has long studied the topic of open-ended evolution, which seeks the principles underlying artificial systems that innovate continually, inspired by biological evolution. Recently, interest has grown within the broader field of AI in a generalization of open-ended evolution, here called open-ended search, wherein such questions of open-endedness are explored for advancing AI, whatever the nature of the underlying search algorithm (e.g. evolutionary or gradient-based). For example, open-ended search might design new architectures for neural networks, new reinforcement learning algorithms, or most ambitiously, aim at designing artificial general intelligence. This paper proposes that open-ended evolution and artificial life have much to contribute towards the understanding of open-ended AI, focusing here in particular on the safety of open-ended search. The idea is that AI systems are increasingly applied in the real world, often producing unintended harms in the process, which motivates the growing field of AI safety. This paper argues that open-ended AI has its own safety challenges, in particular, whether the creativity of open-ended systems can be productively and predictably controlled. This paper explains how unique safety problems manifest in open-ended search, and suggests concrete contributions and research questions to explore them. The hope is to inspire progress towards creative, useful, and safe open-ended search algorithms.
Attribute-aware Identity-hard Triplet Loss for Video-based Person Re-identification

Video-based person re-identification (Re-ID) is an important computer vision task. The batch-hard triplet loss frequently used in video-based person Re-ID suffers from the Distance Variance among Different Positives (DVDP) problem. In this paper, we address this issue by introducing a new metric learning method called Attribute-aware Identity-hard Triplet Loss (AITL), which reduces the intra-class variation among positive samples via calculating attribute distance. To achieve a complete model of video-based person Re-ID, a multi-task framework with Attribute-driven Spatio-Temporal Attention (ASTA) mechanism is also proposed. Extensive experiments on MARS and DukeMTMC-VID datasets shows that both the AITL and ASTA are very effective. Enhanced by them, even a simple light-weighted video-based person Re-ID baseline can outperform existing state-of-the-art approaches. The codes has been published on this https URL.
Entropies of commuting transformations on Hilbert spaces

By establishing Multiplicative Ergodic Theorem for commutative transformations on a separable infinite dimensional Hilbert space, in this paper, we investigate Pesin's entropy formula and SRB measures of a finitely generated random transformations on such space via its commuting generators. Moreover, as an application, we give a formula of Friedland's entropy for certain $C^{2}$ $\mathbb{N}^2$-actions.
Graph-Stega: Semantic Controllable Steganographic Text Generation Guided by Knowledge Graph

Most of the existing text generative steganographic methods are based on coding the conditional probability distribution of each word during the generation process, and then selecting specific words according to the secret information, so as to achieve information hiding. Such methods have their limitations which may bring potential security risks. Firstly, with the increase of embedding rate, these models will choose words with lower conditional probability, which will reduce the quality of the generated steganographic texts; secondly, they can not control the semantic expression of the final generated steganographic text. This paper proposes a new text generative steganography method which is quietly different from the existing models. We use a Knowledge Graph (KG) to guide the generation of steganographic sentences. On the one hand, we hide the secret information by coding the path in the knowledge graph, but not the conditional probability of each generated word; on the other hand, we can control the semantic expression of the generated steganographic text to a certain extent. The experimental results show that the proposed model can guarantee both the quality of the generated text and its semantic expression, which is a supplement and improvement to the current text generation steganography.
Existential Theory of the Reals Completeness of Stationary Nash Equilibria in Perfect Information Stochastic Games

We show that the problem of deciding whether in a multi-player perfect information recursive game (i.e. a stochastic game with terminal rewards) there exists a stationary Nash equilibrium ensuring each player a certain payoff is Existential Theory of the Reals complete. Our result holds for acyclic games, where a Nash equilibrium may be computed efficiently by backward induction, and even for deterministic acyclic games with non-negative terminal rewards. We further extend our results to the existence of Nash equilibria where a single player is surely winning. Combining our result with known gadget games without any stationary Nash equilibrium, we obtain that for cyclic games, just deciding existence of any stationary Nash equilibrium is Existential Theory of the Reals complete. This holds for reach-a-set games, stay-in-a-set games, and for deterministic recursive games.
3D Reconstruction of Novel Object Shapes from Single Images

Accurately predicting the 3D shape of any arbitrary object in any pose from a single image is a key goal of computer vision research. This is challenging as it requires a model to learn a representation that can infer both the visible and occluded portions of any object using a limited training set. A training set that covers all possible object shapes is inherently infeasible. Such learning-based approaches are inherently vulnerable to overfitting, and successfully implementing them is a function of both the architecture design and the training approach. We present an extensive investigation of factors specific to architecture design, training, experiment design, and evaluation that influence reconstruction performance and measurement. We show that our proposed SDFNet achieves state-of-the-art performance on seen and unseen shapes relative to existing methods GenRe [53] and OccNet [29]. We provide the first large-scale evaluation of single image shape reconstruction to unseen objects. The source code, data, and trained models can be found on https://github.com/rehg-lab/3DShapeGen.
Alternating ConvLSTM: Learning Force Propagation with Alternate State Updates

Data-driven simulation is an important step-forward in computational physics when traditional numerical methods meet their limits. Learning-based simulators have been widely studied in past years; however, most previous works view simulation as a general spatial-temporal prediction problem and take little physical guidance in designing their neural network architectures. In this paper, we introduce the alternating convolutional Long Short-Term Memory (Alt-ConvLSTM) that models the force propagation mechanisms in a deformable object with near-uniform material properties. Specifically, we propose an accumulation state, and let the network update its cell state and the accumulation state alternately. We demonstrate how this novel scheme imitates the alternate updates of the first and second-order terms in the forward Euler method of numerical PDE solvers. Benefiting from this, our network only requires a small number of parameters, independent of the number of the simulated particles, and also retains the essential features in ConvLSTM, making it naturally applicable to sequential data with spatial inputs and outputs. We validate our Alt-ConvLSTM on human soft tissue simulation with thousands of particles and consistent body pose changes. Experimental results show that Alt-ConvLSTM efficiently models the material kinetic features and greatly outperforms vanilla ConvLSTM with only the single state update.
Super-Mode OFDM With Index Modulation

Orthogonal frequency division multiplexing (OFDM) with index modulation (OFDM-IM) appears as a promising multi-carrier waveform candidate for beyond 5G due to its attractive advantages such as operational flexibility and ease of implementation. However, OFDM-IM may not be a proper choice for 5G services such as enhanced mobile broadband (eMBB) since achieving high data rates is challenging because of its null subcarriers. One solution to enhance the spectral efficiency of OFDM-IM is the employment of multiple distinguishable constellations (modes) by also exploiting its null subcarriers for data transmission. This article proposes a novel IM technique called super-mode OFDM-IM (SuM-OFDM-IM), where mode activation patterns (MAPs) and subcarrier activation patterns (SAPs) are jointly selected and conventional data symbols are repetition coded over multiple subcarriers to achieve a diversity gain. For the proposed scheme, a low-complexity detector is designed, theoretical analyses are performed and a bit error rate (BER) upper bound is derived. The performance of the proposed system is also investigated through real-time experiments using a software-defined radio (SDR) based prototype. We show that SuM-OFDM-IM exhibits promising results in terms of spectral efficiency and error performance; thus, appears as a potential candidate for 5G and beyond communication systems.
The DeepFake Detection Challenge Dataset

Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. In addition to Deepfakes, a variety of GAN-based face swapping methods have also been published with accompanying code. To counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying DeepFake Detection Challenge (DFDC) Kaggle competition. Importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-swapped dataset. The DFDC dataset is by far the largest currently and publicly available face swap video dataset, with over 100,000 total clips sourced from 3,426 paid actors, produced with several Deepfake, GAN-based, and non-learned methods. In addition to describing the methods used to construct the dataset, we provide a detailed analysis of the top submissions from the Kaggle contest. We show although Deepfake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos. Training, validation and testing corpuses can be downloaded from this https URL.
Attentive Feature Reuse for Multi Task Meta learning

We develop new algorithms for simultaneous learning of multiple tasks (e.g., image classification, depth estimation), and for adapting to unseen task/domain distributions within those high-level tasks (e.g., different environments). First, we learn common representations underlying all tasks. We then propose an attention mechanism to dynamically specialize the network, at runtime, for each task. Our approach is based on weighting each feature map of the backbone network, based on its relevance to a particular task. To achieve this, we enable the attention module to learn task representations during training, which are used to obtain attention weights. Our method improves performance on new, previously unseen environments, and is 1.5x faster than standard existing meta learning methods using similar architectures. We highlight performance improvements for Multi-Task Meta Learning of 4 tasks (image classification, depth, vanishing point, and surface normal estimation), each over 10 to 25 test domains/environments, a result that could not be achieved with standard meta learning techniques like MAML.
A note on community-detection (Kemeny) based testing for COVID-19

The Kemeny constant of a graph can be used to identify and analyse bridges between communities in a graph. Testing, tracking and tracing abilities have been identified as pivotal in helping countries to safely reopen activities after the first wave of the COVID-19 virus. Tracing techniques aim at reconstructing past history of contacts, but can face practical limits in an exponential growth of either testing or overly conservative quarantining. We show how this application of graph theory can be conveniently used to efficiently intercept new virus outbreaks, when they are still in their early stage. Simulations provide promising results in early identification and blocking of possible "super-spreader links that transmit disease between different communities.
Realistic Physics Based Character Controller

Over the course of the last several years there was a strong interest in application of modern optimal control techniques to the field of character animation. This interest was fueled by introduction of efficient learning based algorithms for policy optimization, growth in computation power, and game engine improvements. It was shown that it is possible to generate natural looking control of a character by using two ingredients. First, the simulated agent must adhere to a motion capture dataset. And second, the character aims to track the control input from the user. The paper aims at closing the gap between the researchers and users by introducing an open source implementation of physics based character control in Unity framework that has a low entry barrier and a steep learning curve.
A Multifunction Printer CUI for the Blind

Advances in interface design using touch surfaces creates greater obstacles for blind and visually impaired users of technology. Conversational user interfaces offer a reasonable alternative for interactions and enable greater access and most importantly greater independence for the blind. This paper presents a case study of our work to develop a conversational user interface for accessibility for multifunction printers (MFP). It describes our approach to conversational interfaces in general and the specifics of the solution we created for MFPs. It also presents a user study we performed to assess the solution and guide our future efforts.
Robust Recursive Partitioning for Heterogeneous Treatment Effects with Uncertainty Quantification

Subgroup analysis of treatment effects plays an important role in applications from medicine to public policy to recommender systems. It allows physicians (for example) to identify groups of patients for whom a given drug or treatment is likely to be effective and groups of patients for which it is not. Most of the current methods of subgroup analysis begin with a particular algorithm for estimating individualized treatment effects (ITE) and identify subgroups by maximizing the difference across subgroups of the average treatment effect in each subgroup. These approaches have several weaknesses: they rely on a particular algorithm for estimating ITE, they ignore (in)homogeneity within identified subgroups, and they do not produce good confidence estimates. This paper develops a new method for subgroup analysis, R2P, that addresses all these weaknesses. R2P uses an arbitrary, exogenously prescribed algorithm for estimating ITE and quantifies the uncertainty of the ITE estimation, using a construction that is more robust than other methods. Experiments using synthetic and semi-synthetic datasets (based on real data) demonstrate that R2P constructs partitions that are simultaneously more homogeneous within groups and more heterogeneous across groups than the partitions produced by other methods. Moreover, because R2P can employ any ITE estimator, it also produces much narrower confidence intervals with a prescribed coverage guarantee than other methods.
Akhiezer mechanism dominates relaxation of propagons in amorphous material at room temperature

Propagons play an important role in tuning the thermal conductivity of nanostructured amorphous materials. Although advances have been made to quantitatively evaluate the relaxation time of propagons with molecular dynamics, the underlying relaxation mechanism remains unexplored. Here, we investigate the relaxation process of propagons in amorphous silicon, amorphous silica, and amorphous silicon nitride at room temperature in terms of Akhiezer model, the parameters of which were evaluated by performing lattice dynamics and molecular dynamics analysis. The results show that the Akhiezer model can well reproduce experimental results obtained by various kinds of measurement methods, indicating that Akhiezer mechanism dominates the relaxation process of propagons at room temperature. Moreover, we show that the appropriate sound speed of propagons is around 80% of the Debye sound speed and comparable to that of the sound speed of the transversal modes. We also reveal that the contribution of diffusons to the total thermal conductivity of these amorphous is similar, which is around 1 W/m K, while the contribution of propagons varies significantly depending on the materials, which is 30% in amorphous silicon and silica but can be as high as 70% in amorphous silicon nitride.
Effects of stellar density on the photoevaporation of circumstellar discs

Circumstellar discs are the precursors of planetary systems and develop shortly after their host star has formed. In their early stages, these discs are immersed in an environment rich in gas and neighbouring stars, which can be hostile for their survival. There are several environmental processes that affect the evolution of circumstellar discs, and external photoevaporation is arguably one of the most important ones. Theoretical and observational evidence point to circumstellar discs losing mass quickly when in the vicinity of massive, bright stars. In this work, we simulate circumstellar discs in clustered environments in a range of stellar densities, where the photoevaporation mass-loss process is resolved simultaneously with the stellar dynamics, stellar evolution, and the viscous evolution of the discs. Our results indicate that external photoevaporation is efficient in depleting disc masses and that the degree of its effect is related to stellar density. We find that a local stellar density lower than 100 stars pc−2 is necessary for discs massive enough to form planets to survive for 2.0 Myr. There is an order of magnitude difference in the disc masses in regions of projected density 100 versus 104 stars pc−2. We compare our results to observations of the Lupus clouds, the Orion Nebula Cluster, the Orion Molecular Cloud-2, Taurus, and NGC 2024, and find that the trends observed between region density and disc masses are similar to those in our simulations.
Scattering-free routing of surface plasmon polariton waves with optical null medium

Recently, guiding electromagnetic surface waves without sacrificing scattering losses through paths that have arbitrary shape bumps has gained a lot of interest due to its wealth of advantages in modern photonics and plasmonics devices. In this study, based on transformation optics (TO) methodology, a feasible approach to control the flow of surface plasmon plariton (SPPs) at metal-dielectric interfaces with arbitrary curvature is proposed. The obtained material becomes homogeneous and independent of the bump's geometry. That is, one constant material is required to route SPP waves without scattering the energy into the far-field region, which overcome the bottlenecks encountered in the previous works. Several numerical simulations are carried out to illustrate the capability of the propounded cloak to control the SPP flows at metal/dielectric interfaces. The unique designing approach introduced here may open a new horizon to nano-optics and downscaling of photonic circuits.