Miracle-dz/UniFAR-FTUs huggingface dataset:
Total Generate: Cycle in Cycle Generative Adversarial Networks for Generating Human Faces, Hands, Bodies, and Natural Scenes

We propose a novel and unified Cycle in CycleGenerative Adversarial Network (C2GAN) for generating human faces, hands, bodies, and natural scenes. Our proposed C2GAN is a cross-modal model exploring the joint exploitation of the input image data and guidance data in an interactive manner. C2GAN contains two different generators, i.e., an image-generation generator and a guidance-generation generator. Both generators are mutually connected and trained in an end-to-end fashion and explicitly form three cycled subnets, i.e., one image generation cycle and two guidance generation cycles. Each cycle aims at reconstructing the input domain and simultaneously produces a useful output involved in the generation of another cycle. In this way, the cycles constrain each other implicitly providing complementary information from both image and guidance modalities and bringing an extra supervision gradient across the cycles, facilitating a more robust optimization of the whole model. Extensive results on four guided image-to-image translation subtasks demonstrate that the proposed C2GAN is effective in generating more realistic images compared with state-of-the-art models.
Place recognition survey: An update on deep learning approaches

Autonomous Vehicles (AV) are becoming more capable of navigating in complex environments with dynamic and changing conditions. A key component that enables these intelligent vehicles to overcome such conditions and become more autonomous is the sophistication of the perception and localization systems. As part of the localization system, place recognition has benefited from recent developments in other perception tasks such as place categorization or object recognition, namely with the emergence of deep learning (DL) frameworks. This paper surveys recent approaches and methods used in place recognition, particularly those based on deep learning. The contributions of this work are twofold: surveying recent sensors such as 3D LiDARs and RADARs, applied in place recognition; and categorizing the various DL-based place recognition works into supervised, unsupervised, semi-supervised, parallel, and hierarchical categories. First, this survey introduces key place recognition concepts to contextualize the reader. Then, sensor characteristics are addressed. This survey proceeds by elaborating on the various DL-based works, presenting summaries for each framework. Some lessons learned from this survey include: the importance of NetVLAD for supervised end-to-end learning; the advantages of unsupervised approaches in place recognition, namely for cross-domain applications; or the increasing tendency of recent works to seek, not only for higher performance but also for higher efficiency.
One Million Scenes for Autonomous Driving: ONCE Dataset

Current perception models in autonomous driving have become notorious for greatly relying on a mass of annotated data to cover unseen cases and address the long-tail problem. On the other hand, learning from unlabeled large-scale collected data and incrementally self-training powerful recognition models have received increasing attention and may become the solutions of next-generation industry-level powerful and robust perception models in autonomous driving. However, the research community generally suffered from data inadequacy of those essential real-world scene data, which hampers the future exploration of fully/semi/self-supervised methods for 3D perception. In this paper, we introduce the ONCE (One millioN sCenEs) dataset for 3D object detection in the autonomous driving scenario. The ONCE dataset consists of 1 million LiDAR scenes and 7 million corresponding camera images. The data is selected from 144 driving hours, which is 20x longer than the largest 3D autonomous driving dataset available (e.g. nuScenes and Waymo), and it is collected across a range of different areas, periods and weather conditions. To facilitate future research on exploiting unlabeled data for 3D detection, we additionally provide a benchmark in which we reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset. We conduct extensive analyses on those methods and provide valuable observations on their performance related to the scale of used data. Data, code, and more information are available at https://once-for-auto-driving.github. io/index.html.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

We present a novel neural surface reconstruction method, called NeuS, for reconstructing objects and scenes with high fidelity from 2D image inputs. Existing neural surface reconstruction approaches, such as DVR and IDR, require foreground mask as supervision, easily get trapped in local minima, and therefore struggle with the reconstruction of objects with severe self-occlusion or thin structures. Meanwhile, recent neural methods for novel view synthesis, such as NeRF and its variants, use volume rendering to produce a neural scene representation with robustness of optimization, even for highly complex objects. However, extracting high-quality surfaces from this learned implicit representation is difficult because there are not sufficient surface constraints in the representation. In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation. We observe that the conventional volume rendering method causes inherent geometric errors (i.e. bias) for surface reconstruction, and therefore propose a new formulation that is free of bias in the first order of approximation, thus leading to more accurate surface reconstruction even without the mask supervision. Experiments on the DTU dataset and the BlendedMVS dataset show that NeuS outperforms the state-of-the-arts in high-quality surface reconstruction, especially for objects and scenes with complex structures and self-occlusion.
A literature review of computational thinking in early ages

ABSTRACT Nowadays, technology has become dominant in the daily lives of most people around the world. Technology is present from children to older people, helping in the most diverse daily tasks and allowing accessibility. However, many times these people are just end-users, without any incentive to develop computational thinking (CT). With advances in technologies, the abstraction of coding, programming languages, and the hardware resources involved will become a reality. However, while we have not progressed to this stage, it is necessary to encourage the development of CT teaching from an early age. This work will present the state of the art concerning teaching initiatives and tools on programming, robotics, and other playful tools for the development of CT in the early ages, explicitly filling the gap of CT at the kindergarten level. We present a systematic literature review evaluating more than 60 papers from 2010 to December 2020. The paper’s amount was classified in taxonomy to show CT’s principal tools and initiates applied to children early. To conclude this paper, an extensive discussion about the future trends in this field is present.
Task Attended Meta-Learning for Few-Shot Learning

Meta-learning (ML) has emerged as a promising direction in learning models under constrained resource settings like few-shot learning. The popular approaches for ML either learn a generalizable initial model or a generic parametric optimizer through episodic training. The former approaches leverage the knowledge from a batch of tasks to learn an optimal prior. In this work, we study the importance of a batch for ML. Specifically, we first incorporate a batch episodic training regimen to improve the learning of the generic parametric optimizer. We also hypothesize that the common assumption in batch episodic training that each task in a batch has an equal contribution to learning an optimal meta-model need not be true. We propose to weight the tasks in a batch according to their"importance"in improving the meta-model's learning. To this end, we introduce a training curriculum motivated by selective focus in humans, called task attended meta-training, to weight the tasks in a batch. Task attention is a standalone module that can be integrated with any batch episodic training regimen. The comparisons of the models with their non-task-attended counterparts on complex datasets like miniImageNet and tieredImageNet validate its effectiveness.
Tilt-induced polar order and topological defects in growing bacterial populations

Abstract Rod-shaped bacteria, such as Escherichia coli, commonly live forming mounded colonies. They initially grow two-dimensionally on a surface and finally achieve three-dimensional growth. While it was recently reported that three-dimensional growth is promoted by topological defects of winding number +1/2 in populations of motile bacteria, how cellular alignment plays a role in nonmotile cases is largely unknown. Here, we investigate the relevance of topological defects in colony formation processes of nonmotile E. coli populations, and found that both ±1/2 topological defects contribute to the three-dimensional growth. Analyzing the cell flow in the bottom layer of the colony, we observe that +1/2 defects attract cells and −1/2 defects repel cells, in agreement with previous studies on motile cells, in the initial stage of the colony growth. However, later, cells gradually flow toward −1/2 defects as well, exhibiting a sharp contrast to the existing knowledge. By investigating three-dimensional cell orientations by confocal microscopy, we find that vertical tilting of cells is promoted near the defects. Crucially, this leads to the emergence of a polar order in the otherwise nematic two-dimensional cell orientation. We extend the theory of active nematics by incorporating this polar order and the vertical tilting, which successfully explains the influx toward −1/2 defects in terms of a polarity-induced force. Our work reveals that three-dimensional cell orientations may result in qualitative changes in properties of active nematics, especially those of topological defects, which may be generically relevant in active matter systems driven by cellular growth instead of self-propulsion.
On the Dynamics of Low-viscosity Warped Disks around Black Holes

Accretion disks around black holes can become warped by Lense–Thirring precession. When the disk viscosity is sufficiently small, such that the warp propagates as a wave, then steady-state solutions to the linearized fluid equations exhibit an oscillatory radial profile of the disk tilt angle. Here we show, for the first time, that these solutions are in good agreement with three-dimensional hydrodynamical simulations, in which the viscosity is isotropic and measured to be small compared to the disk angular semi-thickness, and in the case that the disk tilt—and thus the warp amplitude—remains small. We show, using both the linearized fluid equations and hydrodynamical simulations, that the inner disk tilt can be more than several times larger than the original disk tilt, and we provide physical reasoning for this effect. We explore the transition in disk behavior as the misalignment angle is increased, finding increased dissipation associated with regions of strong warping. For large enough misalignments the disk becomes unstable to disk tearing and breaks into discrete planes. For the simulations we present here, we show that the total (physical and numerical) viscosity at the time the disk breaks is small enough that the disk tearing occurs in the wave-like regime, substantiating that disk tearing is possible in this region of parameter space. Our simulations demonstrate that high spatial resolution, and thus low numerical viscosity, is required to accurately model the warp dynamics in this regime. Finally, we discuss the observational implications of our results.
Analytically Tractable Bayesian Deep Q-Learning

Reinforcement learning (RL) has gained increasing interest since the demonstration it was able to reach human performance on video game benchmarks using deep Q-learning (DQN). The current consensus for training neural networks on such complex environments is to rely on gradient-based optimization. Although alternative Bayesian deep learning methods exist, most of them still rely on gradient-based optimization, and they typically do not scale on benchmarks such as the Atari game environment. Moreover none of these approaches allow performing the analytical inference for the weights and biases defining the neural network. In this paper, we present how we can adapt the temporal difference Q-learning framework to make it compatible with the tractable approximate Gaussian inference (TAGI), which allows learning the parameters of a neural network using a closed-form analytical method. Throughout the experiments with on- and off-policy reinforcement learning approaches, we demonstrate that TAGI can reach a performance comparable to backpropagation-trained networks while using fewer hyperparameters, and without relying on gradient-based optimization.
Visual Probing: Cognitive Framework for Explaining Self-Supervised Image Representations

Recently introduced self-supervised methods for image representation learning provide on par or superior results to their fully supervised competitors, yet the corresponding efforts to explain the self-supervised approaches lag behind. Motivated by this observation, we introduce a novel visual probing framework for explaining the self-supervised models by leveraging probing tasks employed previously in natural language processing. The probing tasks require knowledge about semantic relationships between image parts. Hence, we propose a systematic approach to obtain analogs of natural language in vision, such as visual words, context, and taxonomy. Our proposal is grounded in Marr’s computational theory of vision and concerns features like textures, shapes, and lines. We show the effectiveness and applicability of those analogs in the context of explaining self-supervised representations. Our key findings emphasize that relations between language and vision can serve as an effective yet intuitive tool for discovering how machine learning models work, independently of data modality. Our work opens a plethora of research pathways towards more explainable and transparent AI.
Complexity-Free Generalization via Distributionally Robust Optimization

Established approaches to obtain generalization bounds in data-driven optimization and machine learning mostly build on solutions from empirical risk minimization (ERM), which depend crucially on the functional complexity of the hypothesis class. In this paper, we present an alternate route to obtain these bounds on the solution from distributionally robust optimization (DRO), a recent data-driven optimization framework based on worst-case analysis and the notion of ambiguity set to capture statistical uncertainty. In contrast to the hypothesis class complexity in ERM, our DRO bounds depend on the ambiguity set geometry and its compatibility with the true loss function. Notably, when using maximum mean discrepancy as a DRO distance metric, our analysis implies, to the best of our knowledge, the ﬁrst generalization bound in the literature that depends solely on the true loss function, entirely free of any complexity measures or bounds on the hypothesis class.
On the Impact of the Capacity Drop Phenomenon for Freeway Traffic Flow Control

Capacity drop is an empirically observed phenomenon in vehicular traffic flow on freeways whereby, after a critical density is reached, a state of congestion sets in, but the freeway does not become decongested again until the density drops well below the critical density. This introduces a hysteresis effect so that it is easier to enter the congested state than to leave it. However, many existing first-order models of traffic flow, particularly those used for control design, ignore capacity drop, leading to suboptimal controllers. In this paper, we consider a cell transmission model of traffic flow that incorporates capacity drop to study the problem of optimal freeway ramp metering. We show that, if capacity drop is ignored in the control design, then the resulting controller, obtained via a convex program, may be significantly suboptimal. We then propose an alternative model predictive controller that accounts for capacity drop via a mixed integer linear program and show that, for sufficiently large rollout horizon, this controller is optimal. We also compare these approaches to a heuristic handcrafted controller that is viewed as a modification of an integral feedback controller to account for capacity drop. This heuristic controller outperforms the controller that ignores capacity drop but underperforms compared to the proposed alternative model predictive controller. These results suggest that it is generally important to include capacity drop in the controller design process, and we demonstrate this insight on several case studies.
Output, Employment, and Price Effects of U.S. Narrative Tax Changes: A Factor-Augmented Vector Autoregression Approach

This paper examines the short- and long-run effects of U.S. federal personal income and corporate income tax cuts on a wide array of economic policy variables in a data-rich environment. Using a panel of U.S. macroeconomic data set, made up of 132 quarterly macroeconomic series for 1959-2018, the study estimates factor-augmented vector autoregression (FAVARs) models where an extended narrative tax changes dataset combined with unobserved factors. The narrative approach classifies if tax changes are exogenous or endogenous. This paper identifies narrative tax shocks in the vector autoregression model using the sign restrictions with Uhlig's (2005) penalty function. Empirical findings show a significant expansionary effect of tax cuts on the macroeconomic variables. Cuts in personal and corporate income taxes cause a rise in output, investment, employment, and consumption; however, cuts in personal taxes appear to be a more effective fiscal policy tool than the cut in corporate income taxes. Real GDP, employment, investment, and industrial production increase significantly and reach their maximum response values two years after personal income tax cuts. The effects of corporate tax cuts have relatively smaller effects on output and consumption but show immediate and higher effects on fixed investment and price levels.
Pseudo-Relevance Feedback for Multiple Representation Dense Retrieval

Pseudo-relevance feedback mechanisms, from Rocchio to the relevance models, have shown the usefulness of expanding and reweighting the users' initial queries using information occurring in an initial set of retrieved documents, known as the pseudo-relevant set. Recently, dense retrieval -- through the use of neural contextual language models such as BERT for analysing the documents' and queries' contents and computing their relevance scores -- has shown a promising performance on several information retrieval tasks still relying on the traditional inverted index for identifying documents relevant to a query. Two different dense retrieval families have emerged: the use of single embedded representations for each passage and query (e.g. using BERT's [CLS] token), or via multiple representations (e.g. using an embedding for each token of the query and document). In this work, we conduct the first study into the potential for multiple representation dense retrieval to be enhanced using pseudo-relevance feedback. In particular, based on the pseudo-relevant set of documents identified using a first-pass dense retrieval, we extract representative feedback embeddings (using KMeans clustering) -- while ensuring that these embeddings discriminate among passages (based on IDF) -- which are then added to the query representation. These additional feedback embeddings are shown to both enhance the effectiveness of a reranking as well as an additional dense retrieval operation. Indeed, experiments on the MSMARCO passage ranking dataset show that MAP can be improved by upto 26% on the TREC 2019 query set and 10% on the TREC 2020 query set by the application of our proposed ColBERT-PRF method on a ColBERT dense retrieval approach.
Distributed Picard Iteration

—In recent work, we proposed a distributed Picard iteration (DPI) that allows a set of agents, linked by a communication network, to ﬁnd a ﬁxed point of a locally contractive (LC) map that is the average of individual maps held by said agents. In this work, we build upon the DPI and its local linear convergence (LLC) guarantees to make several contributions. We show that Sanger’s algorithm for principal component analysis (PCA) corresponds to the iteration of an LC map that can be written as the average of local maps, each map known to each agent holding a subset of the data. Similarly, we show that a variant of the expectation-maximization (EM) algorithm for parameter estimation from noisy and faulty measurements in a sensor network can be written as the iteration of an LC map that is the average of local maps, each available at just one node. Consequently, via the DPI, we derive two distributed algorithms – distributed EM and distributed PCA – whose LLC guarantees follow from those that we proved for the DPI. The veriﬁcation of the LC condition for EM is challenging, as the underlying operator depends on random samples, thus the LC condition is of probabilistic nature.
Neural network interpretability for forecasting of aggregated renewable generation

With the rapid growth of renewable energy, lots of small photovoltaic (PV) prosumers emerge. Due to the uncertainty of solar power generation, there is a need for aggregated prosumers to predict solar power generation and whether solar power generation will be larger than load. This paper presents two interpretable neural networks to solve the problem: one binary classification neural network and one regression neural network. The neural networks are built using TensorFlow. The global feature importance and local feature contributions are examined by three gradient-based methods: Integrated Gradients, Expected Gradients, and DeepLIFT. Moreover, we detect abnormal cases when predictions might fail by estimating the prediction uncertainty using Bayesian neural networks. Neural networks, which are interpreted by the gradient-based methods and complemented with uncertainty estimation, provide robust and explainable forecasting for decision-makers.
Partial geometric designs having circulant concurrence matrices

We classify small partial geometric designs (PGDs) by spectral characteristics of their concurrence matrices. It is well known that the concurrence matrix of a PGD can have at most three distinct eigenvalues, all of which are nonnegative integers. The matrix contains useful information on the incidence structure of the design. An ordinary 2‐ ( v , k , λ ) $(v,k,\lambda )$ design has a single concurrence λ $\lambda $ , and its concurrence matrix is circulant, a partial geometry has two concurrences 1 and 0, and a transversal design TD λ ( k , u ) ${\text{TD}}_{\lambda }(k,u)$ has two concurrences λ $\lambda $ and 0, and its concurrence matrix is circulant. In this paper, we survey the known PGDs by highlighting their concurrences and constructions. Then we investigate which symmetric circulant matrices are realized as the concurrence matrices of PGDs. In particular, we try to give a list of all PGDs of order up to 12 each of which has a circulant concurrence matrix. We then describe these designs along with their combinatorial properties and constructions. This work is part of the second author's Ph.D. dissertation [46].