Miracle-dz/UniFAR-FTUs huggingface dataset:
ProtoVAE: Prototypical Networks for Unsupervised Disentanglement

Generative modeling and self-supervised learning have in recent years made great strides towards learning from data in a completely unsupervised way. There is still however an open area of investigation into guiding a neural network to encode the data into representations that are interpretable or explainable. The problem of unsupervised disentanglement is of particular importance as it proposes to discover the different latent factors of variation or semantic concepts from the data alone, without labeled examples, and encode them into structurally disjoint latent representations. Without additional constraints or inductive biases placed in the network, a generative model may learn the data distribution and encode the factors, but not necessarily in a disentangled way. Here, we introduce a novel deep generative VAE-based model, ProtoVAE, that leverages a deep metric learning Prototypical network trained using self-supervision to impose these constraints. The prototypical network constrains the mapping of the representation space to data space to ensure that controlled changes in the representation space are mapped to changes in the factors of variations in the data space. Our model is completely unsupervised and requires no a priori knowledge of the dataset, including the number of factors. We evaluate our proposed model on the benchmark dSprites, 3DShapes, and MPI3D disentanglement datasets, showing state of the art results against previous methods via qualitative traversals in the latent space, as well as quantitative disentanglement metrics. We further qualitatively demonstrate the effectiveness of our model on the real-world CelebA dataset.
ARM’s Aerosol–Cloud–Precipitation Research (Aerosol Indirect Effects)

The conception of the Department of Energy’s (DOE’s) Atmospheric Radiation Measurement (ARM) Program over 20 years ago demonstrated prescience on the part of a number of astute scientists, many of whose words fill the pages of this monograph. The early years focused on a handful of cloud and radiation measurements and activities of relatively limited scope. The intervening decades have seen these efforts expanded to some of the finest instrumentation in theworld tomeasure aerosol, clouds, radiation, and precipitation, accompanied by a substantial modeling effort. Together these have allowed theUnited States and international communities to tackle one of the thorniest problems associated with climate change, namely the influence of aerosol particles on cloud microphysics, precipitation, and cloud radiative properties (aerosol indirect effects). ARMresearchwas at the forefront of aerosol indirect efforts from the outset, but because instrumentation was not readily in place and retrieval methodologies were still in their infancy, these were necessarily modeling efforts (e.g., Ghan et al. 1990; Feingold and Heymsfield 1992; Kim and Cess 1993) that addressed subsets of the problem. These early endeavors joined other key studies highlighting the climate forcing potential of tropospheric aerosol (e.g., Charlson et al. 1992) in setting the stage for a research effort that is, to this day, one of the cornerstones of ARM and the Atmospheric System Research Program (ASR). The goal of this chapter is to summarize ARM and ASR efforts in this realm. Because this chapter deals with measurement and modeling capabilities pertaining to each of the components of the aerosol–cloud– precipitation–radiation system, it rests heavily on other chapters that deal more specifically with each individual component. At the outset we note that the term ‘‘aerosol indirect effects’’ is often used loosely to include all aspects of aerosol–cloud interactions, whereas, by definition, the indirect effect is the radiative effect or forcing associated with these interactions. We will therefore introduce the term aerosol–cloud interactions (ACI) when we refer primarily to the microphysical/dynamical aspects of the problem and reserve ‘‘indirect effects’’ for the radiative forcing. ARM’s early focus on atmospheric radiation measurements, followed by years of refinement of microphysical retrievals, has placed it in an excellent position to address both ACI and the associated indirect effects. ACI or aerosol indirect effects are often used to convey a few underlying microphysical processes. The first is the ‘‘albedo effect’’ (Twomey 1977), which states that an increase in the number of aerosol particles results in more cloud condensation nuclei (CCN), a higher droplet concentration, and, all else being equal (particularly liquid water content), smaller drops and a more reflective cloud. It is a fundamental expression of the ability of aerosol particles to generate a larger drop surface area to volume ratio. The second, the ‘‘lifetime effect’’ (Albrecht 1989), proposes that aerosol suppression
Time-Consistent Asset Allocation for Risk Measures in a L\'evy Market

Focusing on gains instead of terminal wealth, we consider an asset allocation problem to maximize time-consistently a mean-risk reward function with a general risk measure which is i) law-invariant, ii) cash- or shift-invariant, and iii) positively homogeneous, and possibly plugged into a general function. We model the market via a generalized version of the multi-dimensional Black-Scholes model using $\alpha$-stable L\'evy processes and give supplementary results for the classical Black-Scholes model. The optimal solution to this problem is a Nash subgame equilibrium given by the solution of an extended Hamilton-Jacobi-Bellman equation. Moreover, we show that the optimal solution is deterministic and unique under appropriate assumptions.
Easy-to-Hard Learning for Information Extraction

Information extraction (IE) systems aim to automatically extract structured information, such as named entities, relations between entities, and events, from unstructured texts. While most existing work addresses a particular IE task, universally modeling various IE tasks with one model has achieved great success recently. Despite their success, they employ a one-stage learning strategy, i.e., directly learning to extract the target structure given the input text, which contradicts the human learning process. In this paper, we propose a unified easy-to-hard learning framework consisting of three stages, i.e., the easy stage, the hard stage, and the main stage, for IE by mimicking the human learning process. By breaking down the learning process into multiple stages, our framework facilitates the model to acquire general IE task knowledge and improve its generalization ability. Extensive experiments across four IE tasks demonstrate the effectiveness of our framework. We achieve new state-of-the-art results on 13 out of 17 datasets. Our code is available at \url{https://github.com/DAMO-NLP-SG/IE-E2H}.
An Offline Time-aware Apprenticeship Learning Framework for Evolving Reward Functions

Apprenticeship learning (AL) is a process of inducing effective decision-making policies via observing and imitating experts' demonstrations. Most existing AL approaches, however, are not designed to cope with the evolving reward functions commonly found in human-centric tasks such as healthcare, where offline learning is required. In this paper, we propose an offline Time-aware Hierarchical EM Energy-based Sub-trajectory (THEMES) AL framework to tackle the evolving reward functions in such tasks. The effectiveness of THEMES is evaluated via a challenging task -- sepsis treatment. The experimental results demonstrate that THEMES can significantly outperform competitive state-of-the-art baselines.
AAAI 2022 Fall Symposium: System-1 and System-2 realized within the Common Model of Cognition

Attempts to import dual-system descriptions of System-1 and System-2 into AI have been hindered by a lack of clarity over their distinction. We address this and other issues by situating System-1 and System-2 within the Common Model of Cognition. Results show that what are thought to be distinctive characteristics of System-1 and 2 instead form a spectrum of cognitive properties. The Common Model provides a comprehensive vision of the computational units involved in System-1 and System-2, their underlying mechanisms, and the implications for learning, metacognition, and emotion.
Unjustified Sample Sizes and Generalizations in Explainable Artificial Intelligence Research: Principles for More Inclusive User Studies

Many ethical frameworks require artificial intelligence (AI) systems to be explainable. Explainable AI (XAI) models are frequently tested for their adequacy in user studies. Since different people may have different explanatory needs, it is important that participant samples in user studies are large enough to represent the target population to enable generalizations. However, it is unclear to what extent XAI researchers reflect on and justify their sample sizes or avoid broad generalizations across people. We analyzed XAI user studies (n = 220) published between 2012 and 2022. Most of the studies did not offer rationales for their sample sizes. Moreover, most of the papers generalized their conclusions beyond their study population, and there was no evidence that broader conclusions in quantitative studies were correlated with larger samples. These methodological problems can impede evaluations of whether XAI systems implement the explainability called for in ethical frameworks. We outline principles for more inclusive XAI user studies.
Joint Channel Estimation and Turbo Equalization of Single-Carrier Systems over Time-Varying Channels

Block transmission systems have been proven successful over frequency-selective channels. For time-varying channel such as in high-speed mobile communication and underwater communication, existing equalizers assume that channels over different data frames are independent. However, the real-world channels over different data frames are correlated, thereby indicating potentials for performance improvement. In this paper, we propose a joint channel estimation and equalization/decoding algorithm for a single-carrier system that exploits temporal correlations of channel between transmitted data frames. Leveraging the concept of dynamic compressive sensing, our method can utilize the information of several data frames to achieve better performance. The information not only passes between the channel and symbol, but also the channels over different data frames. Numerical simulations using an extensively validated underwater acoustic model with a time-varying channel establish that the proposed algorithm outperforms the former bilinear generalized approximate message passing equalizer and classic minimum mean square error turbo equalizer in bit error rate and channel estimation normalized mean square error. The algorithm idea we present can also find applications in other bilinear multiple measurements vector compressive sensing problems.
Online Continual Learning Without the Storage Constraint

Traditional online continual learning (OCL) research has primarily focused on mitigating catastrophic forgetting with fixed and limited storage allocation throughout an agent's lifetime. However, a broad range of real-world applications are primarily constrained by computational costs rather than storage limitations. In this paper, we target such applications, investigating the online continual learning problem under relaxed storage constraints and limited computational budgets. We contribute a simple algorithm, which updates a kNN classifier continually along with a fixed, pretrained feature extractor. We selected this algorithm due to its exceptional suitability for online continual learning. It can adapt to rapidly changing streams, has zero stability gap, operates within tiny computational budgets, has low storage requirements by only storing features, and has a consistency property: It never forgets previously seen data. These attributes yield significant improvements, allowing our proposed algorithm to outperform existing methods by over 20% in accuracy on two large-scale OCL datasets: Continual LOCalization (CLOC) with 39M images and 712 classes and Continual Google Landmarks V2 (CGLM) with 580K images and 10,788 classes, even when existing methods retain all previously seen images. Furthermore, we achieve this superior performance with considerably reduced computational and storage expenses. We provide code to reproduce our results at github.com/drimpossible/ACM.
Smart Policy Control for Securing Federated Learning Management System

The widespread adoption of Internet of Things (IoT) devices in smart cities, intelligent healthcare systems, and various real-world applications have resulted in the generation of vast amounts of data, often analyzed using different Machine Learning (ML) models. Federated learning (FL) has been acknowledged as a privacy-preserving machine learning technology, where multiple parties cooperatively train ML models without exchanging raw data. However, the current FL architecture does not allow for an audit of the training process due to the various data-protection policies implemented by each FL participant. Furthermore, there is no global model verifiability available in the current architecture. This paper proposes a smart contract-based policy control for securing the Federated Learning (FL) management system. First, we develop and deploy a smart contract-based local training policy control on the FL participants’ side. This policy control is used to verify the training process, ensuring that the evaluation process follows the same rules for all FL participants. We then enforce a smart contract-based aggregation policy to manage the global model aggregation process. Upon completion, the aggregated model and policy are stored on blockchain-based storage. Subsequently, we distribute the aggregated global model and the smart contract to all FL participants. Our proposed method uses smart policy control to manage access and verify the integrity of machine learning models. We conducted multiple experiments with various machine learning architectures and datasets to evaluate our proposed framework, such as MNIST and CIFAR-10.
Delayed Dynamics with Transient Resonating Oscillations

Recently, we have studied a delay differential equation which has a coefficient that is a linear function of time. The equation has shown the oscillatory transient dynamics appear and disappear as the delay is increased between zero to asymptotically large delay. We here propose and study another equation that shows similar transient oscillations. It has an extra exponential gaussian factor on the delayed feedback term. It is shown that this equation is analytically tractable with the use of the Lambert $W$ function. This equation is also studied numerically to confirm some of the properties inferred from the analytical solution. We also have found that the amplitude of transient oscillation changes and goes through a maximum as we increase the value of the delay. In this sense, the proposed equation is one of the simplest dynamical equations that brings out a resonant behavior without any external oscillating inputs.
Finite Difference Solution Ansatz approach in Least-Squares Monte Carlo

This article presents a simple but effective and efficient approach to improve the accuracy and stability of Least-Squares Monte Carlo for American-style option pricing as well as expected exposure calculation in valuation adjustments. The key idea is to construct the ansatz of conditional expected continuation payoff using the exact solution from low dimensional finite difference methods, to be used in linear regression. This approach bridges between solving backward partial differential equations and Monte Carlo simulation, aiming at achieving the best of both worlds. We illustrate the technique with realistic examples including Bermudan options, worst of issuer callable notes and expected positive exposure on European options. The method can be considered as a generic numerical scheme across various asset classes, in particular, as an accurate method for pricing and risk-managing American-style derivatives under arbitrary dimensions.
Interactive and incremental learning of spatial object relations from human demonstrations

Humans use semantic concepts such as spatial relations between objects to describe scenes and communicate tasks such as “Put the tea to the right of the cup” or “Move the plate between the fork and the spoon. ” Just as children, assistive robots must be able to learn the sub-symbolic meaning of such concepts from human demonstrations and instructions. We address the problem of incrementally learning geometric models of spatial relations from few demonstrations collected online during interaction with a human. Such models enable a robot to manipulate objects in order to fulfill desired spatial relations specified by verbal instructions. At the start, we assume the robot has no geometric model of spatial relations. Given a task as above, the robot requests the user to demonstrate the task once in order to create a model from a single demonstration, leveraging cylindrical probability distribution as generative representation of spatial relations. We show how this model can be updated incrementally with each new demonstration without access to past examples in a sample-efficient way using incremental maximum likelihood estimation, and demonstrate the approach on a real humanoid robot.
Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares

We propose a differentiable nonlinear least squares framework to account for uncertainty in relative pose estimation from feature correspondences. Specifically, we introduce a symmetric version of the probabilistic normal epipolar constraint, and an approach to estimate the co-variance of feature positions by differentiating through the camera pose estimation procedure. We evaluate our approach on synthetic, as well as the KITTI and EuRoC real-world datasets. On the synthetic dataset, we confirm that our learned covariances accurately approximate the true noise distribution. In real world experiments, we find that our approach consistently outperforms state-of-the-art non-probabilistic and probabilistic approaches, regardless of the feature extraction algorithm of choice.
Anisoplanatic optical turbulence simulation for near-continuous Cn2 profiles without wave propagation

Abstract. For the simulation of anisoplanatic optical turbulence, split-step propagation is the gold standard. Within the context of the degradations being limited to phase distortions, one instead may focus on generating the phase realizations directly, a method that has been utilized in previous so-called multi-aperture simulations. Presently, this modality assumes a constant Cn2 profile. This work presents an alternative derivation for Zernike correlations under anisoplanatic conditions. Multi-aperture simulation easily incorporates these correlations into its framework and achieve a significantly higher degree of accuracy with a minimal increase in time. We additionally use our developed methodology to explain previously reported discrepancies in an empirical implementation of split-step with the analytic tilt correlation. Finally, we outline a major limitation for Zernike-based simulation that still remains.
Density-dependent incompressible Navier--Stokes equations in critical tent spaces

In this article, we prove the existence of global solutions to the inhomogeneous incompressible Navier--Stokes equations, whenever the initial velocity belongs to some subspace of $\mathrm{BMO}^{-1}$, and the initial density is sufficiently close to $1$ in the uniform metric. This is a natural extension to the variable density case of the celebrated result by H. Koch and D. Tataru concerning the classical Navier-Stokes equations.
Full-physics 3-D heterogeneous simulations of electromagnetic induction fields on level and deformed sea ice

Abstract In this paper we explore simulated responses of electromagnetic (EM) signals relative to in situ field surveys and quantify the effects that different values of conductivity in sea ice have on the EM fields. We compute EM responses of ice types with a three-dimensional (3-D) finite-volume discretization of Maxwell’s equations and present 2-D sliced visualizations of their associated EM fields at discrete frequencies. Several interesting observations result: First, since the simulator computes the fields everywhere, each gridcell acts as a receiver within the model volume, and captures the complete, coupled interactions between air, snow, sea ice and sea water as a function of their conductivity; second, visualizations demonstrate how 1-D approximations near deformed ice features are violated. But the most important new finding is that changes in conductivity affect EM field response by modifying the magnitude and spatial patterns (i.e. footprint size and shape) of current density and magnetic fields. These effects are demonstrated through a visual feature we define as ‘null lines’. Null line shape is affected by changes in conductivity near material boundaries as well as transmitter location. Our results encourage the use of null lines as a planning tool for better ground-truth field measurements near deformed ice types.
Dissertation on Applied Microeconomics of Freemium Pricing Strategies in Mobile App Market

In my dissertation, I will analyze how the product market position of a mobile app affects its pricing strategies, which in turn impacts an app's monetization process. Using natural language processing and k-mean clustering on apps' text descriptions, I created a new variable that measures the distinctiveness of an app as compared to its peers. I created four pricing variables, price, cumulative installs, and indicators of whether an app contains in-app ads and purchases. I found that the effect differs for successful apps and less successful apps. I measure the success here using cumulative installs and the firms that developed the apps. Based on third-party rankings and the shape of the distribution of installs, I set two thresholds and divided apps into market-leading and market-follower apps. The market-leading sub-sample consists of apps with high cumulative installs or developed by prestigious firms, and the market-follower sub-sample consists of the rest of the apps. I found that the impact of being niche is smaller in the market-leading apps because of their relatively higher heterogeneity. In addition, being niche also impact utility apps differently from hedonic apps or apps with two-sided market characteristics. For the special gaming category, being niche has some effect but is smaller than in the market follower sub-sample. My research provides novel empirical evidence of digital products to various strands of theoretical research, including the optimal distinctiveness theory, product differentiation, price discrimination in two or multi-sided markets, and consumer psychology.