Miracle-dz/UniFAR-FTUs huggingface dataset:
Provably Adversarially Robust Nearest Prototype Classifiers

Nearest prototype classiﬁers (NPCs) assign to each input point the label of the nearest prototype with respect to a chosen distance metric. A direct advantage of NPCs is that the decisions are interpretable. Previous work could provide lower bounds on the minimal adversarial perturbation in the (cid:96) p -threat model when using the same (cid:96) p distance for the NPCs. In this paper we provide a complete discussion on the complexity when using (cid:96) p -distances for decision and (cid:96) q -threat models for certiﬁcation for p, q ∈ { 1 , 2 , ∞} . In particular we provide scalable algorithms for the exact computation of the minimal adversarial perturbation when using (cid:96) 2 -distance and improved lower bounds in other cases. Using efﬁcient improved lower bounds we train our P rovably adversarially robust NPC (PNPC), for MNIST which have better (cid:96) 2 -robustness guarantees than neural networks. Additionally, we show up to our knowledge the ﬁrst certiﬁcation results w.r. t. to the LPIPS perceptual metric which has been argued to be a more realistic threat model for image classiﬁcation than (cid:96) p -balls. Our PNPC has on CIFAR10 higher certiﬁed robust accuracy than the empirical robust accuracy reported in (Laidlaw et al., 2021). The code is available in our repository.
Dimension-free Complexity Bounds for High-order Nonconvex Finite-sum Optimization

Stochastic high-order methods for ﬁnding ﬁrst-order stationary points in nonconvex ﬁnite-sum optimization have witnessed increasing interest in recent years, and various upper and lower bounds of the oracle complexity have been proved. However, under standard regularity assumptions, existing complexity bounds are all dimension-dependent (e.g., polylogarithmic dependence), which contrasts with the dimension-free complexity bounds for stochastic ﬁrst-order methods and deterministic high-order methods. In this paper, we show that the polylogarithmic dimen-sion dependence gap is not essential and can be closed. More speciﬁcally, we propose stochastic high-order algorithms with novel ﬁrst-order and high-order derivative estimators, which can achieve dimension-free complexity bounds. With the access to p -th order derivatives of the objec-tive function, we prove that our algorithm ﬁnds (cid:15) - stationary points with O ( n (2 p − 1) / (2 p ) /(cid:15) ( p +1) /p ) high-order oracle complexities, where n is the number of individual functions. Our result strictly improves the complexity bounds of existing high-order deterministic methods with respect to the dependence on n , and it is dimension-free compared with existing stochastic high-order methods.
Making Linear MDPs Practical via Contrastive Representation Learning

It is common to address the curse of dimensionality in Markov decision processes (MDPs) by exploiting low-rank representations. This motivates much of the recent theoretical study on linear MDPs. However, most approaches require a given representation under unrealistic assumptions about the normalization of the decomposition or introduce unresolved computational challenges in practice. Instead, we consider an alternative definition of linear MDPs that automatically ensures normalization while allowing efficient representation learning via contrastive estimation. The framework also admits confidence-adjusted index algorithms, enabling an efficient and principled approach to incorporating optimism or pessimism in the face of uncertainty. To the best of our knowledge, this provides the first practical representation learning method for linear MDPs that achieves both strong theoretical guarantees and empirical performance. Theoretically, we prove that the proposed algorithm is sample efficient in both the online and offline settings. Empirically, we demonstrate superior performance over existing state-of-the-art model-based and model-free algorithms on several benchmarks.
Fairness Interventions as (Dis)Incentives for Strategic Manipulation

Although machine learning (ML) algorithms are widely used to make decisions about individuals in various domains, concerns have arisen that (1) these algorithms are vulnerable to strategic manipulation and “gaming the algorithm”; and (2) ML decisions may exhibit bias against certain social groups. Existing works have largely examined these as two separate issues, e.g., by focusing on building ML algorithms robust to strategic manipulation, or on training a fair ML algorithm. In this study, we set out to understand the impact they each have on the other, and examine how to design fair algorithms in the presence of strategic behavior. The strategic interaction between a decision maker and individuals (as decision takers) is modeled as a two-stage (Stackelberg) game; when designing an algorithm, the former anticipates the latter may manipulate their features in order to receive more favorable decisions. We analytically characterize the equilibrium strategies of both, and examine how the algorithms and their resulting fairness properties are affected when the decision maker is strategic (anticipates manipulation), as well as the impact of fairness interventions on equilibrium strategies. In particular, we identify conditions under which anticipation of strategic behavior may mitigate/exacerbate unfairness, and conditions under which fairness interventions can serve as incentives/disincentives for strategic manipulation.
LCANets: Lateral Competition Improves Robustness Against Corruption and Attack

Although Convolutional Neural Networks (CNNs) achieve high accuracy on image recognition tasks, they lack robustness against realistic corruptions and fail catastrophically when deliberately attacked. Previous CNNs with representations similar to primary visual cortex (V1) were more robust to adversarial attacks on images than current adversarial defense techniques, but they required training on large-scale neural recordings or handcrafting neuroscientiﬁc models. Motivated by evidence that neural activity in V1 is sparse, we develop a class of hybrid CNNs, called LCANets, which feature a frontend that performs sparse coding via local lateral competition. We demonstrate that LCANets achieve competitive clean accuracy to standard CNNs on action and image recognition tasks and signiﬁcantly greater accuracy under various image corruptions. We also perform the ﬁrst adversarial attacks with full knowledge of a sparse coding CNN layer by attacking LCANets with white-box and black-box attacks, and we show that, contrary to previous hypotheses, sparse coding layers are not very robust to white-box attacks. Finally, we propose a way to use sparse coding layers as a plug-and-play robust frontend by showing that they signiﬁcantly increase the robustness of adversarially-trained CNNs over corruptions and attacks.
Nonparametric Embeddings of Sparse High-Order Interaction Events

High-order interaction events are common in real-world applications. Learning embeddings that encode the complex relationships of the participants from these events is of great importance in knowledge mining and predictive tasks. Despite the success of existing approaches, e.g. Poisson tensor factorization, they ignore the sparse structure underlying the data, namely the occurred interactions are far less than the possible interactions among all the participants. In this paper, we propose Nonparametric Embeddings of Sparse High-order interaction events (NESH). We hybridize a sparse hypergraph (tensor) process and a matrix Gaussian process to capture both the asymptotic structural sparsity within the interactions and nonlinear temporal relationships between the participants. We prove strong asymptotic bounds (including both a lower and an upper bound) of the sparsity ratio, which reveals the asymptotic properties of the sampled structure. We use batch-normalization, stick-breaking construction, and sparse variational GP approximations to develop an efficient, scalable model inference algorithm. We demonstrate the advantage of our approach in several real-world applications.
COAT: Measuring Object Compositionality in Emergent Representations

Learning representations that can decompose a multi-object scene into its constituent objects and recompose them flexibly is desirable for object-oriented reasoning and planning. Built upon object masks in the pixel space, existing metrics for objectness can only evaluate generative models with an object-specific “slot” structure. We propose to directly measure compositionality in the representation space as a form of objectness, making such evaluations tractable for a wider class of models. Our metric, COAT (Compositional Object Algebra Test), evaluates if a generic representation exhibits certain geometric properties that underpin object compositionality beyond what is already captured by the raw pixel space. Our experiments on the popular CLEVR (Johnson et.al., 2018) domain reveal that existing disentanglement-based generative models are not as compositional as one might expect, suggesting room for further modeling improvements. We hope our work allows for a unified evaluation of object-centric representations, spanning generative as well as discriminative, self-supervised models.
On Implicit Bias in Overparameterized Bilevel Optimization

Many problems in machine learning involve bilevel optimization (BLO), including hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems consist of two nested sub-problems, called the outer and inner problems, respectively. In practice, often at least one of these sub-problems is overparameterized. In this case, there are many ways to choose among optima that achieve equivalent objective values. Inspired by recent studies of the implicit bias induced by optimization algorithms in single-level optimization, we investigate the implicit bias of gradient-based algorithms for bilevel optimization. We delineate two standard BLO methods -- cold-start and warm-start -- and show that the converged solution or long-run behavior depends to a large degree on these and other algorithmic choices, such as the hypergradient approximation. We also show that the inner solutions obtained by warm-start BLO can encode a surprising amount of information about the outer objective, even when the outer parameters are low-dimensional. We believe that implicit bias deserves as central a role in the study of bilevel optimization as it has attained in the study of single-level neural net optimization.
Resilient and Communication Efficient Learning for Heterogeneous Federated Systems

The rise of Federated Learning (FL) is bringing machine learning to edge computing by utilizing data scattered across edge devices. However, the heterogeneity of edge network topologies and the uncertainty of wireless transmission are two major obstructions of FL's wide application in edge computing, leading to prohibitive convergence time and high communication cost. In this work, we propose an FL scheme to address both challenges simultaneously. Specifically, we enable edge devices to learn self-distilled neural networks that are readily prunable to arbitrary sizes, which capture the knowledge of the learning domain in a nested and progressive manner. Not only does our approach tackle system heterogeneity by serving edge devices with varying model architectures, but it also alleviates the issue of connection uncertainty by allowing transmitting part of the model parameters under faulty network connections, without wasting the contributing knowledge of the transmitted parameters. Extensive empirical studies show that under system heterogeneity and network instability, our approach demonstrates significant resilience and higher communication efficiency compared to the state-of-the-art.
Searching for BurgerFormer with Micro-Meso-Macro Space Design

With the success of Transformers in the computer vision field, the automated design of vision Transformers has attracted significant attention. Recently, MetaFormer found that simple average pooling can achieve impressive performance, which naturally raises the question of how to design a search space to search diverse and high-performance Transformer-like architectures. By revisiting typical search spaces, we design micro-meso-macro space to search for Transformer-like architectures, namely Burger-Former. Micro, meso, and macro correspond to the granularity levels of operation, block and stage, respectively. At the microscopic level, we enrich the atomic operations to include various normalizations, activation functions, and basic operations (e.g., multi-head self attention, average pooling). At the mesoscopic level, a ham-burger structure is searched out as the basic Burg-erFormer block. At the macroscopic level, we search for the depth, width, and expansion ratio of the network based on the multi-stage architecture. Meanwhile, we propose a hybrid sampling method for effectively training the supernet. Experimental results demonstrate that the searched BurgerFormer architectures achieve comparable even superior performance compared with current state-of-the-art Transformers on the ImageNet and COCO datasets. The codes can be available at https://github.com/xingxing-123/BurgerFormer.
Quarterly Projection Model for Vietnam: A Hybrid Approach for Monetary Policy Implementation

Papers research in progress by the author(s) and are published to elicit comments and to encourage debate. The views expressed in IMF Working Papers are those of the author(s) and do not necessarily represent the views of the IMF, its Executive Board, or IMF management. ABSTRACT: We present a newly developed Quarterly Projection Model (QPM) for Vietnam. This QPM represents an extended version of the canonical New Keynesian semi-structural model, accounting for Vietnam-specific factors, including a hybrid monetary policy framework. The model incorporates the array of policy instruments, specifically interest rates, indicative nominal credit growth guidance, and exchange rate interventions, that the authorities employ to meet the primary objective of price stability. The calibrated model embeds a theoretically consistent monetary transmission mechanism and demonstrates robust in-sample forecasting accuracy, both of which are important prerequisites for the richer analysis and forecast-based narratives that support a forward-looking monetary policy regime.
D1-D3 (or ) systems with fluxes

In this article we study D1-D3 (or ) brane systems with generic constant electric and magnetic fluxes in IIB string theory. We work out all possible supersymmetric configurations and find out via T-duality all of them and corresponding supersymmetry conditions could be related to the supersymmetric intersecting D1-D1 pairs. And we do D1-D3 (or ) open string quantization for a class of configurations. We find that there are many near massless states in NS sector for near-BPS configurations. Furthermore we calculate open string pair creation rate in generic nonsupersymmetric configurations.
Isoprene and monoterpene simulations using the chemistry–climate model EMAC (v2.55) with interactive vegetation from LPJ-GUESS (v4.0)

Abstract. Earth system models (ESMs) integrate previously separate models of the ocean, atmosphere and vegetation into one comprehensive modelling system enabling the investigation of interactions between different components of the Earth system. Global isoprene and monoterpene emissions from terrestrial vegetation, which represent the most important source of volatile organic compounds (VOCs) in the Earth system, need to be included in global and regional chemical transport models given their major chemical impacts on the atmosphere. Due to the feedback of vegetation activity involving interactions with weather and climate, a coupled modelling system between vegetation and atmospheric chemistry is recommended to address the fate of biogenic volatile organic compounds (BVOCs). In this work, further development in linking LPJ-GUESS, a global dynamic vegetation model, to the atmospheric-chemistry-enabled atmosphere–ocean general circulation model EMAC is presented. New parameterisations are included to calculate the foliar density and leaf area density (LAD) distribution from LPJ-GUESS information. The new vegetation parameters are combined with existing LPJ-GUESS output (i.e. leaf area index and cover fractions) and used in empirically based BVOC modules in EMAC. Estimates of terrestrial BVOC emissions from EMAC's submodels ONEMIS and MEGAN are evaluated using (1) prescribed climatological vegetation boundary conditions at the land–atmosphere interface and (2) dynamic vegetation states calculated in LPJ-GUESS (replacing the “offline” vegetation inputs). LPJ-GUESS-driven global emission estimates for isoprene and monoterpenes from the submodel ONEMIS were 546 and 102 Tg yr−1, respectively. MEGAN determines 657 and 55 Tg of isoprene and monoterpene emissions annually. The new vegetation-sensitive BVOC fluxes in EMAC are in good agreement with emissions from the semi-process-based module in LPJ-GUESS. The new coupled system is used to evaluate the temperature and vegetation sensitivity of BVOC fluxes in doubling CO2 scenarios. This work provides evidence that the new coupled model yields suitable estimates for global BVOC emissions that are responsive to vegetation dynamics. It is concluded that the proposed model set-up is useful for studying land–biosphere–atmosphere interactions in the Earth system.
NOTES D’ÉTUDES ET DE RECHERCHE

This paper investigates the effects of disinflation policies on key macroeconomic variables. Using postwar US data and episode techniques, we identify disinflation shocks as shocks that drive the inflation rate to a lower level in the long–run. We find that in the immediate aftermath of a disinflation policy, the economy enters in a persistent recession. The inflation rate increases above its long–run level and exhibits a positive hump–shaped response. A similar pattern is found for the nominal interest rate, which responds even more strongly in the short–run. We then show that the standard new Keynesian model fails to account for macroeconomic dynamics in disinflationary times. On the contrary a deep habit version of the model successfully accounts for the effects of disinflation policies.
Engagement Challenges as Generative Space

This research offers a contextual snapshot—via the applied tensional analysis (ATA) method— of common barriers that educators face when taking on community engagement work. A six-person research team interviewed 22 faculty and staff actively pursuing community partnerships at three different institutions of higher education in a single region. Participants’ responses were coded to identify common barriers to engagement and the strategies that participants used to overcome those barriers. We draw on ATA (Mease, 2019) as a methodological approach that acknowledges tensions as critical entry points for understanding both the experiences of individual actors and the malleable structures from which those tensions emerge. This article does not attempt to offer one-size-fits-all strategies for community engagement. Instead, it offers ATA as a means for contextualizing specific tensions and strategies within local community engagement settings and for reframing those tensions as generative spaces for community engagement work.
Toward Full Accounting for Leakage Exploitation and Mitigation in Dynamic Encrypted Databases

—Encrypted database draws much attention as it provides privacy-protection services for sensitive data outsourced to a third party. Recent studies show that the security guarantee of encrypted databases are challenged by several leakage-abuse attacks on its search module, and corresponding countermeasures are also proposed. Most of these studies focus on static databases, yet the case for dynamic has not been well investigated. To ﬁll this gap, in this paper, we focus on exploring privacy risks in dynamic encrypted databases and devising effective mitigation techniques. To begin with, we systematically study the exploitable information disclosed during the database querying process, and consider two types of attacks that can recover encrypted queries. The ﬁrst active attack works by injecting encoded ﬁles and correlating ﬁle volume information. The second passive attack works by identifying queries’ unique relational characteristics across updates, assuming certain background knowledge of plaintext databases. To mitigate these attacks, we propose a two-layer encrypted database hardening approach, which obfuscates both search indexes and ﬁles in a continuous way. As a result, the unique characteristics emerging after data updates can be eliminated constantly. We conduct a series of experiments to conﬁrm the severity of our attacks and the effectiveness of our countermeasures.
On bargaining sets for ﬁnite economies

We deﬁne a bargaining set for ﬁnite economies using Aubin’s veto mechanism and show its coincidence with the set of Walrasian allocations. Then, we rewrite our notion in terms of replicated economies showing that, in contrast with Anderson, Trockel and Zhou’s (1997) non-convergence result, this Edgeworth bargaining set shrinks to the set of Walrasian allocations.
ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scale

Incremental learning is one paradigm to enable model building and updating at scale with streaming data. For end-to-end automatic speech recognition (ASR) tasks, the absence of human annotated labels along with the need for privacy preserving policies for model building makes it a daunting challenge. Motivated by these challenges, in this paper we use a cloud based framework for production systems to demonstrate insights from privacy preserving incremental learning for automatic speech recognition (ILASR). By privacy preserving, we mean, usage of ephemeral data which are not human annotated. This system is a step forward for production level ASR models for incremental/continual learning that offers near real-time test-bed for experimentation in the cloud for end-to-end ASR, while adhering to privacy-preserving policies. We show that the proposed system can improve the production models significantly ($3%$) over a new time period of six months even in the absence of human annotated labels with varying levels of weak supervision and large batch sizes in incremental learning. This improvement is $20%$ over test sets with new words and phrases in the new time period. We demonstrate the effectiveness of model building in a privacy-preserving incremental fashion for ASR while further exploring the utility of having an effective teacher model and use of large batch sizes.
Is Bigger Always Better? Lessons Learnt from the Evolution of Is Bigger Always Better? Lessons Learnt from the Evolution of Deep Learning Architectures for Image Classification Deep Learning Architectures for Image Classification

There exist numerous scientific contributions to the design of deep learning networks. However, using the right architecture that is suited for a given business problem with all constraints such as memory and inference time requirements can be cumbersome. We reflect on the evolution of the state-of-the-art of architectures for convolutional neural networks (CNN) for the case of image classification. We compare architectures regarding classification results, model size, and inference time to discuss the choices of designs for CNN architectures. To maintain scientific comprehensibility, the established ILSVRC benchmark is used as a basis for model selection and benchmark data. The quantitative comparison shows that while the model size and the required inference time correlate with result accuracy across all architectures, there are major trade-offs between those factors. The qualitative analysis further depicts that published models always build on previous research and adopt improved components in either evolutionary or revolutionary ways. Finally, we discuss design and result improvement during the evolution of CNN architectures. Further, we derive practical implications for designing deep learning networks.