Miracle-dz/UniFAR-FTUs huggingface dataset:
DP-PCA: Statistically Optimal and Differentially Private PCA

We study the canonical statistical task of computing the principal component from $n$ i.i. d. ~data in $d$ dimensions under $(\varepsilon,\delta)$-differential privacy. Although extensively studied in literature, existing solutions fall short on two key aspects: ($i$) even for Gaussian data, existing private algorithms require the number of samples $n$ to scale super-linearly with $d$, i.e., $n=\Omega(d^{3/2})$, to obtain non-trivial results while non-private PCA requires only $n=O(d)$, and ($ii$) existing techniques suffer from a non-vanishing error even when the randomness in each data point is arbitrarily small. We propose DP-PCA, which is a single-pass algorithm that overcomes both limitations. It is based on a private minibatch gradient ascent method that relies on {\em private mean estimation}, which adds minimal noise required to ensure privacy by adapting to the variance of a given minibatch of gradients. For sub-Gaussian data, we provide nearly optimal statistical error rates even for $n=\tilde O(d)$. Furthermore, we provide a lower bound showing that sub-Gaussian style assumption is necessary in obtaining the optimal error rate.
MIMO Integrated Sensing and Communication with Extended Target: CRB-Rate Tradeoff

This paper studies a multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system, in which a multi-antenna base station (BS) sends unified wireless signals to estimate an extended target and communicate with a multi-antenna communication user (CU) at the same time. We investigate the fundamental tradeoff between the estimation Cramér-Rao bound (CRB) for sensing and the data rate for communication, by characterizing the Pareto boundary of the achievable CRB-rate (C-R) region. Towards this end, we formulate a new MIMO rate maximization problem by optimizing the transmit covariance matrix at the BS, subject to a new form of maximum CRB constraint together with a maximum transmit power constraint. We derive the optimal transmit covariance solution in a semi-closed form, by first implementing the singular-value decomposition (SVD) to diagonalize the communication channel and then properly allocating the transmit power over these subchannels for communication and other orthogonal subchannels (if any) for dedicated sensing. It is shown that the optimal transmit covariance is of full rank, which unifies the conventional rate maximization design with water-filling power allocation and the CRB minimization design with isotropic transmission. Numerical results are provided to validate the performance achieved by our proposed optimal design, in comparison with other benchmark schemes.
NLU for Game-based Learning in Real: Initial Evaluations

Intelligent systems designed for play-based interactions should be contextually aware of the users and their surroundings. Spoken Dialogue Systems (SDS) are critical for these interactive agents to carry out effective goal-oriented communication with users in real-time. For the real-world (i.e., in-the-wild) deployment of such conversational agents, improving the Natural Language Understanding (NLU) module of the goal-oriented SDS pipeline is crucial, especially with limited task-specific datasets. This study explores the potential benefits of a recently proposed transformer-based multi-task NLU architecture, mainly to perform Intent Recognition on small-size domain-specific educational game datasets. The evaluation datasets were collected from children practicing basic math concepts via play-based interactions in game-based learning settings. We investigate the NLU performances on the initial proof-of-concept game datasets versus the real-world deployment datasets and observe anticipated performance drops in-the-wild. We have shown that compared to the more straightforward baseline approaches, Dual Intent and Entity Transformer (DIET) architecture is robust enough to handle real-world data to a large extent for the Intent Recognition task on these domain-specific in-the-wild game datasets.
Unequal Covariance Awareness for Fisher Discriminant Analysis and Its Variants in Classification

Fisher Discriminant Analysis (FDA) is one of the essential tools for feature extraction and classification. In addition, it motivates the development of many improved techniques based on the FDA to adapt to different problems or data types. However, none of these approaches make use of the fact that the assumption of equal covariance matrices in FDA is usually not satisfied in practical situations. Therefore, we propose a novel classification rule for the FDA that accounts for this fact, mitigating the effect of unequal covariance matrices in the FDA. Furthermore, since we only modify the classification rule, the same can be applied to many FDA variants, improving these algorithms further. Theoretical analysis reveals that the new classification rule allows the implicit use of the class covariance matrices while increasing the number of parameters to be estimated by a small amount compared to going from FDA to Quadratic Discriminant Analysis. We illustrate our idea via experiments, which shows the superior performance of the modified algorithms based on our new classification rule compared to the original ones.
UAlberta at SemEval 2022 Task 2: Leveraging Glosses and Translations for Multilingual Idiomaticity Detection

We describe the University of Alberta systems for the SemEval-2022 Task 2 on multilingual idiomaticity detection. Working under the assumption that idiomatic expressions are noncompositional, our first method integrates information on the meanings of the individual words of an expression into a binary classifier. Further hypothesizing that literal and idiomatic expressions translate differently, our second method translates an expression in context, and uses a lexical knowledge base to determine if the translation is literal. Our approaches are grounded in linguistic phenomena, and leverage existing sources of lexical knowledge. Our results offer support for both approaches, particularly the former.
Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts

Conversational Agents (CAs) powered with deep language models (DLMs) have shown tremendous promise in the domain of mental health. Prominently, the CAs have been used to provide informational or therapeutic services (e.g., cognitive behavioral therapy) to patients. However, the utility of CAs to assist in mental health triaging has not been explored in the existing work as it requires a controlled generation of follow-up questions (FQs), which are often initiated and guided by the mental health professionals (MHPs) in clinical settings. In the context of ‘depression’, our experiments show that DLMs coupled with process knowledge in a mental health questionnaire generate 12.54% and 9.37% better FQs based on similarity and longest common subsequence matches to questions in the PHQ-9 dataset respectively, when compared with DLMs without process knowledge support.Despite coupling with process knowledge, we find that DLMs are still prone to hallucination, i.e., generating redundant, irrelevant, and unsafe FQs. We demonstrate the challenge of using existing datasets to train a DLM for generating FQs that adhere to clinical process knowledge. To address this limitation, we prepared an extended PHQ-9 based dataset, PRIMATE, in collaboration with MHPs. PRIMATE contains annotations regarding whether a particular question in the PHQ-9 dataset has already been answered in the user’s initial description of the mental health condition. We used PRIMATE to train a DLM in a supervised setting to identify which of the PHQ-9 questions can be answered directly from the user’s post and which ones would require more information from the user. Using performance analysis based on MCC scores, we show that PRIMATE is appropriate for identifying questions in PHQ-9 that could guide generative DLMs towards controlled FQ generation (with minimal hallucination) suitable for aiding triaging. The dataset created as a part of this research can be obtained from https://github.com/primate-mh/Primate2022
Efficient adiabatic rapid passage in the presence of noise

Adiabatic Rapid Passage (ARP) is a powerful technique for efficient transfer of population between quantum states. In the lab, the efficiency of ARP is often limited by noise on either the energies of the states or the frequency of the driving field. We study ARP in the simple setting of a two-level system subject to sinusoidal fluctuations on the energy level separation by numerically solving the optical Bloch equations in the absence of damping. We investigate the dependence of the efficiency of population transfer on the frequency and amplitude of the perturbation, and find that it is predominantly affected by resonant coupling when the detuning matches the frequency of the noise. We present intuitive principles for when ARP becomes inefficient within this model, and provide a sufficient condition for the population transfer to be above an arbitrary threshold.
Viscoelastic Damping Technologies-Part II: Experimental Identification Procedure and Validation

c � HyperSciences.Publisher Abstract: This is the second of two companion articles addressing an integrated study on the mathematical modeling and assessment of the efficiency of surface mounted or embedded viscoelastic damping treatments, typically used to reduce structural vibration and/or noise radi- ation from structures, incorporating the adequate use and development of viscoelastic (arbitrary frequency dependent) damping models, along with their finite element (FE) implementation, and the experimental identification of the constitutive behavior of viscoelastic materials. In the first article (Vasques, C.M.A. et al., Viscoelastic damping technologies-Part I: Modeling and finite element implementation, Journal of Advanced Research in Mechanical Engineering 1(2): 76-95 (2010)) viscoelastic damping has been tackled from a mathematical point of view and the implementation, at the global FE model level, of time and frequency domain methods, namely the internal variables models, Golla-Hughes-McTavish (GHM) and anelastic displacement fields (ADF), and the complex modulus approach based ones, direct frequency response (DFR), iterative modal strain energy (IMSE) and an iterative complex eigensolution (ICE), respectively, were described and formulated. This second article is a natural extension of the first one. It presents a generic methodology to identify the complex shear modulus of viscoelastic materials. In this case, the complex shear modulus of the well-known viscoelastic material 3M ISD112 is identified and up-to-date values for this material are used and curve-fitted in order to obtain the modeling parameters of the GHM and ADF models. Afterward, a viscoelastic sandwich (three-layered) plate specimen and the correspondent FE model are considered numerically and experimentally. Measured and predicted frequency response functions (FRFs) are compared with the purpose of assessing the performance of the damping models presented in the companion article. The analysis allows to assess the validity of the methodology to determine the frequency dependent complex modulus, the GHM and ADF parameters identification procedure and the outcomes and drawbacks of the DFR, IMSE, ICE, GHM and ADF viscoelastic damping modeling strategies and their FE implementations, with the aim of assisting structural designers in the selection of the most appropriate viscoelastic damping modeling approach for their specific needs.
Improvements of LHC data analysis techniques at Italian WLCG sites. Case-study of the transfer of this technology to other research areas

In 2012, 14 Italian institutions participating in LHC Experiments won a grant from the Italian Ministry of Research (MIUR), with the aim of optimising analysis activities, and in general the Tier2/Tier3 infrastructure. We report on the activities being researched upon, on the considerable improvement in the ease of access to resources by physicists, also those with no specific computing interests. We focused on items like distributed storage federations, access to batch-like facilities, provisioning of user interfaces on demand and cloud systems. R&D on next-generation databases, distributed analysis interfaces, and new computing architectures was also carried on. The project, ending in the first months of 2016, will produce a white paper with recommendations on best practices for data-analysis support by computing centers.
Deep Ensembles for Graphs with Higher-order Dependencies

Graph neural networks (GNNs) continue to achieve state-of-the-art performance on many graph learning tasks, but rely on the assumption that a given graph is a sufficient approximation of the true neighborhood structure. When a system contains higher-order sequential dependencies, we show that the tendency of traditional graph representations to underfit each node's neighborhood causes existing GNNs to generalize poorly. To address this, we propose a novel Deep Graph Ensemble (DGE), which captures neighborhood variance by training an ensemble of GNNs on different neighborhood subspaces of the same node within a higher-order network structure. We show that DGE consistently outperforms existing GNNs on semisupervised and supervised tasks on six real-world data sets with known higher-order dependencies, even under a similar parameter budget. We demonstrate that learning diverse and accurate base classifiers is central to DGE's success, and discuss the implications of these findings for future work on ensembles of GNNs.
Efficient Forecasting of Large Scale Hierarchical Time Series via Multilevel Clustering

We propose a novel approach to the problem of clustering hierarchically aggregated time-series data, which has remained an understudied problem though it has several commercial applications. We first group time series at each aggregated level, while simultaneously leveraging local and global information. The proposed method can cluster hierarchical time series (HTS) with different lengths and structures. For common two-level hierarchies, we employ a combined objective for local and global clustering over spaces of discrete probability measures, using Wasserstein distance coupled with Soft-DTW divergence. For multi-level hierarchies, we present a bottom-up procedure that progressively leverages lower-level information for higher-level clustering. Our final goal is to improve both the accuracy and speed of forecasts for a larger number of HTS needed for a real-world application. To attain this goal, each time series is first assigned the forecast for its cluster representative, which can be considered as a"shrinkage prior"for the set of time series it represents. Then this base forecast can be quickly fine-tuned to adjust to the specifics of that time series. We empirically show that our method substantially improves performance in terms of both speed and accuracy for large-scale forecasting tasks involving much HTS.
Vetting the “Lobster” Diagram: Searching for Unseen Companions in Wide Binaries Using NASA Space Exoplanet Missions

Over the past decade, the number of known wide-binary systems has expanded exponentially, thanks to the release of data from the Gaia Mission. Some of these wide-binary systems are actually higher-order multiples, where one of the components is an unresolved binary itself. One way to search for these systems is by identifying the overluminous components in the systems. In this study, we examine 4947 K+K wide-binary pairs from the SUPERWIDE catalog, and quantify the relative colors and luminosities of the components to find evidence for additional unresolved companions. The method is best illustrated in a graph that we call the “Lobster” diagram. To confirm that the identified overluminous components are close binary systems, we cross-match our wide binaries with the TESS, K2, and Kepler archives, and search for signs of eclipses and fast stellar rotation modulation in the light curves. We find that 78.9% ± 20.7% of the wide binaries that contain an eclipsing system are identified as overluminous in the “Lobster” diagram, and that 73.5% ± 12.4% of the wide binaries that contain a component showing fast rotation (P < 5 days) also show an overluminous component. From these results, we calculate a revised lower limit on the higher-order multiplicity fraction for K+K wide binaries of 40.0% ± 1.6%. We also examine the higher-order multiplicity fraction as a function of projected physical separation and metallicity. The fraction is unusually constant as a function of projected physical separation, while we see no statistically significant evidence that the fraction varies with metallicity.
Mixed Federated Learning: Joint Decentralized and Centralized Learning

Federated learning (FL) enables learning from decentralized privacy-sensitive data, with computations on raw data confined to take place at edge clients. This paper introduces mixed FL, which incorporates an additional loss term calculated at the coordinating server (while maintaining FL's private data restrictions). There are numerous benefits. For example, additional datacenter data can be leveraged to jointly learn from centralized (datacenter) and decentralized (federated) training data and better match an expected inference data distribution. Mixed FL also enables offloading some intensive computations (e.g., embedding regularization) to the server, greatly reducing communication and client computation load. For these and other mixed FL use cases, we present three algorithms: PARALLEL TRAINING, 1-WAY GRADIENT TRANSFER, and 2-WAY GRADIENT TRANSFER. We state convergence bounds for each, and give intuition on which are suited to particular mixed FL problems. Finally we perform extensive experiments on three tasks, demonstrating that mixed FL can blend training data to achieve an oracle's accuracy on an inference distribution, and can reduce communication and computation overhead by over 90%. Our experiments confirm theoretical predictions of how algorithms perform under different mixed FL problem settings.
Meta-Learning Adversarial Bandits

We study online learning with bandit feedback across multiple tasks, with the goal of improving average performance across tasks if they are similar according to some natural task-similarity measure. As the first to target the adversarial setting, we design a unified meta-algorithm that yields setting-specific guarantees for two important cases: multi-armed bandits (MAB) and bandit linear optimization (BLO). For MAB, the meta-algorithm tunes the initialization, step-size, and entropy parameter of the Tsallis-entropy generalization of the well-known Exp3 method, with the task-averaged regret provably improving if the entropy of the distribution over estimated optima-in-hindsight is small. For BLO, we learn the initialization, step-size, and boundary-offset of online mirror descent (OMD) with self-concordant barrier regularizers, showing that task-averaged regret varies directly with a measure induced by these functions on the interior of the action space. Our adaptive guarantees rely on proving that unregularized follow-the-leader combined with multiplicative weights is enough to online learn a non-smooth and non-convex sequence of affine functions of Bregman divergences that upper-bound the regret of OMD.
Fairness in Recommendation: A Survey

Recently, there has been growing attention on fairness considerations in recommender systems with more and more literature on approaches to promote fairness in recommendation. However, the studies are rather fragmented and lack a systematic organization, thus making it difficult to penetrate for new researchers to the domain. This motivates us to provide a systematic survey of existing works on fairness in recommendation. This survey focuses on the foundations for fairness in recommendation literature. It first presents a brief introduction about fairness in basic machine learning tasks such as classification and ranking in order to provide a general overview of fairness research, as well as introduce the more complex situations and challenges that need to be considered when studying fairness in recommender systems. After that, the survey will introduce fairness in recommendation with a focus on the taxonomies of current fairness definitions, the typical techniques for improving fairness, as well as the datasets for fairness studies in recommendation. The survey also talks about the challenges and opportunities in fairness research with the hope of promoting the fair recommendation research area and beyond.
Looking at Creative ML Blindspots with a Sociological Lens

How can researchers from the creative ML/AI community and sociology of culture engage in fruitful collaboration? How do researchers from both fields think (differently) about creativity and the production of creative work? While the ML community considers creativity as a matter of technical expertise and acumen, social scientists have emphasized the role of embeddedness in cultural production. This perspective aims to bridge both disciplines and proposes a conceptual and methodological toolkit for collaboration. We provide a systematic review of recent research in both fields and offer three perspectives around which to structure interdisciplinary research on cultural production: people, processes, and products. We thereby provide necessary grounding work to support multidisciplinary researchers to navigate conceptual and methodological hurdles in their collaboration. Our research will be of interest to ML researchers and sociologists interested in creativity that aim to conduct innovative research by bridging both fields.
Counterfactual Fairness with Partially Known Causal Graph

Fair machine learning aims to avoid treating individuals or sub-populations unfavourably based on \textit{sensitive attributes}, such as gender and race. Those methods in fair machine learning that are built on causal inference ascertain discrimination and bias through causal effects. Though causality-based fair learning is attracting increasing attention, current methods assume the true causal graph is fully known. This paper proposes a general method to achieve the notion of counterfactual fairness when the true causal graph is unknown. To be able to select features that lead to counterfactual fairness, we derive the conditions and algorithms to identify ancestral relations between variables on a \textit{Partially Directed Acyclic Graph (PDAG)}, specifically, a class of causal DAGs that can be learned from observational data combined with domain knowledge. Interestingly, we find that counterfactual fairness can be achieved as if the true causal graph were fully known, when specific background knowledge is provided: the sensitive attributes do not have ancestors in the causal graph. Results on both simulated and real-world datasets demonstrate the effectiveness of our method.