Miracle-dz/UniFAR-FTUs huggingface dataset:
Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly

Machine learning has advanced dramatically, narrowing the accuracy gap to humans in multimodal tasks like visual question answering (VQA). However, while humans can say"I don't know"when they are uncertain (i.e., abstain from answering a question), such ability has been largely neglected in multimodal research, despite the importance of this problem to the usage of VQA in real settings. In this work, we promote a problem formulation for reliable VQA, where we prefer abstention over providing an incorrect answer. We first enable abstention capabilities for several VQA models, and analyze both their coverage, the portion of questions answered, and risk, the error on that portion. For that, we explore several abstention approaches. We find that although the best performing models achieve over 70% accuracy on the VQA v2 dataset, introducing the option to abstain by directly using a model's softmax scores limits them to answering less than 7.5% of the questions to achieve a low risk of error (i.e., 1%). This motivates us to utilize a multimodal selection function to directly estimate the correctness of the predicted answers, which we show can increase the coverage by, for example, 2.3x from 6.8% to 15.6% at 1% risk. While it is important to analyze both coverage and risk, these metrics have a trade-off which makes comparing VQA models challenging. To address this, we also propose an Effective Reliability metric for VQA that places a larger cost on incorrect answers compared to abstentions. This new problem formulation, metric, and analysis for VQA provide the groundwork for building effective and reliable VQA models that have the self-awareness to abstain if and only if they don't know the answer.
ELM: Embedding and Logit Margins for Long-Tail Learning

Long-tail learning is the problem of learning under skewed label distributions, which pose a challenge for standard learners. Several recent approaches for the problem have proposed enforcing a suitable margin in logit space. Such techniques are intuitive analogues of the guiding principle behind SVMs, and are equally applicable to linear models and neural models. However, when applied to neural models, such techniques do not explicitly control the geometry of the learned embeddings. This can be potentially sub-optimal, since embeddings for tail classes may be diffuse, resulting in poor generalization for these classes. We present Embedding and Logit Margins (ELM), a unified approach to enforce margins in logit space, and regularize the distribution of embeddings. This connects losses for long-tail learning to proposals in the literature on metric embedding, and contrastive learning. We theoretically show that minimising the proposed ELM objective helps reduce the generalisation gap. The ELM method is shown to perform well empirically, and results in tighter tail class embeddings.
FedShuffle: Recipes for Better Use of Local Work in Federated Learning

The practice of applying several local updates before aggregation across clients has been empirically shown to be a successful approach to overcoming the communication bottleneck in Federated Learning (FL). Such methods are usually implemented by having clients perform one or more epochs of local training per round while randomly reshuffling their finite dataset in each epoch. Data imbalance, where clients have different numbers of local training samples, is ubiquitous in FL applications, resulting in different clients performing different numbers of local updates in each round. In this work, we propose a general recipe, FedShuffle, that better utilizes the local updates in FL, especially in this regime encompassing random reshuffling and heterogeneity. FedShuffle is the first local update method with theoretical convergence guarantees that incorporates random reshuffling, data imbalance, and client sampling - features that are essential in large-scale cross-device FL. We present a comprehensive theoretical analysis of FedShuffle and show, both theoretically and empirically, that it does not suffer from the objective function mismatch that is present in FL methods that assume homogeneous updates in heterogeneous FL setups, such as FedAvg (McMahan et al., 2017). In addition, by combining the ingredients above, FedShuffle improves upon FedNova (Wang et al., 2020), which was previously proposed to solve this mismatch. Similar to Mime (Karimireddy et al., 2020), we show that FedShuffle with momentum variance reduction (Cutkosky&Orabona, 2019) improves upon non-local methods under a Hessian similarity assumption.
A Near-infrared Look at AGN Feedback in Bulgeless Galaxies

While it is generally believed that supermassive black holes (SMBHs) lie in most galaxies with bulges, few SMBHs have been confirmed in bulgeless galaxies. Identifying such a population could provide important insights to the BH seed population and secular BH growth. To this end, we obtained near-infrared (NIR) spectroscopic observations of a sample of low-redshift bulgeless galaxies with mid-infrared colors suggestive of active galactic nuclei (AGNs). We find additional evidence of AGN activity (such as coronal lines and broad permitted lines) in 69% (9/13) of the sample, demonstrating that mid-infrared selection is a powerful tool to detect AGNs. More than half of the galaxies with confirmed AGN activity show fast outflows in [O iii] in the optical and/or [Si vi] in the NIR, with the latter generally having much faster velocities that are also correlated to their spatial extent. We are also able to obtain virial BH masses for some targets and find they fall within the scatter of other late-type galaxies in the M BH–M stellar relation. The fact that they lack a significant bulge component indicates that secular processes, likely independent of major mergers, grew these BHs to supermassive sizes. Finally, we analyze the rotational gas kinematics and find two notable exceptions: two AGN hosts with outflows that appear to be rotating faster than expected. There is an indication that these two galaxies have stellar masses significantly lower than expected from their dark matter halo masses. This, combined with the observed AGN activity and strong gas outflows, may be evidence of the effects of AGN feedback.
Hybrid Relation Guided Set Matching for Few-shot Action Recognition

Current few-shot action recognition methods reach impressive performance by learning discriminative features for each video via episodic training and designing various temporal alignment strategies. Nevertheless, they are limited in that (a) learning individual features without considering the entire task may lose the most relevant information in the current episode, and (b) these alignment strategies may fail in misaligned instances. To overcome the two limitations, we propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components: hybrid relation module and set matching metric. The purpose of the hybrid relation module is to learn task-specific embeddings by fully exploiting associated relations within and cross videos in an episode. Built upon the task-specific features, we reformulate distance measure between query and support videos as a set matching problem and further design a bidirectional Mean Hausdorff Metric to improve the resilience to misaligned instances. By this means, the proposed HyRSM can be highly informative and flexible to predict query categories under the few-shot settings. We evaluate HyRSM on six challenging benchmarks, and the experimental results show its superiority over the state-of-the-art methods by a convincing margin. Project page: https://hyrsm-cvpr2022.github. io/.
Dust in AGB wind-ISM interaction regions

We aim to constrain the dust mass and grain sizes in the interaction regions between the stellar winds and the ISM around asymptotic giant branch stars. By describing the dust in these regions, we aim to shed light on the role of low mass evolved stars in the origin of dust in galaxies. We use images in the far-infrared at 70 micron and 160 micron to derive dust temperatures and dust masses in the wind-ISM interaction regions around a sample of carbon-rich and oxygen-rich asymptotic giant branch (AGB) stars. The dust temperature and mass are determined in two ways. First directly from the data using the ratio of the measured fluxes and assuming opacities for dust with a constant grain size of 0.1 micron. We then perform 3D dust-radiative transfer models spatially constrained by the observations to consistently calculate the temperature and mass. For the radiative transfer models each model contains one constant grain size, which is varied between 0.01 micron to 5.0 micron. We find that the observed dust mass in the wind-ISM interaction regions is consistent with mass accumulated from the stellar winds. For the carbon-rich sources adding the spatial constraints in the radiative transfer models results in preferentially larger grain sizes (approx. 2 micron). For the oxygen-rich sources the spatial constraints result in too high temperatures in the models, making it impossible to fit the observed far-infrared ratio irrespective of the grain size used, indicating a more complex interplay of grain properties and the stellar radiation field. The results have implications for how likely it is for the grains to survive the transition into the ISM, and the properties of dust particles that later act as seeds for grain growth in the ISM. However, the results for the oxygen-rich sources show that the derivation of dust properties is not straight forward, requiring more complex modelling
The Volatile Carbon-to-oxygen Ratio as a Tracer for the Formation Locations of Interstellar Comets

Based on the occurrence rates implied by the discoveries of 1I/‘Oumuamua and 2I/Borisov, the forthcoming Rubin Observatory Legacy Survey of Space and Time (LSST) should detect ≥one interstellar object every year. We advocate for future measurements of the production rates of H2O, CO2, and CO in these objects to estimate their carbon-to-oxygen ratios, which trace formation locations within their original protoplanetary disks. We review similar measurements for solar system comets, which indicate formation interior to the CO snow line. By quantifying the relative processing in the interstellar medium and solar system, we estimate that production rates will not be representative of primordial compositions for the majority of interstellar comets. Preferential desorption of CO and CO2 relative to H2O in the interstellar medium implies that measured C/O ratios represent lower limits on the primordial ratios. Specifically, production rate ratios of Q(CO)/Q(H2O) < 0.2 and Q(CO)/Q(H2O) > 1 likely indicate formation interior and exterior to the CO snow line, respectively. The high C/O ratio of 2I/Borisov implies that it formed exterior to the CO snow line. We provide an overview of the currently operational facilities capable of obtaining these measurements that will constrain the fraction of ejected comets that formed exterior to the CO snow line. This fraction will provide key insights into the efficiency of and mechanisms for cometary ejection in exoplanetary systems.
DOTIN: Dropping Task-Irrelevant Nodes for GNNs

Scalability is an important consideration for deep graph neural networks. Inspired by the conventional pooling layers in CNNs, many recent graph learning approaches have introduced the pooling strategy to reduce the size of graphs for learning, such that the scalability and efficiency can be improved. However, these pooling-based methods are mainly tailored to a single graph-level task and pay more attention to local information, limiting their performance in multi-task settings which often require task-specific global information. In this paper, departure from these pooling-based efforts, we design a new approach called DOTIN (\underline{D}r\underline{o}pping \underline{T}ask-\underline{I}rrelevant \underline{N}odes) to reduce the size of graphs. Specifically, by introducing $K$ learnable virtual nodes to represent the graph embeddings targeted to $K$ different graph-level tasks, respectively, up to 90\% raw nodes with low attentiveness with an attention model -- a transformer in this paper, can be adaptively dropped without notable performance decreasing. Achieving almost the same accuracy, our method speeds up GAT by about 50\% on graph-level tasks including graph classification and graph edit distance (GED) with about 60\% less memory, on D\&D dataset. Code will be made publicly available in https://github.com/Sherrylone/DOTIN.
Attention Based Neural Networks for Wireless Channel Estimation

In this paper, we deploy the self-attention mechanism to achieve improved channel estimation for orthogonal frequency-division multiplexing waveforms in the downlink. Specifically, we propose a new hybrid encoder-decoder structure (called HA02) for the first time which exploits the attention mechanism to focus on the most important input information. In particular, we implement a transformer encoder block as the encoder to achieve the sparsity in the input features and a residual neural network as the decoder respectively, inspired by the success of the attention mechanism. Using 3GPP channel models, our simulations show superior estimation performance compared with other candidate neural network methods for channel estimation.
A Hamiltonian Dysthe equation for deep-water gravity waves with constant vorticity

Abstract This paper is a study of the water wave problem in a two-dimensional domain of infinite depth in the presence of non-zero constant vorticity. A goal is to describe the effects of uniform shear flow on the modulation of weakly nonlinear quasi-monochromatic surface gravity waves. Starting from the Hamiltonian formulation of this problem and using techniques from Hamiltonian transformation theory, we derive a Hamiltonian Dysthe equation for the time evolution of the wave envelope. Consistent with previous studies, we observe that the uniform shear flow tends to enhance or weaken the modulational instability of Stokes waves depending on its direction and strength. Our method also provides a non-perturbative procedure to reconstruct the surface elevation from the wave envelope, based on the Birkhoff normal form transformation to eliminate all non-resonant triads. This model is tested against direct numerical simulations of the full Euler equations and against a related Dysthe equation derived recently by Curtis, Carter & Kalisch (J. Fluid Mech., vol. 855, 2018, pp. 322–350) in the context of constant vorticity. Very good agreement is found for a range of values of the vorticity.
How social influence affects the wisdom of crowds in influence networks

A long-standing debate is whether social influence improves the collective wisdom of a crowd or undermines it. This paper addresses this question based on a naive learning setting in influence systems theory: in our models individuals evolve their estimates of an unknown truth according to the weighted-average opinion dynamics. A formal mathematization is provided with rigorous theoretical analysis. We obtain various conditions for improving, optimizing and undermining the crowd accuracy, respectively. We prove that if the wisdom of finite-size group is improved, then the collective estimate converges to the truth as group size increases, provided individuals' variances are finite. We show that whether social influence improves or undermines the wisdom is determined by the social power allocation of the influence system: if the influence system allocates relatively larger social power to relatively more accurate individuals, it improves the wisdom; on the contrary, if the influence system assigns less social power to more accurate individuals, it undermines the wisdom. At a population level, individuals' susceptibilities to interpersonal influence and network centralities are both crucial. To improve the wisdom, more accurate individuals should be less susceptible and have larger network centralities. Particularly, in democratic influence networks, if relatively more accurate individuals are relatively less susceptible, the wisdom is improved; if more accurate individuals are more susceptible, the wisdom is undermined, which is consistent with the reported empirical evidence. Our investigation provides a theoretical framework for understanding the role social influence plays in the emergence of collective wisdom.
D3: A Massive Dataset of Scholarly Metadata for Analyzing the State of Computer Science Research

DBLP is the largest open-access repository of scientific articles on computer science and provides metadata associated with publications, authors, and venues. We retrieved more than 6 million publications from DBLP and extracted pertinent metadata (e.g., abstracts, author affiliations, citations) from the publication texts to create the DBLP Discovery Dataset (D3). D3 can be used to identify trends in research activity, productivity, focus, bias, accessibility, and impact of computer science research. We present an initial analysis focused on the volume of computer science research (e.g., number of papers, authors, research activity), trends in topics of interest, and citation patterns. Our findings show that computer science is a growing research field (15% annually), with an active and collaborative researcher community. While papers in recent years present more bibliographical entries in comparison to previous decades, the average number of citations has been declining. Investigating papers’ abstracts reveals that recent topic trends are clearly reflected in D3. Finally, we list further applications of D3 and pose supplemental research questions. The D3 dataset, our findings, and source code are publicly available for research purposes.
On tuning a mean-field model for semi-supervised classification

Semi-supervised learning (SSL) has become an interesting research area due to its capacity for learning in scenarios where both labeled and unlabeled data are available. In this work, we focus on the task of transduction—when the objective is to label all data presented to the learner—with a mean-field approximation to the Potts model. Aiming at this particular task we study how classification results depend on β and find that the optimal phase depends highly on the amount of labeled data available. In the same study, we also observe that more stable classifications regarding small fluctuations in β are related to configurations of high probability and propose a tuning approach based on such observation. This method relies on a novel parameter γ and we then evaluate two different values of the said quantity in comparison with classical methods in the field. This evaluation is conducted by changing the amount of labeled data available and the number of nearest neighbors in the similarity graph. Empirical results show that the tuning method is effective and allows NMF to outperform other approaches in datasets with fewer classes. In addition, one of the chosen values for γ also leads to results that are more resilient to changes in the number of neighbors, which might be of interest to practitioners in the field of SSL.
Divisible Codes for Quantum Computation

Divisible codes are defined by the property that codeword weights share a common divisor greater than one. They are used to design signals for communications and sensing, and this paper explores how they can be used to protect quantum information as it is transformed by logical gates. Given a CSS code $\mathcal{C}$, we derive conditions that are both necessary and sufficient for a transversal diagonal physical operator $U_Z$ to preserve $\mathcal{C}$ and induce $U_L$. The group of $Z$-stabilizers in a CSS code $\mathcal{C}$ is determined by the dual of a classical $[n, k_1]$ binary code $\mathcal{C}_1$, and the group of $X$-stabilizers is determined by a classical $[n, k_2]$ binary code $\mathcal{C}_2$ that is contained in $\mathcal{C}_1$. The requirement that a diagonal physical operator $U_Z$ fixes a CSS code $\mathcal{C}$ leads to constraints on the congruence of weights in cosets of $\mathcal{C}_2$. These constraints are a perfect fit to divisible codes, and represent an opportunity to take advantage of the extensive literature on classical codes with two or three weights. We construct new families of CSS codes using cosets of the first order Reed Muller code defined by quadratic forms. We provide a simple alternative to the standard method of deriving the coset weight distributions (based on Dickson normal form) that may be of independent interest. Finally, we develop an approach to circumventing the Eastin-Knill Theorem which states that no QECC can implement a universal set of logical gates through transversal gates alone. The essential idea is to design stabilizer codes in layers, with $N_1$ inner qubits and $N_2$ outer qubits, and to assemble a universal set of fault tolerant gates on the inner qubits.
Quantum evolution of the Hawking state for black holes

We give a general description of the evolving quantum state of a Schwarzschild black hole, in the quantum ﬁeld theory approximation. Such a time-dependent description is based on introducing a choice of time slices. We in particular consider slices that smoothly cross the horizon, and introduction of “stationary” such slices simpliﬁes the analysis. This analysis goes beyond standard derivations of Hawking radiation that focus on asymptotic excitations, and in particular gives an evolving state that is regular at the horizon, with no explicit transplanckian dependence, and that can in principle be generalized to incorporate interacting ﬁelds. It is also expected to be useful in connecting to information-theoretic investigation of black hole evolution. The description of the evolving state depends on the choice of slices as well as coordinates on the slices and mode bases; these choices give diﬀerent “pictures” analogous to that of Schr¨odinger. Evolution does have a simpler appearance in an energy eigenbasis, but such a basis is also singular at the horizon; evolution of regular modes has a more complicated appearance, whose properties may be inferred by comparing with the energy eigenbasis. In a regular description, Hawking quanta are produced in a black hole atmosphere, at scales comparable to the horizon size. This approach is also argued to extend to more general asymptotics, such as that of anti de Sitter space. In the latter context, this analysis provides a description of the hamiltonian and evolution of a black hole that may be compared to the large- N dynamics of the proposed dual CFT.
Modest holography and bulk reconstruction in asymptotically flat spacetimes

In this work we present a"modest"holographic reconstruction of the bulk geometry in asymptotically flat spacetime using the two-point correlators of boundary quantum field theory (QFT) in asymptotically flat spacetime. The boundary QFT lives on the null boundary of the spacetime, namely null infinity and/or the Killing horizons. The bulk reconstruction relies on two unrelated results: (i) there is a bulk-to-boundary type correspondence between free quantum fields living in the bulk manifold and free quantum fields living on its null boundary, and (ii) one can construct the metric by making use of the Hadamard expansion of the field living in the bulk. This holographic reconstruction is"modest"in that the fields used are non-interacting and not strong-weak holographic duality in the sense of AdS/CFT, but it works for generic asymptotically flat spacetime subject to some reasonably mild conditions.