Miracle-dz/UniFAR-FTUs huggingface dataset:
IcebergHT: High Performance PMEM Hash Tables Through Stability and Low Associativity

Modern hash table designs strive to minimize space while maximizing speed. The most important factor in speed is the number of cache lines accessed during updates and queries. This is especially important on PMEM, which is slower than DRAM and in which writes are more expensive than reads. This paper proposes two stronger design objectives: stability and low-associativity. A stable hash table doesn't move items around, and a hash table has low associativity if there are only a few locations where an item can be stored. Low associativity ensures that queries need to examine only a few memory locations, and stability ensures that insertions write to very few cache lines. Stability also simplifies scaling and crash safety. We present IcebergHT, a fast, crash-safe, concurrent, and space-efficient hash table for PMEM based on the design principles of stability and low associativity. IcebergHT combines in-memory metadata with a new hashing technique, iceberg hashing, that is (1) space efficient, (2) stable, and (3) supports low associativity. In contrast, existing hash-tables either modify numerous cache lines during insertions (e.g. cuckoo hashing), access numerous cache lines during queries (e.g. linear probing), or waste space (e.g. chaining). Moreover, the combination of (1)-(3) yields several emergent benefits: IcebergHT scales better than other hash tables, supports crash-safety, and has excellent performance on PMEM (where writes are particularly expensive).
Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Network

Sufficient dimension reduction is a powerful tool to extract core information hidden in the high-dimensional data and has potentially many important applications in machine learning tasks. However, the existing nonlinear sufficient dimension reduction methods often lack the scalability necessary for dealing with large-scale data. We propose a new type of stochastic neural network under a rigorous probabilistic framework and show that it can be used for sufficient dimension reduction for large-scale data. The proposed stochastic neural network is trained using an adaptive stochastic gradient Markov chain Monte Carlo algorithm, whose convergence is rigorously studied in the paper as well. Through extensive experiments on real-world classification and regression problems, we show that the proposed method compares favorably with the existing state-of-the-art sufficient dimension reduction methods and is computationally more efficient for large-scale data.
Imaginary-time Quantum Relaxation Critical Dynamics with Semi-ordered Initial States

We explore the imaginary-time relaxation dynamics near quantum critical points with semi-ordered initial states. Different from the case with homogeneous ordered initial states, in which the order parameter $M$ decays homogeneously as $M\propto \tau^{-\beta/\nu z}$, here $M$ depends on the location $x$, showing rich scaling behaviors. Similar to the classical relaxation dynamics with an initial domain wall in Model A, which describes the purely dissipative dynamics, here as the imaginary time evolves, the domain wall expands into an interfacial region with growing size. In the interfacial region, the local order parameter decays as $M\propto \tau^{-\beta_1/\nu z}$, with $\beta_1$ being an additional dynamic critical exponent. Far away from the interfacial region the local order parameter decays as $M\propto \tau^{-\beta/\nu z}$ in the short-time stage, then crosses over to the scaling behavior of $M\propto \tau^{-\beta_1/\nu z}$ when the location $x$ is absorbed in the interfacial region. A full scaling form characterizing these scaling properties is developed. The quantum Ising model in both one and two dimensions are taken as examples to verify the scaling theory. In addition, we find that for the quantum Ising model the scaling function is an analytical function and $\beta_1$ is not an independent exponent.
Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognition

The success of most advanced facial expression recognition works relies heavily on large-scale annotated datasets. However, it poses great challenges in acquiring clean and consistent annotations for facial expression datasets. On the other hand, self-supervised contrastive learning has gained great popularity due to its simple yet effective instance discrimination training strategy, which can potentially circumvent the annotation issue. Nevertheless, there remain inherent disadvantages of instance-level discrimination, which are even more challenging when faced with complicated facial representations. In this paper, we revisit the use of self-supervised contrastive learning and explore three core strategies to enforce expression-specific representations and to minimize the interference from other facial attributes, such as identity and face styling. Experimental results show that our proposed method outperforms the current state-of-the-art self-supervised learning methods, in terms of both categorical and dimensional facial expression recognition tasks.
CO AND H2 ABSORPTION IN THE AA TAURI CIRCUMSTELLAR DISK

The direct study of molecular gas in inner protoplanetary disks is complicated by uncertainties in the spatial distribution of the gas, the time variability of the source, and the comparison of observations across a wide range of wavelengths. Some of these challenges can be mitigated with far-ultraviolet spectroscopy. Using new observations obtained with the Hubble Space Telescope Cosmic Origins Spectrograph, we measure column densities and rovibrational temperatures for CO and H2 observed on the line of sight through the AA Tauri circumstellar disk. CO A – X absorption bands are observed against the far-UV continuum. The CO absorption is characterized by log10(N(12CO)) = 17.5 ± 0.5 cm−2 and T rot(CO) = 500+500 −200 K, although this rotational temperature may underestimate the local kinetic temperature of the CO-bearing gas. We also detect 13CO in absorption with an isotopic ratio of ∼20. We do not observe H2 absorption against the continuum; however, hot H2 (v > 0) is detected in absorption against the Lyα emission line. We measure the column densities in eight individual rovibrational states, determining a total log10(N(H2)) = 17.9+0.6 −0.3 cm−2 with a thermal temperature of T(H2) = 2500+800 −700 K. The high temperature of the molecules, the relatively small H2 column density, and the high inclination of the AA Tauri disk suggest that the absorbing gas resides in an inner disk atmosphere. If the H2 and CO are cospatial within a molecular layer ∼0.6 AU thick, this region is characterized by ∼ 105 cm−3 with an observed 〈CO/H2〉 ratio of ∼0.4. We also find evidence for a departure from a purely thermal H2 distribution, suggesting that excitation by continuum photons and H2 formation may be altering the level populations in the molecular gas.
A Posteriori Error Estimate and Adaptivity for QM/MM Models of Crystalline Defects

Hybrid quantum/molecular mechanics (QM/MM) models play a pivotal role in molecular simulations. These models provide a balance between accuracy, surpassing pure MM models, and computational efficiency, offering advantages over pure QM models. Adaptive approaches have been developed to further improve this balance by allowing on-the-fly selection of the QM and MM subsystems as necessary. We propose a novel and robust adaptive QM/MM method for practical material defect simulations. To ensure mathematical consistency with the QM reference model, we employ machine-learning interatomic potentials (MLIPs) as the MM models. Our adaptive QM/MM method utilizes a residual-based error estimator that provides both upper and lower bounds for the approximation error, thus indicating its reliability and efficiency. Furthermore, we introduce a novel adaptive algorithm capable of anisotropically updating the QM/MM partitions. This update is based on the proposed residual-based error estimator and involves solving a free interface motion problem, which is efficiently achieved using the fast marching method. We demonstrate the robustness of our approach via numerical tests on a wide range of crystalline defects.
ConstGCN: Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction

Document-level relation extraction with graph neural networks faces a fundamental graph construction gap between training and inference - the golden graph structure only available during training, which causes that most methods adopt heuristic or syntactic rules to construct a prior graph as a pseudo proxy. In this paper, we propose $\textbf{ConstGCN}$, a novel graph convolutional network which performs knowledge-based information propagation between entities along with all specific relation spaces without any prior graph construction. Specifically, it updates the entity representation by aggregating information from all other entities along with each relation space, thus modeling the relation-aware spatial information. To control the information flow passing through the indeterminate relation spaces, we propose to constrain the propagation using transmitting scores learned from the Noise Contrastive Estimation between fact triples. Experimental results show that our method outperforms the previous state-of-the-art (SOTA) approaches on the DocRE dataset.
Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation

How can we learn from all available data when training machine-learnt climate models, without incurring any extra cost at simulation time? Typically, the training data comprises coarse-grained high-resolution data. But only keeping this coarse-grained data means the rest of the high-resolution data is thrown out. We use a transfer learning approach, which can be applied to a range of machine learning models, to leverage all the high-resolution data. We use three chaotic systems to show it stabilises training, gives improved generalisation performance and results in better forecasting skill. Our anonymised code is at https://www.dropbox.
Deep Clustering: A Comprehensive Survey

Cluster analysis plays an indispensable role in machine learning and data mining. Learning a good data representation is crucial for clustering algorithms. Recently, deep clustering, which can learn clustering-friendly representations using deep neural networks, has been broadly applied in a wide range of clustering tasks. Existing surveys for deep clustering mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering. To address this issue, in this paper we provide a comprehensive survey for deep clustering in views of data sources. With different data sources and initial conditions, we systematically distinguish the clustering methods in terms of methodology, prior knowledge, and architecture. Concretely, deep clustering methods are introduced according to four categories, i.e., traditional single-view deep clustering, semi-supervised deep clustering, deep multi-view clustering, and deep transfer clustering. Finally, we discuss the open challenges and potential future opportunities in different fields of deep clustering.
A multi-physics method for fracture and fragmentation at high strain-rates

This work outlines a diffuse interface method for the study of fracture and fragmentation in ductile metals at high strain-rates in Eulerian finite volume simulations. The work is based on an existing diffuse interface method capable of simulating a broad range of different multi-physics applications, including multi-material interaction, damage and void opening. The work at hand extends this method with a technique to model realistic material inhomogeneities, and examines the performance of the method on a selection of challenging problems. Material inhomogeneities are included by evolving a scalar field that perturbs a material's plastic yield stress. This perturbation results in non-uniform fragments with a measurable statistical distribution, allowing for underlying defects in a material to be modelled. As the underlying numerical scheme is three dimensional, parallelisable and multi-physics-capable, the scheme can be tested on a range of strenuous problems. These problems especially include a three-dimensional explosively driven fracture study, with an explicitly resolved condensed phase explosive. The new scheme compares well with both experiment and previous numerical studies.
SDA: Simple Discrete Augmentation for Contrastive Sentence Representation Learning

Contrastive learning has recently achieved compelling performance in unsupervised sentence representation. As an essential element, data augmentation protocols, however, have not been well explored. The pioneering work SimCSE resorting to a simple dropout mechanism (viewed as continuous augmentation) surprisingly dominates discrete augmentations such as cropping, word deletion, and synonym replacement as reported. To understand the underlying rationales, we revisit existing approaches and attempt to hypothesize the desiderata of reasonable data augmentation methods: balance of semantic consistency and expression diversity. We then develop three simple yet effective discrete sentence augmentation schemes: punctuation insertion, modal verbs, and double negation. They act as minimal noises at lexical level to produce diverse forms of sentences. Furthermore, standard negation is capitalized on to generate negative samples for alleviating feature suppression involved in contrastive learning. We experimented extensively with semantic textual similarity on diverse datasets. The results support the superiority of the proposed methods consistently.
On the Evaluation Metrics for Paraphrase Generation

In this paper we revisit automatic metrics for paraphrase evaluation and obtain two findings that disobey conventional wisdom: (1) Reference-free metrics achieve better performance than their reference-based counterparts. (2) Most commonly used metrics do not align well with human annotation.Underlying reasons behind the above findings are explored through additional experiments and in-depth analyses.Based on the experiments and analyses, we propose ParaScore, a new evaluation metric for paraphrase generation. It possesses the merits of reference-based and reference-free metrics and explicitly models lexical divergence. Based on our analysis and improvements, our proposed reference-based outperforms than reference-free metrics.Experimental results demonstrate that ParaScore significantly outperforms existing metrics.
Dynamic Dialogue Policy for Continual Reinforcement Learning

Continual learning is one of the key components of human learning and a necessary requirement of artificial intelligence. As dialogue can potentially span infinitely many topics and tasks, a task-oriented dialogue system must have the capability to continually learn, dynamically adapting to new challenges while preserving the knowledge it already acquired. Despite the importance, continual reinforcement learning of the dialogue policy has remained largely unaddressed. The lack of a framework with training protocols, baseline models and suitable metrics, has so far hindered research in this direction. In this work we fill precisely this gap, enabling research in dialogue policy optimisation to go from static to dynamic learning. We provide a continual learning algorithm, baseline architectures and metrics for assessing continual learning models. Moreover, we propose the dynamic dialogue policy transformer (DDPT), a novel dynamic architecture that can integrate new knowledge seamlessly, is capable of handling large state spaces and obtains significant zero-shot performance when being exposed to unseen domains, without any growth in network parameter size. We validate the strengths of DDPT in simulation with two user simulators as well as with humans.
Graph Neural Networks as Gradient Flows: understanding graph convolutions via energy

Gradient ﬂows are diﬀerential equations that minimize an energy functional and constitute the main descriptors of physical systems. We apply this formalism to Graph Neural Networks (GNNs) to develop new frameworks for learning on graphs as well as provide a better theoretical understanding of existing ones. We derive GNNs as a gradient ﬂow equation of a parametric energy that provides a physics-inspired interpretation of GNNs as learning particle dynamics in the feature space. In particular, we show that in graph convolutional models (GCN), the positive/negative eigenvalues of the channel mixing matrix correspond to attractive/repulsive forces between adjacent features. We rigorously prove how the channel-mixing can learn to steer the dynamics towards low or high frequencies, which allows to deal with heterophilic graphs. We show that the same class of energies is decreasing along a larger family of GNNs; albeit not gradient ﬂows, they retain their inductive bias. We experimentally evaluate an instance of the gradient ﬂow framework that is principled, more eﬃcient than GCN, and achieves competitive performance on graph datasets of varying homophily often outperforming recent baselines speciﬁcally designed to target heterophily.
A Privacy Preserving IoT Data Marketplace Using IOTA Smart Contracts

In recent years, the volume of data generated by IoT devices has increased dramatically. Using this data can improve decision-making in the public and private sectors and increase productivity. Many attempts have been made to enhance and adapt businesses to exploit this IoT data. Among these, IoT data trading is the most popular approach. To this end, ongoing projects are currently focused on developing decentralized data marketplaces for IoT using blockchain and cryptocurrencies. Here we explore how a decentralized data marketplace could be created using IOTA tangle and IOTA smart contract chains (SC chains). We also consider the advantages of such architecture in terms of cost, scalability, and privacy over current designs and introduce the various elements it should have.
Floorplan-Aware Camera Poses Refinement

Processing large indoor scenes is a challenging task, as scan registration and camera trajectory estimation methods accumulate errors across time. As a result, the quality of reconstructed scans is insufficient for some applications, such as visual-based localization and navigation, where the correct position of walls is crucial. For many indoor scenes, there exists an image of a technical ftoorplan that contains information about the geometry and main structural elements of the scene, such as walls, partitions, and doors. We argue that such a ftoorplan is a useful source of spatial information, which can guide a 3D model optimization. The standard RGB-D 3D reconstruction pipeline consists of a tracking module applied to an RGB-D sequence and a bundle adjustment (BA) module that takes the posed RGB-D sequence and corrects the camera poses to improve consistency. We propose a novel optimization algorithm expanding conventional BA that leverages the prior knowledge about the scene structure in the form of a ftoorplan. Our experiments on the Redwood dataset and our self-captured data demonstrate that utilizing ftoorplan improves accuracy of 3D reconstructions.
See, Plan, Predict: Language-guided Cognitive Planning with Video Prediction

Cognitive planning is the structural decomposition of complex tasks into a sequence of future behaviors. In the computational setting, performing cognitive planning entails grounding plans and concepts in one or more modalities in order to leverage them for low level control. Since real-world tasks are often described in natural language, we devise a cognitive planning algorithm via language-guided video prediction. Current video prediction models do not support conditioning on natural language instructions. Therefore, we propose a new video prediction architecture which leverages the power of pre-trained transformers.The network is endowed with the ability to ground concepts based on natural language input with generalization to unseen objects. We demonstrate the effectiveness of this approach on a new simulation dataset, where each task is defined by a high-level action described in natural language. Our experiments compare our method again stone video generation baseline without planning or action grounding and showcase significant improvements. Our ablation studies highlight an improved generalization to unseen objects that natural language embeddings offer to concept grounding ability, as well as the importance of planning towards visual"imagination"of a task.
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks

In this work, we explore the maximum-margin bias of quasi-homogeneous neural networks trained with gradient flow on an exponential loss and past a point of separability. We introduce the class of quasi-homogeneous models, which is expressive enough to describe nearly all neural networks with homogeneous activations, even those with biases, residual connections, and normalization layers, while structured enough to enable geometric analysis of its gradient dynamics. Using this analysis, we generalize the existing results of maximum-margin bias for homogeneous networks to this richer class of models. We find that gradient flow implicitly favors a subset of the parameters, unlike in the case of a homogeneous model where all parameters are treated equally. We demonstrate through simple examples how this strong favoritism toward minimizing an asymmetric norm can degrade the robustness of quasi-homogeneous models. On the other hand, we conjecture that this norm-minimization discards, when possible, unnecessary higher-order parameters, reducing the model to a sparser parameterization. Lastly, by applying our theorem to sufficiently expressive neural networks with normalization layers, we reveal a universal mechanism behind the empirical phenomenon of Neural Collapse.
ELECTRON HEATING AND ACCELERATION BY MAGNETIC RECONNECTION IN HOT ACCRETION FLOWS

Both analytical and numerical works show that magnetic reconnection must occur in hot accretion flows. This process will effectively heat and accelerate electrons. In this paper, we use the numerical hybrid simulation of magnetic reconnection plus the test-electron method to investigate the electron acceleration and heating due to magnetic reconnection in hot accretion flows. We consider fiducial values of density, temperature, and magnetic parameter βe (defined as the ratio of the electron pressure to the magnetic pressure) of the accretion flow as n0 ∼ 106 cm-3, T0e ∼ 2 × 109 K, and βe = 1. We find that electrons are heated to a higher temperature Te = 5 × 109 K, and a fraction η ∼ 8% of electrons are accelerated into a broken power-law distribution, dN(γ) ∝ γ−p, with p ≈ 1.5 and 4 below and above ∼1 MeV, respectively. We also investigate the effect of varying β and n0. We find that when βe is smaller or n0 is larger, i.e., the magnetic field is stronger, Te, η, and p all become larger.
Systematic Evaluation and User Study of Privacy of Default Apps in Apple's Mobile Ecosystem

Users need to configure default apps when they first start using their devices. The privacy configurations of the default apps do not always match what users think they have initially enabled. We first systematically evaluated the privacy configurations of default apps. We discovered serious issues with the documentation of the default apps. Based on these findings, we explored users' experiences with an interview study (N=15). Our findings from both studies show that: the instructions of setting privacy configurations of default apps are vague and lack required steps; users were unable to disable default apps from accessing their personal information; users assumed they were being tracked by some default apps; default apps may cause tensions in family relationships because of information sharing. Our results illuminate on the privacy and security implications of configuring the privacy of default apps and how users perceive and understand the mobile ecosystem.