Miracle-dz/UniFAR-FTUs huggingface dataset:
A Memory-Augmented Neural Network Model of Abstract Rule Learning

Human intelligence is characterized by a remarkable ability to infer abstract rules from experience and apply these rules to novel domains. As such, designing neural network algorithms with this capacity is an important step toward the development of deep learning systems with more human-like intelligence. However, doing so is a major outstanding challenge, one that some argue will require neural networks to use explicit symbol-processing mechanisms. In this work, we focus on neural networks' capacity for arbitrary role-filler binding, the ability to associate abstract "roles" to context-specific "fillers," which many have argued is an important mechanism underlying the ability to learn and apply rules abstractly. Using a simplified version of Raven's Progressive Matrices, a hallmark test of human intelligence, we introduce a sequential formulation of a visual problem-solving task that requires this form of binding. Further, we introduce the Emergent Symbol Binding Network (ESBN), a recurrent neural network model that learns to use an external memory as a binding mechanism. This mechanism enables symbol-like variable representations to emerge through the ESBN's training process without the need for explicit symbol-processing machinery. We empirically demonstrate that the ESBN successfully learns the underlying abstract rule structure of our task and perfectly generalizes this rule structure to novel fillers.
Electrostatic interaction between SARS-CoV-2 virus and charged electret fibre

While almost any kind of face mask offers some protection against particles and pathogens of different sizes, the most efficient ones make use of a layered structure where one or more layers are electrically charged. This electret layer is essential to efficient filtration of difficult-to-capture small particles, yet the exact nature of electrostatic capture with respect to both the charge on the particles and the electret fibres as well as the effect of immediate environment remains unclear. Here, we explore in detail the electrostatic interaction between the surface of a single charged electret fibre and a model of SARS-CoV-2 virus. Using Poisson-Boltzmann electrostatics coupled to a detailed spike protein charge regulation model, we show how pH and salt concentration drastically change both the scale and the sign of the interaction. Furthermore, the configuration of the few spike proteins closest to the electret fibre turns out to be as important for the strength of the interaction as their total number on the virus, a direct consequence of spike protein charge regulation. The results of our work elucidate the details of virus electrostatics and contribute to the general understanding of efficient virus filtration mechanisms.
DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation

Public resources and services (e.g., datasets, training platforms, pre-trained models) have been widely adopted to ease the development of Deep Learning-based applications. However, if the third-party providers are untrusted, they can inject poisoned samples into the datasets or embed backdoors in those models. Such an integrity breach can cause severe consequences, especially in safety- and security-critical applications. Various backdoor attack techniques have been proposed for higher effectiveness and stealthiness. Unfortunately, existing defense solutions are not practical to thwart those attacks in a comprehensive way. In this paper, we investigate the effectiveness of data augmentation techniques in mitigating backdoor attacks and enhancing DL models' robustness. An evaluation framework is introduced to achieve this goal. Specifically, we consider a unified defense solution, which (1) adopts a data augmentation policy to fine-tune the infected model and eliminate the effects of the embedded backdoor; (2) uses another augmentation policy to preprocess input samples and invalidate the triggers during inference. We propose a systematic approach to discover the optimal policies for defending against different backdoor attacks by comprehensively evaluating 71 state-of-the-art data augmentation functions. Extensive experiments show that our identified policy can effectively mitigate eight different kinds of backdoor attacks and outperform five existing defense methods. We envision this framework can be a good benchmark tool to advance future DNN backdoor studies.
StackRec: Efficient Training of Very Deep Sequential Recommender Models by Layer Stacking

Deep learning has brought great progress for the sequential recommendation (SR) tasks. With the structure of advanced residual networks, sequential recommender models can be stacked with many hidden layers, e.g., up to 100 layers on real-world SR datasets. Training such a deep network requires expensive computation and longer training time, especially in situations when there are tens of billions of user-item interactions. To deal with such a challenge, we present StackRec, a simple but very efficient training framework for deep SR models by layer stacking. Specifically, we first offer an important insight that residual layers/blocks in a well-trained deep SR model have similar distribution. Enlightened by this, we pro-pose progressively stacking such pre-trained residual layers/blocks so as to yield a deeper but easier-to-train SR model. We validate the proposed StackRec by instantiating it with two state-of-the-art SR models in three practical scenarios and real-world datasets. Extensive experiments show that StackRec achieves not only comparable performance, but also substantial acceleration in training time, compared to SR models that are trained from scratch.
Accelerating high-throughput virtual screening through molecular pool-based active learning†

Structure-based virtual screening is an important tool in early stage drug discovery that scores the interactions between a target protein and candidate ligands. As virtual libraries continue to grow (in excess of 108 molecules), so too do the resources necessary to conduct exhaustive virtual screening campaigns on these libraries. However, Bayesian optimization techniques, previously employed in other scientific discovery problems, can aid in their exploration: a surrogate structure–property relationship model trained on the predicted affinities of a subset of the library can be applied to the remaining library members, allowing the least promising compounds to be excluded from evaluation. In this study, we explore the application of these techniques to computational docking datasets and assess the impact of surrogate model architecture, acquisition function, and acquisition batch size on optimization performance. We observe significant reductions in computational costs; for example, using a directed-message passing neural network we can identify 94.8% or 89.3% of the top-50 000 ligands in a 100M member library after testing only 2.4% of candidate ligands using an upper confidence bound or greedy acquisition strategy, respectively. Such model-guided searches mitigate the increasing computational costs of screening increasingly large virtual libraries and can accelerate high-throughput virtual screening campaigns with applications beyond docking.
Entropy inequality and energy dissipation of inertial Qian–Sheng model for nematic liquid crystals

For the inertial Qian-Sheng model of nematic liquid crystals in the $Q$-tensor framework, we illustrate the roles played by the entropy inequality and energy dissipation in the well-posedness of smooth solutions when we employ energy method. We first derive the coefficients requirements from the entropy inequality, and point out the entropy inequality is insufficient to guarantee energy dissipation. We then introduce a novel Condition (H) which ensures the energy dissipation. We prove that when both the entropy inequality and Condition (H) are obeyed, the local in time smooth solutions exist for large initial data. Otherwise, we can only obtain small data local solutions. Furthermore, to extend the solutions globally in time and obtain the decay of solutions, we require at least one of the two conditions: entropy inequality, or $\tilde{\mu}_2= \mu_2$, which significantly enlarge the range of the coefficients in previous works.
A CT-based numerical characterization of tensile behavior of additively manufactured octet-truss structures and its experimental validation

The production of lightweight metal lattice structures has received much attention due to the recent developments in additive manufacturing (AM). The design flexibility comes, however, with the complexity of the underlying physics. In fact, metal additive manufacturing introduces process-induced geometrical defects that mainly result in deviations of the effective geometry from the nominal one. This kind of defects is the major cause of the gap between as-designed and as-manufactured mechanical behavior of AM products. Thus, the possibility to incorporate the precise manufactured geometries into the computational analysis is crucial for the quality and performance assessment of the final parts. Computed tomography (CT) is an accurate method for the acquisition of the manufactured shape. However, it is often not feasible to integrate the CT-based geometrical information into the traditional computational analysis due to the complexity of the meshing procedure for such high-resolution geometrical models and the prohibitive numerical costs. In this work, an embedded numerical framework is applied to efficiently simulate and compare the mechanical behavior of as-designed to as-manufactured octet-truss lattice structures. Employing an immersed boundary method, namely the Finite Cell Method (FCM), we perform direct numerical simulations (DNS) and first-order numerical homogenization analysis of a tensile test for a 3D printed octet-truss structure. Numerical results based on CT scan (as-manufactured geometry) show an excellent agreement with experimental measurements, whereas both DNS and first-order numerical homogenization performed directly on the 3D virtual model (as-designed geometry) of the structure show a significant deviation from experimental data.
Extracting Training Data from Large Language Models

It has become common to publish large (billion parameter) language models that have been trained on private datasets. This paper demonstrates that in such settings, an adversary can perform a training data extraction attack to recover individual training examples by querying the language model. We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data. These extracted examples include (public) personally identifiable information (names, phone numbers, and email addresses), IRC conversations, code, and 128-bit UUIDs. Our attack is possible even though each of the above sequences are included in just one document in the training data. We comprehensively evaluate our extraction attack to understand the factors that contribute to its success. For example, we find that larger models are more vulnerable than smaller models. We conclude by drawing lessons and discussing possible safeguards for training large language models.
A Virgo Environmental Survey Tracing Ionised Gas Emission (VESTIGE)

We observed the late-type peculiar galaxy NGC 4424 during the Virgo Environmental Survey Tracing Galaxy Evolution (VESTIGE), a blind narrow-band Hα+[NII] imaging survey of the Virgo cluster carried out with MegaCam at the Canada-French-Hawaii Telescope (CFHT). The presence of a ∼110 kpc (in projected distance) HI tail in the southern direction indicates that this galaxy is undergoing a ram pressure stripping event. The deep narrow-band image revealed a low surface brightness (Σ(Hα) ≃ 4 × 10−18 erg s−1 cm−2 arcsec−2) ionised gas tail ∼10 kpc in length extending from the centre of the galaxy to the north-west, thus in the direction opposite to the HI tail. Chandra and XMM X-rays data do not show a compact source in the nucleus or an extended tail of hot gas, while IFU spectroscopy (MUSE) indicates that the gas is photo-ionised in the inner regions and shock-ionised in the outer parts. Medium-resolution (MUSE) and high-resolution (Fabry-Perot) IFU spectroscopy confirms that the ionised gas is kinematically decoupled from the stellar component and indicates the presence of two kinematically distinct structures in the stellar disc. The analysis of the SED of the galaxy indicates that the activity of star formation was totally quenched in the outer disc ∼250–280 Myr ago, while only reduced by ∼80% in the central regions. All this observational evidence suggests that NGC 4424 is the remnant of an unequal-mass merger that occurred ≲500 Myr ago when the galaxy was already a member of the Virgo cluster, and is now undergoing a ram pressure stripping event that has removed the gas and quenched the activity of star formation in the outer disc. The tail of ionised gas probably results from the outflow produced by a central starburst fed by the collapse of gas induced by the merging episode. This outflow is sufficiently powerful to overcome the ram pressure induced by the intracluster medium on the disc of the galaxy crossing the cluster. This analysis thus suggests that feedback can participate in the quenching process of galaxies in high-density regions.
Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks

We investigate segmenting and clustering speech into low-bitrate phone-like sequences without supervision. We specifically constrain pretrained self-supervised vector-quantized (VQ) neural networks so that blocks of contiguous feature vectors are assigned to the same code, thereby giving a variable-rate segmentation of the speech into discrete units. Two segmentation methods are considered. In the first, features are greedily merged until a prespecified number of segments are reached. The second uses dynamic programming to optimize a squared error with a penalty term to encourage fewer but longer segments. We show that these VQ segmentation methods can be used without alteration across a wide range of tasks: unsupervised phone segmentation, ABX phone discrimination, same-different word discrimination, and as inputs to a symbolic word segmentation algorithm. The penalized method generally performs best. While results are only comparable to the state-of-the-art in some cases, in all tasks a reasonable competing approach is outperformed at a substantially lower bitrate.
Noisy Linear Convergence of Stochastic Gradient Descent for CV@R Statistical Learning under Polyak-Łojasiewicz Conditions

Conditional Value-at-Risk ($\mathrm{CV@R}$) is one of the most popular measures of risk, which has been recently considered as a performance criterion in supervised statistical learning, as it is related to desirable operational features in modern applications, such as safety, fairness, distributional robustness, and prediction error stability. However, due to its variational definition, $\mathrm{CV@R}$ is commonly believed to result in difficult optimization problems, even for smooth and strongly convex loss functions. We disprove this statement by establishing noisy (i.e., fixed-accuracy) linear convergence of stochastic gradient descent for sequential $\mathrm{CV@R}$ learning, for a large class of not necessarily strongly-convex (or even convex) loss functions satisfying a set-restricted Polyak-Lojasiewicz inequality. This class contains all smooth and strongly convex losses, confirming that classical problems, such as linear least squares regression, can be solved efficiently under the $\mathrm{CV@R}$ criterion, just as their risk-neutral versions. Our results are illustrated numerically on such a risk-aware ridge regression task, also verifying their validity in practice.
Stickiness of randomly rough surfaces with high fractal dimension: is there a fractal limit?

Two surfaces are ”sticky” if breaking their mutual contact requires a finite tensile force. At low fractal dimensions D, there is consensus stickiness does not depend on the upper truncation frequency of roughness spectrum (or ”magnification”). As debate is still open for the case at high D, we exploit BAM theory of Ciavarella and Persson-Tosatti theory, to derive criteria for all fractal dimensions. For high D, we show that stickiness is more influenced by short wavelength roughness with respect to the low D case. BAM converges at high magnifications to a simple criterion which depends only on D, in agreement with theories that includes Lennard-Jones traction-gap law, while Persson-Tosatti disagrees because of its simplifying approximations.
Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery

Open, or non-laparoscopic surgery, represents the vast majority of all operating room procedures, but few tools exist to objectively evaluate these techniques at scale. Current efforts involve human expert-based visual assessment. We leverage advances in computer vision to introduce an automated approach to video analysis of surgical execution. A state-of-the-art convolutional neural network architecture for object detection was used to detect operating hands in open surgery videos. Automated assessment was expanded by combining model predictions with a fast object tracker to enable surgeon-specific hand tracking. To train our model, we used publicly available videos of open surgery from YouTube and annotated these with spatial bounding boxes of operating hands. Our model's spatial detections of operating hands significantly outperforms the detections achieved using pre-existing hand-detection datasets, and allow for insights into intra-operative movement patterns and economy of motion.
Parallel decoding of multiple logical qubits in tensor-network codes

We consider tensor-network stabilizer codes and show that their tensor-network decoder has the property that independent logical qubits can be decoded in parallel. As long as the error rate is below threshold, we show that this parallel decoder is essentially optimal. As an application, we verify this for the max-rate holographic Steane (heptagon) code. For holographic codes this tensor-network decoder was shown to be efficient with complexity polynomial in n, the number of physical qubits. Here we show that, by using the parallel decoding scheme, the complexity is also linear in k, the number of logical qubits. Because the tensor-network contraction is computationally efficient, this allows us to exactly contract tensor networks corresponding to codes with up to half a million qubits. Finally, we calculate the bulk threshold (the threshold for logical qubits a fixed distance from the code centre) under depolarizing noise for the max-rate holographic Steane code to be 9.4%.
The Relationship Between Board Diversity and Earnings Management Practices: Literature Review

The purpose of this paper is to provide a critical review of the existing studies in the literature related to the board diversity aspects and its association with earnings management practices. In general, there are inconsistent findings regarding whether demographic diversity at corporate boards can influence different earnings management practices. Besides, previous studies focused on accrual-based earnings management method and there is a need to include other earnings management methods. The proportion of the minority group on boards should also be taken into consideration when testing the board diversity. The current study contributes to the literature by addressing different future research topics.
Breaking the Expressive Bottlenecks of Graph Neural Networks

Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to measure the expressiveness of graph neural networks (GNNs), showing that the neighborhood aggregation GNNs were at most as powerful as 1-WL test in distinguishing graph structures. There were also improvements proposed in analogy to $k$-WL test ($k>1$). However, the aggregators in these GNNs are far from injective as required by the WL test, and suffer from weak distinguishing strength, making it become expressive bottlenecks. In this paper, we improve the expressiveness by exploring powerful aggregators. We reformulate aggregation with the corresponding aggregation coefficient matrix, and then systematically analyze the requirements of the aggregation coefficient matrix for building more powerful aggregators and even injective aggregators. It can also be viewed as the strategy for preserving the rank of hidden features, and implies that basic aggregators correspond to a special case of low-rank transformations. We also show the necessity of applying nonlinear units ahead of aggregation, which is different from most aggregation-based GNNs. Based on our theoretical analysis, we develop two GNN layers, ExpandingConv and CombConv. Experimental results show that our models significantly boost performance, especially for large and densely connected graphs.
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the prediction capacity. However, there are several severe issues with Transformer that prevent it from being directly applicable to LSTF, including quadratic time complexity, high memory usage, and inherent limitation of the encoder-decoder architecture. To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a ProbSparse self-attention mechanism, which achieves O(L log L) in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences. (iii) the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference speed of long-sequence predictions. Extensive experiments on four large-scale datasets demonstrate that Informer significantly outperforms existing methods and provides a new solution to the LSTF problem.
Optimal maintenance schedule for a wind power turbine with Weibull components

Wind power technology is one of the most important sources of the renewable energy available today. A large part of the wind energy cost is due to the cost of maintaining the wind power equipment, especially for offshore wind farms. To further reduce the maintenance cost, one can improve the design of the wind turbine components, making them more reliable. One can also reduce the maintenance costs by optimal scheduling of the component replacements. The latter task is the main motivation for this paper. When a wind turbine component fails to function, it might need to be replaced under the less than ideal circumstances. This is known as corrective maintenance. To minimise the unnecessary costs, a more active maintenance policy based on life expectancy of the key components is preferred. Optimal scheduling of preventive maintenance activities requires advanced mathematical modelling. In this paper an optimisation model is developed using the renewal-reward theorem. In the multi-component setting, our approach involves a new idea of virtual maintenance which allows us to treat each replacement event as a renewal event even if some components are not replaced by new ones. The proposed optimisation algorithm is applied to a four-component model of a wind turbine and the optimal maintenance plans are computed for various initial conditions.