Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Power of Gradual Network Alignment Using Dual-Perception Similarities

Network alignment (NA) is the task of finding the correspondence of nodes between two networks based on the network structure and node attributes. Our study is motivated by the fact that, since most of existing NA methods have attempted to discover all node pairs <italic>at once</italic>, they do not harness information enriched through <italic>interim</italic> discovery of node correspondences to more accurately find the next correspondences during the node matching. To tackle this challenge, we propose <inline-formula><tex-math notation="LaTeX">${\sf Grad\text{-}Align}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">Grad</mml:mi><mml:mtext mathvariant="sans-serif">-</mml:mtext><mml:mi mathvariant="sans-serif">Align</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="shin-ieq1-3300877.gif"/></alternatives></inline-formula>, a new NA method that <italic>gradually</italic> discovers node pairs by making full use of node pairs exhibiting strong consistency, which are easy to be discovered in the early stage of gradual matching. Specifically, <inline-formula><tex-math notation="LaTeX">${\sf Grad\text{-}Align}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">Grad</mml:mi><mml:mtext mathvariant="sans-serif">-</mml:mtext><mml:mi mathvariant="sans-serif">Align</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="shin-ieq2-3300877.gif"/></alternatives></inline-formula> first generates node embeddings of the two networks based on graph neural networks along with our <italic>layer-wise reconstruction loss</italic>, a loss built upon capturing the first-order and higher-order neighborhood structures. Then, nodes are gradually aligned by computing <italic>dual-perception similarity</italic> measures including the <italic>multi-layer embedding similarity</italic> as well as the <italic>Tversky similarity</italic>, an asymmetric set similarity using the Tversky index applicable to networks with different scales. Additionally, we incorporate an edge augmentation module into <inline-formula><tex-math notation="LaTeX">${\sf Grad\text{-}Align}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">Grad</mml:mi><mml:mtext mathvariant="sans-serif">-</mml:mtext><mml:mi mathvariant="sans-serif">Align</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="shin-ieq3-3300877.gif"/></alternatives></inline-formula> to reinforce the structural consistency. Through comprehensive experiments using real-world and synthetic datasets, we empirically demonstrate that <inline-formula><tex-math notation="LaTeX">${\sf Grad\text{-}Align}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">Grad</mml:mi><mml:mtext mathvariant="sans-serif">-</mml:mtext><mml:mi mathvariant="sans-serif">Align</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="shin-ieq4-3300877.gif"/></alternatives></inline-formula> consistently outperforms state-of-the-art NA methods.
The BrainScaleS-2 Accelerated Neuromorphic System With Hybrid Plasticity

Since the beginning of information processing by electronic components, the nervous system has served as a metaphor for the organization of computational primitives. Brain-inspired computing today encompasses a class of approaches ranging from using novel nano-devices for computation to research into large-scale neuromorphic architectures, such as TrueNorth, SpiNNaker, BrainScaleS, Tianjic, and Loihi. While implementation details differ, spiking neural networks—sometimes referred to as the third generation of neural networks—are the common abstraction used to model computation with such systems. Here we describe the second generation of the BrainScaleS neuromorphic architecture, emphasizing applications enabled by this architecture. It combines a custom analog accelerator core supporting the accelerated physical emulation of bio-inspired spiking neural network primitives with a tightly coupled digital processor and a digital event-routing network.
Natural Language Descriptions of Deep Visual Features

Some neurons in deep networks specialize in recognizing highly specific perceptual, structural, or semantic features of inputs. In computer vision, techniques exist for identifying neurons that respond to individual concept categories like colors, textures, and object classes. But these techniques are limited in scope, labeling only a small subset of neurons and behaviors in any network. Is a richer characterization of neuron-level computation possible? We introduce a procedure (called MILAN, for mutual-information-guided linguistic annotation of neurons) that automatically labels neurons with open-ended, compositional, natural language descriptions. Given a neuron, MILAN generates a description by searching for a natural language string that maximizes pointwise mutual information with the image regions in which the neuron is active. MILAN produces fine-grained descriptions that capture categorical, relational, and logical structure in learned features. These descriptions obtain high agreement with human-generated feature descriptions across a diverse set of model architectures and tasks, and can aid in understanding and controlling learned models. We highlight three applications of natural language neuron descriptions. First, we use MILAN for analysis, characterizing the distribution and importance of neurons selective for attribute, category, and relational information in vision models. Second, we use MILAN for auditing, surfacing neurons sensitive to human faces in datasets designed to obscure them. Finally, we use MILAN for editing, improving robustness in an image classifier by deleting neurons sensitive to text features spuriously correlated with class labels.
The VISCACHA survey -- IV. The SMC West Halo in 8D

The structure of the Small Magellanic Cloud (SMC) is very complex, in particular in the periphery that suffers more from the interactions with the Large Magellanic Cloud (LMC). A wealth of observational evidence has been accumulated revealing tidal tails and bridges made up of gas, stars and star clusters. Nevertheless, a full picture of the SMC outskirts is only recently starting to emerge with a 6D phase-space map plus age and metallicity using star clusters as tracers. In this work, we continue our analysis of another outer region of the SMC, the so-called West Halo, and combined it with the previously analysed Northern Bridge. We use both structures to define the Bridge and Counter-bridge trailing and leading tidal tails. These two structures are moving away from each other, roughly in the SMC-LMC direction. The West Halo form a ring around the SMC inner regions that goes up to the background of the Northern Bridge shaping an extended layer of the Counter-bridge. Four old Bridge clusters were identified at distances larger than 8 kpc from the SMC centre moving towards the LMC, which is consistent with the SMC-LMC closest distance of 7.5 kpc when the Magellanic Bridge was formed about 150Myr ago; this shows that the Magellanic Bridge was not formed only by pulled gas, but it also removed older stars from the SMC during its formation. We also found age and metallicity radial gradients using projected distances on sky, which are vanished when we use the real 3D distances.
Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?

Neural retrieval models are generally regarded as fundamentally different from the retrieval techniques used in the late 1990's when the TREC ad hoc test collections were constructed. They thus provide the opportunity to empirically test the claim that pooling-built test collections can reliably evaluate retrieval systems that did not contribute to the construction of the collection (in other words, that such collections can be reusable). To test the reusability claim, we asked TREC assessors to judge new pools created from new search results for the TREC-8 ad hoc collection. These new search results consisted of five new runs (one each from three transformer-based models and two baseline runs that use BM25) plus the set of TREC-8 submissions that did not previously contribute to pools. The new runs did retrieve previously unseen documents, but the vast majority of those documents were not relevant. The ranking of all runs by mean evaluation score when evaluated using the official TREC-8 relevance judgment set and the newly expanded relevance set are almost identical, with Kendall's tau correlations greater than 0.99. Correlations for individual topics are also high. The TREC-8 ad hoc collection was originally constructed using deep pools over a diverse set of runs, including several effective manual runs. Its judgment budget, and hence construction cost, was relatively large. However, it does appear that the expense was well-spent: even with the advent of neural techniques, the collection has stood the test of time and remains a reliable evaluation instrument as retrieval techniques have advanced.
Enabling Deep Learning on Edge Devices through Filter Pruning and Knowledge Transfer

Deep learning models have introduced various intelligent applications to edge devices, such as image classification, speech recognition, and augmented reality. There is an increasing need of training such models on the devices in order to deliver personalized, responsive, and private learning. To address this need, this paper presents a new solution for deploying and training state-of-the-art models on the resource-constrained devices. First, the paper proposes a novel filter-pruning-based model compression method to create lightweight trainable models from large models trained in the cloud, without much loss of accuracy. Second, it proposes a novel knowledge transfer method to enable the on-device model to update incrementally in real time or near real time using incremental learning on new data and enable the on-device model to learn the unseen categories with the help of the in-cloud model in an unsupervised fashion. The results show that 1) our model compression method can remove up to 99.36% parameters of WRN-28-10, while preserving a Top-1 accuracy of over 90% on CIFAR-10; 2) our knowledge transfer method enables the compressed models to achieve more than 90% accuracy on CIFAR-10 and retain good accuracy on old categories; 3) it allows the compressed models to converge within real time (three to six minutes) on the edge for incremental learning tasks; 4) it enables the model to classify unseen categories of data (78.92% Top-1 accuracy) that it is never trained with.
The fhd polarised imaging pipeline: A new approach to widefield interferometric polarimetry

Abstract We describe a new polarised imaging pipeline implemented in the fhd software package. The pipeline is based on the optimal mapmaking imaging approach and performs horizon-to-horizon image reconstruction in all polarisation modes. We discuss the formalism behind the pipeline’s polarised analysis, describing equivalent representations of the polarised beam response, or Jones matrix. We show that, for arrays where antennas have uniform polarisation alignments, defining a non-orthogonal instrumental polarisation basis enables accurate and efficient image reconstruction. Finally, we present a new calibration approach that leverages widefield effects to perform fully polarised calibration. This analysis pipeline underlies the analysis of Murchison Widefield Array data in Byrne et al. (2022, MNRAS, 510, 2011).
The value of point of sales information in upstream supply chain forecasting: an empirical investigation

Traditionally, manufacturers use past orders (received from some downstream supply chain level) to forecast future ones, before turning such forecasts into appropriate inventory and production optimisation decisions. With recent advances in information sharing technologies, upstream supply chain (SC) companies may have access to downstream point of sales (POS) data. Such data can be used as an alternative source of information for forecasting. There are a few studies that investigate the benefits of using orders versus POS data in upstream SC forecasting; the results are mixed and empirical evidence is lacking, particularly in the context of multi-echelon SCs and in the presence of promotions. We investigate an actual three-echelon SC with 684 series where the manufacturer aims to forecast orders received from distribution centres (DCs) using either aggregated POS data at DCs level or historical orders received from the DCs. Our results show that the order-based methods outperform the POS-based ones by 6–15%. We find that low values of mean, variance, non-linearity and entropy of POS data, and promotion presence negatively impact the performance of the POS-based forecasts. Such findings are useful for determining the appropriate source of data and the impact of series characteristics for order forecasting in SCs.
CodeRetriever: Unimodal and Bimodal Contrastive Learning

In this paper, we propose the CodeRetriever 001 model, which combines the unimodal and bi-002 modal contrastive learning to train function-003 level code semantic representations, specif-004 ically for the code search task. For uni-005 modal contrastive learning, we design a 006 semantic-guided method to build positive code 007 pairs based on the documentation and func-008 tion name. For bimodal contrastive learn-009 ing, we leverage the documentation and in-010 line comments of code to build text-code 011 pairs. Both contrastive objectives can fully 012 leverage the large-scale code corpus for pre-013 training. Experimental results on several pub-014 lic benchmarks, (i.e., CodeSearch, CoSQA, 015 etc.) demonstrate the effectiveness of CodeRe-016 triever in the zero-shot setting. By ﬁne-tuning 017 with domain/language speciﬁed downstream 018 data, CodeRetriever achieves the new state-of-019 the-art performance with signiﬁcant improve-020 ment over existing code pre-trained models. 021 We will make the code, model checkpoint, and 022 constructed datasets publicly available. 023
Exploiting Semantic Epsilon Greedy Exploration Strategy in Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) can model many real world applications. However, many MARL approaches rely on epsilon greedy for exploration, which may discourage visiting advantageous states in hard scenarios. In this paper, we propose a new approach QMIX(SEG) for tackling MARL. It makes use of the value function factorization method QMIX to train per-agent policies and a novel Semantic Epsilon Greedy (SEG) exploration strategy. SEG is a simple extension to the conventional epsilon greedy exploration strategy, yet it is experimentally shown to greatly improve the performance of MARL. We first cluster actions into groups of actions with similar effects and then use the groups in a bi-level epsilon greedy exploration hierarchy for action selection. We argue that SEG facilitates semantic exploration by exploring in the space of groups of actions, which have richer semantic meanings than atomic actions. Experiments show that QMIX(SEG) largely outperforms QMIX and leads to strong performance competitive with current state-of-the-art MARL approaches on the StarCraft Multi-Agent Challenge (SMAC) benchmark.
Class-Aware Generative Adversarial Transformers for Medical Image Segmentation

Transformers have made remarkable progress towards modeling long-range dependencies within the medical image analysis domain. However, current transformer-based models suffer from several disadvantages: (1) existing methods fail to capture the important features of the images due to the naive tokenization scheme; (2) the models suffer from information loss because they only consider single-scale feature representations; and (3) the segmentation label maps generated by the models are not accurate enough without considering rich semantic contexts and anatomical tex-tures. In this work, we present CASTformer , a novel type of generative adversarial transformers, for 2D medical image segmentation. First, we take advantage of the pyramid structure to construct multi-scale representations and handle multi-scale variations. We then design a novel class-aware transformer module to better learn the discriminative regions of objects with semantic structures. Lastly, we utilize an adversarial training strategy that boosts segmentation accuracy and correspondingly allows a transformer-based discriminator to capture high-level semantically correlated contents and low-level anatomical features. Our experiments demonstrate that CASTformer dramatically outperforms previous state-of-the-art transformer-based approaches on three benchmarks, obtaining 2.54%-5.88% absolute improvements in Dice over previous models. Further qualitative experiments provide a more detailed picture of the model’s inner workings, shed light on the challenges in improved transparency, and demonstrate that transfer learning can greatly improve performance and reduce the size of medical image datasets in training, making CASTformer a strong starting point for downstream medical image analysis tasks. Codes and models will be available to the public.
Exploiting Hybrid Models of Tensor-Train Networks For Spoken Command Recognition

This work aims to design a low complexity spoken command recognition (SCR) system by considering different trade-offs between the number of model parameters and classification accuracy. More specifically, we exploit a deep hybrid architecture of a tensor-train (TT) network to build an end-to-end SRC pipeline. Our command recognition system, namely CNN+(TT-DNN), is composed of convolutional layers at the bottom for spectral feature extraction and TT layers at the top for command classification. Compared with a traditional end-to-end CNN baseline for SCR, our proposed CNN+(TTDNN) model replaces fully connected (FC) layers with TT ones and it can substantially reduce the number of model parameters while maintaining the baseline performance of the CNN model. We initialize the CNN+(TT-DNN) model in a randomized manner or based on a well-trained CNN+DNN, and assess the CNN+(TT-DNN) models on the Google Speech Command Dataset. Our experimental results show that the proposed CNN+(TT-DNN) model attains a competitive accuracy of 96.31% with 4 times fewer model parameters than the CNN model. Furthermore, the CNN+(TT-DNN) model can obtain a 97.2% accuracy when the number of parameters is increased.
Fat Tails and Optimal Liability Driven Portfolios

We look at optimal liability-driven portfolios in a family of fat-tailed and extremal risk measures, especially in the context of pension fund and insurance fixed cashflow liability profiles, but also those arising in derivatives books such as delta one books or options books in the presence of stochastic volatilities. In the extremal limit, we recover a new tail risk measure, Extreme Deviation (XD), an extremal risk measure significantly more sensitive to extremal returns than CVaR. Resulting optimal portfolios optimize the return per unit of XD, with portfolio weights consisting of a liability hedging contribution, and a risk contribution seeking to generate positive risk-adjusted return. The resulting allocations are analyzed qualitatively and quantitatively in a number of different limits.
Variational Model Inversion Attacks

Given the ubiquity of deep neural networks, it is important that these models do not reveal information about sensitive data that they have been trained on. In model inversion attacks, a malicious user attempts to recover the private dataset used to train a supervised neural network. A successful model inversion attack should generate realistic and diverse samples that accurately describe each of the classes in the private dataset. In this work, we provide a probabilistic interpretation of model inversion attacks, and formulate a variational objective that accounts for both diversity and accuracy. In order to optimize this variational objective, we choose a variational family defined in the code space of a deep generative model, trained on a public auxiliary dataset that shares some structural similarity with the target dataset. Empirically, our method substantially improves performance in terms of target attack accuracy, sample realism, and diversity on datasets of faces and chest X-ray images.
Post-training Quantization for Neural Networks with Provable Guarantees

While neural networks have been remarkably successful in a wide array of applications, implementing them in resource-constrained hardware remains an area of intense research. By replacing the weights of a neural network with quantized (e.g., 4-bit, or binary) counterparts, massive savings in computation cost, memory, and power consumption are attained. To that end, we generalize a post-training neural-network quantization method, GPFQ, that is based on a greedy path-following mechanism. Among other things, we propose modifications to promote sparsity of the weights, and rigorously analyze the associated error. Additionally, our error analysis expands the results of previous work on GPFQ to handle general quantization alphabets, showing that for quantizing a single-layer network, the relative square error essentially decays linearly in the number of weights -- i.e., level of over-parametrization. Our result holds across a range of input distributions and for both fully-connected and convolutional architectures thereby also extending previous results. To empirically evaluate the method, we quantize several common architectures with few bits per weight, and test them on ImageNet, showing only minor loss of accuracy compared to unquantized models. We also demonstrate that standard modifications, such as bias correction and mixed precision quantization, further improve accuracy.
On Brown's Problem, Poincare' models for the classifying spaces for proper actions and Nielsen Realization

There is the problem, whether for a given virtually torsionfree discrete group Γ there exists a cocompact proper topological Γ-manifold, which is equivariantly homotopy equivalent to the classifying space for proper actions. It is related to Nielsen’s Realization and to the problem of Brown, whether there is a d-dimensional model for the classifying space for proper actions, if the underlying group has virtually cohomological dimension d. Assuming that the expected manifold model has a zero-dimensional singular set, we solve the problem in the Poincaré category and obtain new results about Brown’s problem under certain conditions concerning the underlying group, for instance if it is hyperbolic. In a sequel paper together with James Davis we will deal with this on the level of topological manifolds.
Pillai's conjecture for polynomials

In this paper we study the polynomial version of Pillai's conjecture on the exponential Diophantine equation -17ex p^n - q^m = f. We prove that for any non-constant polynomial \( f \) there are only finitely many quadruples \( (n,m,\deg p,\deg q) \) consisting of integers \( n,m \geq 2 \) and non-constant polynomials \( p,q \) such that Pillai's equation holds. Moreover, we will give some examples that there can still be infinitely many possibilities for the polynomials \( p,q \).
A model for the spread of fire blight during bloom

We derive an infectious disease model to describe the spread of fire blight during bloom. By coupling the disease dynamics of the stationary host with the transport of the pathogen by vectors we are able to investigate the blossom blight disease cycle in a spatial setting. We use Schauder's fixed point theorem in combination with the method of upper and lower solutions to prove the existence of travelling wave solutions when the model parameters satisfy certain constraints. By studying numerical simulations we argue that it is likely that travelling waves exist for model parameters that do not satisfy these constraints. Through a sensitivity analysis we show that ooze-carrying vectors and orchard density both play a significant role in the severity of a blossom blight epidemic when compared to the growth rate of the pathogen and the mobility of ooze-carrying vectors.