Miracle-dz/UniFAR-FTUs huggingface dataset:
Distribution Embedding Networks for Meta-Learning with Heterogeneous Covariate Spaces

We propose Distribution Embedding Networks (DEN) for classiﬁcation with small data using meta-learning techniques. Unlike existing meta-learning approaches that focus on image recognition tasks and require the training and target tasks to be similar, DEN is speciﬁcally designed to be trained on a diverse set of training tasks and applied on tasks whose number and distribution of covariates diﬀer vastly from its training tasks. Such property of DEN is enabled by its three-block architecture: a covariate transformation block followed by a distribution embedding block and then a classiﬁcation block. We provide theoretical insights to show that this architecture allows the embedding and classiﬁcation blocks to be ﬁxed after pre-training on a diverse set of tasks; only the covariate transformation block with relatively few parameters needs to be updated for each new task. To facilitate the training of DEN, we also propose an approach to synthesize binary classiﬁcation training tasks, and demonstrate that DEN outperforms existing methods in a number of synthetic and real tasks in numerical studies.
Modularization, Composition, and Hierarchization of Petri Nets with Heraklit

It is known for decades that computer-based systems cannot be understood without a concept of modularization and decomposition. We suggest a universal, expressive, intuitively attractive composition operator for Petri nets, combined with a refinement concept and an algebraic representation of nets and their composition. Case studies show exemplarily, how large systems can be composed from tiny net snippets. In the future, more field studies are needed to better understand the consequences of the proposed ideas in the real world.
Monoidal Streams for Dataflow Programming

We introduce monoidal streams: a generalization of causal stream functions to monoidal categories. In the same way that streams provide semantics to dataflow programming with pure functions, monoidal streams provide semantics to dataflow programming with theories of processes represented by a symmetric monoidal category. At the same time, monoidal streams form a feedback monoidal category, which can be used to interpret signal flow graphs. As an example, we study a stochastic dataflow language.
An Investigation into the Kinetics and Mechanism of Phase Transitions in Optical Phase Change Ternary Alloy $Ge_2Sb_2Te_5$

Ternary alloys of Ge-Sb-Te (GST) have been extensively studied due to their unique ability display a reversible change in their phase upon stimulation by optical pulses i.e amorphous (a-GST) to crystalline (c-GST) and vice-versa. The two phases exhibit remarkably diﬀerent electrical and optical properties like conductivity, reﬂectivity, refractive index and optical loss, this coupled with their high phase switching speeds, low power phase switching, large switching cycles, large measur-able optical and electrical contrast, and phase stability makes GST alloys stand out from other phase change materials (PCM) . GST alloys have already found ex-tensive use in optical disks and electronic memories due to their non-volatility and zero static power consumption, but the precise mechanism of the phase change is not clearly understood. The phase change mechanism has usually been attributed to the optical pulse, usually a high power short pulse laser, heating up the c-GST alloy to above its melting temperature ( T m ) after which, if it is cooled rapidly enough to below the glass transition temperature ( T g ), the atoms are ﬁxed in place due to the drastic reduction in their mobility, resulting in a phase which exhibits structure similar to a frozen liquid and lacks long range order i.e amorphous. If al-ternatively the a-GST is heated above T g with a intermediate power laser pulse for a signiﬁcant amount of time to induce nucleation, then this favours the shift back to the energetically favourable crystalline phase. With the growing interest in next generation data storage technology for photonic and neuromorphic computing, the research into understanding and improving the properties of GST alloys has also been rekindled. In this term paper we hope to investigate and gain a deeper understanding and appreciation of the kinetics and underlying atomistic mechanism of this phase transition from c-GST to a-GST and vice-versa which is only now becoming clearer.
Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain–computer interfaces

Objective. Covariance matrices of noisy multichannel electroencephalogram (EEG) time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain–computer interfaces (BCI) based on event-related potentials (ERP) and a linear discriminant analysis (LDA) classifier, the state of the art covariance estimation uses shrinkage regularization. As this is a general covariance regularization approach, we aim at improving LDA further by better exploiting the domain-specific characteristics of the EEG to regularize the covariance estimates. Approach. We propose to enforce a block-Toeplitz structure for the covariance matrix of the LDA, which implements an assumption of signal stationarity in short time windows for each channel. Main results. An offline re-analysis of data collected from 213 subjects under 13 different event-related potential BCI protocols showed a significantly increased binary classification performance of this ‘ToeplitzLDA’ compared to shrinkage regularized LDA (up to 6 AUC points, p < 0.001) and Riemannian classification approaches (up to 2 AUC points, p < 0.001). In an unsupervised visual speller application, this improvement would translate to a relative reduction of spelling errors by 81% on average for 25 subjects. Additionally, aside from lower memory and reduced time complexity for LDA training, ToeplitzLDA proves to be robust against drastic increases of the number of temporal features. Significance. The proposed covariance estimation allows BCI researchers to improve classification rates and reduce calibration times of BCI protocols using event-related potentials and thus support the usability of corresponding applications. Its lower computational and memory needs could make it a valuable algorithm especially for mobile BCIs.
Flow Redirection for Epidemic Reaction-DIffusion Control

We show we can control an epidemic reaction-diffusion on a directed, and heterogeneous, network by redirecting the flows, thanks to the optimisation of well-designed loss functions, in particular the basic reproduction number of the model. We provide a final size relation linking the basic reproduction number to the epidemic final sizes, for diffusions around a reference diffusion with basic reproduction number less than 1. Experimentally, we show control is possible for different topologies, network heterogeneity levels, and speeds of diffusion. Our experimental results highlight the relevance of the basic reproduction number loss, compared to more straightforward losses.
PSO-PINN: Physics-Informed Neural Networks Trained with Particle Swarm Optimization

Physics-informed neural networks (PINN) have recently emerged as a promising application of deep learning in a wide range of engineering and scientiﬁc problems based on partial differential equation (PDE) models. However, evidence shows that PINN training by gradient descent displays pathologies that often prevent convergence when solving PDEs with irregular solutions. In this paper, we propose the use of a particle swarm optimization (PSO) approach to train PINNs. The resulting PSO-PINN algorithm not only mitigates the undesired behaviors of PINNs trained with standard gradient descent but also presents an ensemble approach to PINN that affords the possibility of robust predictions with quantiﬁed uncertainty. We also propose PSO-BP-CD (PSO with Back-Propagation and Coefﬁcient Decay), a hybrid PSO variant that combines swarm optimization with gradient descent, putting more weight on the latter as training progresses and the swarm zeros in on a good local optimum. Comprehensive experimental results show that PSO-PINN with the proposed PSO-BP-CD algorithm outperforms PINN ensembles trained with other PSO variants or with pure gradient descent.
From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer

Knowledge graph completion aims to address the problem of extending a KG with missing triples. In this paper, we provide an approach GenKGC, which converts knowledge graph completion to sequence-to-sequence generation task with the pre-trained language model. We further introduce relation-guided demonstration and entity-aware hierarchical decoding for better representation learning and fast inference. Experimental results on three datasets show that our approach can obtain better or comparable performance than baselines and achieve faster inference speed compared with previous methods with pre-trained language models. We also release a new large-scale Chinese knowledge graph dataset OpenBG500 for research purpose1.
EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators

Dilated and transposed convolutions are widely used in modern convolutional neural networks (CNNs). These kernels are used extensively during CNN training and inference of applications such as image segmentation and high-resolution image generation. Although these kernels have grown in popularity, they stress current compute systems due to their high memory intensity, exascale compute demands, and large energy consumption. We find that commonly-used low-power CNN inference accelerators based on spatial architectures are not optimized for both of these convolutional kernels. Dilated and transposed convolutions introduce significant zero padding when mapped to the underlying spatial architecture, significantly degrading performance and energy efficiency. Existing approaches that address this issue require significant design changes to the otherwise simple, efficient, and well-adopted architectures used to compute direct convolutions. To address this challenge, we propose EcoFlow, a new set of dataflows and mapping algorithms for dilated and transposed convolutions. These algorithms are tailored to execute efficiently on existing low-cost, small-scale spatial architectures and requires minimal changes to the network-on-chip of existing accelerators. EcoFlow eliminates zero padding through careful dataflow orchestration and data mapping tailored to the spatial architecture. EcoFlow enables flexible and high-performance transpose and dilated convolutions on architectures that are otherwise optimized for CNN inference. We evaluate the efficiency of EcoFlow on CNN training workloads and Generative Adversarial Network (GAN) training workloads. Experiments in our new cycle-accurate simulator show that EcoFlow 1) reduces end-to-end CNN training time between 7-85%, and 2) improves end-to-end GAN training performance between 29-42%, compared to state-of-the-art CNN inference accelerators.
Flexible Triggering Kernels for Hawkes Process Modeling

Recently proposed encoder-decoder structures for modeling Hawkes processes use transformer-inspired architectures, which encode the history of events via embeddings and self-attention mechanisms. These models deliver better prediction and goodness-of-fit than their RNN-based counterparts. However, they often require high computational and memory complexity requirements and sometimes fail to adequately capture the triggering function of the underlying process. So motivated, we introduce an efficient and general encoding of the historical event sequence by replacing the complex (multilayered) attention structures with triggering kernels of the observed data. Noting the similarity between the triggering kernels of a point process and the attention scores, we use a triggering kernel to replace the weights used to build history representations. Our estimate for the triggering function is equipped with a sigmoid gating mechanism that captures local-in-time triggering effects that are otherwise challenging with standard decaying-over-time kernels. Further, taking both event type representations and temporal embeddings as inputs, the model learns the underlying triggering type-time kernel parameters given pairs of event types. We present experiments on synthetic and real data sets widely used by competing models, while further including a COVID-19 dataset to illustrate a scenario where longitudinal covariates are available. Results show the proposed model outperforms existing approaches while being more efficient in terms of computational complexity and yielding interpretable results via direct application of the newly introduced kernel.
Systems for Memory Disaggregation: Challenges & Opportunities

Memory disaggregation addresses memory imbalance in a cluster by decoupling CPU and memory allocations of applications while also increasing the effective memory capacity for (memory-intensive) applications beyond the local memory limit imposed by traditional fixed-capacity servers. As the network speeds in the tightly-knit environments like modern datacenters inch closer to the DRAM speeds, there has been a recent proliferation of work in this space ranging from software solutions that pool memory of traditional servers for the shared use of the cluster to systems targeting the memory disaggregation in the hardware. In this report, we look at some of these recent memory disaggregation systems and study the important factors that guide their design, such as the interface through which the memory is exposed to the application, their runtime design and relevant optimizations to retain the near-native application performance, various approaches they employ in managing cluster memory to maximize utilization, etc. and we analyze the associated trade-offs. We conclude with a discussion on some open questions and potential future directions that can render disaggregation more amenable for adoption.
Stochastic effects in axion inflation and primordial black hole formation

We revisit the model of axion inflation in the context of stochastic inflation and investigate the effects of the stochastic noises associated to the electromagnetic fields. Because of the parity violating interaction, one polarization of the gauge field is amplified inducing large curvature perturbation power spectrum. Taking into account the stochastic kicks arising from the short modes at the time of horizon crossing we obtain the corresponding Langevin equations for the long modes of the electromagnetic and axion fields. It is shown that a mean-reverting process governs the dynamics of the electromagnetic fields such that the tachyonic growth of the gauge fields is balanced by the diffusion forces. As the instability parameter grows towards the end of inflation, the large curvature perturbations induced from gauge field perturbations lead to copious production of small mass primordial black holes (PBHs). It is shown that the produced PBHs follow a Gaussian statistics. Imposing the observational constraints on PBHs formation relaxes the previous bounds on the instability parameter by about fifty percents.
Even Simpler Deterministic Matrix Sketching

This paper provides a one-line proof of Frequent Directions (FD) for sketching streams of matrices. It simplifies the main results in [1] and [2]. The simpler proof arises from sketching the covariance of the stream of matrices rather than the stream itself. Introduction Let Xt ∈ R d×nt be a stream of matrices. Let C = ∑ T t=1 XtX T t ∈ R d×d be their covariance matrix. Frequent Directions [1] maintains a rank deficient approximate covariance matrix C̃t ∈ R d×d using Algorithm 1. Set C̃0 ∈ R d×d to be the all zeros matrix. Then, at time t = 1, . , T compute C̃t = Update(C̃t−1, Xt, l). Algorithm 1 Frequent Directions (FD) Update 1: function Update(C̃t−1, Xt, l) 2: UtΛtU T t = C̃t−1 +XtX T t 3: return C̃t = U ·max(Λ − I · λ t l , 0) · U 4: end function Above, UtΛtU T t is the eigen-decomposition of C̃t−1+XtX T t and λ l is the its l’th largest eigenvalue. Note that the rank of C̃t is at most l − 1 for all t by construction. It can therefore be stored in O(dl) space. Assuming nt < l, the update operation itself also consumes at most O(dl) space. Lemma 1 (simplified from [2] and [1]). Let C̃ denote the approximated covariance produced by FD and λi be the eigenvalues of the exact covariance C in descending order. For any l and simultaneously for all k < l we have ‖C − C̃‖ ≤ 1 l− k d
Beware of Greeks bearing entanglement? Quantum covert channels, information flow and non-local games

Can quantum entanglement increase the capacity of (classical) covert channels? To one familiar with Holevo's Theorem it is tempting to think that the answer is obviously no. However, in this work we show: quantum entanglement can in fact increase the capacity of a classical covert channel, in the presence of an active adversary; on the other hand, a zero-capacity channel is not improved by entanglement, so entanglement cannot create ‘purely quantum’ covert channels; the problem of determining the capacity of a given channel in the presence of entanglement is undecidable; but there is an algorithm to bound the entangled capacity of a channel from above, adapted from the semi-definite hierarchy from the theory of non-local games, whose close connection to channel capacity is at the core of all of our results.
Interactive Mobile App Navigation with Uncertain or Under-specified Natural Language Commands

We introduce Mobile app Tasks with Iterative Feedback (MoTIF), a new dataset where the goal is to complete a nat-ural language query in a mobile app. Current datasets for related tasks in interactive question answering, visual common sense reasoning, and question-answer plausibility prediction do not support research in resolving ambiguous nat-ural language requests or operating in diverse digital domains. As a result, they fail to capture complexities of real question answering or interactive tasks. In contrast, MoTIF contains natural language requests that are not satisﬁable, the ﬁrst such work to investigate this issue for interactive vision-language tasks. MoTIF also contains follow up questions for ambiguous queries to enable research on task uncertainty resolution. We introduce task feasibility prediction and propose an initial model which obtains an F1 score of 61.1. We next benchmark task automation with our dataset and ﬁnd adaptations of prior work perform poorly due to our realistic language requests, obtaining an accuracy of only 20.2% when mapping commands to grounded actions. We analyze performance and gain insight for future work that may bridge the gap between current model ability and what is needed for successful use in application.
Color Image Inpainting via Robust Pure Quaternion Matrix Completion: Error Bound and Weighted Loss

In this paper, we study color image inpainting as a pure quaternion matrix completion problem. In the literature, the theoretical guarantee for quaternion matrix completion is not well-established. Our main aim is to propose a new minimization problem with an objective combining nuclear norm and a quadratic loss weighted among three channels. To fill the theoretical vacancy, we obtain the error bound in both clean and corrupted regimes, which relies on some new results of quaternion matrices. A general Gaussian noise is considered in robust completion where all observations are corrupted. Motivated by the error bound, we propose to handle unbalanced or correlated noise via a cross-channel weight in the quadratic loss, with the main purpose of rebalancing noise level, or removing noise correlation. Extensive experimental results on synthetic and color image data are presented to confirm and demonstrate our theoretical findings.
Modeling unknown dynamical systems with hidden parameters

We present a data-driven numerical approach for modeling unknown dynamical systems with miss-ing/hidden parameters. The method is based on training a deep neural network (DNN) model for the unknown system using its trajectory data. A key feature is that the unknown dynamical system contains system parameters that are completely hidden, in the sense that no information about the parameters is available through either the measurement trajectory data or our prior knowledge of the system. We demonstrate that by training a DNN using the trajectory data with sufﬁcient time history, the resulting DNN model can accurately model the unknown dynamical system. For new initial conditions associated with new, and unknown, system parameters, the DNN model can produce accurate system predictions over longer time.
Twitter Referral Behaviours on News Consumption with Ensemble Clustering of Click-Stream Data in Turkish Media

Click-stream data, which comes with a massive volume generated by the human activities on the websites, has become a prominent feature to identify readers’ characteristics by the newsrooms after the digitization of the news outlets. It is essential to have elastic architectures to process the streaming data, particularly for unprecedented traﬃc, enabling conducting more comprehensive analyses such as recommending mostly related articles to the readers. Although the nature of click-stream data has a similar logic within the websites, it has inherent limitations to recognize human behaviors when looking from a broad perspective, which brings the need of limiting the problem in niche areas. This study investigates the anonymized readers’ click activities in the organizations’ websites to identify news consumption patterns following referrals from Twitter, who incidentally reach but propensity is mainly the routed news content. The investigation is widened to a broad perspective by linking the log data with news content to enrich the insights rather than sticking into the web journey. The methodologies on ensemble cluster analysis with mixed-type embedding strategies are applied and compared to ﬁnd similar reader groups and interests independent from time. Our results demonstrate that the quality of clustering mixed-type data set approaches to optimal internal validation scores when embedded by Uniform Manifold Approximation and Projection (UMAP) and using consensus function as a key to access the most applicable hyper parameter conﬁgurations in the given ensemble rather than using consensus function results directly. Evaluation of the resulting clusters highlights speciﬁc clusters repeatedly present in the samples, which provide insights to the news organizations and overcome the degradation of the modeling behaviors due to the change in the interest over time.
Gain-dependent Purcell enhancement, breakdown of Einstein's relations, and superradiance in nanolasers

Light emitters in a single-mode nanolaser interact with the same cavity field, that gives rise to polarization correlations which transform the cavity mode. Usually these correlations are ignored, however, collective phenomena can lead to the distinct suband superradiance, whose fully quantum description is challenging. Here, we develop a simple yet rigorous picture of radiative transitions in single-mode nanolasers that accounts for polarization correlations. We show that the collective behavior of emitters modifies the photonic density of states leading to gain-dependent Purcell enhancement of spontaneous emission. Moreover, the stimulated emission rate is dependent on both the photon number and the laser lineshape. As the laser line narrows, stimulated emission becomes stronger than predicted by Einstein’s relations and the nanolaser reaches the threshold earlier. Finally, we provide concise, ready-to-use expressions for spontaneous and stimulated emission rates seamlessly describing both conventional and superradiant nanolasers.