Miracle-dz/UniFAR-FTUs huggingface dataset:
Estimating longterm power spectral densities in AGN from simulations

The power spectral density (PSD) represents a key property quantifying the stochastic or random noise type fluctuations in variable sources like Active Galactic Nuclei (AGN). In recent years, estimates of the PSD have been refined by improvements in both, the quality of observed lightcurves and modeling them with simulations. This has aided in quantifying the variability including evaluating the significance of quasi-periodic oscillations. A central assumption in making such estimates is that of weak non-stationarity. This is violated for sources with a power-law PSD index steeper than one as the integral power diverges. As a consequence, estimates of the flux probability density function (PDF) and PSD are interlinked. In general, for evaluating parameters of both properties from lightcurves, one cannot avoid a multi-dimensional, multi-parameter model which is complex and computationally expensive, as well as harder to constrain and interpret. However, if we only wish to compute the PSD index as is often the case, we can use a simpler model. We explore a bending power-law model instead of a simple power-law as input to time-series simulations to test the quality of reconstruction. Examining the longterm variability of the classical blazar Mrk 421, extending to multiple years as is typical of Fermi-LAT or Swift-BAT lightcurves, we find that a transition from pink (PSD index one) to white noise at a characteristic timescale, $t_b \sim 500-1000$ years, comparable to the viscous timescale at the disk truncation radius, seems to provide a good model for simulations. This is both a physically motivated as well as a computationally efficient model that can be used to compute the PSD index.
No Spurious Local Minima: on the Optimization Landscapes of Wide and Deep Neural Networks

Empirical studies suggest that wide neural networks are comparably easy to optimize, but mathematical support for this observation is scarce. In this paper, we analyze the optimization landscapes of deep learning with wide networks. We prove especially that constraint and unconstraint empirical-risk minimization over such networks has no spurious local minima. Hence, our theories substantiate the common belief that increasing network widths not only improves the expressiveness of deep-learning pipelines but also facilitates their optimizations.
Weight Encode Reconstruction Network for Computed Tomography in a Semi-Case-Wise and Learning-Based Way

Classic algebraic reconstruction technology (ART) for computed tomography requires pre-determined weights of the voxels for the projected pixel values to build the equations. However, such weights cannot be accurately obtained due to the high physical complexity and computation resources required. In this study, we propose a semi-case-wise learning-based method named Weight Encode Reconstruction Network (WERNet) to co-learn the target voxel values and intrinsic physics of the case in a self-supervised manner without labeling the target voxel set. With the help of gradient normalization, the WERNet reconstructed voxel set with a high accuracy and showed a higher capability of denoising compared to the classic ART methods. Moreover, the encoder of the network is transferable from a voxel set with complex structures to unseen cases without the deduction of the accuracy. Our method can be applied in tomography-related applications and similar inversion problems even with unclear intrinsic physics.
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale

We study the zero-shot transfer capabilities of text matching models on a massive scale, by self-supervised training on 140 source domains from community question answering forums in English. We investigate the model performances on nine benchmarks of answer selection and question similarity tasks, and show that all 140 models transfer surprisingly well, where the large majority of models substantially outperforms common IR baselines. We also demonstrate that considering a broad selection of source domains is crucial for obtaining the best zero-shot transfer performances, which contrasts the standard procedure that merely relies on the largest and most similar domains. In addition, we extensively study how to best combine multiple source domains. We propose to incorporate self-supervised with supervised multi-task learning on all available source domains. Our best zero-shot transfer model considerably outperforms in-domain BERT and the previous state of the art on six benchmarks. Fine-tuning of our model with in-domain data results in additional large gains and achieves the new state of the art on all nine benchmarks.
Note: An alternative proof of the vulnerability of k-NN classifiers in high intrinsic dimensionality regions

This document proposes an alternative proof of the result contained in article "High intrinsic dimensionality facilitates adversarial attack: Theoretical evidence", Amsaleg et a. The proof is simpler to understand (I believe) and leads to a more precise statement about the asymptotical distribution of the relative amount of perturbation.
DIRV: Dense Interaction Region Voting for End-to-End Human-Object Interaction Detection

Recent years, human-object interaction (HOI) detection has achieved impressive advances. However, conventional two-stage methods are usually slow in inference. On the other hand, existing one-stage methods mainly focus on the union regions of interactions, which introduce unnecessary visual information as disturbances to HOI detection. To tackle the problems above, we propose a novel one-stage HOI detection approach DIRV in this paper, based on a new concept called interaction region for the HOI problem. Unlike previous methods, our approach concentrates on the densely sampled interaction regions across different scales for each human-object pair, so as to capture the subtle visual features that is most essential to the interaction. Moreover, in order to compensate for the detection flaws of a single interaction region, we introduce a novel voting strategy that makes full use of those overlapped interaction regions in place of conventional Non-Maximal Suppression (NMS). Extensive experiments on two popular benchmarks: V-COCO and HICO-DET show that our approach outperforms existing state-of-the-arts by a large margin with the highest inference speed and lightest network architecture. Our code is publicly available at www.github. com/MVIG-SJTU/DIRV.
Revisiting Runtime Dynamic Optimization for Join Queries in Big Data Management Systems

Effective query optimization remains an open problem for Big Data Management Systems. In this work, we revisit an old idea, runtime dynamic optimization, and adapt it to a big data management system, AsterixDB. The approach runs in stages (re-optimization points), starting by first executing all predicates local to a single dataset. The intermediate result created by a stage is then used to re-optimize the remaining query. This re-optimization approach avoids inaccurate intermediate result cardinality estimates, thus leading to much better execution plans. While it introduces overhead for materializing intermediate results, experiments show that this overhead is relatively small and is an acceptable price to pay given the optimization benefits.
Super-Nyquist Co-Prime Sensing.

The theory of co-prime arrays has been studied in the past. Nyquist rate estimation of second order statistics using the combined difference set was demonstrated with low latency. This paper proposes a novel method to reconstruct the second order statistics at a rate that is twice the Nyquist rate using the same sub-Nyquist co-prime samplers. We analyse the difference set, and derive the closed-form expressions for the weight function and the bias of the correlogram estimate. The main lobe width of the bias window is approximately half of the width obtained using the prototype co-prime sampler. Since the proposed scheme employs the same rate prototype co-prime samplers; the number of samples acquired in one co-prime period and hardware cost are unaffected. Super-Nyquist estimation with multiple co-prime periods is also described. Furthermore, n-tuple or multi-level co-prime structure is presented from a super-Nyquist perspective. Here, estimation at a rate q times higher than Nyquist is possible, where q is the number of sub-samplers.
Encoded Prior Sliced Wasserstein AutoEncoder for learning latent manifold representations

While variational autoencoders have been successful generative models for a variety of tasks, the use of conventional Gaussian or Gaussian mixture priors are limited in their ability to capture topological or geometric properties of data in the latent representation. In this work, we introduce an Encoded Prior Sliced Wasserstein AutoEncoder (EPSWAE) wherein an additional prior-encoder network learns an unconstrained prior to match the encoded data manifold. The autoencoder and prior-encoder networks are iteratively trained using the Sliced Wasserstein Distance (SWD), which efficiently measures the distance between two $\textit{arbitrary}$ sampleable distributions without being constrained to a specific form as in the KL divergence, and without requiring expensive adversarial training. Additionally, we enhance the conventional SWD by introducing a nonlinear shearing, i.e., averaging over random $\textit{nonlinear}$ transformations, to better capture differences between two distributions. The prior is further encouraged to encode the data manifold by use of a structural consistency term that encourages isometry between feature space and latent space. Lastly, interpolation along $\textit{geodesics}$ on the latent space representation of the data manifold generates samples that lie on the manifold and hence is advantageous compared with standard Euclidean interpolation. To this end, we introduce a graph-based algorithm for identifying network-geodesics in latent space from samples of the prior that maximize the density of samples along the path while minimizing total energy. We apply our framework to 3D-spiral, MNIST, and CelebA datasets, and show that its latent representations and interpolations are comparable to the state of the art on equivalent architectures.
A Survey of the State of Explainable AI for Natural Language Processing

Recent years have seen important advances in the quality of state-of-the-art models, but this has come at the expense of models becoming less interpretable. This survey presents an overview of the current state of Explainable AI (XAI), considered within the domain of Natural Language Processing (NLP). We discuss the main categorization of explanations, as well as the various ways explanations can be arrived at and visualized. We detail the operations and explainability techniques currently available for generating explanations for NLP model predictions, to serve as a resource for model developers in the community. Finally, we point out the current gaps and encourage directions for future work in this important research area.
Discontinuous Ground States for NLSE on $\mathbb{R}$ with a F\"{u}l\"{o}p-Tsutsui $\delta$ interaction

We analyse the existence and the stability of the ground states of the one-dimensional nonlinear Schrodinger equation with a focusing power nonlinearity and a defect located at the origin. In this paper a ground state is defined as a global minimizer of the action functional on the Nehari manifold and the defect considered is a Fulop-Tsutsui $\delta$ type, namely a $\delta$ condition that allows discontinuities. The existence of ground states is proved by variational techniques, while the stability results from the Grillakis-Shatah-Strauss theory.
Error trade-off relations for two-parameter unitary model with commuting generators

We investigate whether a trade-off relation between the diagonal elements of the mean square error matrix exists for the two-parameter unitary models with mutually commuting generators. We show that the error trade-off relation which exists in our models of a finite dimension system is a generic phenomenon in the sense that it occurs with a finite volume in the spate space. We analyze a qutrit system to show that there can be an error trade-off relation given by the SLD and RLD Cramer-Rao bounds that intersect each other. First, we analyze an example of the reference state showing the non-trivial trade-off relation numerically, and find that its eigenvalues must be in a certain range to exhibit the trade-off relation. For another example, one-parameter family of reference states, we analytically show that the non-trivial relation always exists and that the range where the trade-off relation exists is up to about a half of the possible range.
Exact Penalties for Decomposable Optimization Problems

We consider a general decomposable convex optimization problem. By using right-hand side allocation technique, it can be transformed into a collection of small dimensional optimization problems. The master problem is a convex non-smooth optimization problem. We propose to apply the exact non-smooth penalty method, which gives a solution of the initial problem under some fixed penalty parameter and provides the consistency of lower level problems. The master problem is suggested to be solved by a two-speed subgradient projection method, which enhances the step-size selection. Preliminary results of computational experiments confirm its efficiency.
A variable metric mini-batch proximal stochastic recursive gradient algorithm with diagonal Barzilai-Borwein stepsize

Variable metric proximal gradient methods with different metric selections have been widely used in composite optimization. Combining the Barzilai-Borwein (BB) method with a diagonal selection strategy for the metric, the diagonal BB stepsize can keep low per-step computation cost as the scalar BB stepsize and better capture the local geometry of the problem. In this paper, we propose a variable metric mini-batch proximal stochastic recursive gradient algorithm VM-mSRGBB, which updates the metric using a new diagonal BB stepsize. The linear convergence of VM-mSRGBB is established for strongly convex, non-strongly convex and convex functions. Numerical experiments on standard data sets show that VM-mSRGBB is better than or comparable to some variance reduced stochastic gradient methods with best-tuned scalar stepsizes or BB stepsizes. Furthermore, the performance of VM-mSRGBB is superior to some advanced mini-batch proximal stochastic gradient methods.
Effects of high CO2 levels on the ecophysiology of the diatom Thalassiosira weissflogii differ depending on the iron nutritional status

Iron availability in seawater, namely the concentrationofdissolved inorganic iron ([Fe′]), is affectedby changes inpH. Such changes in the availabilityof iron shouldbe taken intoaccountwhen investigating theeffects of oceanacidificationonphytoplanktonecophysiologybecause ironplays a key role in phytoplankton metabolism. However, changes in iron availability in response to changes in ocean acidity are difficult to quantify specifically using natural seawater because these factors change simultaneously. In the present study, the availability of iron and carbonate chemistryweremanipulated individually and simultaneously in the laboratory to examine the effect of each factor on phytoplankton ecophysiology. The effects of various pCO2 conditions ( 390, 600, and 800 matm)onthegrowth, cell size, andelemental stoichiometry (carbon [C],nitrogen[N], phosphorus [P], and silicon [Si]) of the diatom Thalassiosira weissflogii under high iron ([Fe′] 1⁄4 240 pmol l) and low iron ([Fe′] 1⁄4 24 pmol l) conditions were investigated. Cell volume decreased with increasing pCO2, whereas intracellular C, N, and P concentrations increased with increasing pCO2 only under high iron conditions. Si:C, Si:N, and Si:P ratios decreasedwith increasing pCO2. It reflects higher production of net C,N, and Pwithno corresponding change in net Si production under high pCO2 and high iron conditions. In contrast, significant linear relationships between measured parameters and pCO2 were rarely detected under low iron conditions.We conclude that the increasingCO2 levels could affect on the biogeochemical cycling of bioelements selectively under the iron-replete conditions in the coastal ecosystems.
Attention-Based Clustering: Learning a Kernel from Context

In machine learning, no data point stands alone. We believe that context is an underappreciated concept in many machine learning methods. We propose Attention-Based Clustering (ABC), a neural architecture based on the attention mechanism, which is designed to learn latent representations that adapt to context within an input set, and which is inherently agnostic to input sizes and number of clusters. By learning a similarity kernel, our method directly combines with any out-of-the-box kernel-based clustering approach. We present competitive results for clustering Omniglot characters and include analytical evidence of the effectiveness of an attention-based approach for clustering.
The M Supergiant High-mass X-Ray Binary 4U 1954+31

The X-ray binary 4U 1954+31 has been classified as a low-mass X-ray binary containing an M giant and a neutron star (NS). It has also been included in the rare class of X-ray symbiotic binaries. The Gaia parallax, infrared colors, spectral type, abundances, and orbital properties of the M star demonstrate that the cool star in this system is not a low-mass giant but a high-mass M supergiant. Thus, 4U 1954+31 is a high-mass X-ray binary (HMXB) containing a late-type supergiant. It is the only known binary system of this type. The mass of the M I is M ⊙ giving an age of this system in the range 12–50 Myr with the NS no more than 43 Myr old. The spin period of the NS is one of the longest known, 5 hr. The existence of M I plus NS binary systems is in accord with stellar evolution theory, with this system a more evolved member of the HMXB population.
Efficient Estimation of Graph Trussness

A $k$-truss is an edge-induced subgraph $H$ such that each of its edges belongs to at least $k-2$ triangles of $H$. This notion has been introduced around ten years ago in social network analysis and security, as a form of cohesive subgraph that is rich of triangles and less stringent than the clique. The \emph{trussness} of a graph is the maximum $k$ such that a $k$-truss exists. The problem of computing $k$-trusses has been largely investigated from the practical and engineering point of view. On the other hand, the theoretical side of the problem has received much less attention, despite presenting interesting challenges. The existing methods share a common design, based on iteratively removing the edge with smallest support, where the support of an edge is the number of triangles containing it. The aim of this paper is studying algorithmic aspects of graph trussness. While it is possible to show that the time complexity of computing exactly the graph trussness and that of counting/listing all triangles is inherently the same, we provide efficient algorithms for estimating its value, under suitable conditions, with significantly lower complexity than the exact approach. In particular, we provide a $(1 \pm \epsilon)$-approximation algorithm that is asymptotically faster than the exact approach, on graphs which contain $\omega(m \, polylog(n))$ triangles, and has the same running time on graphs that do not. For the latter case, we also show that it is impossible to obtain an approximation algorithm with faster running time than the one of the exact approach when the number of triangles is $O(m)$, unless well known conjectures on triangle-freeness and Boolean matrix multiplication are false.
Towards privacy aware deep learning for embedded systems

Memorization of training data by deep neural networks enables an adversary to mount successful membership inference attacks. Here, an adversary with blackbox query access to the model can infer whether an individual's data record was part of the model's sensitive training data using only the output predictions. This violates the data confidentiality, by inferring samples from proprietary training data, and privacy of the individual whose sensitive record was used to train the model. This privacy threat is profound in commercial embedded systems with on-device processing. Addressing this problem requires neural networks to be inherently private by design while conforming to the memory, power and computation constraints of embedded systems. This is lacking in literature. We present the first work towards membership privacy by design in neural networks while reconciling privacy-accuracy-efficiency trade-offs for embedded systems. We conduct an extensive privacy-centred neural network design space exploration to understand the membership privacy risks of well adopted state-of-the-art techniques: model compression via pruning, quantization, and off-the-shelf efficient architectures. We study the impact of model capacity on memorization of training data and show that compressed models (after retraining) leak more membership information compared to baseline uncompressed models while off-the-shelf architectures do not satisfy all efficiency requirements. Based on these observations, we identify quantization as a potential design choice to address the three dimensional trade-off. We propose Gecko training methodology where we explicitly add resistance to membership inference attacks as a design objective along with memory, computation, and power constraints of the embedded devices. We show that models trained using Gecko are comparable to prior defences against blackbox membership attacks in terms of accuracy and privacy while additionally providing efficiency. This enables Gecko models to be deployed on embedded systems while providing membership privacy.
Hypergraph Lambek Calculus

It is known that context-free grammars can be extended to generating graphs resulting in graph grammars; one of such fundamental approaches is hyperedge replacement grammars. On the other hand there are type-logical grammars which also serve to describe string languages. In this paper, we investigate how to extend the Lambek calculus ($\mathrm{L}$) and grammars based on it to graphs. The resulting approach is called hypergraph Lambek calculus ($\mathrm{HL}$). It is a logical sequential calculus whose sequents are graphs; it naturally extends the Lambek calculus and also allows one to embed its variants (commutative $\mathrm{L}$, $\mathrm{NL\diamondsuit}$, $\mathrm{L}_{\mathbf{1}}^\ast$). Besides, many properties of the Lambek calculus (cut elimination, counters, models) can be lifted to $\mathrm{HL}$. However, while Lambek grammars are equivalent to context-free grammars in the string case, hypergraph Lambek grammars are much more powerful than hyperedge replacement grammars. Particularly, the former can generate the language of all graphs without isolated nodes; the language of all bipartite graphs; finite intersections of languages generated by hyperedge replacement grammars. Nevertheless, the derivability problem in $\mathrm{HL}$ and the membership problem for grammars based on $\mathrm{HL}$ are NP-complete as well as the membership problem for hyperedge replacement grammars.