Miracle-dz/UniFAR-FTUs huggingface dataset:
Explicit Convergence Rates of Greedy and Random Quasi-Newton Methods

Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In this paper, we consider the explicit superlinear convergence rates of quasi-Newton methods and address two open problems mentioned by Rodomanov and Nesterov. First, we extend Rodomanov and Nesterov's results to random quasi-Newton methods, which include common DFP, BFGS, SR1 methods. Such random methods adopt a random direction for updating the approximate Hessian matrix in each iteration. Second, we focus on the specific quasi-Newton methods: SR1 and BFGS methods. We provide improved versions of greedy and random methods with provable better explicit (local) superlinear convergence rates. Our analysis is closely related to the approximation of a given Hessian matrix, unconstrained quadratic objective, as well as the general strongly convex, smooth, and strongly self-concordant functions.
3 1 O ct 2 02 1 High-Fidelity Quantum Information Transmission using a Nonrefrigerated Lossy Microwave Waveguide at Room Temperature

Quantum microwave transmission is key to realizing modular superconducting quantum computers [1] and distributed quantum networks [2]. However, a large number of incoherent photons are thermally generated in the microwave frequency spectrum. Hence, high fidelity quantum microwave transmission has long been believed to be infeasible without refrigeration [3, 4]. In this work, we propose a novel method for high fidelity quantum microwave transmission using a lossy waveguide at room temperature. The proposed scheme considers two cryogenic nodes (i.e., a transmitter and a receiver) connected by a room-temperature lossy microwave waveguide. First, a cryogenic pre-amplification is provided before transmission. Second, at the receiver side, a cryogenic loop antenna coupled to an LC harmonic oscillator inside the output port of the waveguide is implemented, while the LC harmonic oscillator is located outside the waveguide. The loop antenna converts the quantum microwave fields (which contain both signal and noise photons) to a quantum voltage across the coupled LC harmonic oscillator. The induced noise photons across the LC oscillator include the amplification noise, the thermal occupation of the waveguide, and the fluctuation-dissipation noise. The loop antenna detector at the receiver is designed to extensively suppress the induced noise photons across the LC oscillator to achieve effectivly cooled waveguide. We show that by properly designing the pre-amplification gain, along with the loop antenna at the receiver, a high fidelity quantum transmission (i.e., more than 0.97) can be achieved for transmission distances reaching 100 m.
Intrusion Detection using Spatial-Temporal features based on Riemannian Manifold

Network traffic data is a combination of different data bytes packets under different network protocols. These traffic packets have complex time-varying non-linear relationships. Existing state-of-the-art methods rise up to this challenge by fusing features into multiple subsets based on correlations and using hybrid classification techniques that extract spatial and temporal characteristics. This often requires high computational cost and manual support that limit them for real-time processing of network traffic. To address this, we propose a new novel feature extraction method based on covariance matrices that extract spatial-temporal characteristics of network traffic data for detecting malicious network traffic behavior. The covariance matrices in our proposed method not just naturally encode the mutual relationships between different network traffic values but also have well-defined geometry that falls in the Riemannian manifold. Riemannian manifold is embedded with distance metrics that facilitate extracting discriminative features for detecting malicious network traffic. We evaluated our model on NSL-KDD and UNSW-NB15 datasets and showed our proposed method significantly outperforms the conventional method and other existing studies on the dataset.
Simulating Realistic MRI variations to Improve Deep Learning model and visual explanations using GradCAM

In the medical field, landmark detection in MRI plays an important role in reducing medical technician efforts in tasks like scan planning, image registration, etc. First, 88 landmarks spread across the brain anatomy in the three respective views- sagittal, coronal, and axial are manually annotated, later guidelines from the expert clinical technicians are taken subanatomy-wise, for better localization of the existing landmarks, in order to identify and locate the important atlas landmarks even in oblique scans. To overcome limited data availability, we implement realistic data augmentation to generate synthetic 3D volumetric data. We use a modified HighRes3DNet model for solving brain MRI volumetric landmark detection problem. In order to visually explain our trained model on unseen data, and discern a stronger model from a weaker model, we implement Gradient-weighted Class Activation Mapping (GradCAM) which produces a coarse localization map highlighting the regions the model is focusing. Our experiments show that the proposed method shows favorable results, and the overall pipeline can be extended to a variable number of landmarks and other anatomies.
Quantum state transfer between distant optomechanical interfaces via shortcut to adiabaticity

We propose a protocol to realize fast high-fidelity quantum state transfer between distant optomechanical interfaces connected by a continuum waveguide. The scheme consists of three steps: two accelerating adiabatic processes joined by a population conversion process. In comparison to the traditional adiabatic technique, our method reaches a higher transfer fidelity with a shorter time. Numerical results show that the fidelity of this transfer scheme in the dissipative system mainly depends on the protocol speed and the coupling strength of the waveguide and cavities. Assisted by inverting the pulse sequence, a bidirectional transfer can be implemented, indicating the potential to build a quantum network.
Causal Discovery in Linear Structural Causal Models with Deterministic Relations

Linear structural causal models (SCMs)– in which each observed variable is generated by a subset of the other observed variables as well as a subset of the exogenous sources– are pervasive in causal inference and casual discovery. However, for the task of causal discovery, existing work almost exclusively focus on the submodel where each observed variable is associated with a distinct source with non-zero variance. This results in the restriction that no observed variable can deterministically depend on other observed variables or latent confounders. In this paper, we extend the results on structure learning by focusing on a subclass of linear SCMs which do not have this property, i.e., models in which observed variables can be causally affected by any subset of the sources, and are allowed to be a deterministic function of other observed variables or latent confounders. This allows for a more realistic modeling of inﬂuence or information propagation in systems. We focus on the task of causal discovery form observational data generated from a member of this subclass. We derive a set of necessary and sufﬁcient conditions for unique identiﬁability of the causal structure. To the best of our knowledge, this is the ﬁrst work that gives identiﬁability results for causal discovery under both latent confounding and deterministic relationships. Further, we propose an algorithm for recovering the underlying causal structure when the aforementioned conditions are satisﬁed. We validate our theoretical results both on synthetic and real datasets. linear DS-P-SCM in causal discovery, we show that there are models that can be correctly explained by a linear P-SCM, while linear DS-P-SCMs either are inapplicable or
Property-directed reachability as abstract interpretation in the monotone theory

Inferring inductive invariants is one of the main challenges of formal verification. The theory of abstract interpretation provides a rich framework to devise invariant inference algorithms. One of the latest breakthroughs in invariant inference is property-directed reachability (PDR), but the research community views PDR and abstract interpretation as mostly unrelated techniques. This paper shows that, surprisingly, propositional PDR can be formulated as an abstract interpretation algorithm in a logical domain. More precisely, we define a version of PDR, called Λ-PDR, in which all generalizations of counterexamples are used to strengthen a frame. In this way, there is no need to refine frames after their creation, because all the possible supporting facts are included in advance. We analyze this algorithm using notions from Bshouty’s monotone theory, originally developed in the context of exact learning. We show that there is an inherent overapproximation between the algorithm’s frames that is related to the monotone theory. We then define a new abstract domain in which the best abstract transformer performs this overapproximation, and show that it captures the invariant inference process, i.e., Λ-PDR corresponds to Kleene iterations with the best transformer in this abstract domain. We provide some sufficient conditions for when this process converges in a small number of iterations, with sometimes an exponential gap from the number of iterations required for naive exact forward reachability. These results provide a firm theoretical foundation for the benefits of how PDR tackles forward reachability.
Explainable Artificial Intelligence for Smart City Application: A Secure and Trusted Platform

Artificial Intelligence (AI) is one of the disruptive technologies that is shaping the future. It has growing applications for data-driven decisions in major smart city solutions, including transportation, education, healthcare, public governance, and power systems. At the same time, it is gaining popularity in protecting critical cyber infrastructure from cyber threats, attacks, damages, or unauthorized access. However, one of the significant issues of those traditional AI technologies (e.g., deep learning) is that the rapid progress in complexity and sophistication propelled and turned out to be uninterpretable black boxes. On many occasions, it is very challenging to understand the decision and bias to control and trust systems' unexpected or seemingly unpredictable outputs. It is acknowledged that the loss of control over interpretability of decision-making becomes a critical issue for many data-driven automated applications. But how may it affect the system's security and trustworthiness? This chapter conducts a comprehensive study of machine learning applications in cybersecurity to indicate the need for explainability to address this question. While doing that, this chapter first discusses the black-box problems of AI technologies for Cybersecurity applications in smart city-based solutions. Later, considering the new technological paradigm, Explainable Artificial Intelligence (XAI), this chapter discusses the transition from black-box to white-box. This chapter also discusses the transition requirements concerning the interpretability, transparency, understandability, and Explainability of AI-based technologies in applying different autonomous systems in smart cities. Finally, it has presented some commercial XAI platforms that offer explainability over traditional AI technologies before presenting future challenges and opportunities.
Optimal Sketching for Trace Estimation

Matrix trace estimation is ubiquitous in machine learning applications and has traditionally relied on Hutchinson's method, which requires $O(\log(1/\delta)/\epsilon^2)$ matrix-vector product queries to achieve a $(1 \pm \epsilon)$-multiplicative approximation to $\text{tr}(A)$ with failure probability $\delta$ on positive-semidefinite input matrices $A$. Recently, the Hutch++ algorithm was proposed, which reduces the number of matrix-vector queries from $O(1/\epsilon^2)$ to the optimal $O(1/\epsilon)$, and the algorithm succeeds with constant probability. However, in the high probability setting, the non-adaptive Hutch++ algorithm suffers an extra $O(\sqrt{\log(1/\delta)})$ multiplicative factor in its query complexity. Non-adaptive methods are important, as they correspond to sketching algorithms, which are mergeable, highly parallelizable, and provide low-memory streaming algorithms as well as low-communication distributed protocols. In this work, we close the gap between non-adaptive and adaptive algorithms, showing that even non-adaptive algorithms can achieve $O(\sqrt{\log(1/\delta)}/\epsilon + \log(1/\delta))$ matrix-vector products. In addition, we prove matching lower bounds demonstrating that, up to a $\log \log(1/\delta)$ factor, no further improvement in the dependence on $\delta$ or $\epsilon$ is possible by any non-adaptive algorithm. Finally, our experiments demonstrate the superior performance of our sketch over the adaptive Hutch++ algorithm, which is less parallelizable, as well as over the non-adaptive Hutchinson's method.
MFNet: Multiclass Few-Shot Segmentation Network With Pixel-Wise Metric Learning

In visual recognition tasks, few-shot learning requires the ability to learn object categories with few support examples. Its re-popularity in light of the deep learning development is mainly in image classification. This work focuses on few-shot semantic segmentation, which is still a largely unexplored field. A few recent advances are often restricted to single-class few-shot segmentation. In this paper, we first present a novel multi-way (class) encoding and decoding architecture which effectively fuses multi-scale query information and multi-class support information into one query-support embedding. Multi-class segmentation is directly decoded upon this embedding. For better feature fusion, a multi-level attention mechanism is proposed within the architecture, which includes the attention for support feature modulation and attention for multi-scale combination. Last, to enhance the embedding space learning, an additional pixel-wise metric learning module is introduced with triplet loss formulated on the pixel-level of the query-support embedding. Extensive experiments on standard benchmarks PASCAL-<inline-formula> <tex-math notation="LaTeX">$5^{i}$ </tex-math></inline-formula> and COCO-<inline-formula> <tex-math notation="LaTeX">$20^{i}$ </tex-math></inline-formula> show clear benefits of our method over the state of the art in multi-class few-shot segmentation.
Visual Explanations for Convolutional Neural Networks via Latent Traversal of Generative Adversarial Networks

Lack of explainability in artificial intelligence, specifically deep neural networks, remains a bottleneck for implementing models in practice. Popular techniques such as Gradient-weighted Class Activation Mapping (Grad-CAM) provide a coarse map of salient features in an image, which rarely tells the whole story of what a convolutional neural network(CNN) learned. Using COVID-19 chest X-rays, we present a method for interpreting what a CNN has learned by utilizing Generative Adversarial Networks (GANs). Our GAN framework disentangles lung structure from COVID-19 features. Using this GAN, we can visualize the transition of a pair of COVID negative lungs in a chest radiograph to a COVID positive pair by interpolating in the latent space of the GAN, which provides fine-grained visualization of how the CNN responds to varying features within the lungs.
CO
 kinematics unveil outflows plausibly driven by a young jet in the gigahertz peaked radio core of
 NGC 6328

1National Observatory of Athens (NOA), Institute for Astronomy, Astrophysics, Space Applications and Remote Sensing (IAASARS), GR–15236, Greece 2National and Kapodistrian University of Athens, Department of Astrophysics, Astronomy & Mechanics, Faculty of Physics, Panepistimioupolis Zografou, 15784, Greece 3Istituto di Astrofisica e Planetologia Spaziali (INAF–IAPS), Fosso del Cavaliere 100, I–00133 Roma, Italy 4Instituto de Astrofísica de Canarias, Calle Vía Láctea, s/n, E-38205 La Laguna, Spain 5School of Physics & Astronomy, Cardiff University, Queens Buildings, The Parade, Cardiff CF24 3AA, UK 6Observatoire de Paris, LERMA, Collège de France, CNRS, PSL Univ., Sorbonne Univ., F-75014 Paris, France
Domain-adaptation of spherical embeddings

Domain adaptation of embedding models, updating a generic embedding to the language of a specific domain, is a proven technique for domains that have insufficient data to train an effective model from scratch. Chemistry publications is one such domain, where scientific jargon and overloaded terminology inhibit the performance of a general language model. The recent spherical embedding model (JoSE) proposed in arXiv:1911.01196 jointly learns word and document embeddings during training on the multi-dimensional unit sphere, which performs well for document classification and word correlation tasks. But, we show a non-convergence caused by global rotations during its training prevents it from domain adaptation. In this work, we develop methods to counter the global rotation of the embedding space and propose strategies to update words and documents during domain specific training. Two new document classification data-sets are collated from general and chemistry scientific journals to compare the proposed update training strategies with benchmark models. We show that our strategies are able to reduce the performance cost of domain adaptation to a level similar to Word2Vec.
Nonlinear interferometry beyond classical limit facilitated by cyclic dynamics

Qi Liu, ∗ Ling-Na Wu, ∗ Jia-Hao Cao, Tian-Wei Mao, Xin-Wei Li, Shuai-Feng Guo, Meng Khoon Tey, 3 and Li You 3, † State Key Laboratory of Low Dimensional Quantum Physics, Department of Physics, Tsinghua University, Beijing 100084, China Institut für Theoretische Physik, Technische Universität Berlin, Hardenbergstraße 36, Berlin 10623, Germany Frontier Science Center for Quantum Information, Beijing 100193, China (Dated: November 2, 2021)
Modelling and simulating spatial extremes by combining extreme value theory with generative adversarial networks

Modelling dependencies between climate extremes is important for climate risk assessment, for instance when allocating emergency management funds. In statistics, multivariate extreme value theory is often used to model spatial extremes. However, most commonly used approaches require strong assumptions and are either too simplistic or over-parameterized. From a machine learning perspective, Generative Adversarial Networks (GANs) are a powerful tool to model dependencies in high-dimensional spaces. Yet in the standard setting, GANs do not well represent dependencies in the extremes. Here we combine GANs with extreme value theory (evtGAN) to model spatial dependencies in summer maxima of temperature and winter maxima in precipitation over a large part of western Europe. We use data from a stationary 2000-year climate model simulation to validate the approach and explore its sensitivity to small sample sizes. Our results show that evtGAN outperforms classical GANs and standard statistical approaches to model spatial extremes. Already with about 50 years of data, which corresponds to commonly available climate records, we obtain reasonably good performance. In general, dependencies between temperature extremes are better captured than dependencies between precipitation extremes due to the high spatial coherence in temperature fields. Our approach can be applied to other climate variables and can *Corresponding author: younes.boulaguiem@unige.ch. 1 ar X iv :2 11 1. 00 26 7v 2 [ st at . A P] 1 9 Ja n 20 22 be used to emulate climate models when running very long simulations to determine dependencies in the extremes is deemed infeasible.
Efficient Inference Without Trading-off Regret in Bandits: An Allocation Probability Test for Thompson Sampling

Using bandit algorithms to conduct adaptive randomised experiments can minimise regret, but it poses major challenges for statistical inference (e.g., biased estimators, inflated type-I error and reduced power). Recent attempts to address these challenges typically impose restrictions on the exploitative nature of the bandit algorithm$-$trading off regret$-$and require large sample sizes to ensure asymptotic guarantees. However, large experiments generally follow a successful pilot study, which is tightly constrained in its size or duration. Increasing power in such small pilot experiments, without limiting the adaptive nature of the algorithm, can allow promising interventions to reach a larger experimental phase. In this work we introduce a novel hypothesis test, uniquely based on the allocation probabilities of the bandit algorithm, and without constraining its exploitative nature or requiring a minimum experimental size. We characterise our $Allocation\ Probability\ Test$ when applied to $Thompson\ Sampling$, presenting its asymptotic theoretical properties, and illustrating its finite-sample performances compared to state-of-the-art approaches. We demonstrate the regret and inferential advantages of our approach, particularly in small samples, in both extensive simulations and in a real-world experiment on mental health aspects.
DetectorNet: Transformer-enhanced Spatial Temporal Graph Neural Network for Traffic Prediction

Detectors with high coverage have direct and far-reaching benefits for road users in route planning and avoiding traffic congestion, but utilizing these data presents unique challenges including: the dynamic temporal correlation, and the dynamic spatial correlation caused by changes in road conditions. Although the existing work considers the significance of modeling with spatial-temporal correlation, what it has learned is still a static road network structure, which cannot reflect the dynamic changes of roads, and eventually loses much valuable potential information. To address these challenges, we propose DetectorNet enhanced by Transformer. Differs from previous studies, our model contains a Multi-view Temporal Attention module and a Dynamic Attention module, which focus on the long-distance and short-distance temporal correlation, and dynamic spatial correlation by dynamically updating the learned knowledge respectively, so as to make accurate prediction. In addition, the experimental results on two public datasets and the comparison results of four ablation experiments proves that the performance of DetectorNet is better than the eleven advanced baselines.
URIR: Recommendation algorithm of user RNN encoder and item encoder based on knowledge graph

Due to a large amount of information, it is difficult for users to find what they are interested in among the many choices. In order to improve users' experience, recommendation systems have been widely used in music recommendations, movie recommendations, online shopping, and other scenarios. Recently, Knowledge Graph (KG) has been proven to be an effective tool to improve the performance of recommendation systems. However, a huge challenge in applying knowledge graphs for recommendation is how to use knowledge graphs to obtain better user codes and item codes. In response to this problem, this research proposes a user Recurrent Neural Network (RNN) encoder and item encoder recommendation algorithm based on Knowledge Graph (URIR). This study encodes items by capturing high-level neighbor information to generate items' representation vectors and applies an RNN and items' representation vectors to encode users to generate users' representation vectors, and then perform inner product operation on users' representation vectors and items' representation vectors to get probabilities of users interaction with items. Numerical experiments on three real-world datasets demonstrate that URIR is superior performance to state-of-the-art algorithms in indicators such as AUC, Precision, Recall, and MRR. This implies that URIR can effectively use knowledge graph to obtain better user codes and item codes, thereby obtaining better recommendation results.