Miracle-dz/UniFAR-FTUs huggingface dataset:
Permutation Equivariant Graph Framelets for Heterophilous Graph Learning

The nature of heterophilous graphs is significantly different from that of homophilous graphs, which causes difficulties in early graph neural network (GNN) models and suggests aggregations beyond the one-hop neighborhood. In this article, we develop a new way to implement multiscale extraction via constructing Haar-type graph framelets with desired properties of permutation equivariance, efficiency, and sparsity, for deep learning tasks on graphs. We further design a graph framelet neural network model permutation equivariant graph framelet augmented network (PEGFAN) based on our constructed graph framelets. The experiments are conducted on a synthetic dataset and nine benchmark datasets to compare the performance with other state-of-the-art models. The result shows that our model can achieve the best performance on certain datasets of heterophilous graphs (including the majority of heterophilous datasets with relatively larger sizes and denser connections) and competitive performance on the remaining.
Bayesian Spiked Laplacian Graphs

In network analysis, it is common to work with a collection of graphs that exhibit heterogeneity. For example, neuroimaging data from patient cohorts are increasingly available. A critical analytical task is to identify communities, and graph Laplacian-based meth-ods are routinely used. However, these methods are currently limited to a single network and also do not provide measures of uncertainty on the community assignment. In this work, we ﬁrst propose a probabilistic network model called the “Spiked Laplacian Graph” that considers an observed network as a transform of the Laplacian and degree matrices of the network generating process, with the Laplacian eigenvalues modeled by a modiﬁed spiked structure. This eﬀectively reduces the number of parameters in the eigenvectors, and their sign patterns allow eﬃcient estimation of the underlying community structure. Further, the posterior distribution of the eigenvectors provides uncertainty quantiﬁcation for the community estimates. Second, we introduce a Bayesian non-parametric approach to address the issue of heterogeneity in a collection of graphs. Theoretical results are established on the posterior consistency of the procedure and provide insights on the trade-oﬀ between model resolution and accuracy. We illustrate the performance of the methodology on synthetic data sets, as well as a neuroscience study related to brain activity in working memory.
Attacks against Federated Learning Defense Systems and their Mitigation

The susceptibility of federated learning (FL) to attacks from untrustworthy endpoints has led to the design of several defense systems. FL defense systems enhance the federated optimization algorithm using anomaly detection, scaling the updates from endpoints depending on their anomalous behavior. However, the defense systems themselves may be exploited by the endpoints with more sophisticated attacks. First, this paper proposes three categories of attacks and shows that they can eﬀectively deceive some well-known FL defense systems. In the ﬁrst two categories, referred to as on-oﬀ attacks, the adversary toggles between being honest and engaging in attacks. We analyse two such on-oﬀ attacks, label ﬂipping and free riding, and show their impact against existing FL defense systems. As a third category, we propose attacks based on “good mouthing” and “bad mouthing”, to boost or diminish inﬂuence of the victim endpoints on the global model. Secondly, we propose a new federated optimization algorithm, Viceroy, that can successfully mitigate all the proposed attacks. The proposed attacks and the mitigation strategy have been tested on a number of diﬀerent experiments establishing their eﬀectiveness in comparison with other contemporary methods. The proposed algorithm has also been made available as open source. Finally, in the appendices, we provide an induction proof for the on-oﬀ model poisoning attack, and the proof of convergence and adversarial tolerance for the new federated optimization algorithm.
Cutting Planes Width and the Complexity of Graph Isomorphism Refutations

The width complexity measure plays a central role in Resolution and other propositional proof systems like Polynomial Calculus (under the name of degree). The study of width lower bounds is the most extended method for proving size lower bounds, and it is known that for these systems, proofs with small width also imply the existence of proofs with small size. Not much has been studied, however, about the width parameter in the Cutting Planes (CP) proof system, a measure that was introduced by Dantchev and Martin in 2011 under the name of CP cutwidth. In this paper, we study the width complexity of CP refutations of graph isomorphism formulas. For a pair of non-isomorphic graphs G and H , we show a direct connection between the Weisfeiler– Leman differentiation number WL ( G, H ) of the graphs and the width of a CP refutation for the corresponding isomorphism formula Iso( G, H ). In particular, we show that if WL ( G, H ) ≤ k , then there is a CP refutation of Iso( G, H ) with width k , and if WL ( G, H ) > k , then there are no CP refutations of Iso( G, H ) with width k − 2. Similar results are known for other proof systems, like Resolution, Sherali–Adams, or Polynomial Calculus. We also obtain polynomial-size CP refutations from our width bound for isomorphism formulas for graphs with constant WL-dimension.
Optimal Convergence Rates for Distributed Nystroem Approximation

The distributed kernel ridge regression (DKRR) has shown great potential in processing complicated tasks. However, DKRR only made use of the local samples that failed to capture the global characteristics. Besides, the existing optimal learning guarantees were provided in expectation and only pertain to the attainable case that the target regression lies exactly in the kernel space. In this paper, we propose distributed learning with globally-shared Nystr¨om centers ( DNystr¨om ), which utilizes global information across the local clients. We also study the statistical properties of DNystr¨om in expectation and in probability, respectively, and obtain several state-of-the-art results with the minimax optimal learning rates. Note that, the optimal convergence rates for DNystr¨om pertain to the non-attainable case, while the statistical results allow more partitions and require fewer Nystr¨om centers. Finally, we conduct experiments on several real-world datasets to validate the eﬀectiveness of the proposed algorithm, and the empirical results coincide with our theoretical ﬁndings.
Self-Improvement for Circuit-Analysis Problems

Many results in fine-grained complexity reveal intriguing consequences from solving various SAT problems even slightly faster than exhaustive search. We prove a self-improving (or “bootstrapping”) theorem for Circuit-SAT, # Circuit-SAT, and its fully-quantified version: solving one of these problems faster for “large” circuit sizes implies a significant speed-up for “smaller” circuit sizes. Our general arguments work for a variety of models solving circuit-analysis problems, including non-uniform circuits and randomized models of computation. We derive striking consequences for the complexities of these problems. For example, we show that certain fine-grained improvements on the runtime exponents of polynomial-time versions of Circuit-SAT would imply subexponential-time algorithms for Circuit-SAT on 2 o ( n ) -size circuits, refuting the Exponential Time Hypothesis. We also show how slightly faster # Circuit-SAT algorithms on large circuits can be used to prove lower bounds against uniform circuits with symmetric gates for functions in deterministic linear time. Our result suggests an “algorithmic method” approach for uniform circuit lower bounds, which trades non-uniformity for a substantial reduction in the complexity of the hard function.
Topological Convolutional Layers for Deep Learning

This work introduces the Topological CNN (TCNN), which encompasses several topologically deﬁned convolutional methods. Manifolds with important relationships to the natural image space are used to parameterize image ﬁlters which are used as convolutional weights in a TCNN. These manifolds also parameterize slices in layers of a TCNN across which the weights are localized. We show evidence that TCNNs learn faster, on less data, with fewer learned parameters, and with greater generalizability and interpretability than conventional CNNs. We introduce and explore TCNN layers for both image and video data. We propose extensions to 3D images and 3D video.
Online Optimization over Riemannian Manifolds

Online optimization has witnessed a massive surge of research attention in recent years. In this paper, we propose online gradient descent and online bandit algorithms over Riemannian manifolds in full information and bandit feedback settings respectively, for both geodesically convex and strongly geodesically convex functions. We establish a series of upper bounds on the regrets for the proposed algorithms over Hadamard manifolds. We also find a universal lower bound for achievable regret on Hadamard manifolds. Our analysis shows how time horizon, dimension, and sectional curvature bounds have impact on the regret bounds. When the manifold permits positive sectional curvature, we prove similar regret bound can be established by handling non-constrictive project maps. In addition, numerical studies on problems defined on symmetric positive definite matrix manifold, hyperbolic spaces, and Grassmann manifolds are provided to validate our theoretical findings, using synthetic and real-world data. ∗. A preliminary version of the paper is scheduled for presentation at NeurIPS-2021 (Wang et al., 2021). †. Correspondence author (G. Shi, Ross Street Building, Darlington NSW 2006, Sydney, Australia; +6102-8627 8037). c ©2023 Xi Wang, Zhipeng Tu, Yiguang Hong, Yingyi Wu and Guodong Shi. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v24/21-1308.html. Wang, Tu, Hong, Wu, and Shi
Recursive Quantile Estimation: Non-Asymptotic Confidence Bounds

This paper considers the recursive estimation of quantiles using the stochastic gradient descent (SGD) algorithm with Polyak-Ruppert averaging. The algorithm oﬀers a computationally and memory eﬃcient alternative to the usual empirical estimator. Our focus is on studying the non-asymptotic behavior by providing exponentially decreasing tail probability bounds under mild assumptions on the smoothness of the density functions. This novel non-asymptotic result is based on a bound of the moment generating function of the SGD estimate. We apply our result to the problem of best arm identiﬁcation in a multi-armed stochastic bandit setting under quantile preferences.
SQLFlow: An Extensible Toolkit Integrating DB and AI

Integrating AI algorithms into databases is an ongoing eﬀort in both academia and industry. We introduce SQLFlow, a toolkit seamlessly combining data manipulations and AI operations that can be run locally or remotely. SQLFlow extends SQL syntax to support typical AI tasks including model training, inference, interpretation, and mathematical optimization. It is compatible with a variety of database management systems (DBMS) and AI engines, including MySQL, TiDB, MaxCompute, and Hive, as well as TensorFlow, scikit-learn, and XGBoost. Documentations and case studies are available at https://sqlflow.org . The source code and additional details can be found at https://github.com/sql-machine-learning/sqlflow .
Label Distribution Changing Learning with Sample Space Expanding

With the evolution of data collection ways, label ambiguity has arisen from various applications. How to reduce its uncertainty and leverage its eﬀectiveness is still a challenging task. As two types of representative label ambiguities, Label Distribution Learning (LDL), which annotates each instance with a label distribution, and Emerging New Class (ENC), which focuses on model reusing with new classes, have attached extensive attentions. Nevertheless, in many applications, such as emotion distribution recognition and facial age estimation, we may face a more complicated label ambiguity scenario, i.e., label distribution changing with sample space expanding owing to the new class. To solve this crucial but rarely studied problem, we propose a new framework named as Label Distribution Changing Learning (LDCL) in this paper, together with its theoretical guarantee with generalization error bound. Our approach expands the sample space by re-scaling previous distribution and then estimates the emerging label value via scaling constraint factor. For demonstration, we present two special cases within the framework, together with their optimizations and convergence analyses. Besides evaluating LDCL on most of the existing 13 data sets, we also apply it in the application of emotion distribution recognition. Experimental results demonstrate the eﬀectiveness of our approach in both tackling label ambiguity problem and estimating facial emotion.
Proof Simulation via Round-based Strategy Extraction for QBF

The proof complexity of Quantiﬁed Boolean Formulas (QBF) relates to both QBF solving and QBF certiﬁcation. One method to p-simulate a QBF proof system is by formalising the soundness of its strategy extraction in propositional logic [7]. In this work we illustrate how to use extended QBF Frege [4] to simulate LD-Q( D rrs )-Res, a proof system that combines conﬂict driven clause learning with dependency schemes [22], us-ing such a method. The round-based technique is the most common way to show a QBF proof system has strategy extraction, originally shown for Q-resolution [11] and later used for LD-Q-Resolution [10], LQU-Resolution [2], expansion based systems [5] and dependency-scheme based systems [24]. Many of these proof systems were already shown to be simulated by extended QBF Frege, but simulation had to use a specialised local strategy extraction technique. Here we simulate the remaining systems, by formalising the soundness of LD-Q( D rrs )-Res’s round-based strategy extraction in propositional logic. This is a positive result for certiﬁcation, and further suggests the feasibility of using Extended QU-Resolution or QRAT to certify QCDCL solvers.
QCDCL vs QBF Resolution: Further Insights

We continue the investigation on the relations of QCDCL and QBF resolution systems. In particular, we introduce QCDCL versions that tightly characterise QU-Resolution and (a slight variant of) long-distance Q-Resolution. We show that most QCDCL variants – parameterised by different policies for decisions, unit propagations and reductions – lead to incomparable systems for almost all choices of these policies.
Risk Bounds for Positive-Unlabeled Learning Under the Selected At Random Assumption

Positive-Unlabeled learning (PU learning) is a special case of semi-supervised binary classi-ﬁcation where only a fraction of positive examples is labeled. The challenge is then to ﬁnd the correct classiﬁer despite this lack of information. Recently, new methodologies have been introduced to address the case where the probability of being labeled may depend on the covariates. In this paper, we are interested in establishing risk bounds for PU learning under this general assumption. In addition, we quantify the impact of label noise on PU learning compared to the standard classiﬁcation setting. Finally, we provide a lower bound on the minimax risk proving that the upper bound is almost optimal
Optimizing ROC Curves with a Sort-Based Surrogate Loss for Binary Classification and Changepoint Detection

Receiver Operating Characteristic (ROC) curves are useful for evaluating binary classiﬁcation models, but diﬃcult to use for learning since the Area Under the Curve (AUC) is a piecewise constant function of predicted values. ROC curves can also be used in other problems with false positive and true positive rates such as changepoint detection. We show that in this more general context, the ROC curve can have loops, points with highly sub-optimal error rates, and AUC greater than one. This observation motivates a new optimization objective: rather than maximizing the AUC, we would like a monotonic ROC curve with AUC=1 that avoids points with large values for Min(FP,FN). We propose an L1 relaxation of this objective that results in a new surrogate loss function called the AUM, short for Area Under Min(FP, FN). Whereas previous loss functions are based on summing over all labeled examples or pairs, the AUM requires a sort and a sum over the sequence of points on the ROC curve. We show that AUM directional derivatives can be eﬃciently computed and used in a gradient descent learning algorithm. In our empirical study of supervised binary classiﬁcation and changepoint detection problems, we show that our new AUM minimization learning algorithm results in improved AUC and speed relative to previous baselines.
Interactions between astrocytes and extracellular matrix structures contribute to neuroinflammation-associated epilepsy pathology

Often considered the “housekeeping” cells of the brain, astrocytes have of late been rising to the forefront of neurodegenerative disorder research. Identified as crucial components of a healthy brain, it is undeniable that when astrocytes are dysfunctional, the entire brain is thrown into disarray. We offer epilepsy as a well-studied neurological disorder in which there is clear evidence of astrocyte contribution to diseases as evidenced across several different disease models, including mouse models of hippocampal sclerosis, trauma associated epilepsy, glioma-associated epilepsy, and beta-1 integrin knockout astrogliosis. In this review we suggest that astrocyte-driven neuroinflammation, which plays a large role in the pathology of epilepsy, is at least partially modulated by interactions with perineuronal nets (PNNs), highly structured formations of the extracellular matrix (ECM). These matrix structures affect synaptic placement, but also intrinsic neuronal properties such as membrane capacitance, as well as ion buffering in their immediate milieu all of which alters neuronal excitability. We propose that the interactions between PNNs and astrocytes contribute to the disease progression of epilepsy vis a vis neuroinflammation. Further investigation and alteration of these interactions to reduce the resultant neuroinflammation may serve as a potential therapeutic target that provides an alternative to the standard anti-seizure medications from which patients are so frequently unable to benefit.
Elevation, aspect, and slope influence woody vegetation structure and composition but not species richness in a human-influenced landscape in northwestern Yunnan, China

Elevation-for-latitude substitution offers a tool for studying the influence of temperature and precipitation variability on vegetation structure and composition. Understanding how elevation, aspect, and slope influence vegetation patterns may help in predicting how climate change influences human forest usage and in developing strategies for ensuring the sustained provision of ecosystem services. However, most ecological studies have been carried out in protected areas, leaving forest areas used by humans to lesser attention. Therefore, we asked how elevation, aspect, and slope impact the vegetation on a human-influenced mountain. We measured woody vegetation size, richness, and composition on a mountain with plots set systematically in four cardinal directions at 100-m elevational intervals from the peak, from 1900 to 4200 m above sea level, in the Hengduan Mountains in eastern Himalaya, southwestern China. We quantified how tree maximum height, basal area, aboveground biomass (AGB), tree and shrub species richness, and woody species composition changed with elevation, aspect, and slope. Based on generalized linear models, the maximum tree height, tree basal area, and woody species AGB followed a unimodal trend along elevational gradients, with tree height and basal area peaking at 3100 m, while AGB was highest at 3300 m and somewhat higher on the southern slope. Basal area increased with slope degree. Neither tree nor shrub species richness was influenced by elevation, aspect, or slope. According to canonical correspondence analysis and TWINSPAN classification, elevation and north-south orientation of the slope were major factors influencing woody species compositions, and vegetation was classified into five types of communities. Our results indicated that the influences of elevation, aspect, and slope on woody vegetation structure were similar in a human-influenced forested mountain area as in protected mountain landscapes based on the literature. However, as forests in this area are used more intensively at low and middle elevations of the southern and western slopes, where aridity restricts tree size and AGB, climate change is likely to challenge traditional harvesting practices and place pressure on moving forest usage to higher altitudes.
The effect of web 2.0 based technology applications on speaking skills and speaking anxiety in teaching Turkish as a foreign language: Voki example

The aim of this study is to examine the effect of Voki, one of the web 2.0-based technology applications, on Turkish learners’ speaking skills and speaking anxiety. In the study, exploratory sequential design, one of the mixed design types in which quantitative and qualitative approaches are used together, was adopted. The study group of the research consisted of 61 students at A2 level (31 experimental group, 30 control group) who learn Turkish as a foreign language at the Turkish Language Teaching Center of a university located in the south of Turkey. Speaking Anxiety Scale and Speaking Skill Assessment Form were used as data collection tools. During the 6-week intervention, the experimental group used Voki in speaking lessons, while the control group did not use any technology-based web 2.0 tool. Descriptive statistics, chi-square analysis, dependent and independent groups t-test were used to analyze the quantitative data collected in the study. Descriptive analysis and content analysis were used to analyze the qualitative data collected through a semi-structured interview form. As a result of the study, it was determined that the Voki application improved the speaking skills of the students in the experimental group and reduced their speaking anxiety. It was also determined that the students in the experimental group expressed positive opinions about the application. Therefore, the use of Voki application in speaking activities in foreign language teaching is recommended.
Cry1 expression during postnatal development is critical for the establishment of normal circadian period

The mammalian circadian system generates an approximate 24-h rhythm through a complex autoregulatory feedback loop. Four genes, Period1 (Per1), Period2 (Per2), Cryptochrome1 (Cry1), and Cryptochrome2 (Cry2), regulate the negative feedback within this loop. Although these proteins have distinct roles within the core circadian mechanism, their individual functions are poorly understood. Here, we used a tetracycline trans-activator system (tTA) to examine the role of transcriptional oscillations in Cry1 and Cry2 in the persistence of circadian activity rhythms. We demonstrate that rhythmic Cry1 expression is an important regulator of circadian period. We then define a critical period from birth to postnatal day 45 (PN45) where the level of Cry1 expression is critical for setting the endogenous free running period in the adult animal. Moreover, we show that, although rhythmic Cry1 expression is important, in animals with disrupted circadian rhythms overexpression of Cry1 is sufficient to restore normal behavioral periodicity. These findings provide new insights into the roles of the Cryptochrome proteins in circadian rhythmicity and further our understanding of the mammalian circadian clock.