Miracle-dz/UniFAR-FTUs huggingface dataset:
An Efficient Alternating Riemannian/Projected Gradient Descent Ascent Algorithm for Fair Principal Component Analysis

Fair principal component analysis (FPCA), a ubiquitous dimensionality reduction technique in signal processing and machine learning, aims to find a low-dimensional representation for a high-dimensional dataset in view of fairness. The FPCA problem involves optimizing a non-convex and non-smooth function over the Stiefel manifold. The state-of-the-art methods for solving the problem are subgradient methods and semidefinite relaxation-based methods. However, these two types of methods have their obvious limitations and thus are only suitable for efficiently solving the FPCA problem in special scenarios. This paper aims at developing efficient algorithms for solving the FPCA problem in general, especially large-scale, settings. In this paper, we first transform FPCA into a smooth non-convex linear minimax optimization problem over the Stiefel manifold. To solve the above general problem, we propose an efficient alternating Riemannian/projected gradient descent ascent (ARPGDA) algorithm, which performs a Riemannian gradient descent step and an ordinary projected gradient ascent step at each iteration. We prove that ARPGDA can find an $\varepsilon$-stationary point of the above problem within $\mathcal{O}(\varepsilon^{-3})$ iterations. Simulation results show that, compared with the state-of-the-art methods, our proposed ARPGDA algorithm can achieve a better performance in terms of solution quality and speed for solving the FPCA problems.
Three-Dimensional Propagation of Kink Wave Trains in Solar Coronal Slabs

Impulsively excited wave trains are of considerable interest in solar coronal seismology. To our knowledge, however, it remains to examine the three-dimensional (3D) dispersive propagation of impulsive kink waves in straight, field-aligned, symmetric, low-beta, slab equilibria that are structured only in one transverse direction. We offer a study here, starting with an analysis of linear oblique kink modes from an eigenvalue problem perspective. Two features are numerically found for continuous and step structuring alike, one being that the group and phase velocities may lie on opposite sides of the equilibrium magnetic field (B0), and the other being that the group trajectories extend only to a limited angle from B0. We justify these features by making analytical progress for the step structuring. More importantly, we demonstrate by a 3D time-dependent simulation that these features show up in the intricate interference patterns of kink wave trains that arise from a localized initial perturbation. In a plane perpendicular to the direction of inhomogeneity, the large-time slab-guided patterns are confined to a narrow sector about B0, with some wavefronts propagating toward B0. We conclude that the phase and group diagrams lay the necessary framework for understanding the complicated time-dependent behavior of impulsive waves.
Stability of fixed points in Poisson geometry and higher Lie theory

We provide a uniform approach to obtain sufficient criteria for a (higher order) fixed point of a given bracket structure on a manifold to be stable under deformations. Examples of bracket structures include Lie algebroids, Lie n-algebroids, singular foliations, Lie bialgebroids, Courant algebroids and Dirac structures in split Courant algebroids admitting a Dirac complement. We show that the stability problems are specific instances of the following problem: given a differential graded Lie algebra g, a differential graded Lie subalgebra h of degreewise finite codimension in g and a Maurer-Cartan element Q ∈ h, when are Maurer-Cartan elements near Q in g gauge equivalent to elements of h? We show that the vanishing of a finite-dimensional cohomology group associated to g, h and Q implies a positive answer to the question above, and therefore implies stability of fixed points of the geometric structures described above. In particular, we recover the stability results of CrainicFernandes for zero-dimensional leaves, as well as the stability results for higher order singularities of Dufour-Wade.
Oblique Quasi-kink Modes in Solar Coronal Slabs Embedded in an Asymmetric Magnetic Environment: Resonant Damping, Phase and Group Diagrams

There has been considerable interest in magnetoacoustic waves in static, straight, field-aligned, 1D equilibria where the exteriors of a magnetic slab are different between the two sides. We focus on trapped, transverse fundamental, oblique quasi-kink modes in pressureless setups, where the density varies continuously from a uniform interior (with density ρ i) to a uniform exterior on either side (with density ρ L or ρ R), assuming ρ L ≤ ρ R ≤ ρ i. The continuous structuring and oblique propagation make our study new, relative to pertinent studies, and lead to wave damping via the Alfvén resonance. We compute resonantly damped quasi-kink modes as resistive eigenmodes, and isolate the effects of system asymmetry by varying ρ i/ρ R from the “Fully Symmetric” (ρ i/ρ R = ρ i/ρ L) to the “Fully Asymmetric” limit (ρ i/ρ R = 1). We find that the damping rates possess a nonmonotonic ρ i/ρ R-dependence as a result of the difference between the two Alfvén continua, and resonant absorption occurs only in one continuum when ρ i/ρ R is below some threshold. We also find that the system asymmetry results in two qualitatively different regimes for the phase and group diagrams. The phase and group trajectories lie essentially on the same side (different sides) relative to the equilibrium magnetic field when the configuration is not far from a “Fully Asymmetric” (“Fully Symmetric”) one. Our numerical results are understood by making analytical progress in the thin-boundary limit, and discussed for imaging observations of axial standing modes and impulsively excited wavetrains.
"Rust Belt"Across America: An Application of a Nationwide, Block-Group-Level Deprivation Index

In the United States, large post-industrial cites such as Detroit are well-known for high levels of socioeconomic deprivation. But while Detroit is an exceptional case, similar levels of deprivation can still be found in other large cities, as well as in smaller towns and rural areas. This study calculates a standardized measure for all block groups in the lower 48 states and DC, before isolating"high-deprivation"areas that exceed Detroit's median value. These block groups are investigated and mapped for the 83 cities with populations above 250,000, as well as at the state level and for places of all sizes. Detroit is shown to indeed be unique not only for its levels of deprivation (which are higher than 95 percent of the country), but also for the dispersion of highly-deprived block groups throughout the city. Smaller, more concentrated pockets of high deprivation can be found in nearly every large city, and some cities below 20,000 residents have an even larger share of high-deprivation areas. Cities' percentages of high-deprivation areas are positively related to overall poverty, population density, and the percentage of White residents, and negatively related to the share of Black residents.
Mining contrast sets in classiﬁcation, regression, and survival data by fusing separate and conquer models

Identifying diﬀerences between groups is one of the most important knowledge discovery problems. The procedure, also known as contrast sets mining, is applied in a wide range of areas like medicine, industry, or economics. In the paper we present RuleKit-CS, an algorithm for contrast set mining based on a sequential covering – a well established heuristic for decision rule induction. The fusion of multiple passes accompanied with an attribute pe-nalization scheme allows generation of contrast sets describing same examples with diﬀerent attributes, distinguishing presented approach from the standard sequential covering. The ability to identify contrast sets in regression and survival data sets, the feature not provided by the existing algorithms, further extends the usability of RuleKit-CS. Experiments on over 130 data sets from various areas and detailed analysis of selected cases conﬁrmed RuleKit-CS to be a useful tool for discovering diﬀerences between deﬁned groups. The algorithm was implemented as a part of the RuleKit suite available at GitHub under GNU AGPL 3 licence ( https://github.com/adaa-polsl/RuleKit ).
Intelligence and global bias in the stock market

Trade is one of the essential features of human intelligence. The securities market is the ultimate expression of it. The fundamental indicators of stocks include information about the effects of noise and bias on stock prices; however, distinguishing between them is generally hard. In this article, I present the fundamentals hypothesis based on rational expectations and detect the global bias components from the actual fundamental indicators using a log-normal distribution model based on the fundamentals hypothesis. The analysis results show that biases generally exhibit the same characteristics, strongly supporting our theory. Notably, the positive price-to-cash flows from the investing activities ratio are proxies for the fundamentals. The answer is simple: "Cash is a fact, and profit is an opinion. " Namely, opinions of management and accounting added noise to fundamentals. As a result, we obtain the Kesten process and the Pareto distribution. This result means the market knows this noise and shows a stable global bias in the stock market.
System Network Analytics: Evolution and Stable Rules of a State Series

System Evolution Analytics on a system that evolves is a challenge because it makes a State Series SS = {S1, S2…SN} (i.e., a set of states ordered by time) with several inter-connected entities changing over time. We present stability characteristics of interesting evolution rules occurring in multiple states. We defined an evolution rule with its stability as the fraction of states in which the rule is interesting. Extensively, we defined stable rule as the evolution rule having stability that exceeds a given threshold minimum stability (minStab). We also defined persistence metric, a quantitative measure of persistent entity-connections. We explain this with an approach and algorithm for System Network Analytics (SysNet-Analytics), which uses minStab to retrieve Network Evolution Rules (NERs) and Stable NERs (SNERs). The retrieved information is used to calculate a proposed System Network Persistence (SNP) metric. This work is automated as a SysNet-Analytics Tool to demonstrate application on real world systems including: software system, natural-language system, retail market system, and IMDb system. We quantified stability and persistence of entity-connections in a system state series. This results in evolution information, which helps in system evolution analytics based on knowledge discovery and data mining.
Predicting phase transition pressure in solids: a semiclassical possibility

: This is a short review of the physical ideas, algorithm for calculations of the phase transition pressure, and some results of a semi-classical theory of the behavior of materials under high pressure proposed by P. Savi} and R. Ka{anin. It is based on the Coulomb interaction, supplemented by a microscopic selection rule and a set of experimentally founded postulates. The theory has been applied to cases ranging from DAC experiments to the calculation of models of planetary internal structure.
Polarization and reliability of news sources in Wikipedia

PurposeWikipedia's inclusive editorial policy permits unrestricted participation, enabling individuals to contribute and disseminate their expertise while drawing upon a multitude of external sources. News media outlets constitute nearly one-third of all citations within Wikipedia. However, embracing such a radically open approach also poses the challenge of the potential introduction of biased content or viewpoints into Wikipedia. The authors conduct an investigation into the integrity of knowledge within Wikipedia, focusing on the dimensions of source political polarization and trustworthiness. Specifically, the authors delve into the conceivable presence of political polarization within the news media citations on Wikipedia, identify the factors that may influence such polarization within the Wikipedia ecosystem and scrutinize the correlation between political polarization in news media sources and the factual reliability of Wikipedia's content.Design/methodology/approachThe authors conduct a descriptive and regression analysis, relying on Wikipedia Citations, a large-scale open dataset of nearly 30 million citations from English Wikipedia. Additionally, this dataset has been augmented with information obtained from the Media Bias Monitor (MBM) and the Media Bias Fact Check (MBFC). FindingsThe authors find a moderate yet significant liberal bias in the choice of news media sources across Wikipedia. Furthermore, the authors show that this effect persists when accounting for the factual reliability of the news media.Originality/valueThe results contribute to Wikipedia’s knowledge integrity agenda in suggesting that a systematic effort would help to better map potential biases in Wikipedia and find means to strengthen its neutral point of view policy.
Anisotropic flow: A case for bottomonia

Studies of [Formula: see text] [Formula: see text] at RHIC and LHC energies have provided important elements toward the understanding on the production mechanisms and thermalization of charm quarks. Bottomonia has an advantage since it is a cleaner probe. A brief discussion has been provided for [Formula: see text] [Formula: see text], which can become the new probe for QGP, including the necessity of studies for small systems.
Code4ML: a large-scale dataset of annotated Machine Learning code

The use of program code as a data source is increasingly expanding among data scientists. The purpose of the usage varies from the semantic classification of code to the automatic generation of programs. However, the machine learning model application is somewhat limited without annotating the code snippets. To address the lack of annotated datasets, we present the Code4ML corpus. It contains code snippets, task summaries, competitions, and dataset descriptions publicly available from Kaggle—the leading platform for hosting data science competitions. The corpus consists of ~2.5 million snippets of ML code collected from ~100 thousand Jupyter notebooks. A representative fraction of the snippets is annotated by human assessors through a user-friendly interface specially designed for that purpose. Code4ML dataset can help address a number of software engineering or data science challenges through a data-driven approach. For example, it can be helpful for semantic code classification, code auto-completion, and code generation for an ML task specified in natural language.
Planning with Spatial and Temporal Abstraction from Point Clouds for Deformable Object Manipulation

: Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels. Previous methods typically either focus on short-horizon tasks or make the strong assumption that full-state information is available. However, full states of deformable objects are often unavailable. In this paper, we propose PlAnning with Spatial and Temporal Abstraction (PASTA), which incorporates both spatial abstraction (reasoning about objects and their relations to each other) and temporal abstraction (reasoning over skills instead of low-level actions). Our framework maps high-dimension 3D point clouds into a set of latent vectors and plans skill sequences with the latent set representation. Our method can solve challenging, novel sequential deformable object manipulation tasks in the real world, which require combining multiple tool-use skills such as cutting with a knife, pushing with a pusher, and spreading dough with a roller. Additional materials can be found on our project website. 1
Deep Learning Object Detection Approaches to Source Identification

Traditionally source identiﬁcation is solved using threshold based energy detection algorithms. These algorithms frequently sum up the activity in regions, and consider regions above a speciﬁc activity threshold to be sources. While these algorithms work for the majority of cases, they often fail to detect signals that occupy small frequency bands, fail to distinguish sources with overlapping frequency bands, and cannot detect any signals under a speciﬁed signal to noise ratio. Through the conversion of raw signal data to spectrogram, source identiﬁcation can be framed as an object detection problem. By leveraging modern advancements in deep learning based object detection, we propose a system that manages to alleviate the failure cases encountered when using traditional source identiﬁcation algorithms. Our contributions include framing source identiﬁcation as an object detection problem, the publication of a spectrogram object detection dataset, and evaluation of the RetinaNet and YOLOv5 object detection models trained on the dataset. Our ﬁnal models achieve Mean Average Precisions of up to 0.906. With such a high Mean Average Precision, these models are sufﬁciently robust for use in real world applications.
Low-Complexity Channel Estimation for Massive MIMO Systems With Decentralized Baseband Processing

The traditional centralized baseband processing architecture is faced with the bottlenecks of high computation complexity and excessive fronthaul communication, especially when the number of antennas at the base station (BS) is large. To cope with these two challenges, the decentralized baseband processing (DBP) architecture has been proposed, where the BS antennas are partitioned into multiple clusters and each is connected to a local baseband unit (BBU). In this paper, we are interested in the low-complexity distributed channel estimation (CE) method under such DBP architecture, which is rarely studied in the literature. The aim is to devise distributed CE algorithms that can perform as well as the centralized scheme but with a small inter-BBU communication cost. Specifically, based on the low-complexity diagonal minimum mean square error channel estimator, we propose two distributed CE algorithms, namely the aggregate-then-estimate algorithm and the estimate-then-aggregate algorithm. In contrast to the existing distributed CE algorithm which requires iterative information exchanges among the nodes, our algorithms only require one roundtrip communication among BBUs. Extensive experiment results are presented to demonstrate the advantages of the proposed distributed CE algorithms in terms of estimation accuracy, inter-BBU communication cost and computation complexity.
Joint Semantic Transfer Network for IoT Intrusion Detection

In this article, we propose a joint semantic transfer network (JSTN) toward effective intrusion detection (ID) for large-scale scarcely labeled Internet of Things (IoT) domain. As a multisource heterogeneous domain adaptation (MS-HDA) method, the JSTN integrates a knowledge-rich network intrusion (NI) domain and another small-scale IoT intrusion (II) domain as source domains and preserves intrinsic semantic properties to assist target II domain ID. The JSTN jointly transfers the following three semantics to learn a domain-invariant and discriminative feature representation. The scenario semantic endows source NI and II domains with characteristics from each other to ease the knowledge transfer process via a confused domain discriminator and categorical distribution knowledge preservation. It also reduces the source–target discrepancy to make the shared feature space domain invariant. Meanwhile, the weighted implicit semantic transfer boosts discriminability via a fine-grained knowledge preservation, which transfers the source categorical distribution to the target domain. The source–target divergence guides the importance weighting during knowledge preservation to reflect the degree of knowledge learning. Additionally, the hierarchical explicit semantic alignment performs centroid-level and representative-level alignment with the help of a geometric similarity-aware pseudo-label refiner, which exploits the value of the unlabeled target II domain and explicitly aligns feature representations from a global and local perspective in a concentrated manner. Comprehensive experiments on various tasks verify the superiority of the JSTN against state-of-the-art comparing methods, on average a 10.3% of accuracy boost is achieved. The statistical soundness of each constituting component and the computational efficiency is also verified.
Instance-Optimal Differentially Private Estimation

In this work, we study local minimax convergence estimation rates subject to ε -diﬀerential privacy. Unlike worst-case rates, which may be conservative, algorithms that are locally minimax optimal must adapt to easy instances of the problem. We construct locally minimax diﬀerentially private estimators for one-parameter exponential families and estimating the tail rate of a distribution. In these cases, we show that optimal algorithms for simple hypothesis testing, namely the recent optimal private testers of Canonne et al. (2019), directly inform the design of locally minimax estimation algorithms.
Probing for targeted syntactic knowledge through grammatical error detection

Targeted studies testing knowledge of subject-verb agreement (SVA) indicate that pre-trained language models encode syntactic information. We assert that if models robustly encode subject-verb agreement, they should be able to identify when agreement is correct and when it is incorrect. To that end, we propose grammatical error detection as a diagnostic probe to evaluate token-level contextual representations for their knowledge of SVA. We evaluate contextual representations at each layer from five pre-trained English language models: BERT, XLNet, GPT-2, RoBERTa and ELECTRA. We leverage public annotated training data from both English second language learners and Wikipedia edits, and report results on manually crafted stimuli for subject-verb agreement. We find that masked language models linearly encode information relevant to the detection of SVA errors, while the autoregressive models perform on par with our baseline. However, we also observe a divergence in performance when probes are trained on different training sets, and when they are evaluated on different syntactic constructions, suggesting the information pertaining to SVA error detection is not robustly encoded.
Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport

We address key challenges in long-horizon embodied exploration and navigation by proposing a new object transport task and a novel modular framework for temporally extended navigation. Our first contribution is the design of a novel Long-HOT environment focused on deep exploration and long-horizon planning where the agent is required to efficiently find and pick up target objects to be carried and dropped at a goal location, with load constraints and optional access to a container if it finds one. Further, we propose a modular hierarchical transport policy (HTP) that builds a topological graph of the scene to perform exploration with the help of weighted frontiers. Our hierarchical approach uses a combination of motion planning algorithms to reach point goals within explored locations and object navigation policies for moving towards semantic targets at unknown locations. Experiments on both our proposed Habitat transport task and on MultiOn benchmarks show that our method significantly outperforms baselines and prior works. Further, we validate the effectiveness of our modular approach for long-horizon transport by demonstrating meaningful generalization to much harder transport scenes with training only on simpler versions of the task.