Miracle-dz/UniFAR-FTUs huggingface dataset:
Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Games

We consider the problem of learning discriminative representations for data in a high-dimensional space with distribution supported on or around multiple low-dimensional linear subspaces. That is, we wish to compute a linear injective map of the data such that the features lie on multiple orthogonal subspaces. Instead of treating this learning problem using multiple PCAs, we cast it as a sequential game using the closed-loop transcription (CTRL) framework recently proposed for learning discriminative and generative representations for general low-dimensional submanifolds. We prove that the equilibrium solutions to the game indeed give correct representations. Our approach unifies classical methods of learning subspaces with modern deep learning practice, by showing that subspace learning problems may be provably solved using the modern toolkit of representation learning. In addition, our work provides the first theoretical justification for the CTRL framework, in the important case of linear subspaces. We support our theoretical findings with compelling empirical evidence. We also generalize the sequential game formulation to more general representation learning problems. Our code, including methods for easy reproduction of experimental results, is publically available on GitHub.
PHN: Parallel heterogeneous network with soft gating for CTR prediction

The Click-though Rate (CTR) prediction task is a basic task in recommendation system. Most of the previous researches of CTR models built based on Wide \&deep structure and gradually evolved into parallel structures with different modules. However, the simple accumulation of parallel structures can lead to higher structural complexity and longer training time. Based on the Sigmoid activation function of output layer, the linear addition activation value of parallel structures in the training process is easy to make the samples fall into the weak gradient interval, resulting in the phenomenon of weak gradient, and reducing the effectiveness of training. To this end, this paper proposes a Parallel Heterogeneous Network (PHN) model, which constructs a network with parallel structure through three different interaction analysis methods, and uses Soft Selection Gating (SSG) to feature heterogeneous data with different structure. Finally, residual link with trainable parameters are used in the network to mitigate the influence of weak gradient phenomenon. Furthermore, we demonstrate the effectiveness of PHN in a large number of comparative experiments, and visualize the performance of the model in training process and structure.
Nested bandits

In many online decision processes, the optimizing agent is called to choose between large numbers of alternatives with many inherent similarities; in turn, these similarities imply closely correlated losses that may confound standard discrete choice models and bandit algorithms. We study this question in the context of nested bandits, a class of adversarial multi-armed bandit problems where the learner seeks to minimize their regret in the presence of a large number of distinct alternatives with a hierarchy of embedded (non-combinatorial) similarities. In this setting, optimal algorithms based on the exponential weights blueprint (like Hedge, EXP3, and their variants) may incur significant regret because they tend to spend excessive amounts of time exploring irrelevant alternatives with similar, suboptimal costs. To account for this, we propose a nested exponential weights (NEW) algorithm that performs a layered exploration of the learner's set of alternatives based on a nested, step-by-step selection method. In so doing, we obtain a series of tight bounds for the learner's regret showing that online learning problems with a high degree of similarity between alternatives can be resolved efficiently, without a red bus / blue bus paradox occurring.
DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessment

Compared with spatial counterparts, temporal relationships between frames and their influences on video quality assessment (VQA) are still relatively under-studied in existing works. These relationships lead to two important types of effects for video quality. Firstly, some meaningless temporal variations (such as shaking, flicker, and unsmooth scene transitions) cause temporal distortions that degrade quality of videos. Secondly, the human visual system often has different attention to frames with different contents, resulting in their different importance to the overall video quality. Based on prominent time-series modeling ability of transformers, we propose a novel and effective transformer-based VQA method to tackle these two issues. To better differentiate temporal variations and thus capture the temporal distortions, we design the Spatial-Temporal Distortion Extraction (STDE) module that extracts multi-level spatial-temporal features with a video swin transformer tiny (Swin-T) backbone and uses temporal difference layer to further capture these distortions. To tackle with temporal quality attention, we propose the encoder-decoder-like temporal content transformer (TCT). We also introduce the temporal sampling on features to reduce the input length for the TCT, so as to improve the learning effectiveness and efficiency of this module. Consisting of the STDE and the TCT, the proposed Temporal Distortion-Content Transformers for Video Quality Assessment (DisCoVQA) reaches state-of-the-art performance on several VQA benchmarks without any extra pre-training datasets and up to 10% better generalization ability than existing methods. We also conduct extensive ablation experiments to prove the effectiveness of each part in our proposed model, and provide visualizations to prove that the proposed modules achieve our intention on modeling these temporal issues. Our code is published at https://github.com/QualityAssessment/DisCoVQA.
NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search

Graph neural architecture search (GraphNAS) has recently aroused considerable attention in both academia and industry. However, two key challenges seriously hinder the further research of GraphNAS. First, since there is no consensus for the experimental setting, the empirical results in different research papers are often not comparable and even not reproducible, leading to unfair comparisons. Secondly, GraphNAS often needs extensive computations, which makes it highly inefficient and inaccessible to researchers without access to large-scale computation. To solve these challenges, we propose NAS-Bench-Graph, a tailored benchmark that supports unified, reproducible, and efficient evaluations for GraphNAS. Specifically, we construct a unified, expressive yet compact search space, covering 26,206 unique graph neural network (GNN) architectures and propose a principled evaluation protocol. To avoid unnecessary repetitive training, we have trained and evaluated all of these architectures on nine representative graph datasets, recording detailed metrics including train, validation, and test performance in each epoch, the latency, the number of parameters, etc. Based on our proposed benchmark, the performance of GNN architectures can be directly obtained by a look-up table without any further computation, which enables fair, fully reproducible, and efficient comparisons. To demonstrate its usage, we make in-depth analyses of our proposed NAS-Bench-Graph, revealing several interesting findings for GraphNAS. We also showcase how the benchmark can be easily compatible with GraphNAS open libraries such as AutoGL and NNI. To the best of our knowledge, our work is the first benchmark for graph neural architecture search.
Channel Estimation for Delay Alignment Modulation

Delay alignment modulation (DAM) is a promising technology for inter-symbol interference (ISI)-free communication without relying on sophisticated channel equalization or multi-carrier transmissions. The key ideas of DAM are delay precompensation and path-based beamforming, so that the multi-path signal components will arrive at the receiver simultaneously and constructively, rather than causing the detrimental ISI. However, the practical implementation of DAM requires channel state information (CSI) at the transmitter side. Therefore, in this letter, we study an efficient channel estimation method for DAM based on block orthogonal matching pursuit (BOMP) algorithm, by exploiting the block sparsity of the channel vector. Based on the imperfectly estimated CSI, the delay pre-compensations and tap-based beamforming are designed for DAM, and the resulting performance is studied. Simulation results demonstrate that with the BOMP-based channel estimation method, the CSI can be effectively acquired with low training overhead, and the performance of DAM based on estimated CSI is comparable to the ideal case with perfect CSI.
Outage in PMD compensated optical fiber line due to uncertainty of PMD estimation

The effect of uncertain control of a first order PMD compensator on outage probability in fiber optic communication line is analyzed. For a PMD compensator – PMD monitor set a characteristics is proposed that is capable of presenting limits of compensation. Relation between outage probability and uncertainty of estimated PMD parameters, used to control the compensator, is quantified by numerical calculations. Consequences of the uncertainty for maximum transmission reach are evaluated. Streszczenie. Przeanalizowano wpływ niepewności sterowania kompensatorem PMD pierwszego rzędu na prawdopodobieństwo niedostępności w światłowodowej linii telekomunikacyjnej. Dla układu kompensator PMD – monitor PMD zaproponowano charakterystykę umożliwiającą prezentację granic kompensacji. Numerycznie przeprowadzono ocenę ilościową relacji pomiędzy prawdopodobieństwem niedostępności a niepewnością estymacji parametrów PMD, zastosowanych do sterowania kompensatorem. Oceniono skutki niepewności dla maksymalnego zasięgu transmisji. (Niedostępność w światłowodowej linii optycznej z kompensacją PMD w wyniku niepewności estymacji PMD)
The (partially) massless spin-3/2 and spin-5/2 fields in de Sitter spacetime as unitary and non-unitary representations of the de Sitter algebra

The divergence-free and gamma-traceless vector-spinor eigenfunctions, as well as the divergence-free and gamma-traceless rank-2 symmetric tensor-spinor eigenfunctions, of the Dirac operator on the $N$-sphere ($S^{N}$) are written down explicitly for $N \geq 3$. The spin-3/2 and spin-5/2 eigenmodes of the Dirac operator with arbitrary imaginary mass parameter on $N$-dimensional ($N\geq 3$) de Sitter spacetime ($dS_{N}$) are obtained by analytic continuation. Their transformation properties under the de Sitter algebra spin$(N,1)$ are studied. For $N$ odd, we show that there is no de Sitter (dS) invariant scalar product for these eigenmodes. For $N$ even, although dS invariant scalar products exist, positive-definiteness of the norm occurs only for the strictly and partially massless theories in $N=4$ dimensions. For $N=4$, the way in which the eigenmodes form unitary strictly and partially massless representations of spin$(4,1)$ is emphasised. The analysis presented in this paper reveals previously unknown features of the gauge-invariant theories with spin 3/2 and 5/2 on $dS_{N}$ ($N \geq 3$): the strictly massless spin-3/2 field theory, as well as the strictly and partially massless spin-5/2 field theories, are unitary only for $N=4$. In particular, a unitary theory for the gravitino field on $dS_{N}$ does not exist unless $N=4$.
Spatially Resolved Ionized Outflows Extending to ∼ 2 kpc in Seyfert 1 Galaxy NGC 7469 Revealed by the Very Large Telescope/MUSE

The Seyfert 1 galaxy NGC 7469 possesses a prominent nuclear starburst ring and a luminous active galactic nucleus (AGN). Evidence of an outflow in the innermost nuclear region has been found in previous works. We detect the ionized gas outflow on a larger scale in the galaxy using the archival Very Large Telescope/MUSE and Chandra observations. The optical emission lines are modeled using two Gaussian components, and a nonparametric approach is applied to measure the kinematics of [O iii] and Hα emitting gas. Line ratio diagnostics and spatially resolved maps are derived to examine the origin of the outflow. The kiloparsec-scale kinematics of [O iii] are dominated by a blueshifted component whereas the velocity map of Hα shows a rotational disk with a complex nonrotational substructure. The starburst wind around the circumnuclear ring is confirmed, and we find evidence of an AGN-driven outflow extending to a radial distance of ∼ 2 kpc from the nucleus, with a morphology consistent with a nearly face-on ionization cone. The previously reported circumnuclear outflow resembles part of the bright base. We derive mass and energy outflow rates for both the starburst wind and the AGN-driven outflow. The estimated kinetic coupling efficiency of the kiloparsec-scale AGN outflow is Ėout/Lbol∼0.1% , lower than the threshold predicted by the “two-stage” theoretical model for effective feedback. Our results reinforce the importance of spatially resolved study to disentangle feedback where AGNs and starbursts coexist, which may be common during the cosmic noon of black hole and galaxy growth.
KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Few-Shot NLP

This paper focuses on text data augmentation for few-shot NLP tasks. The existing data augmentation algorithms either leverage task-independent heuristic rules (e.g., Synonym Replacement) or ﬁne-tune general-purpose pre-trained language models (e.g., GPT2) using a small training set to produce new synthetic data. Consequently, these methods have trivial task-speciﬁc knowledge and are limited to yielding low-quality synthetic data for weak baselines in simple tasks. To combat this issue, we propose Know ledge Mixture D ata A ugmentation Model ( KnowDA ): an encoder-decoder LM pretrained on a mixture of diverse NLP tasks using K nowledge M ixture T raining (KoMT). KoMT is a training procedure that reformulates input examples from various heterogeneous NLP tasks into a uniﬁed text-to-text format, and employs denoising objectives in different granularity to learn to generate partial or complete samples. With the aid of KoMT, KnowDA could combine required task-speciﬁc knowledge implicitly from the learned mixture of tasks and quickly grasp the inherent synthesis law of the target task through a few given instances. To the best of our knowledge, we are the ﬁrst attempt to scale the number of tasks to 100+ in multi-task co-training for data augmentation. Extensive experiments show that i) KnowDA successfully improves the performance of Albert and Deberta by a large margin on the FewGLUE benchmark, outperforming previous state-of-the-art data augmentation baselines; ii) KnowDA could for data augmentation in few-shot language learning for the ﬁrst time. We demonstrate that the proposed Knowledge Mixture enables pre-trained language models the capability of generating proper synthetic instances from scratch for complicated tasks (i.e., the data sample has long sequences or multiple sentences). the effectiveness of our and KnowDA outperforms state-of-the-art
Marginal Tail-Adaptive Normalizing Flows

Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of the distribution. In this paper, we focus on improving the ability of normalizing flows to correctly capture the tail behavior and, thus, form more accurate models. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. This theoretical insight leads us to a novel type of flows based on flexible base distributions and data-driven linear layers. An empirical analysis shows that the proposed method improves on the accuracy -- especially on the tails of the distribution -- and is able to generate heavy-tailed data. We demonstrate its application on a weather and climate example, in which capturing the tail behavior is essential.
Fock–Goncharov dual cluster varieties and Gross–Siebert mirrors

Abstract Cluster varieties come in pairs: for any 𝒳 {\mathcal{X}} cluster variety there is an associated Fock–Goncharov dual 𝒜 {\mathcal{A}} cluster variety. On the other hand, in the context of mirror symmetry, associated with any log Calabi–Yau variety is its mirror dual, which can be constructed using the enumerative geometry of rational curves in the framework of the Gross–Siebert program. In this paper we bridge the theory of cluster varieties with the algebro-geometric framework of Gross–Siebert mirror symmetry. Particularly, we show that the mirror to the 𝒳 {\mathcal{X}} cluster variety is a degeneration of the Fock–Goncharov dual 𝒜 {\mathcal{A}} cluster variety and vice versa. To do this, we investigate how the cluster scattering diagram of Gross, Hacking, Keel and Kontsevich compares with the canonical scattering diagram defined by Gross and Siebert to construct mirror duals in arbitrary dimensions. Consequently, we derive an enumerative interpretation of the cluster scattering diagram. Along the way, we prove the Frobenius structure conjecture for a class of log Calabi–Yau varieties obtained as blow-ups of toric varieties.
Learning Multiscale Transformer Models for Sequence Generation

Multiscale feature hierarchies have been witnessed the success in the computer vision area. This further motivates researchers to design multiscale Transformer for natural language processing, mostly based on the self-attention mechanism. For example, restricting the receptive field across heads or extracting local fine-grained features via convolutions. However, most of existing works directly modeled local features but ignored the word-boundary information. This results in redundant and ambiguous attention distributions, which lacks of interpretability. In this work, we define those scales in different linguistic units, including sub-words, words and phrases. We built a multiscale Transformer model by establishing relationships among scales based on word-boundary information and phrase-level prior knowledge. The proposed \textbf{U}niversal \textbf{M}ulti\textbf{S}cale \textbf{T}ransformer, namely \textsc{Umst}, was evaluated on two sequence generation tasks. Notably, it yielded consistent performance gains over the strong baseline on several test sets without sacrificing the efficiency.
Collaboration and Integration in Project-Based Supply Chains in the Construction Industry

: This study investigates whether integrative and collaborative practices of the construction industry can be exclusively attributed to integrated project-delivery methods or whether traditional project-delivery methods also foster integration within project-based design teams. Project managers assessed team collaboration and the integration of teams into 46 construction industry projects in The Netherlands. Explanatory factor analysis (EFA) was used to identify the components that explained collaboration and integration within project design teams. Using analysis of variance of factor scores, the main ﬁ nding of the study was that, in the construction industry, collaboration is an independent component in integrative and collaborative practices that can be reliably assessed in research. Furthermore, this study provides evidence suggesting that both traditional and integrated project-delivery methods might lead to collaboration over time. The third ﬁ nding is that different project-delivery methods were not signi ﬁ cantly different in terms of the dimensions of integration and collaboration, except in the component of inclusive decision making within the building team and for strategic partnering. The ﬁ ndings suggest that relying on the type of project-delivery method is not suf ﬁ cient for managers communicating about the level of supply chain integration and collaboration. DOI: 10.1061/(ASCE)ME.1943-5479.0000592. © 2018 American Society of Civil Engineers.
Proﬁling of Patrons’ Interest Areas from Library’s Circulation Records – An Approach to Knowledge Management for University Students –

—The concept of knowledge management is important not only in industries but also in educational organizations like universities. Considering the importance of this concept, it is not surprising that many universities have introduced the database system for saving the proﬁles and history information of students and utilize them in order to improve their educational abilities. In order to make the information more effective in education, it is preferable to collect not only the information in raw but also the knowledge that is found by the data with data analysis and data mining. In this paper, as a new approach to knowledge mining in education as a part of educational knowledge management, we deal with the circulation records of a university library as the target data. The library’s circulation records show the relationship between the patrons and the books, which are usable to know the patrons about their ﬁelds of interest, knowledge levels, and other information. In this paper, we put special emphasis on the investigation of the proﬁling of students as a knowledge management. As a part of this, we deal with the interest area of a student and explore the measuring methods for the proﬁling of the student patrons.
Framework for continuous agile technology roadmap updating

Purpose Roadmapping has been used as an approach to support market, product and technology-integrated planning, resulting in a document commonly known as a roadmap. Despite the gains made in relation to the technique, recent studies indicate that most users leave or have difficulties in sustaining the process (i.e. maintaining the updated roadmaps). This paper aims to present a framework for continuous roadmap updating that incorporates principles from agile management fields. Design/methodology/approach The framework was developed through action research in a manufacturing firm in the construction industry. Findings The results demonstrate a positive impact on the degree of continuous information monitoring, roadmap credibility and use of the roadmap during innovation strategy decisions. Originality/value The key contribution of this framework is the demonstration of a new strategy for carrying out the maps in which information is internalized by the organization itself, using agile teams, without commissioned specialists and as part of the work standards.