Miracle-dz/UniFAR-FTUs huggingface dataset:
A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentation

Effectively modeling discriminative spatio-temporal information is essential for segmenting activities in long action sequences. However, we observe that existing methods are limited in weak spatio-temporal modeling capability due to two forms of decoupled modeling: (i) cascaded interaction couples spatial and temporal modeling, which over-smooths motion modeling over the long sequence, and (ii) joint-shared temporal modeling adopts shared weights to model each joint, ignoring the distinct motion patterns of different joints. We propose a Decoupled Spatio-Temporal Framework (DeST) to address the above issues. Firstly, we decouple the cascaded spatio-temporal interaction to avoid stacking multiple spatio-temporal blocks, while achieving sufficient spatio-temporal interaction. Specifically, DeST performs once unified spatial modeling and divides the spatial features into different groups of subfeatures, which then adaptively interact with temporal features from different layers. Since the different sub-features contain distinct spatial semantics, the model could learn the optimal interaction pattern at each layer. Meanwhile, inspired by the fact that different joints move at different speeds, we propose joint-decoupled temporal modeling, which employs independent trainable weights to capture distinctive temporal features of each joint. On four large-scale benchmarks of different scenes, DeST significantly outperforms current state-of-the-art methods with less computational complexity.
Building rigid networks with prestress and selective pruning

The mechanical properties of biopolymer networks depend on their mean coordination number and stress state. Models based on spring networks have been shown to capture some properties of materials such as the actin cortex, but the effect of prestress combined with filament pruning, known to exist in the cortex, has not been studied. We show that in central-force spring networks below isostatic coordination that are rigidified by prestress, details of the pruning method significantly affect mechanical properties: networks pruned by a tension-inhibited method not only have a larger shear modulus than randomly-pruned networks, but require far smaller initial prestrains in order to remain rigid at biologically-relevant coordinations. These findings suggest a possible reason for the common motif of tension-inhibited filament-cleaving proteins in biopolymer networks.
Adaptive Compression of the Latent Space in Variational Autoencoders

Variational Autoencoders (VAEs) are powerful generative models that have been widely used in various fields, including image and text generation. However, one of the known challenges in using VAEs is the model's sensitivity to its hyperparameters, such as the latent space size. This paper presents a simple extension of VAEs for automatically determining the optimal latent space size during the training process by gradually decreasing the latent size through neuron removal and observing the model performance. The proposed method is compared to traditional hyperparameter grid search and is shown to be significantly faster while still achieving the best optimal dimensionality on four image datasets. Furthermore, we show that the final performance of our method is comparable to training on the optimal latent size from scratch, and might thus serve as a convenient substitute.
NLLG Quarterly arXiv Report 09/23: What are the most influential current AI Papers?

Artificial Intelligence (AI) has witnessed rapid growth, especially in the subfields Natural Language Processing (NLP), Machine Learning (ML) and Computer Vision (CV). Keeping pace with this rapid progress poses a considerable challenge for researchers and professionals in the field. In this arXiv report, the second of its kind, which covers the period from January to September 2023, we aim to provide insights and analysis that help navigate these dynamic areas of AI. We accomplish this by 1) identifying the top-40 most cited papers from arXiv in the given period, comparing the current top-40 papers to the previous report, which covered the period January to June; 2) analyzing dataset characteristics and keyword popularity; 3) examining the global sectoral distribution of institutions to reveal differences in engagement across geographical areas. Our findings highlight the continued dominance of NLP: while only 16% of all submitted papers have NLP as primary category (more than 25% have CV and ML as primary category), 50% of the most cited papers have NLP as primary category, 90% of which target LLMs. Additionally, we show that i) the US dominates among both top-40 and top-9k papers, followed by China; ii) Europe clearly lags behind and is hardly represented in the top-40 most cited papers; iii) US industry is largely overrepresented in the top-40 most influential papers.
Relevance Feedback with Brain Signals

The Relevance Feedback (RF) process relies on accurate and real-time relevance estimation of feedback documents to improve retrieval performance. Since collecting explicit relevance annotations imposes an extra burden on the user, extensive studies have explored using pseudo-relevance signals and implicit feedback signals as substitutes. However, such signals are indirect indicators of relevance and suffer from complex search scenarios where user interactions are absent or biased. Recently, the advances in portable and high-precision brain-computer interface (BCI) devices have shown the possibility to monitor user’s brain activities during search process. Brain signals can directly reflect user’s psychological responses to search results and thus it can act as additional and unbiased RF signals. To explore the effectiveness of brain signals in the context of RF, we propose a novel RF framework that combines BCI-based relevance feedback with pseudo-relevance signals and implicit signals to improve the performance of document re-ranking. The experimental results on the user study dataset show that incorporating brain signals leads to significant performance improvement in our RF framework. Besides, we observe that brain signals perform particularly well in several hard search scenarios, especially when implicit signals as feedback are missing or noisy. This reveals when and how to exploit brain signals in the context of RF.
Observations of the migrating semidiurnal and quaddiurnal tides from the RAIDS/NIRS instrument

In this paper we analyze temperature data from the Near‐Infrared Spectrometer (NIRS) instrument on Remote Atmospheric and Ionospheric Detection System experiment on the International Space Station and the Sounding of the Atmosphere using Broadband Emission Radiometry (SABER) on the Thermosphere Ionosphere Mesosphere Energetics Dynamics satellite during June and July 2010 to investigate structures of the migrating semidiurnal (12 h) and quaddiurnal (6 h) tides in the upper mesosphere and lower thermosphere. Temperature measurements from the NIRS and SABER instruments allow us to examine the tides from the stratosphere to the lower thermosphere. We find that the amplitude of the migrating 6 h tide grows from ~5 K near 100 km altitude to ~30 K near 130 km. The amplitudes of the tide at altitudes accessible by NIRS are much larger than those previously reported at lower altitudes from the Aura Microwave Limb Sounder and the SABER instruments. The amplitude of the 12 h tide in the NIRS data shows two peaks in the lower thermosphere (between 95 and 130 km) with a maximum around 60 K occurring in the winter hemisphere near 20° latitude and a second maximum around 40 K occurring in the summer hemisphere near 30° latitude. The structure of the migrating terdiurnal (8 h) tide is also investigated in the NIRS data and shows increasing amplitude with altitude over a broad range of latitudes, roughly between 50°N and 30°S. Altitudinal variations seen in the 6, 8, and 12 h tides suggest an evolving mix of various Hough modes.
LLM360: Towards Fully Transparent Open-Source LLMs

The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers. However, most LLMs have only released partial artifacts, such as the final model weights or inference code, and technical reports increasingly limit their scope to high-level design choices and surface statistics. These choices hinder progress in the field by degrading transparency into the training of LLMs and forcing teams to rediscover many details in the training process. We present LLM360, an initiative to fully open-source LLMs, which advocates for all training code and data, model checkpoints, and intermediate results to be made available to the community. The goal of LLM360 is to support open and collaborative AI research by making the end-to-end LLM training process transparent and reproducible by everyone. As a first step of LLM360, we release two 7B parameter LLMs pre-trained from scratch, Amber and CrystalCoder, including their training code, data, intermediate checkpoints, and analyses (at https://www.llm360.ai). We are committed to continually pushing the boundaries of LLMs through this open-source effort. More large-scale and stronger models are underway and will be released in the future.
Profinite completions of free-by-free groups contain everything

Given an arbitrary, finitely presented, residually finite group Γ, one can construct a finitely generated, residually finite, free-by-free group $M_\Gamma = F_\infty\rtimes F_4$ and an embedding $M_\Gamma \hookrightarrow (F_4\ast \Gamma)\times F_4$ that induces an isomorphism of profinite completions. In particular, there is a free-by-free group whose profinite completion contains $\widehat{\Gamma}$ as a retract.
Gated Linear Attention Transformers with Hardware-Efficient Training

Transformers with linear attention allow for efficient parallel training but can simultaneously be formulated as an RNN with 2D (matrix-valued) hidden states, thus enjoying linear-time inference complexity. However, linear attention generally underperforms ordinary softmax attention. Moreover, current implementations of linear attention lack I/O-awareness and are thus slower than highly optimized implementations of softmax attention. This work describes a hardware-efficient algorithm for linear attention that trades off memory movement against parallelizability. The resulting implementation, dubbed FLASHLINEARATTENTION, is faster than FLASHATTENTION-2(Dao, 2023) as a standalone layer even at short sequence lengths (e.g., 1K). We then generalize this algorithm to a more expressive variant of linear attention with data-dependent gates. When used as a replacement for the standard attention layer in Transformers, the resulting gated linear attention (GLA) Transformer is found to perform competitively against the LLaMA-architecture Transformer (Touvron et al., 2023) as well recent linear-time-inference baselines such as RetNet(Sun et al., 2023a) and Mamba (Gu&Dao, 2023) on moderate-scale language modeling experiments. GLA Transformer is especially effective at length generalization, enabling a model trained on 2K to generalize to 28K on PG19 without significant perplexity degradations. For training speed, the GLA Transformer has higher throughput than a similarly-sized Mamba model.
Numerical Simulation of the Cleaning Process of Microchannel by an External Stationary Flow

Clogging of microchannels due to particle adhesion on the walls is a serious problem in many industrial processes. As such, the service life of microfluidic systems is determined by their ability to maintain flow without interruption over a long period of time. Quite often, cleaning of microfluidic and filtration systems by flushing them with a clean liquid flow is used to extend their operating time. A hydrodynamic approach is required for modeling of the cleaning process. In this paper, a mathematical model of microchannel cleaning from fine particles due to interaction with the flow is proposed. The interaction of particles with the flow is described within the Stokes approximation, a random force caused by diffusion is taken into account. The interaction between particles is neglected. The interaction with the channel walls is caused by van der Waals force only. The equations describing the dynamics of particles and flow changes inside the microchannel during their detachment are obtained. The problem is solved numerically in the framework of the random walk model with statistical processing of obtaining data. The flow characteristics and the time of microchannel cleaning depending on the model parameters are obtained
Enforcing conserved quantities in Galerkin truncation and finite volume discretization

Finite-dimensional truncations are routinely used to approximate partial differential equations (PDEs), either to obtain numerical solutions or to derive reduced-order models. The resulting discretized equations are known to violate certain physical properties of the system. In particular, first integrals of the PDE may not remain invariant after discretization. Here, we use the method of reduced-order nonlinear solutions (RONS) to ensure that the conserved quantities of the PDE survive its finite-dimensional truncation. In particular, we develop two methods: Galerkin RONS and finite volume RONS. Galerkin RONS ensures the conservation of first integrals in Galerkin-type truncations, whether used for direct numerical simulations or reduced-order modeling. Similarly, finite volume RONS conserves any number of first integrals of the system, including its total energy, after finite volume discretization. Both methods are applicable to general time-dependent PDEs and can be easily incorporated in existing Galerkin-type or finite volume code. We demonstrate the efficacy of our methods on two examples: direct numerical simulations of the shallow water equation and a reduced-order model of the nonlinear Schrodinger equation. As a byproduct, we also generalize RONS to phenomena described by a system of PDEs.
Vivisecting the Dissection: On the Role of Trusted Components in BFT Protocols

A recent paper by Gupta et al. (EuroSys'23) challenged the usefulness of trusted component (TC) based Byzantine fault-tolerant (BFT) protocols to lower the replica group size from $3f+1$ to $2f+1$, identifying three limitations of such protocols and proposing that TCs should be used instead to improve the performance of BFT protocols. Here, we point out flaws in both arguments and advocate that the most worthwhile use of TCs in BFT protocols is indeed to make them as resilient as crash fault-tolerant (CFT) protocols, which can tolerate up to $f$ faulty replicas using $2f+1$ replicas.
Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers

Social media platforms serve as accessible outlets for individuals to express their thoughts and experiences, resulting in an influx of user-generated data spanning all age groups. While these platforms enable free expression, they also present significant challenges, including the proliferation of hate speech and offensive content. Such objectionable language disrupts objective discourse and can lead to radicalization of debates, ultimately threatening democratic values. Consequently, organizations have taken steps to monitor and curb abusive behavior, necessitating automated methods for identifying suspicious posts. This paper contributes to Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages (HASOC) 2023 shared tasks track. We, team Z-AGI Labs, conduct a comprehensive comparative analysis of hate speech classification across five distinct languages: Bengali, Assamese, Bodo, Sinhala, and Gujarati. Our study encompasses a wide range of pre-trained models, including Bert variants, XLM-R, and LSTM models, to assess their performance in identifying hate speech across these languages. Results reveal intriguing variations in model performance. Notably, Bert Base Multilingual Cased emerges as a strong performer across languages, achieving an F1 score of 0.67027 for Bengali and 0.70525 for Assamese. At the same time, it significantly outperforms other models with an impressive F1 score of 0.83009 for Bodo. In Sinhala, XLM-R stands out with an F1 score of 0.83493, whereas for Gujarati, a custom LSTM-based model outshined with an F1 score of 0.76601. This study offers valuable insights into the suitability of various pre-trained models for hate speech detection in multilingual settings. By considering the nuances of each, our research contributes to an informed model selection for building robust hate speech detection systems.
Asynchronous Distributed Optimization with Delay-free Parameters

Existing asynchronous distributed optimization algorithms often use diminishing step-sizes that cause slow practical convergence, or use fixed step-sizes that depend on and decrease with an upper bound of the delays. Not only are such delay bounds hard to obtain in advance, but they also tend to be large and rarely attained, resulting in unnecessarily slow convergence. This paper develops asynchronous versions of two distributed algorithms, Prox-DGD and DGD-ATC, for solving consensus optimization problems over undirected networks. In contrast to alternatives, our algorithms can converge to the fixed point set of their synchronous counterparts using step-sizes that are independent of the delays. We establish convergence guarantees for strongly and weakly convex problems under both partial and total asynchrony. We also show that the convergence speed of the two asynchronous methods adapts to the actual level of asynchrony rather than being constrained by the worst-case. Numerical experiments demonstrate a strong practical performance of our asynchronous algorithms.
Multi-dimensional Fair Federated Learning

Federated learning (FL) has emerged as a promising collaborative and secure paradigm for training a model from decentralized data without compromising privacy. Group fairness and client fairness are two dimensions of fairness that are important for FL. Standard FL can result in disproportionate disadvantages for certain clients, and it still faces the challenge of treating different groups equitably in a population. The problem of privately training fair FL models without compromising the generalization capability of disadvantaged clients remains open. In this paper, we propose a method, called mFairFL, to address this problem and achieve group fairness and client fairness simultaneously. mFairFL leverages differential multipliers to construct an optimization objective for empirical risk minimization with fairness constraints. Before aggregating locally trained models, it first detects conflicts among their gradients, and then iteratively curates the direction and magnitude of gradients to mitigate these conflicts. Theoretical analysis proves mFairFL facilitates the fairness in model development. The experimental evaluations based on three benchmark datasets show significant advantages of mFairFL compared to seven state-of-the-art baselines.
EFFECT OF PARSLEY SEEDS (Petroselinum crispum) AS FEED ADDITIVE ON PRODUCTIVE PERFORMANCE IN JAPANESE LAYING QUAILS

This study was conducted in the Animal farm, department of Animal Resources, College of Agriculture, University of Tikrit during the period from 11 March,2014 to 11 May,2014 to investigate the effect of adding Parsley seeds (Petroselinum crispum) in the diets on feed consumption, feed efficiency and egg production in the performance of Japanese quails (Coturnix coturnix japonica). Ninety six laying quails (72 females and 24 males) at 48 weeks of age were reared in cages and randomly allocated with 4 dietary treatments one was the control group (without any additives) (T1); the others were (0.5 %, 1.0 % and 1.5 % parsley) added to (T2, T3 and T4) respectively, each group consist of 24 birds and divided to 3 Replicates (8 birds). Water and feed were provided ad libitum during the experimental period. There were no significant difference in egg weight and feed efficiency among the groups while there were significant in hen day egg production (H.D %), total cumulative egg number, egg mass and feed consumption due to parsley seeds in quail compared with control. Concluded from this study that use of parsley to improve productive performance of Japanese quails.
Recent Advances in Deterministic Human Motion Prediction: A Review

In recent years, with the continuous advancement of deep learning and the emergence of large-scale human motion datasets, human motion prediction technology has gradually gained prominence in various fields such as human-computer interaction, autonomous driving, sports analysis, and personnel tracking. This article introduces common model architectures in this domain along with their respective advantages and disadvantages. It also systematically summarizes recent research innovations, focusing on in-depth discussions of relevant papers in these areas, thereby highlighting forward-looking insights into the field's development. Furthermore, this paper provides a comprehensive overview of existing methods, commonly used datasets, and evaluation metrics in this field. Finally, it discusses some of the current limitations in the field and proposes potential future research directions to address these challenges and promote further advancements in human motion prediction.
Modyn: A Platform for Model Training on Dynamic Datasets With Sample-Level Data Selection

Machine learning training data is often dynamic in real-world use cases, i.e., data is added or removed and may experience distribution shifts over time. Models must incorporate this evolving training data to improve generalization, adapt to potential distribution shifts, and adhere to privacy regulations. However, the cost of model (re)training is proportional to how often the model trains and on how much data it trains on. While ML research explores these topics in isolation, there is no end-to-end open-source platform to facilitate the exploration of model retraining and data selection policies and the deployment these algorithms efficiently at scale. We present Modyn, a platform for model training on dynamic datasets that enables sample-level data selection and triggering policies. Modyn orchestrates continuous training pipelines while optimizing the underlying system infrastructure to support fast access to arbitrary data samples for efficient data selection. Modyn's extensible architecture allows users to run training pipelines without modifying the platform code, and enables researchers to effortlessly extend the system. We evaluate Modyn's training throughput, showing that even in memory-bound recommendation systems workloads, Modyn is able to reach 80 to 100 % of the throughput compared to loading big chunks of data locally without sample-level data selection. Additionally, we showcase Modyn's functionality with three different data selection policies.