Miracle-dz/UniFAR-FTUs huggingface dataset:
A Survey on Large Language Models from Concept to Implementation

Recent advancements in Large Language Models (LLMs), particularly those built on Transformer architectures, have significantly broadened the scope of natural language processing (NLP) applications, transcending their initial use in chatbot technology. This paper investigates the multifaceted applications of these models, with an emphasis on the GPT series. This exploration focuses on the transformative impact of artificial intelligence (AI) driven tools in revolutionizing traditional tasks like coding and problem-solving, while also paving new paths in research and development across diverse industries. From code interpretation and image captioning to facilitating the construction of interactive systems and advancing computational domains, Transformer models exemplify a synergy of deep learning, data analysis, and neural network design. This survey provides an in-depth look at the latest research in Transformer models, highlighting their versatility and the potential they hold for transforming diverse application sectors, thereby offering readers a comprehensive understanding of the current and future landscape of Transformer-based LLMs in practical applications.
Evaluation of Transit cosmological model in $f(R,T^{\phi})$ theory of gravity

We have explored a transitioning cosmic model, depicting late-time accelerated expansion in $f(R,T^{\phi})$ theory of gravity for an isotropic and homogeneous universe, where the trace of energy-momentum tensor $T^{\phi}$ is the function of the self-interacting scalar field $\phi$. We have proposed an explicit solution to the derived model by utilizing a scale factor of the hybrid form $a(t) = t^{\alpha} e^{\beta t}$, where $\alpha$ and $\beta$ are constants. To evaluate the best-fit values of free parameters of the suggested model, the statistical analysis based on the Markov Chain Monte Carlo (MCMC) method has been employed on 57 OHD points. We have described the dynamical features of the model like energy density, cosmic pressure, and equation of state parameter in the context of scalar field $\phi$. We have also described the potential and behavior of the scalar field for quintessence and phantom scenarios. The deceleration parameter depicts a transitioning universe with signature flipping at $z_t = 0.82$ with the present value of deceleration parameter $q_0=-0.41$. The violation of SEC for the derived model indicates the cosmic expansion at a faster rate. We have used statefinders to diagnose the model. The findings for our theoretical model indicate that the derived model agrees with observed findings within a particular range of limitations.
Simulating Relational Event Histories -- Why and How

Many important social phenomena result from repeated interactions among individuals over time such as email exchanges in an organization, or face-to-face interactions in a classroom. Insights into the mechanisms underlying the dynamics of these interactions can be achieved through simulations of networks on a fine temporal granularity. In this paper, we present statistical frameworks to simulate relational event networks under dyadic and actor-oriented relational event models. These simulators have a broad applicability in temporal social network research such as model fit assessment, theory building, network intervention planning, making predictions, understanding the impact of network structures, to name a few. We show this in three extensive applications. First, it is shown why simulation-based techniques are crucial for relational event model assessment, for example to investigate how past events affect future interactions in the network. Second, we demonstrate how simulation techniques contribute to a better understanding of the longevity of network interventions. Third, we show how simulation techniques are important when building and extending theories about social phenomena such as understanding social identity dynamics using optimal distinctiveness theory.
A Real-Time Framework for Domain-Adaptive Underwater Object Detection with Image Enhancement

In recent years, significant progress has been made in the field of underwater image enhancement (UIE). However, its practical utility for high-level vision tasks, such as underwater object detection (UOD) in Autonomous Underwater Vehicles (AUVs), remains relatively unexplored. It may be attributed to several factors: (1) Existing methods typically employ UIE as a pre-processing step, which inevitably introduces considerable computational overhead and latency. (2) The process of enhancing images prior to training object detectors may not necessarily yield performance improvements. (3) The complex underwater environments can induce significant domain shifts across different scenarios, seriously deteriorating the UOD performance. To address these challenges, we introduce EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capability. Specifically, both the UIE and UOD task heads share the same network backbone and utilize a lightweight design. Furthermore, to ensure balanced training for both tasks, we present a multi-stage training strategy aimed at consistently enhancing their performance. Additionally, we propose a novel domain-adaptation strategy to align feature embeddings originating from diverse underwater environments. Comprehensive experiments demonstrate that our framework not only achieves state-of-the-art (SOTA) performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios. Our efficiency analysis further highlights the substantial potential of our framework for onboard deployment.
Stau pairs from natural SUSY at high luminosity LHC

Natural supersymmetry (SUSY) with light higgsinos is perhaps the most plausible of all weak scale SUSY models while a variety of motivations point to (right) tau sleptons as the lightest of all the sleptons. We examine a SUSY model line with rather light right-staus embedded within natural SUSY. For light stau_1 of a few hundred GeV, then the decays stau_1 ->\tau\tchi_{1,2}^0 and \nu_\tau\tchi_1^- occur at comparable rates where the (higgsino-like) \tchi_1^\pm and \tchi_2^0 release only small visible energy: in this case, the expected \tau^+\tau^- +\eslt signature is diminished from usual expectations due to the presence of the nearly invisible decay mode \ttau_1 ->\nu_\tau\tchi_1^-. However, once m_{\ttau_1}>~m(bino), then decays to binos such as \ttau_1 ->\tau\tchi_3^0 open up where \tchi_3^0 decays to higgsinos plus W^\pm, Z^0 and h at comparable rates. For these heavier staus, then stau pair production gives rise to diboson+\eslt events which may contain 0, 1 or 2 additional hard \tau leptons. From these considerations, we examine the potential for future discovery of tau-slepton pair production at high-luminosity LHC. While we do not find a 5\sigma HL-LHC discovery reach for 3000 fb^{-1}, we do find a 95\% CL exclusion reach, ranging between m_{\ttau_1}:100-450 GeV for m_{\tchi_1^0}~ 100 GeV. This latter reach disappears for m_{\tchi_1^0}>~ 200 GeV.
A Review of Neuroscience-Inspired Machine Learning

One major criticism of deep learning centers around the biological implausibility of the credit assignment schema used for learning -- backpropagation of errors. This implausibility translates into practical limitations, spanning scientific fields, including incompatibility with hardware and non-differentiable implementations, thus leading to expensive energy requirements. In contrast, biologically plausible credit assignment is compatible with practically any learning condition and is energy-efficient. As a result, it accommodates hardware and scientific modeling, e.g. learning with physical systems and non-differentiable behavior. Furthermore, it can lead to the development of real-time, adaptive neuromorphic processing systems. In addressing this problem, an interdisciplinary branch of artificial intelligence research that lies at the intersection of neuroscience, cognitive science, and machine learning has emerged. In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks, discussing the solutions they provide for different scientific fields as well as their advantages on CPUs, GPUs, and novel implementations of neuromorphic hardware. We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.
Cleaning data with Swipe

The repair problem for functional dependencies is the problem where an input database needs to be modified such that all functional dependencies are satisfied and the difference with the original database is minimal. The output database is then called an optimal repair. If the allowed modifications are value updates, finding an optimal repair is NP-hard. A well-known approach to find approximations of optimal repairs builds a Chase tree in which each internal node resolves violations of one functional dependency and leaf nodes represent repairs. A key property of this approach is that controlling the branching factor of the Chase tree allows to control the trade-off between repair quality and computational efficiency. In this paper, we explore an extreme variant of this idea in which the Chase tree has only one path. To construct this path, we first create a partition of attributes such that classes can be repaired sequentially. We repair each class only once and do so by fixing the order in which dependencies are repaired. This principle is called priority repairing and we provide a simple heuristic to determine priority. The algorithms for attribute partitioning and priority repair are combined in the Swipe algorithm. An empirical study on four real-life data sets shows that Swipe is in the range of one to three orders of magnitude faster than multi-sequence Chase-based approaches, whereas the quality of repairs is comparable or better. Moreover, we provide a scalability analysis of the Swipe algorithm.
Tessellating Urban Space based on Street Intersections and Barriers to Movement

This paper presents a novel method to partition urban space with respect to its mobility network rather than abstract administrative boundaries or simple grids. The main goal of our method is to support mobility infrastructure planning and optimization in an urban environment. To this end, our tessellation, which is based on constrained Voronoi tessellation, generates cells that are compact and centred around street intersections. Furthermore, it takes into account unsuitable areas and barriers that obstruct movement through urban space. The resulting cells are fine-grained planning units that reflect the characteristics of the underlying mobility network. Potential applications include demand modelling, location planning and optimization for bicycleand car-sharing systems, or public transport stops.
Influence of BOLD Contributions to Diffusion fMRI Activation of the Visual Cortex

Reliance on the hemodynamic response as a surrogate marker of neural activity imposes an intrinsic limit on the spatial specificity of functional MRI. An alternative approach based on diffusion-weighted functional MRI (DfMRI) has been reported as a contrast less reliant on hemodynamic effects, however current evidence suggests that both hemodynamic and unique neural sources contribute to the diffusion signal. Here we compare activation patterns obtained with the standard blood oxygenation level-dependent (BOLD) contrast to DfMRI in order to gain a deeper understanding of how the BOLD proportion contributes to the observable diffusion signal. Both individual and group-level activation patterns obtained with DfMRI and BOLD to a visual field stimulation paradigm were analyzed. At the individual level, the DfMRI contrast showed a strong, positive relationship between the volumes of cortex activated in response to quadrant- and hemi-field visual stimulation. This was not observed in the corresponding BOLD experiment. Overall, the DfMRI response indicated less between-subject variability, with random effects analyses demonstrating higher statistical values at the peak voxel for DfMRI. Furthermore, the spatial extent of the activation was more restricted to the primary visual region for DfMRI than BOLD. However, the diffusion signal was sensitive to the hemodynamic response in a manner dependent on experimental manipulation. It was also limited by its low signal-to-noise ratio (SNR), demonstrating lower sensitivity than BOLD. Together these findings both support DfMRI as a contrast that bears a closer spatial relationship to the underlying neural activity than BOLD, and raise important caveats regarding its utilization. Models explaining the DfMRI signal change need to consider the dynamic vascular contributions that may vary with neural activity.
A noise-tolerant, resource-saving probabilistic binary neural network implemented by the SOT-MRAM compute-in-memory system

We report a spin-orbit torque(SOT) magnetoresistive random-access memory(MRAM)-based probabilistic binary neural network(PBNN) for resource-saving and hardware noise-tolerant computing applications. With the presence of thermal fluctuation, the non-destructive SOT-driven magnetization switching characteristics lead to a random weight matrix with controllable probability distribution. In the meanwhile, the proposed CIM architecture allows for the concurrent execution of the probabilistic vector-matrix multiplication (PVMM) and binarization. Furthermore, leveraging the effectiveness of random binary cells to propagate multi-bit probabilistic information, our SOT-MRAM-based PBNN system achieves a 97.78\% classification accuracy under a 7.01\% weight variation on the MNIST database through 10 sampling cycles, and the number of bit-level computation operations is reduced by a factor of 6.9 compared to that of the full-precision LeNet-5 network. Our work provides a compelling framework for the design of reliable neural networks tailored to the applications with low power consumption and limited computational resources.
Ghost cycles exhibit increased entrainment and richer dynamics in response to external forcing compared to slow-fast systems

Many natural, living and engineered systems display oscillations that are characterized by multiple timescales. Typically, such systems are described as slow-fast systems, where the slow dynamics result from a hyperbolic slow manifold that guides the movement of the system trajectories. Recently, we have provided an alternative description in which the slow dynamics result from a non-hyperbolic and Lyapunov-unstable attracting sets from connected dynamical ghosts that form a closed orbit (termed ghost cycles). Here we investigate the response properties of both type of systems to external forcing. Using the classical Van-der-Pol oscillator and two modified versions of this model that correspond to a 1-ghost and a 2-ghost cycle, respectively, we find that ghost cycles are characterized by significant increase especially in the 1:1 entrainment regions as demonstrated by the corresponding Arnold tongues and exhibit richer dynamics (bursting, chaos) in contrast to the classical slow-fast system. Phase plane analysis reveals that these features result from the continuous remodeling of the attractor landscape of the ghost cycles models characteristic for non-autonomous systems, whereas the attractor landscape of the corresponding slow-fast system remains qualitatively unaltered. We propose that systems containing ghost cycles display increased flexibility and responsiveness to continuous environmental changes.
OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion

We present OAKINK2, a dataset of bimanual object manipulation tasks for complex daily activities. In pursuit of constructing the complex tasks into a structured representation, OAKINK2 introduces three level of abstraction to organize the manipulation tasks: Affordance, Primitive Task, and Complex Task. OAKINK2 features on an object-centric perspective for decoding the complex tasks, treating them as a sequence of object affordance fulfillment. The first level, Affordance, outlines the functionalities that objects in the scene can afford, the second level, Primitive Task, describes the minimal interaction units that humans interact with the object to achieve its affordance, and the third level, Complex Task, illustrates how Primitive Tasks are composed and interdependent. OAKINK2 dataset provides multi-view image streams and precise pose annotations for the human body, hands and various interacting objects. This extensive collection supports applications such as interaction reconstruction and motion synthesis. Based on the 3-level abstraction of OAKINK2, we explore a task-oriented framework for Complex Task Completion (CTC). CTC aims to generate a sequence of bimanual manipulation to achieve task objectives. Within the CTC framework, we employ Large Language Models (LLMs) to decompose the complex task objectives into sequences of Primitive Tasks and have developed a Motion Fulfillment Model that generates bimanual hand motion for each Primitive Task. OAKINK2 datasets and models are available at https://oakink.net/v2.
RiEMann: Near Real-Time SE(3)-Equivariant Robot Manipulation without Point Cloud Segmentation

We present RiEMann, an end-to-end near Real-time SE(3)-Equivariant Robot Manipulation imitation learning framework from scene point cloud input. Compared to previous methods that rely on descriptor field matching, RiEMann directly predicts the target poses of objects for manipulation without any object segmentation. RiEMann learns a manipulation task from scratch with 5 to 10 demonstrations, generalizes to unseen SE(3) transformations and instances of target objects, resists visual interference of distracting objects, and follows the near real-time pose change of the target object. The scalable action space of RiEMann facilitates the addition of custom equivariant actions such as the direction of turning the faucet, which makes articulated object manipulation possible for RiEMann. In simulation and real-world 6-DOF robot manipulation experiments, we test RiEMann on 5 categories of manipulation tasks with a total of 25 variants and show that RiEMann outperforms baselines in both task success rates and SE(3) geodesic distance errors on predicted poses (reduced by 68.6%), and achieves a 5.4 frames per second (FPS) network inference speed. Code and video results are available at https://riemann-web.github. io/.
Kinectofon: Performing with Shapes in Planes

The paper presents the Kinectofon, an instrument for creating sounds through free-hand interaction in a 3D space. The instrument is based on the RGB and depth image streams retrieved from a Microsoft Kinect sensor device. These two image streams are used to create different types of motiongrams, which, again, are used as the source material for a sonification process based on inverse FFT. The instrument is intuitive to play, allowing the performer to create sound by “touching” a virtual sound wall.
Enhancing Efficiency in Sparse Models with Sparser Selection

Sparse models, including sparse Mixture-of-Experts (MoE) models, have emerged as an effective approach for scaling Transformer models. However, they often suffer from computational inefficiency since a significant number of parameters are unnecessarily involved in computations via multiplying values by zero or low activation values. To address this issue, we present \tool, a novel MoE designed to enhance both the efficacy and efficiency of sparse MoE models. \tool leverages small experts and a threshold-based router to enable tokens to selectively engage only essential parameters. Our extensive experiments on language modeling and machine translation tasks demonstrate that \tool can enhance model performance while decreasing the computation load at MoE layers by over 50\% without sacrificing performance. Furthermore, we present the versatility of \tool by applying it to dense models, enabling sparse computation during inference. We provide a comprehensive analysis and make our code available at https://anonymous.4open. science/r/XMoE.
WALT3D: Generating Realistic Training Data from Time-Lapse Imagery for Reconstructing Dynamic Objects under Occlusion

Current methods for 2D and 3D object understanding struggle with severe occlusions in busy urban environments, partly due to the lack of large-scale labeled ground-truth annotations for learning occlusion. In this work, we introduce a novel framework for automatically generating a large, realistic dataset of dynamic objects under occlusions using freely available time-lapse imagery. By leveraging off-the-shelf 2D (bounding box, segmentation, keypoint) and 3D (pose, shape) predictions as pseudo-groundtruth, unoccluded 3D objects are identified automatically and composited into the background in a clip-art style, ensuring realistic appearances and physically accurate occlusion configurations. The resulting clip-art image with pseudo-groundtruth enables efficient training of object reconstruction methods that are robust to occlusions. Our method demonstrates significant improvements in both 2D and 3D reconstruction, particularly in scenarios with heavily occluded objects like vehicles and people in urban scenes.
The impact of electronic government on democracy: e-democracy through e-participation

Electronic government is the use of Information and Communication Technology in the transformation of government; primarily aiming to the improvement of accessibility, effectiveness and responsibility. It is based on the diffusion of the information and the information policy development. Electronic government guides to increasing citizens' participation and active citizens' development affecting the mechanisms of democracy. Electronic democracy (e-democracy) includes technological innovations that allow improvement and empowerment of democratic institutions with or without the use of internet. E-democracy aims the production of functional democratic processes like communication, information providing and decision-making after electronic public dialogue and voting. E-democracy is the cornerstone in a social system as citizens can have an active participation in the public issues. This paper describes the electronic government implementations promoting electronic democracy through electronic participation (e-participation) and appearing benefits for the citizens, the enterprises, the public administration and the society.
On Bootstrapping Lasso in Generalized Linear Models and the Cross Validation

Generalized linear models or GLM constitutes an important set of models which generalizes the ordinary linear regression by connecting the response variable with the covariates through arbitrary link functions. On the other hand, Lasso is a popular and easy to implement penalization method in regression when all the covariates are not relevant. However, Lasso generally has non-tractable asymptotic distribution and hence development of an alternative method of distributional approximation is required for the purpose of statistical inference. In this paper, we develop a Bootstrap method which works as an approximation of the distribution of the Lasso estimator for all the sub-models of GLM. To connect the distributional approximation theory based on the proposed Bootstrap method with the practical implementation of Lasso, we explore the asymptotic properties of K-fold cross validation-based penalty parameter. The results established essentially justifies drawing valid statistical inference regarding the unknown parameters based on the proposed Bootstrap method for any sub model of GLM after selecting the penalty parameter using K-fold cross validation. Good finite sample properties are also shown through a moderately large simulation study. The method is also implemented on a real data set.