Miracle-dz/UniFAR-FTUs huggingface dataset:
Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts

Top-K sparse softmax gating mixture of experts has been widely used for scaling up massive deep-learning architectures without increasing the computational cost. Despite its popularity in real-world applications, the theoretical understanding of that gating function has remained an open problem. The main challenge comes from the structure of the top-K sparse softmax gating function, which partitions the input space into multiple regions with distinct behaviors. By focusing on a Gaussian mixture of experts, we establish theoretical results on the effects of the top-K sparse softmax gating function on both density and parameter estimations. Our results hinge upon defining novel loss functions among parameters to capture different behaviors of the input regions. When the true number of experts $k_{\ast}$ is known, we demonstrate that the convergence rates of density and parameter estimations are both parametric on the sample size. However, when $k_{\ast}$ becomes unknown and the true model is over-specified by a Gaussian mixture of $k$ experts where $k>k_{\ast}$, our findings suggest that the number of experts selected from the top-K sparse softmax gating function must exceed the total cardinality of a certain number of Voronoi cells associated with the true parameters to guarantee the convergence of the density estimation. Moreover, while the density estimation rate remains parametric under this setting, the parameter estimation rates become substantially slow due to an intrinsic interaction between the softmax gating and expert functions.
Beam Squint Assisted User Localization in Near-Field Integrated Sensing and Communications Systems

Integrated sensing and communication (ISAC) has been regarded as a key technology for 6G wireless communications, in which large-scale multiple input and multiple output (MIMO) array with higher and wider frequency bands will be adopted. However, recent studies show that the beam squint phenomenon can not be ignored in wideband MIMO system, which generally deteriorates the communications performance. In this paper, we find that with the aid of true-time-delay lines (TTDs), the range and trajectory of the beam squint in near-field communications systems can be freely controlled, and hence it is possible to reversely utilize the beam squint for user localization. We derive the trajectory equation for near-field beam squint points and design a way to control such trajectory. With the proposed design, beamforming from different subcarriers would purposely point to different angles and different distances, such that users from different positions would receive the maximum power at different subcarriers. Hence, one can simply localize multiple users from the beam squint effect in frequency domain, and thus reduce the beam sweeping overhead as compared to the conventional time domain beam search based approach. Furthermore, we utilize the phase difference of the maximum power subcarriers received by the user at different frequencies in several times beam sweeping to obtain a more accurate distance estimation result, ultimately realizing high accuracy and low beam sweeping overhead user localization. Simulation results demonstrate the effectiveness of the proposed schemes.
FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning

Exemplar-free class-incremental learning (CIL) poses several challenges since it prohibits the rehearsal of data from previous tasks and thus suffers from catastrophic forgetting. Recent approaches to incrementally learning the classifier by freezing the feature extractor after the first task have gained much attention. In this paper, we explore prototypical networks for CIL, which generate new class prototypes using the frozen feature extractor and classify the features based on the Euclidean distance to the prototypes. In an analysis of the feature distributions of classes, we show that classification based on Euclidean metrics is successful for jointly trained features. However, when learning from non-stationary data, we observe that the Euclidean metric is suboptimal and that feature distributions are heterogeneous. To address this challenge, we revisit the anisotropic Mahalanobis distance for CIL. In addition, we empirically show that modeling the feature covariance relations is better than previous attempts at sampling features from normal distributions and training a linear classifier. Unlike existing methods, our approach generalizes to both many- and few-shot CIL settings, as well as to domain-incremental settings. Interestingly, without updating the backbone network, our method obtains state-of-the-art results on several standard continual learning benchmarks. Code is available at https://github.com/dipamgoswami/FeCAM.
Affective Game Computing: A Survey

This article surveys the current state-of-the-art in affective computing (AC) principles, methods, and tools as applied to games. We review this emerging field, namely affective game computing, through the lens of the four core phases of the affective loop: game affect elicitation, game affect sensing, game affect detection, and game affect adaptation. In addition, we provide a taxonomy of terms, methods, and approaches used across the four phases of the affective game loop and situate the field within this taxonomy. We continue with a comprehensive review of available affect data collection methods with regard to gaming interfaces, sensors, annotation protocols, and available corpora. This article concludes with a discussion on the current limitations of affective game computing and our vision for the most promising future research directions in the field.
Cloudy Forecast: How Predictable is Communication Latency in the Cloud?

Many systems and services rely on timing assumptions for performance and availability to perform critical aspects of their operation, such as various timeouts for failure detectors or optimizations to concurrency control mechanisms. Many such assumptions rely on the ability of different components to communicate on time -- a delay in communication may trigger the failure detector or cause the system to enter a less-optimized execution mode. Unfortunately, these timing assumptions are often set with little regard to actual communication guarantees of the underlying infrastructure -- in particular, the variability of communication delays between processes in different nodes/servers. The higher communication variability holds especially true for systems deployed in the public cloud since the cloud is a utility shared by many users and organizations, making it prone to higher performance variance due to noisy neighbor syndrome. In this work, we present Cloud Latency Tester (CLT), a simple tool that can help measure the variability of communication delays between nodes to help engineers set proper values for their timing assumptions. We also provide our observational analysis of running CLT in three major cloud providers and share the lessons we learned.
Decreasing the human coding burden in randomized trials with text-based outcomes via model-assisted impact analysis

For randomized trials that use text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by trained human raters. This process, the current standard, is both time-consuming and limiting: even the largest human coding efforts are typically constrained to measure only a small set of dimensions across a subsample of available texts. In this work, we present an inferential framework that can be used to increase the power of an impact assessment, given a fixed human-coding budget, by taking advantage of any ``untapped"observations -- those documents not manually scored due to time or resource constraints -- as a supplementary resource. Our approach, a methodological combination of causal inference, survey sampling methods, and machine learning, has four steps: (1) select and code a sample of documents; (2) build a machine learning model to predict the human-coded outcomes from a set of automatically extracted text features; (3) generate machine-predicted scores for all documents and use these scores to estimate treatment impacts; and (4) adjust the final impact estimates using the residual differences between human-coded and machine-predicted outcomes. As an extension to this approach, we also develop a strategy for identifying an optimal subset of documents to code in Step 1 in order to further enhance precision. Through an extensive simulation study based on data from a recent field trial in education, we show that our proposed approach can be used to reduce the scope of a human-coding effort while maintaining nominal power to detect a significant treatment impact.
Towards Robust Robot 3D Perception in Urban Environments: The UT Campus Object Dataset

We introduce the UT Campus Object Dataset (CODa), a mobile robot egocentric perception dataset collected on the University of Texas Austin Campus. Our dataset contains 8.5 hours of multimodal sensor data: synchronized 3D point clouds and stereo RGB video from a 128-channel 3D LiDAR and two 1.25MP RGB cameras at 10 fps; RGB-D videos from an additional 0.5MP sensor at 7 fps, and a 9-DOF IMU sensor at 40 Hz. We provide 58 minutes of ground-truth annotations containing 1.3 million 3D bounding boxes with instance IDs for 53 semantic classes, 5000 frames of 3D semantic annotations for urban terrain, and pseudo-ground truth localization. We repeatedly traverse identical geographic locations for a wide range of indoor and outdoor areas, weather conditions, and times of the day. Using CODa, we empirically demonstrate that: 1) 3D object detection performance in urban settings is significantly higher when trained using CODa compared to existing datasets even when employing state-of-the-art domain adaptation approaches, 2) sensor-specific fine-tuning improves 3D object detection accuracy and 3) pretraining on CODa improves cross-dataset 3D object detection performance in urban settings compared to pretraining on AV datasets. Using our dataset and annotations, we release benchmarks for 3D object detection and 3D semantic segmentation using established metrics. In the future, the CODa benchmark will include additional tasks like unsupervised object discovery and re-identification. We publicly release CODa on the Texas Data Repository, pre-trained models, dataset development package, and interactive dataset viewer on our website at https://amrl.cs. utexas.edu/coda. We expect CODa to be a valuable dataset for research in egocentric 3D perception and planning for autonomous navigation in urban environments.
OCCURRENCE AND SIGNIFICANCE OF SERPENTINITE-HOSTED, TALC- AND AMPHIBOLE-RICH FAULT ROCKS IN MODERN OCEANIC SETTINGS AND OPHIOLITE COMPLEXES: AN OVERVIEW

This paper reviews the occurrence and significance of talc- and amphibole-rich fault rocks developed in mafic-ultramafic sequences and evaluates their role in deformation and alteration of the oceanic lithosphere from different tectonic settings (from spreading mid-ocean ridges up to orogenic belts). Recently, talc and amphibole-rich fault rocks have been sampled and studied from detachment fault surfaces along slow and ultra-slow spreading mid-ocean ridges, and constraining the conditions of deformation and strain localization during the evolution of oceanic core complexes. These rocks are documented not only in oceanic core complexes, but also in other oceanic fracture zones where ultramafic rocks are exposed on the seafloor, while only few occurrences have been reported in ophiolite sequences. Samples recovered in situ in oceanic settings record heterogeneous deformation (crystal-plastic to cataclastic) under greenschist- facies conditions and are commonly restricted to localized shear zones (< 200 m) and are associated with intense talc-amphibole metasomatism. The presence of mechanically weak minerals, such as talc, serpentine and chlorite, may be critical to the development of such fault zones and may enhance unroofing of upper mantle peridotites and lower crustal gabbroic rocks during seafloor spreading. Talc in particular may be influential in lubricating and softening mylonitic shear zones and can lead to strain localization and focused hydrothermal circulation along such faults. The rheology of these rocks, and its evolution during dehydration reactions could play an important role also in subduction-zone processes and during the formation of ultramafic orogenic belts. Here, we review the occurrence and significance of talc- and amphibole-rich fault rocks in different tectonic settings on the seafloor and evaluate their role in deformation and alteration of the oceanic lithosphere.
Handbook of atmospheric electrodynamics

Ion Chemistry and Composition of the Atmosphere, A.A. Viggiano and F. Arnold Meteorologic Aspects of Thunderstorms, E.R. Williams Thunderstorm Electrification, C.P.R. Saunders Lightning Currents, T. Ogawa Lightning Detection from Ground and Space, R.E. Orville Artificially Triggered Lightning, K. Horii and M. Nakano Ball Lightning, H. Kikuchi Lightning and Atmospheric Chemistry: The Rate of Atmospheric NO Production, M.G. Lawrence, W.L. Chameides, P.S. Kasibhatla, H.Levy II, and W. Moxim Lightning Within Planetary Atmospheres, K. Rinnert Quasistatic Electromagnetic Phenomena in the Atmosphere and Ionosphere, R.H. Holzworth Schumann Resonances, D.D. Sentman Low-Frequency Radio Noise, A.C. Frazer-Smith Radio Noise Above 300 kHz due to Natural Causes, D.E. Proctor Atmospheric Noise and Its Effects of Telecommunication System Performance, A.D. Spaulding
Internet Protocol Geolocation: Development of a Delay-Based Hybrid Methodology for Locating the Geographic Location of a Network Node

Abstract : Internet Protocol Geolocation (IP Geolocation), the process of determining the approximate geographic location of an IP addressable node, has proven useful in a wide variety of commercial applications. Commercial applications of IP Geolocation include market research, redirection for performance enhancement, restricting content, and combating fraud. The potential for military applications include securing remote access via geographic authentication, intelligence collection, and cyber attack attribution. IP Geolocation methods can be divided into three basic categories based upon which information is used to determine the geographic location of the given IP address: 1) Information contained in databases, 2) information that is leaked during connections with the IP of interest, and 3) network-based routing and timing information. This thesis focused upon an analysis in the third category: delay-based methods for IP Geolocation. Specifically, a comparative analysis of three existing delay-based IP Geolocation methods: Upper-bound Multilateration (UBM), Constraint Based Geolocation (CBG), and Time to Location Heuristic (TTLH) is conducted using a simulated network. Based upon analysis of the results, a new hybrid methodology is proposed to improve the accuracy when conducting IP Geolocation. Simulations of the new hybrid methodology show that the hybrid methodology is superior to all existing delay-based methods for IP Geolocation.
Full Duplex Millimeter Wave Radio over Fiber with 24 Gbps 4-QAM Downstream Signal in W Band (75-110 GHz)

We propose a millimeter-wave radio over fiber transmission system in the W-band (75–110 GHz) with quadrature amplitude modulation and wavelength reuse ability. Optical millimeter-wave carriers are generated at the central station by frequency quadrupling a 16.75GHz local oscillator using two LiNb Mach-Zehnder modulators. 24 Gbps of downlink data is single side band modulated with one of the tones generated by LiNb Mach-Zehnder modulators and sent over a 30 km long single mode fiber. At the base station, beating of tones occurs at a high speed photodetector and generates an 87GHz millimeter wave containing the downlink data. Another local oscillator millimeter wave at 67GHz is generated with the use of fiber Bragg reflectors. This millimeter wave is used to down convert the uplink data received wirelessly from the user station to 4 GHz. The down converted uplink data is double side band modulated with the second tone which was received from the base station and partially reflected by fiber Bragg reflector and sent to the central station, thus keeping the access units simple. The simulation analysis for 4 and 16 quadrature amplitude modulation shows that the full duplex model has good performance with bit error rate below forward error correction limits even after 30 km standard single mode fiber transmission at 24 Gbps without any inline dispersion compensation.
Weakly Supervised Reasoning by Neuro-Symbolic Approaches

Deep learning has largely improved the performance of various natural language processing (NLP) tasks. However, most deep learning models are black-box machinery, and lack explicit interpretation. In this chapter, we will introduce our recent progress on neuro-symbolic approaches to NLP, which combines different schools of AI, namely, symbolism and connectionism. Generally, we will design a neural system with symbolic latent structures for an NLP task, and apply reinforcement learning or its relaxation to perform weakly supervised reasoning in the downstream task. Our framework has been successfully applied to various tasks, including table query reasoning, syntactic structure reasoning, information extraction reasoning, and rule reasoning. For each application, we will introduce the background, our approach, and experimental results.
On the Posterior Distribution in Denoising: Application to Uncertainty Quantification

Denoisers play a central role in many applications, from noise suppression in low-grade imaging sensors, to empowering score-based generative models. The latter category of methods makes use of Tweedie's formula, which links the posterior mean in Gaussian denoising (\ie the minimum MSE denoiser) with the score of the data distribution. Here, we derive a fundamental relation between the higher-order central moments of the posterior distribution, and the higher-order derivatives of the posterior mean. We harness this result for uncertainty quantification of pre-trained denoisers. Particularly, we show how to efficiently compute the principal components of the posterior distribution for any desired region of an image, as well as to approximate the full marginal distribution along those (or any other) one-dimensional directions. Our method is fast and memory-efficient, as it does not explicitly compute or store the high-order moment tensors and it requires no training or fine tuning of the denoiser. Code and examples are available on the project webpage in https://hilamanor.github. io/GaussianDenoisingPosterior/ .
From Text to Source: Results in Detecting Large Language Model-Generated Content

The widespread use of Large Language Models (LLMs), celebrated for their ability to generate human-like text, has raised concerns about misinformation and ethical implications. Addressing these concerns necessitates the development of robust methods to detect and attribute text generated by LLMs. This paper investigates"Cross-Model Detection,"by evaluating whether a classifier trained to distinguish between source LLM-generated and human-written text can also detect text from a target LLM without further training. The study comprehensively explores various LLM sizes and families, and assesses the impact of conversational fine-tuning techniques, quantization, and watermarking on classifier generalization. The research also explores Model Attribution, encompassing source model identification, model family, and model size classification, in addition to quantization and watermarking detection. Our results reveal several key findings: a clear inverse relationship between classifier effectiveness and model size, with larger LLMs being more challenging to detect, especially when the classifier is trained on data from smaller models. Training on data from similarly sized LLMs can improve detection performance from larger models but may lead to decreased performance when dealing with smaller models. Additionally, model attribution experiments show promising results in identifying source models and model families, highlighting detectable signatures in LLM-generated text, with particularly remarkable outcomes in watermarking detection, while no detectable signatures of quantization were observed. Overall, our study contributes valuable insights into the interplay of model size, family, and training data in LLM detection and attribution.
In-Domain GAN Inversion for Faithful Reconstruction and Editability

Generative Adversarial Networks (GANs) have significantly advanced image synthesis through mapping randomly sampled latent codes to high-fidelity synthesized images. However, applying well-trained GANs to real image editing remains challenging. A common solution is to find an approximate latent code that can adequately recover the input image to edit, which is also known as GAN inversion. To invert a GAN model, prior works typically focus on reconstructing the target image at the pixel level, yet few studies are conducted on whether the inverted result can well support manipulation at the semantic level. This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model. In this way, we manage to sufficiently reuse the knowledge learned by GANs for image reconstruction, facilitating a wide range of editing applications without any retraining. We further make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property. Such a trade-off sheds light on how a GAN model represents an image with various semantics encoded in the learned latent distribution. Code, models, and demo are available at the project page https://genforce.github. io/idinvert/.
A Contextual Topic Modeling and Content Analysis of Iranian laws and Regulations

A constitution is the highest legal document of a country and serves as a guide for the establishment of other laws. The constitution defines the political principles, structure, hierarchy, position, and limits of the political power of a country's government. It determines and guarantees the rights of citizens. This study aimed at topic modeling of Iranian laws. As part of this research, 11760 laws were collected from the Dotic website. Then, topic modeling was conducted on the title and content of the regularizations using LDA. Data analysis with topic modeling led to the identification of 10 topics including Economic, Customs, Housing and Urban Development, Agriculture, Insurance, Legal and judicial, Cultural, Information Technology, Political, and Government. The largest topic, Economic, accounts for 29% of regulations, while the smallest are Political and Government, accounting for 2%. This research utilizes a topic modeling method in exploring law texts and identifying trends in regularizations from 2016-2023. In this study, it was found that regularizations constitute a significant percentage of law, most of which are related to economics and customs. Cultural regularizations have increased in 2023. It can be concluded any law enacted each year can reflect society's conditions and legislators' top concerns.