Miracle-dz/UniFAR-FTUs huggingface dataset:
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization

The sheer size of modern neural networks makes model serving a serious computational challenge. A popular class of compression techniques overcomes this challenge by pruning or sparsifying the weights of pretrained networks. While useful, these techniques often face serious tradeoffs between computational requirements and compression quality. In this work, we propose a novel optimization-based pruning framework that considers the combined effect of pruning (and updating) multiple weights subject to a sparsity constraint. Our approach, CHITA, extends the classical Optimal Brain Surgeon framework and results in significant improvements in speed, memory, and performance over existing optimization-based approaches for network pruning. CHITA's main workhorse performs combinatorial optimization updates on a memory-friendly representation of local quadratic approximation(s) of the loss function. On a standard benchmark of pretrained models and datasets, CHITA leads to significantly better sparsity-accuracy tradeoffs than competing methods. For example, for MLPNet with only 2% of the weights retained, our approach improves the accuracy by 63% relative to the state of the art. Furthermore, when used in conjunction with fine-tuning SGD steps, our method achieves significant accuracy gains over the state-of-the-art approaches.
Efficient Masked Autoencoders with Self-Consistency

Inspired by masked language modeling (MLM) in natural language processing, masked image modeling (MIM) has been recognized as a strong and popular self-supervised pre-training method in computer vision. However, its high random mask ratio would result in two serious problems: 1) the data are not efficiently exploited, which brings inefficient pre-training (\eg, 1600 epochs for MAE $vs.$ 300 epochs for the supervised), and 2) the high uncertainty and inconsistency of the pre-trained model, \ie, the prediction of the same patch may be inconsistent under different mask rounds. To tackle these problems, we propose efficient masked autoencoders with self-consistency (EMAE), to improve the pre-training efficiency and increase the consistency of MIM. In particular, we progressively divide the image into K non-overlapping parts, each of which is generated by a random mask and has the same mask ratio. Then the MIM task is conducted parallelly on all parts in an iteration and generates predictions. Besides, we design a self-consistency module to further maintain the consistency of predictions of overlapping masked patches among parts. Overall, the proposed method is able to exploit the data more efficiently and obtains reliable representations. Experiments on ImageNet show that EMAE achieves even higher results with only 300 pre-training epochs under ViT-Base than MAE (1600 epochs). EMAE also consistently obtains state-of-the-art transfer performance on various downstream tasks, like object detection, and semantic segmentation.
Sparkling Silence: Practices and Challenges of Livestreaming Among Deaf or Hard of Hearing Streamers

Understanding livestream platforms’ accessibility challenges for minority groups, such as people with disabilities, is critical to increasing the diversity and inclusion of those platforms. While prior work investigated the experiences of streamers with vision or motor loss, little is known about the experiences of deaf or hard of hearing (DHH) streamers who must work with livestreaming platforms that heavily depend on audio. We conducted semi-structured interviews with DHH streamers to learn why they livestream, how they navigate livestream platforms and related challenges. Our findings revealed their desire to break the stereotypes towards the DHH groups via livestream and the intense interplay between interaction methods, such as sign language, texts, lip language, background music, and viewer characteristics. Major accessibility challenges include the lack of real-time captioning, the small sign language reading window, and misinterpretation of sign language. We present design considerations for improving the accessibility of the livestream platforms.
The remarkable role of the vector-like quark doublet in the Cabibbo angle and $W$-mass anomalies

The combination of precise determinations of $V_{us}$ and $V_{ud}$ hints towards a violation of the CKM first row unitarity at about $3\sigma$ level. Conversely, the recent measurement of the $W$-boson mass by the CDF Collaboration exhibits significant tension with the SM prediction, intensifying the conflict that may arise in models addressing the unitarity violation. We demonstrate that one vector-like $SU(2)_L$ doublet with mass of a few TeV mixing with light quarks and the top quark can simultaneously account for the two anomalies, without conflicting with flavour-changing phenomena and electroweak observables. Moreover, another tension in the value of the Cabibbo angle is reported also at $3\sigma$ level, between two determinations of $V_{us}$ obtained from semi-leptonic $K\ell3$ and leptonic $K\mu 2$ kaon decays. We show that the vector-like doublet can be at the origin of this discrepancy and the substantial positive shift in the $W$-boson mass. This unique feature of the vector-like quark doublet may render it a crucial puzzle piece in new physics scenarios addressing the CKM unitarity problem. The model can be potentially probed in future colliders.
Reconstruction-based Out-of-Distribution Detection for Short-Range FMCW Radar

Out-of-distribution (OOD) detection recently has drawn attention due to its critical role in the safe deployment of modern neural network architectures in real-world applications. The OOD detectors aim to distinguish samples that lie outside the training distribution in order to avoid the overconfident predictions of machine learning models on OOD data. Existing detectors, which mainly rely on the logit, intermediate feature space, softmax score, or reconstruction loss, manage to produce promising results. However, most of these methods are developed for the image domain. In this study, we propose a novel reconstruction-based OOD detector to operate on the radar domain. Our method exploits an autoencoder (AE) and its latent representation to detect the OOD samples. We propose two scores incorporating the patch-based reconstruction loss and the energy value calculated from the latent representations of each patch. We achieve an AUROC of 90.72% on our dataset collected by using 60 GHz short-range FMCW Radar. The experiments demonstrate that, in terms of AUROC and AUPR, our method outperforms the baseline (AE) and the other state-of-the-art methods. Also, thanks to its model size of 641 kB, our detector is suitable for embedded usage.
Differentially Private Distributed Convex Optimization

This paper considers distributed optimization (DO) where multiple agents cooperate to minimize a global objective function, expressed as a sum of local objectives, subject to some constraints. In DO, each agent iteratively solves a local optimization model constructed by its own data and communicates some information (e.g., a local solution) with its neighbors until a global solution is obtained. Even though locally stored data are not shared with other agents, it is still possible to reconstruct the data from the information communicated among agents, which could limit the practical usage of DO in applications with sensitive data. To address this issue, we propose a privacy-preserving DO algorithm for constrained convex optimization models, which provides a statistical guarantee of data privacy, known as differential privacy, and a sequence of iterates that converges to an optimal solution in expectation. The proposed algorithm generalizes a linearized alternating direction method of multipliers by introducing a multiple local updates technique to reduce communication costs and incorporating an objective perturbation method in the local optimization models to compute and communicate randomized feasible local solutions that cannot be utilized to reconstruct the local data, thus preserving data privacy. Under the existence of convex constraints, we show that, while both algorithms provide the same level of data privacy, the objective perturbation used in the proposed algorithm can provide better solutions than does the widely adopted output perturbation method that randomizes the local solutions by adding some noise. We present the details of privacy and convergence analyses and numerically demonstrate the effectiveness of the proposed algorithm by applying it in two different applications, namely, distributed control of power flow and federated learning, where data privacy is of concern.
Investigation of Cryogenic Current-Voltage Anomalies in SiGe HBTs: Role of Base-Emitter Junction Inhomogeneities

The deviations of cryogenic collector current-voltage characteristics of SiGe heterojunction bipolar transistors (HBTs) from ideal drift-diffusion theory have been a topic of investigation for many years. Recent work indicates that direct tunneling across the base contributes to the non-ideal current in highly-scaled devices. However, cryogenic discrepancies have been observed even in older-generation devices for which direct tunneling is negligible, suggesting another mechanism may also contribute. Although similar non-ideal current-voltage characteristics have been observed in Schottky junctions and were attributed to spatial inhomogeneities in the base-emitter junction potential, this explanation has not been considered for SiGe HBTs. Here, we experimentally investigate this hypothesis by characterizing the collector current ideality factor and built-in potential of a SiGe HBT versus temperature using a cryogenic probe station. The temperature-dependence of the ideality factor and the relation between the built-in potential as measured by capacitance-voltage and current-voltage characteristics are in good qualitative agreement with the predictions of a theory of electrical transport across a junction with a Gaussian distribution of potential barrier heights. These observations suggest that lateral inhomogeneities in the base-emitter junction potential may contribute to the cryogenic non-idealities. This work helps to identify the physical mechanisms limiting the cryogenic microwave noise performance of SiGe HBTs.
Strongly coupled simulation of incompressible fluid and rigid bodies with velocity-based constraints using particle method

This paper presents a novel particle method to compute strongly coupled incompressible fluid and rigid bodies. The method adopts a velocity-based formulation and utilizes the linear complementarity problem for the incompressibility constraint. Since all the constraints for incompressibility, inter-rigid-body contacts, and interaction between incompressible fluid and rigid bodies are mathematically compatible, strongly coupled simulation is achieved using the method, where the shapes of the rigid bodies are represented by particles as well. The abstract concept of velocity-based constraints is presented, which generalizes the formulations of the incompressibility constraint and inter-rigid-body contacts and provides a generic way to achieve strongly coupled simulation. Several numerical examples are presented to verify the method, which includes rigid-body computation, hydrostatic pressure, dam-break computation, and circular parch computation for incompressible fluid, buoyancy and seesaw computation for interaction of incompressible fluid and rigid bodies, and complex-scene computation for overall behavior and stability.
Improving Model Generalization by On-manifold Adversarial Augmentation in the Frequency Domain

Deep neural networks (DNNs) may suffer from significantly degenerated performance when the training and test data are of different underlying distributions. Despite the importance of model generalization to out-of-distribution (OOD) data, the accuracy of state-of-the-art (SOTA) models on OOD data can plummet. Recent work has demonstrated that regular or off-manifold adversarial examples, as a special case of data augmentation, can be used to improve OOD generalization. Inspired by this, we theoretically prove that on-manifold adversarial examples can better benefit OOD generalization. Nevertheless, it is nontrivial to generate on-manifold adversarial examples because the real manifold is generally complex. To address this issue, we proposed a novel method of Augmenting data with Adversarial examples via a Wavelet module (AdvWavAug), an on-manifold adversarial data augmentation technique that is simple to implement. In particular, we project a benign image into a wavelet domain. With the assistance of the sparsity characteristic of wavelet transformation, we can modify an image on the estimated data manifold. We conduct adversarial augmentation based on AdvProp training framework. Extensive experiments on different models and different datasets, including ImageNet and its distorted versions, demonstrate that our method can improve model generalization, especially on OOD data. By integrating AdvWavAug into the training process, we have achieved SOTA results on some recent transformer-based models.
Bayesian Nagaoka-Hayashi Bound for Multiparameter Quantum-State Estimation Problem

In this work we propose a Bayesian version of the Nagaoka-Hayashi bound when estimating a parametric family of quantum states. This lower bound is a generalization of a recently proposed bound for point estimation to Bayesian estimation. We then show that the proposed lower bound can be efficiently computed as a semidefinite programming problem. As a lower bound, we also derive a Bayesian version of the Holevo-type bound from the Bayesian Nagaoka-Hayashi bound. Lastly, we prove that the new lower bound is tighter than the Bayesian quantum Cramer-Rao bounds.
Relation between the transport coefficients and the internal entropy of simple systems

The transport coefficients of hard-sphere, soft-sphere, Lennard-Jones, and one-component-plasma systems are expressed in terms of the corresponding internal entropies, to reveal some universal characteristics. The transport of all the inverse $n\mathrm{th}$-power repulsive potentials appears to be related, and can be numerically estimated by the hard-sphere model, although the hard-sphere transport is different. The entropy dependence of the coefficients of self-diffusion and (to a lesser extent) shear viscosity, for systems with purely repulsive potentials, is only slightly affected by the addition of an attractive potential.
An Algorithm and Complexity Results for Causal Unit Selection

The unit selection problem aims to identify objects, called units, that are most likely to exhibit a desired mode of behavior when subjected to stimuli (e.g., customers who are about to churn but would change their mind if encouraged). Unit selection with counterfactual objective functions was introduced relatively recently with existing work focusing on bounding a specific class of objective functions, called the benefit functions, based on observational and interventional data -- assuming a fully specified model is not available to evaluate these functions. We complement this line of work by proposing the first exact algorithm for finding optimal units given a broad class of causal objective functions and a fully specified structural causal model (SCM). We show that unit selection under this class of objective functions is $\text{NP}^\text{PP}$-complete but is $\text{NP}$-complete when unit variables correspond to all exogenous variables in the SCM. We also provide treewidth-based complexity bounds on our proposed algorithm while relating it to a well-known algorithm for Maximum a Posteriori (MAP) inference.
A distance comparison principle for curve shortening flow with free boundary

We introduce a reflected chord-arc profile for curves with orthogonal boundary condition and obtain a chord-arc estimate for embedded free boundary curve shortening flows in a convex planar domain. As a consequence, we are able to prove that any such flow either converges in infinite time to a (unique) ``critical chord'', or contracts in finite time to a ``round half-point'' on the boundary.
Energy Efficiency and Throughput of Random Access Protocols for RIS-Aided IoT Networks

Green Internet of Things (IoT) aims to enable a sustainable smart world by making energy efficiency (EE) the main performance indicator for IoT hardware and software. With respect to network design, this implies in developing energy-efficient communication protocols and network architectures adapted to the ubiquitousness of the IoT machine-type devices (MTDs) and the sporadic traffic generated by them, keeping a low power consumption at the MTDs-side. In this sense, reconfigurable intelligent surfaces (RISs) have presented the capacity of significantly improving the network coverage using mostly passive reflecting elements, drastically reducing the power expenditure. In this paper, we develop a realistic power consumption model and an expression for the overall system EE for RIS-aided IoT networks that adopt a two time-scale random access (RA) protocol to handle the uplink transmissions. Specifically, during each time slot of the RA protocol, the RIS covers a specific area of interest in the communication cell with a predefined set of phase-shift configurations, changing the channel qualities of the contending MTDs. Numerical results comparing the RA protocol performance reveal that access policies that exploit information of the channel qualities are suitable for green IoT networks, simultaneously attaining competitive EE and throughput combined with low power consumption at the MTDs-side.
Effect of preferential concentration on turbulent collision rates

The effect of particle inertia on the interparticle collision rates of a turbulent aerosol was investigated recently by Sundaram and Collins (1997) using direct numerical simulation (DNS). They observed that for values of the particle Stokes number (here defined as the ratio of the particle response time to Kolmogorov time scale) near unity, the collision frequency was enhanced by between one and two orders of magnitude. This enhancement was attributed in part to the local enrichment of the particle concentration in low-vorticity regions of the flow due to the centrifuge effect commonly referred to as preferential concentration (Eaton and Fessler 1994). Sundaram and Collins (1997) showed that the correction factor for the collision kernel in a preferentially concentrated system is g(σ), where g(r) is the particle radial distribution function and σ is the collision diameter. This paper uses DNS, in combination with statistical analysis, to study the dependence of the radial distribution function on the tur.
Overview and Performance Analysis of Various Waveforms in High Mobility Scenarios

In the high-mobility scenarios of next-generation wireless communication systems (beyond 5G/6G), the performance of orthogonal frequency division multiplexing (OFDM) deteriorates drastically due to the loss of orthogonality between the subcarriers caused by large Doppler frequency shifts. Various emerging waveforms have been proposed for fast time-varying channels with excellent results. In this paper, we classify these waveforms from the perspective of their modulation domain and establish a unified framework to provide a comprehensive system structure comparison. Then we analyze bit error rate (BER) performance of each waveform in doubly selective channels. Through the discussions on their complexity and compatibility with OFDM systems, we finally give the candidate waveform suggestions.
Incremental Learning of Acoustic Scenes and Sound Events

In this paper, we propose a method for incremental learning of two distinct tasks over time: acoustic scene classification (ASC) and audio tagging (AT). We use a simple convolutional neural network (CNN) model as an incremental learner to solve the tasks. Generally, incremental learning methods catastrophically forget the previous task when sequentially trained on a new task. To alleviate this problem, we propose independent learning and knowledge distillation (KD) between the timesteps in learning. Experiments are performed on TUT 2016/2017 dataset, containing 4 acoustic scene classes and 25 sound event classes. The proposed incremental learner first solves the ASC task with an accuracy of 94.0%. Next, it learns to solve the AT task with an F1 score of 54.4%. At the same time, its performance on the previous ASC task decreases only by 5.1 percentage points due to the additional learning of the AT task.
Hierarchical Reinforcement Learning in Complex 3D Environments

Hierarchical Reinforcement Learning (HRL) agents have the potential to demonstrate appealing capabilities such as planning and exploration with abstraction, transfer, and skill reuse. Recent successes with HRL across different domains provide evidence that practical, effective HRL agents are possible, even if existing agents do not yet fully realize the potential of HRL. Despite these successes, visually complex partially observable 3D environments remained a challenge for HRL agents. We address this issue with Hierarchical Hybrid Offline-Online (H2O2), a hierarchical deep reinforcement learning agent that discovers and learns to use options from scratch using its own experience. We show that H2O2 is competitive with a strong non-hierarchical Muesli baseline in the DeepMind Hard Eight tasks and we shed new light on the problem of learning hierarchical agents in complex environments. Our empirical study of H2O2 reveals previously unnoticed practical challenges and brings new perspective to the current understanding of hierarchical agents in complex domains.
High Sensitive Α-Fe2o3 Nano-Structured Gas Sensor Fabricated Through Annealing Technique for Detecting Ethanol

On the way to advance the sensing technology, various strategies based on the nano-materials have been introduced to improve the performance of the gas sensors. In this study, we have introduced a facile fabrication procedure for annealing hematite to monitor ethanol gas. Due to large specific area of this conductive platform, more available target molecules (ethanol gas) are detected. To construct this platform, initially, a large scale well-separated iron oxide (hematite) nanostructures are created via an annealing process. The morphology of the nanostructure is optimized through annealing temperature and operating temperature for ethanol gas. The best response for the gas sensor is achieved for hematite nanostructures fabricated through the annealing temperature of 500{\deg}C and the operating temperature of 225$^\circ$C. The fabricated nanostructures is tested in air ambient conditions for various ethanol gas concentrations from 50 to 1000 ppm. The modified sensor exhibited acceptable reproducibility and good selectivity with no interference-effect. To examine the X-ray diffraction patterns of hematite nanostructures at three annealed temperatures, the Rietveld method and FullProf software are used in which the average crystal size has been obtained. Obviously, with increasing the annealing temperature, the average size of the crystals has been increased. The results of ultraviolet-visible spectroscopy show that the energy gap decreased with increasing annealing temperature. The maximum sensing response of hematite flower-like nanostructures is obtained at a concentration of 1000 ppm. Also, the shortest response time is estimated at about 27 seconds for ethanol at a concentration of 200 ppm. The detection limit of this sensor is obtained at 50 ppm.