Miracle-dz/UniFAR-FTUs huggingface dataset:
Classification Diffusion Models

A prominent family of methods for learning data distributions relies on density ratio estimation (DRE), where a model is trained to $\textit{classify}$ between data samples and samples from some reference distribution. These techniques are successful in simple low-dimensional settings but fail to achieve good results on complex high-dimensional data, like images. A different family of methods for learning distributions is that of denoising diffusion models (DDMs), in which a model is trained to $\textit{denoise}$ data samples. These approaches achieve state-of-the-art results in image, video, and audio generation. In this work, we present $\textit{Classification Diffusion Models}$ (CDMs), a generative technique that adopts the denoising-based formalism of DDMs while making use of a classifier that predicts the amount of noise added to a clean signal, similarly to DRE methods. Our approach is based on the observation that an MSE-optimal denoiser for white Gaussian noise can be expressed in terms of the gradient of a cross-entropy-optimal classifier for predicting the noise level. As we illustrate, CDM achieves better denoising results compared to DDM, and leads to at least comparable FID in image generation. CDM is also capable of highly efficient one-step exact likelihood estimation, achieving state-of-the-art results among methods that use a single step. Code is available on the project's webpage in https://shaharYadin.github. io/CDM/ .
Node Duplication Improves Cold-start Link Prediction

Graph Neural Networks (GNNs) are prominent in graph machine learning and have shown state-of-the-art performance in Link Prediction (LP) tasks. Nonetheless, recent studies show that GNNs struggle to produce good results on low-degree nodes despite their overall strong performance. In practical applications of LP, like recommendation systems, improving performance on low-degree nodes is critical, as it amounts to tackling the cold-start problem of improving the experiences of users with few observed interactions. In this paper, we investigate improving GNNs' LP performance on low-degree nodes while preserving their performance on high-degree nodes and propose a simple yet surprisingly effective augmentation technique called NodeDup. Specifically, NodeDup duplicates low-degree nodes and creates links between nodes and their own duplicates before following the standard supervised LP training scheme. By leveraging a ''multi-view'' perspective for low-degree nodes, NodeDup shows significant LP performance improvements on low-degree nodes without compromising any performance on high-degree nodes. Additionally, as a plug-and-play augmentation module, NodeDup can be easily applied to existing GNNs with very light computational cost. Extensive experiments show that NodeDup achieves 38.49%, 13.34%, and 6.76% improvements on isolated, low-degree, and warm nodes, respectively, on average across all datasets compared to GNNs and state-of-the-art cold-start methods.
Feasibility of borehole ambient noise interferometry for permanent reservoir monitoring

The analysis of seismic ambient noise acquired during temporary or permanent microseismic monitoring campaigns (e.g., improved/enhanced oil recovery monitoring, surveillance of induced seismicity) is potentially well suited for time‐lapse studies based on seismic interferometry. No additional data acquisition required, ambient noise processing can be automatized to a high degree, and seismic interferometry is very sensitive to small medium changes. Thus there is an opportunity for detection and monitoring of velocity variations in a reservoir at negligible additional cost and effort.
3C 273 host galaxy with Hubble Space Telescope coronagraphy

The close-in regions of bright quasars' host galaxies have been difficult to image due to the overwhelming light coming from quasars. With coronagraphic observations in visible light using the Space Telescope Imaging Spectrograph (STIS) on the Hubble Space Telescope we removed 3C 273 quasar light using color-matching reference stars. The observations revealed the host galaxy from $60 to $0 with nearly full angular coverage. Isophote modeling has revealed a new core jet, a core blob, and multiple smaller-scale blobs within $2 The blobs could potentially be satellite galaxies or infalling materials towards the central quasar. Using archival STIS data, we constrained the apparent motion of its large scale jets over a $22$ yr timeline. By resolving the 3C 273 host galaxy with STIS, our study validates the use of coronagraphs on extragalactic sources for obtaining new insights into the central (at sim $kpc scales) regions of quasar hosts.
Rank two Artin-Schelter regular algebras and non commuting derivations

If $\Delta$ and $\Gamma$ are two derivations of a commutative algebra A such that $\Delta$$\Gamma$-$\Gamma$$\Delta$ = $\Delta$ is locally nilpotent, one can endow A with a new product * whose filtered semiclassical limit is the Poisson structure $\Delta$ $\land$ $\Gamma$. In this article we first study theses (Poisson) algebras from an algebraic point of view, and when A is a polynomial algebra, we investigate their homological properties. In particular, if the derivations $\Delta$ and $\Gamma$ are linear, we prove that the algebras (A, *) are Artin-Schelter regular algebras. Assuming furthermore a technical condition on $\Gamma$, we show that the algebra (A, *) is Calabi-Yau if and only if the trace of $\Gamma$ is equal to 1 if and only if the Poisson algebra (A, $\Delta$ $\land$ $\Gamma$) is unimodular.
SAWEC: Sensing-Assisted Wireless Edge Computing

Emerging mobile virtual reality (VR) systems will require to continuously perform complex computer vision tasks on ultra-high-resolution video frames through the execution of deep neural networks (DNNs)-based algorithms. Since state-of-the-art DNNs require computational power that is excessive for mobile devices, techniques based on wireless edge computing (WEC) have been recently proposed. However, existing WEC methods require the transmission and processing of a high amount of video data which may ultimately saturate the wireless link. In this paper, we propose a novel Sensing-Assisted Wireless Edge Computing (SAWEC) paradigm to address this issue. SAWEC leverages knowledge about the physical environment to reduce the end-to-end latency and overall computational burden by transmitting to the edge server only the relevant data for the delivery of the service. Our intuition is that the transmission of the portion of the video frames where there are no changes with respect to previous frames can be avoided. Specifically, we leverage wireless sensing techniques to estimate the location of objects in the environment and obtain insights about the environment dynamics. Hence, only the part of the frames where any environmental change is detected is transmitted and processed. We evaluated SAWEC by using a 10K 360$^{\circ}$ camera with a Wi-Fi 6 sensing system operating at 160 MHz and performing localization and tracking. We perform experiments in an anechoic chamber and a hall room with two human subjects in six different setups. Experimental results show that SAWEC reduces the channel occupation, and end-to-end latency by 93.81%, and 96.19% respectively while improving the instance segmentation performance by 46.98% with respect to state-of-the-art WEC approaches. For reproducibility purposes, we pledge to share our whole dataset and code repository.
Indoor overheating and mitigation of converted lofts in London, UK

In the UK, there has been an increase in the number of loft conversions, driven by demands for increased floor areas of dwellings to accommodate more individuals or increase property values. While rooms directly underneath roofs are known to have increased overheating risks, there is little research available that quantifies this risk, and how to mitigate it cost-effectively. This paper seeks to evaluate overheating risks in loft conversions, using integrated environmental solutions virtual environment to dynamically simulate indoor temperatures in a semi-detached dwelling in London, UK, under current and future (2050s and 2080s medium and high emissions) climate scenarios. Adaptive overheating risk and energy consumption are calculated with and without passive overheating adaptations that reduce solar gains, increase ventilation, or add thermal insulation. Marginal abatement cost curves (MACC) are then used to select the most cost-effective adaptations based on installation and ongoing energy consumption costs. Results estimate 11,340–12,210 more summertime Category I overheating degree-hours for the loft than conventional bedrooms in the dwelling under the current climate; total category I loft overheating degree-hours may increase to 20,319 by 2080. While external shutters and night-purge ventilation were the most effective at reducing overheating degree-hours (96% and 89%, respectively), the most cost-effective solutions considering capital and ongoing costs are ventilation strategies, including night-time purge ventilation, advance ventilation and cross ventilation. Passive adaptations are not capable of eliminating overheating entirely, and by the 2080s active cooling is likely to be required to maintain comfortable indoor conditions in lofts. Practical application: Converted lofts – present in 5.8% of English and 10.8% of London dwellings – are at significantly elevated risk of high indoor temperatures relative to conventional rooms. Passive adaptations such as ventilation and shading can effectively mitigate loft overheating until around 2080, after which active measures become necessary. When capital and ongoing costs are considered, the most cost-effective heat mitigating adaptations are night and advance ventilation and internal curtains/blinds. Heat mitigating adaptations for converted lofts should become mandatory, and such spaces should not be occupied by the vulnerable or elderly during hot weather.
Inverse Faraday effect in superconductors with potential impurities

I consider a nonlinear response of conventional superconductors contaminated with potential impurities or imperfections to a circular polarized light. I focus on dc contributions to the induced current density which describe the emergence of the static magnetization in a superconductor. This effect is known as inverse Faraday effect. By employing quasiclassical theory of superconductivity I derive an expression for the induced static magnetization as a function of frequency of an external ac field and disorder scattering rate. The scattering of electrons off potential impurities is taken into account within the framework of the self-consistent Born approximation. It is found that the magnitude of the inverse Faraday effect decreases with an increase in disorder scattering rate. I have also discovered that the value of the induced magnetization has a characteristic minimum at a frequency which approximately equals twice the value of the pairing gap in a clean superconductor and shifts to higher values with an increase in disorder scattering rate.
Self-Supervised Learning of Visual Robot Localization Using LED State Prediction as a Pretext Task

We propose a novel self-supervised approach for learning to visually localize robots equipped with controllable LEDs. We rely on a few training samples labeled with position ground truth and many training samples in which only the LED state is known, whose collection is cheap. We show that using LED state prediction as a pretext task significantly helps to learn the visual localization end task. The resulting model does not require knowledge of LED states during inference. We instantiate the approach to visual relative localization of nano-quadrotors: experimental results show that using our pretext task significantly improves localization accuracy (from 68.3% to 76.2%) and outperforms alternative strategies, such as a supervised baseline, model pre-training, and an autoencoding pretext task. We deploy our model aboard a 27-g Crazyflie nano-drone, running at 21 fps, in a position-tracking task of a peer nano-drone. Our approach, relying on position labels for only 300 images, yields a mean tracking error of 4.2 cm versus 11.9 cm of a supervised baseline model trained without our pretext task.
A distributive lattice of model structures relating to spectral sequences

The $S$-model category structures on filtered chain complexes and bicomplexes were introduced by Cirici, Egas Santander, Livernet and Whitehouse and later generalised by this author. In this paper we show they are left proper, cellular and stable model categories. We use these properties and the Cellularization Principle of Greenlees and Shipley to show that an adjunction with right adjoint the product totalisation functor from bicomplexes to filtered chains is a Quillen equivalence. Combined with other known Quillen equivalences between filtered chains this shows these model categories all present the same homotopy category. We also construct a distributive lattice whose elements are the $S$-model categories of filtered chain complexes.
Instruction Tuning for Secure Code Generation

Modern language models (LMs) have gained widespread acceptance in everyday and professional contexts, particularly in programming. An essential procedure enabling this adoption is instruction tuning, which substantially enhances LMs' practical utility by training them to follow user instructions and human preferences. However, existing instruction tuning schemes overlook a crucial aspect: the security of generated code. As a result, even the state-of-the-art instruction-tuned LMs frequently produce unsafe code, posing significant security risks. In this work, we introduce SafeCoder to address this gap. SafeCoder performs security-centric fine-tuning using a diverse and high-quality dataset that we collected using an automated pipeline. We integrate the security fine-tuning with standard instruction tuning, to facilitate a joint optimization of both security and utility. Despite its simplicity, we show that SafeCoder is effective across a variety of popular LMs and datasets. It is able to drastically improve security (by about 30%), while preserving utility.
WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensing

WiFi-based human sensing has exhibited remarkable potential to analyze user behaviors in a non-intrusive and device-free manner, benefiting applications as diverse as smart homes and healthcare. However, most previous works focus on single-user sensing, which has limited practicability in scenarios involving multiple users. Although recent studies have begun to investigate WiFi-based multi-user sensing, there remains a lack of benchmark datasets to facilitate reproducible and comparable research. To bridge this gap, we present WiMANS, to our knowledge, the first dataset for multi-user sensing based on WiFi. WiMANS contains over 9.4 hours of dual-band WiFi Channel State Information (CSI), as well as synchronized videos, monitoring simultaneous activities of multiple users. We exploit WiMANS to benchmark the performance of state-of-the-art WiFi-based human sensing models and video-based models, posing new challenges and opportunities for future work. We believe WiMANS can push the boundaries of current studies and catalyze the research on WiFi-based multi-user sensing.
WERank: Towards Rank Degradation Prevention for Self-Supervised Learning Using Weight Regularization

A common phenomena confining the representation quality in Self-Supervised Learning (SSL) is dimensional collapse (also known as rank degeneration), where the learned representations are mapped to a low dimensional subspace of the representation space. The State-of-the-Art SSL methods have shown to suffer from dimensional collapse and fall behind maintaining full rank. Recent approaches to prevent this problem have proposed using contrastive losses, regularization techniques, or architectural tricks. We propose WERank, a new regularizer on the weight parameters of the network to prevent rank degeneration at different layers of the network. We provide empirical evidence and mathematical justification to demonstrate the effectiveness of the proposed regularization method in preventing dimensional collapse. We verify the impact of WERank on graph SSL where dimensional collapse is more pronounced due to the lack of proper data augmentation. We empirically demonstrate that WERank is effective in helping BYOL to achieve higher rank during SSL pre-training and consequently downstream accuracy during evaluation probing. Ablation studies and experimental analysis shed lights on the underlying factors behind the performance gains of the proposed approach.
CodeMind: A Framework to Challenge Large Language Models for Code Reasoning

Solely relying on test passing to evaluate Large Language Models (LLMs) for code synthesis may result in unfair assessment or promoting models with data leakage. As an alternative, we introduce CodeMind, a framework designed to gauge the code reasoning abilities of LLMs. CodeMind currently supports three code reasoning tasks: Independent Execution Reasoning (IER), Dependent Execution Reasoning (DER), and Specification Reasoning (SR). The first two evaluate models to predict the execution output of an arbitrary code or code the model could correctly synthesize. The third one evaluates the extent to which LLMs implement the specified expected behavior. Our extensive evaluation of nine LLMs across five benchmarks in two different programming languages using CodeMind shows that LLMs fairly follow control flow constructs and, in general, explain how inputs evolve to output, specifically for simple programs and the ones they can correctly synthesize. However, their performance drops for code with higher complexity, non-trivial logical and arithmetic operators, non-primitive types, and API calls. Furthermore, we observe that, while correlated, specification reasoning (essential for code synthesis) does not imply execution reasoning (essential for broader programming tasks such as testing and debugging): ranking LLMs based on test passing can be different compared to code reasoning.
Improved Lower Bounds for Approximating Parameterized Nearest Codeword and Related Problems under ETH

In this paper we present a new gap-creating randomized self-reduction for parameterized Maximum Likelihood Decoding problem over $\mathbb{F}_p$ ($k$-MLD$_p$). The reduction takes a $k$-MLD$_p$ instance with $k\cdot n$ vectors as input, runs in time $f(k)n^{O(1)}$ for some computable function $f$, outputs a $(3/2-\varepsilon)$-Gap-$k'$-MLD$_p$ instance for any $\varepsilon>0$, where $k'=O(k^2\log k)$. Using this reduction, we show that assuming the randomized Exponential Time Hypothesis (ETH), no algorithms can approximate $k$-MLD$_p$ (and therefore its dual problem $k$-NCP$_p$) within factor $(3/2-\varepsilon)$ in $f(k)\cdot n^{o(\sqrt{k/\log k})}$ time for any $\varepsilon>0$. We then use reduction by Bhattacharyya, Ghoshal, Karthik and Manurangsi (ICALP 2018) to amplify the $(3/2-\varepsilon)$-gap to any constant. As a result, we show that assuming ETH, no algorithms can approximate $k$-NCP$_p$ and $k$-MDP$_p$ within $\gamma$-factor in $f(k)n^{o(k^{\varepsilon_\gamma})}$ time for some constant $\varepsilon_\gamma>0$. Combining with the gap-preserving reduction by Bennett, Cheraghchi, Guruswami and Ribeiro (STOC 2023), we also obtain similar lower bounds for $k$-MDP$_p$, $k$-CVP$_p$ and $k$-SVP$_p$. These results improve upon the previous $f(k)n^{\Omega(\mathsf{poly} \log k)}$ lower bounds for these problems under ETH using reductions by Bhattacharyya et al. (J.ACM 2021) and Bennett et al. (STOC 2023).
Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shifts

We study the robustness of deep reinforcement learning algorithms against distribution shifts within contextual multi-stage stochastic combinatorial optimization problems from the operations research domain. In this context, risk-sensitive algorithms promise to learn robust policies. While this field is of general interest to the reinforcement learning community, most studies up-to-date focus on theoretical results rather than real-world performance. With this work, we aim to bridge this gap by formally deriving a novel risk-sensitive deep reinforcement learning algorithm while providing numerical evidence for its efficacy. Specifically, we introduce discrete Soft Actor-Critic for the entropic risk measure by deriving a version of the Bellman equation for the respective Q-values. We establish a corresponding policy improvement result and infer a practical algorithm. We introduce an environment that represents typical contextual multi-stage stochastic combinatorial optimization problems and perform numerical experiments to empirically validate our algorithm's robustness against realistic distribution shifts, without compromising performance on the training distribution. We show that our algorithm is superior to risk-neutral Soft Actor-Critic as well as to two benchmark approaches for robust deep reinforcement learning. Thereby, we provide the first structured analysis on the robustness of reinforcement learning under distribution shifts in the realm of contextual multi-stage stochastic combinatorial optimization problems.
How to Train Data-Efficient LLMs

The training of large language models (LLMs) is expensive. In this paper, we study data-efficient approaches for pre-training LLMs, i.e., techniques that aim to optimize the Pareto frontier of model quality and training resource/data consumption. We seek to understand the tradeoffs associated with data selection routines based on (i) expensive-to-compute data-quality estimates, and (ii) maximization of coverage and diversity-based measures in the feature space. Our first technique, Ask-LLM, leverages the zero-shot reasoning capabilities of instruction-tuned LLMs to directly assess the quality of a training example. To target coverage, we propose Density sampling, which models the data distribution to select a diverse sample. In our comparison of 19 samplers, involving hundreds of evaluation tasks and pre-training runs, we find that Ask-LLM and Density are the best methods in their respective categories. Coverage sampling can recover the performance of the full data, while models trained on Ask-LLM data consistently outperform full-data training -- even when we reject 90% of the original dataset, while converging up to 70% faster.