OpenRaiser/Intern-Atlas huggingface dataset:
Paralleling and Accelerating Arc Consistency Enforcement with Recurrent Tensor Computations
Venue: arXiv cs.DC 2024
Year: 2024

We propose a new arc consistency enforcement paradigm that transforms arc consistency enforcement into recurrent tensor operations. In each iteration of the recurrence, all involved processes can be fully parallelized with tensor operations. And the number of iterations is quite small. Based on these benefits, the resulting algorithm fully leverages the power of parallelization and GPU, and therefore is extremely efficient on large and densely connected constraint networks.
Mapping savannah woody vegetation at the species level with multispecral drone and hyperspectral EnMAP data
Venue: arXiv cs.LG 2024
Year: 2024

Savannahs are vital ecosystems whose sustainability is endangered by the spread of woody plants. This research targets the accurate mapping of fractional woody cover (FWC) at the species level in a South African savannah, using EnMAP hyperspectral data. Field annotations were combined with very high-resolution multispectral drone data to produce land cover maps that included three woody species. The high-resolution labelled maps were then used to generate FWC samples for each woody species class at the 30-m spatial resolution of EnMAP. Four machine learning regression algorithms were tested for FWC mapping on dry season EnMAP imagery. The contribution of multitemporal information was also assessed by incorporating as additional regression features, spectro-temporal metrics from Sentinel-2 data of both the dry and wet seasons. The results demonstrated the suitability of our approach for accurately mapping FWC at the species level. The highest accuracy rates achieved from the combined EnMAP and Sentinel-2 experiments highlighted their synergistic potential for species-level vegetation mapping.
Multi-Channel Masked Autoencoder and Comprehensive Evaluations for Reconstructing 12-Lead ECG from Arbitrary Single-Lead ECG
Venue: arXiv cs.LG 2024
Year: 2024

Electrocardiogram (ECG) has emerged as a widely accepted diagnostic instrument for cardiovascular diseases (CVD). The standard clinical 12-lead ECG configuration causes considerable inconvenience and discomfort, while wearable devices offers a more practical alternative. To reduce information gap between 12-lead ECG and single-lead ECG, this study proposes a multi-channel masked autoencoder (MCMA) for reconstructing 12-Lead ECG from arbitrary single-lead ECG, and a comprehensive evaluation benchmark, ECGGenEval, encompass the signal-level, feature-level, and diagnostic-level evaluations. MCMA can achieve the state-of-the-art performance. In the signal-level evaluation, the mean square errors of 0.0317 and 0.1034, Pearson correlation coefficients of 0.7885 and 0.7420. In the feature-level evaluation, the average standard deviation of the mean heart rate across the generated 12-lead ECG is 1.0481, the coefficient of variation is 1.58%, and the range is 3.2874. In the diagnostic-level evaluation, the average F1-score with two generated 12-lead ECG from different single-lead ECG are 0.8233 and 0.8410.
Haze-Aware Attention Network for Single-Image Dehazing
Venue: arXiv cs.CV 2024
Year: 2024

Single-image dehazing is a pivotal challenge in computer vision that seeks to remove haze from images and restore clean background details. Recognizing the limitations of traditional physical model-based methods and the inefficiencies of current attention-based solutions, we propose a new dehazing network combining an innovative Haze-Aware Attention Module (HAAM) with a Multiscale Frequency Enhancement Module (MFEM). The HAAM is inspired by the atmospheric scattering model, thus skillfully integrating physical principles into high-dimensional features for targeted dehazing. It picks up on latent features during the image restoration process, which gives a significant boost to the metrics, while the MFEM efficiently enhances high-frequency details, thus sidestepping wavelet or Fourier transform complexities. It employs multiscale fields to extract and emphasize key frequency components with minimal parameter overhead. Integrated into a simple U-Net framework, our Haze-Aware Attention Network (HAA-Net) for single-image dehazing significantly outperforms existing attention-based and transformer models in efficiency and effectiveness. Tested across various public datasets, the HAA-Net sets new performance benchmarks. Our work not only advances the field of image dehazing but also offers insights into the design of attention mechanisms for broader applications in computer vision.
Introducing Total Harmonic Resistance for Graph Robustness under Edge Deletions
Venue: arXiv cs.SI 2024
Year: 2024

Assessing and improving the robustness of a graph $G$ are critical steps in network design and analysis. To this end, we consider the optimisation problem of removing $k$ edges from $G$ such that the resulting graph has minimal robustness, simulating attacks or failures. In this paper, we propose total harmonic resistance as a new robustness measure for this purpose - and compare it to the recently proposed forest index [Zhu et al., IEEE Trans.\ Inf.\ Forensics and Security, 2023]. Both measures are related to the established total effective resistance measure, but their advantage is that they can handle disconnected graphs. This is also important for originally connected graphs due to the removal of the $k$ edges. To compare our measure with the forest index, we first investigate exact solutions for small examples. The best $k$ edges to select when optimizing for the forest index lie at the periphery. Our proposed measure, in turn, prioritizes more central edges, which should be beneficial for most applications. Furthermore, we adapt a generic greedy algorithm to our optimization problem with the total harmonic resistance. With this algorithm, we perform a case study on the Berlin road network and also apply the algorithm to established benchmark graphs. The results are similar as for the small example graphs above and indicate the higher suitability of the new measure.
Optimal Constant-Weight and Mixed-Weight Conflict-Avoiding Codes
Venue: arXiv cs.IT 2024
Year: 2024

A conflict-avoiding code (CAC) is a deterministic transmission scheme for asynchronous multiple access without feedback. When the number of simultaneously active users is less than or equal to $w$, a CAC of length $L$ with weight $w$ can provide a hard guarantee that each active user has at least one successful transmission within every consecutive $L$ slots. In this paper, we generalize some previously known constructions of constant-weight CACs, and then derive several classes of optimal CACs by the help of Kneser's Theorem and some techniques in Additive Combinatorics. Another spotlight of this paper is to relax the identical-weight constraint in prior studies to study mixed-weight CACs for the first time, for the purpose of increasing the throughput and reducing the access delay of some potential users with higher priority. As applications of those obtained optimal CACs, we derive some classes of optimal mixed-weight CACs.
Demonstration of real-time event camera to collaborative robot communication
Venue: arXiv cs.RO 2024
Year: 2024

Real-time robot actuation is one of the main challenges to overcome in human-robot interaction. Most visual sensors are either too slow or their data are too complex to provide meaningful information and low latency input to a robotic system. Data output of an event camera is high-frequency and extremely lightweight, with only 8 bytes per event. To evaluate the hypothesis of using event cameras as data source for a real-time robotic system, the position of a waving hand is acquired from the event data and transmitted to a collaborative robot as a movement command. A total time delay of 110 ms was measured between the original movement and the robot movement, where much of the delay is caused by the robot dynamics.
Federated Learning Forecasting for Strengthening Grid Reliability and Enabling Markets for Resilience
Venue: arXiv cs.LG 2024
Year: 2024

We propose a comprehensive approach to increase the reliability and resilience of future power grids rich in distributed energy resources. Our distributed scheme combines federated learning-based attack detection with a local electricity market-based attack mitigation method. We validate the scheme by applying it to a real-world distribution grid rich in solar PV. Simulation results demonstrate that the approach is feasible and can successfully mitigate the grid impacts of cyber-physical attacks.
Interactions with Generative Information Retrieval Systems
Venue: arXiv cs.IR 2024
Year: 2024

At its core, information access and seeking is an interactive process. In existing search engines, interactions are limited to a few pre-defined actions, such as "requery", "click on a document", "scrolling up/down", "going to the next result page", "leaving the search engine", etc. A major benefit of moving towards generative IR systems is enabling users with a richer expression of information need and feedback and free-form interactions in natural language and beyond. In other words, the actions users take are no longer limited by the clickable links and buttons available on the search engine result page and users can express themselves freely through natural language. This can go even beyond natural language, through images, videos, gestures, and sensors using multi-modal generative IR systems. This chapter briefly discusses the role of interaction in generative IR systems. We will first discuss different ways users can express their information needs by interacting with generative IR systems. We then explain how users can provide explicit or implicit feedback to generative IR systems and how they can consume such feedback. Next, we will cover how users interactively can refine retrieval results. We will expand upon mixed-initiative interactions and discuss clarification and preference elicitation in more detail. We then discuss proactive generative IR systems, including context-aware recommendation, following up past conversations, contributing to multi-party conversations, and feedback requests. Providing explanation is another interaction type that we briefly discuss in this chapter. We will also briefly describe multi-modal interactions in generative information retrieval. Finally, we describe emerging frameworks and solutions for user interfaces with generative AI systems.
Estimating the Energy Footprint of Software Systems: a Primer
Venue: arXiv cs.SE 2024
Year: 2024

In Green Software Development, quantifying the energy footprint of a software system is one of the most basic activities. This documents provides a high-level overview of how the energy footprint of a software system can be estimated to support Green Software Development. We introduce basic concepts in the area, highlight methodological issues that must be accounted for when conducting experiments, discuss trade-offs associated with different estimation approaches, and make some practical considerations. This document aims to be a starting point for researchers who want to begin conducting work in this area.
PyTond: Efficient Python Data Science on the Shoulders of Databases
Venue: arXiv cs.DB 2024
Year: 2024

Python data science libraries such as Pandas and NumPy have recently gained immense popularity. Although these libraries are feature-rich and easy to use, their scalability limitations require more robust computational resources. In this paper, we present PyTond, an efficient approach to push the processing of data science workloads down into the database engines that are already known for their big data handling capabilities. Compared to the previous work, by introducing TondIR, our approach can capture a more comprehensive set of workloads and data layouts. Moreover, by doing IR-level optimizations, we generate better SQL code that improves the query processing by the underlying database engine. Our evaluation results show promising performance improvement compared to Python and other alternatives for diverse data science workloads.
Beware of Validation by Eye: Visual Validation of Linear Trends in Scatterplots
Venue: arXiv cs.CV 2024
Year: 2024

Visual validation of regression models in scatterplots is a common practice for assessing model quality, yet its efficacy remains unquantified. We conducted two empirical experiments to investigate individuals' ability to visually validate linear regression models (linear trends) and to examine the impact of common visualization designs on validation quality. The first experiment showed that the level of accuracy for visual estimation of slope (i.e., fitting a line to data) is higher than for visual validation of slope (i.e., accepting a shown line). Notably, we found bias toward slopes that are "too steep" in both cases. This lead to novel insights that participants naturally assessed regression with orthogonal distances between the points and the line (i.e., ODR regression) rather than the common vertical distances (OLS regression). In the second experiment, we investigated whether incorporating common designs for regression visualization (error lines, bounding boxes, and confidence intervals) would improve visual validation. Even though error lines reduced validation bias, results failed to show the desired improvements in accuracy for any design. Overall, our findings suggest caution in using visual model validation for linear trends in scatterplots.
Scaling Diffusion Transformers to 16 Billion Parameters
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we present DiT-MoE, a sparse version of the diffusion Transformer, that is scalable and competitive with dense networks while exhibiting highly optimized inference. The DiT-MoE includes two simple designs: shared expert routing and expert-level balance loss, thereby capturing common knowledge and reducing redundancy among the different routed experts. When applied to conditional image generation, a deep analysis of experts specialization gains some interesting observations: (i) Expert selection shows preference with spatial position and denoising time step, while insensitive with different class-conditional information; (ii) As the MoE layers go deeper, the selection of experts gradually shifts from specific spacial position to dispersion and balance. (iii) Expert specialization tends to be more concentrated at the early time step and then gradually uniform after half. We attribute it to the diffusion process that first models the low-frequency spatial information and then high-frequency complex information. Based on the above guidance, a series of DiT-MoE experimentally achieves performance on par with dense networks yet requires much less computational load during inference. More encouragingly, we demonstrate the potential of DiT-MoE with synthesized image data, scaling diffusion model at a 16.5B parameter that attains a new SoTA FID-50K score of 1.80 in 512$\times$512 resolution settings. The project page: https://github.com/feizc/DiT-MoE.
Speed-robust scheduling revisited
Venue: arXiv cs.DS 2024
Year: 2024

Speed-robust scheduling is the following two-stage problem of scheduling $n$ jobs on $m$ uniformly related machines. In the first stage, the algorithm receives the value of $m$ and the processing times of $n$ jobs; it has to partition the jobs into $b$ groups called bags. In the second stage, the machine speeds are revealed and the bags are assigned to the machines, i.e., the algorithm produces a schedule where all the jobs in the same bag are assigned to the same machine. The objective is to minimize the makespan (the length of the schedule). The algorithm is compared to the optimal schedule and it is called $ρ$-robust, if its makespan is always at most $ρ$ times the optimal one. Our main result is an improved bound for equal-size jobs for $b=m$. We give an upper bound of $1.6$. This improves previous bound of $1.8$ and it is almost tight in the light of previous lower bound of $1.58$. Second, for infinitesimally small jobs, we give tight upper and lower bounds for the case when $b\geq m$. This generalizes and simplifies the previous bounds for $b=m$. Finally, we introduce a new special case with relatively small jobs for which we give an algorithm whose robustness is close to that of infinitesimal jobs and thus gives better than $2$-robust for a large class of inputs.
Video-Language Alignment via Spatio-Temporal Graph Transformer
Venue: arXiv cs.CV 2024
Year: 2024

Video-language alignment is a crucial multi-modal task that benefits various downstream applications, e.g., video-text retrieval and video question answering. Existing methods either utilize multi-modal information in video-text pairs or apply global and local alignment techniques to promote alignment precision. However, these methods often fail to fully explore the spatio-temporal relationships among vision tokens within video and across different video-text pairs. In this paper, we propose a novel Spatio-Temporal Graph Transformer module to uniformly learn spatial and temporal contexts for video-language alignment pre-training (dubbed STGT). Specifically, our STGT combines spatio-temporal graph structure information with attention in transformer block, effectively utilizing the spatio-temporal contexts. In this way, we can model the relationships between vision tokens, promoting video-text alignment precision for benefiting downstream tasks. In addition, we propose a self-similarity alignment loss to explore the inherent self-similarity in the video and text. With the initial optimization achieved by contrastive learning, it can further promote the alignment accuracy between video and text. Experimental results on challenging downstream tasks, including video-text retrieval and video question answering, verify the superior performance of our method.
Identifying Coordinated Activities on Online Social Networks Using Contrast Pattern Mining
Venue: arXiv cs.SI 2024
Year: 2024

The proliferation of misinformation and disinformation on social media networks has become increasingly concerning. With a significant portion of the population using social media on a regular basis, there are growing efforts by malicious organizations to manipulate public opinion through coordinated campaigns. Current methods for identifying coordinated user accounts typically rely on either similarities in user behaviour, latent coordination in activity traces, or classification techniques. In our study, we propose a framework based on the hypothesis that coordinated users will demonstrate abnormal growth in their behavioural patterns over time relative to the wider population. Specifically, we utilize the EPClose algorithm to extract contrasting patterns of user behaviour during a time window of malicious activity, which we then compare to a historical time window. We evaluated the effectiveness of our approach using real-world data, and our results show a minimum increase of 10% in the F1 score compared to existing approaches.
Novel Artistic Scene-Centric Datasets for Effective Transfer Learning in Fragrant Spaces
Venue: arXiv cs.CV 2024
Year: 2024

Olfaction, often overlooked in cultural heritage studies, holds profound significance in shaping human experiences and identities. Examining historical depictions of olfactory scenes can offer valuable insights into the role of smells in history. We show that a transfer-learning approach using weakly labeled training data can remarkably improve the classification of fragrant spaces and, more generally, artistic scene depictions. We fine-tune Places365-pre-trained models by querying two cultural heritage data sources and using the search terms as supervision signal. The models are evaluated on two manually corrected test splits. This work lays a foundation for further exploration of fragrant spaces recognition and artistic scene classification. All images and labels are released as the ArtPlaces dataset at https://zenodo.org/doi/10.5281/zenodo.11584328.
Bidding efficiently in Simultaneous Ascending Auctions with incomplete information using Monte Carlo Tree Search and determinization
Venue: arXiv cs.GT 2024
Year: 2024

For decades, Simultaneous Ascending Auction (SAA) has been the most widely used mechanism for spectrum auctions, and it has recently gained popularity for allocating 5G licenses in many countries. Despite its relatively simple rules, SAA introduces a complex strategic game with an unknown optimal bidding strategy. Given the high stakes involved, with billions of euros sometimes on the line, developing an efficient bidding strategy is of utmost importance. In this work, we extend our previous method, a Simultaneous Move Monte-Carlo Tree Search (SM-MCTS) based algorithm named $SMS^α$ to incomplete information framework. For this purpose, we compare three determinization approaches which allow us to rely on complete information SM-MCTS. This algorithm addresses, in incomplete framework, the four key strategic issues of SAA: the exposure problem, the own price effect, budget constraints, and the eligibility management problem. Through extensive numerical experiments on instances of realistic size with an uncertain framework, we show that $SMS^α$ largely outperforms state-of-the-art algorithms by achieving higher expected utility while taking less risks, no matter which determinization method is chosen.
Exploring Quantization for Efficient Pre-Training of Transformer Language Models
Venue: arXiv cs.LG 2024
Year: 2024

The increasing scale of Transformer models has led to an increase in their pre-training computational requirements. While quantization has proven to be effective after pre-training and during fine-tuning, applying quantization in Transformers during pre-training has remained largely unexplored at scale for language modeling. This study aims to explore the impact of quantization for efficient pre-training of Transformers, with a focus on linear layer components. By systematically applying straightforward linear quantization to weights, activations, gradients, and optimizer states, we assess its effects on model efficiency, stability, and performance during training. By offering a comprehensive recipe of effective quantization strategies to be applied during the pre-training of Transformers, we promote high training efficiency from scratch while retaining language modeling ability. Code is available at https://github.com/chandar-lab/EfficientLLMs.