OpenRaiser/Intern-Atlas huggingface dataset:
Cross-Table Pretraining towards a Universal Function Space for Heterogeneous Tabular Data
Venue: arXiv cs.LG 2024
Year: 2024

Tabular data from different tables exhibit significant diversity due to varied definitions and types of features, as well as complex inter-feature and feature-target relationships. Cross-dataset pretraining, which learns reusable patterns from upstream data to support downstream tasks, have shown notable success in various fields. Yet, when applied to tabular data prediction, this paradigm faces challenges due to the limited reusable patterns among diverse tabular datasets (tables) and the general scarcity of tabular data available for fine-tuning. In this study, we fill this gap by introducing a cross-table pretrained Transformer, XTFormer, for versatile downstream tabular prediction tasks. Our methodology insight is pretraining XTFormer to establish a "meta-function" space that encompasses all potential feature-target mappings. In pre-training, a variety of potential mappings are extracted from pre-training tabular datasets and are embedded into the "meta-function" space, and suited mappings are extracted from the "meta-function" space for downstream tasks by a specified coordinate positioning approach. Experiments show that, in 190 downstream tabular prediction tasks, our cross-table pretrained XTFormer wins both XGBoost and Catboost on 137 (72%) tasks, and surpasses representative deep learning models FT-Transformer and the tabular pre-training approach XTab on 144 (76%) and 162 (85%) tasks.
A Closer Look at Logical Reasoning with LLMs: The Choice of Tool Matters
Venue: arXiv cs.CL 2024
Year: 2024

The emergence of Large Language Models (LLMs) has demonstrated promising progress in solving logical reasoning tasks effectively. Several recent approaches have proposed to change the role of the LLM from the reasoner into a translator between natural language statements and symbolic representations which are then sent to external symbolic solvers to resolve. This paradigm has established the current state-of-the-art result in logical reasoning (i.e., deductive reasoning). However, it remains unclear whether the variance in performance of these approaches stems from the methodologies employed or the specific symbolic solvers utilized. There is a lack of consistent comparison between symbolic solvers and how they influence the overall reported performance. This is important, as each symbolic solver also has its own input symbolic language, presenting varying degrees of challenge in the translation process. To address this gap, we perform experiments on 3 deductive reasoning benchmarks with LLMs augmented with widely used symbolic solvers: Z3, Pyke, and Prover9. The tool-executable rates of symbolic translation generated by different LLMs exhibit a near 50% performance variation. This highlights a significant difference in performance rooted in very basic choices of tools. The almost linear correlation between the executable rate of translations and the accuracy of the outcomes from Prover9 highlight a strong alignment between LLMs ability to translate into Prover9 symbolic language, and the correctness of those translations.
Phasor-Driven Acceleration for FFT-based CNNs
Venue: arXiv cs.CV 2024
Year: 2024

Recent research in deep learning (DL) has investigated the use of the Fast Fourier Transform (FFT) to accelerate the computations involved in Convolutional Neural Networks (CNNs) by replacing spatial convolution with element-wise multiplications on the spectral domain. These approaches mainly rely on the FFT to reduce the number of operations, which can be further decreased by adopting the Real-Valued FFT. In this paper, we propose using the phasor form, a polar representation of complex numbers, as a more efficient alternative to the traditional approach. The experimental results, evaluated on the CIFAR-10, demonstrate that our method achieves superior speed improvements of up to a factor of 1.376 (average of 1.316) during training and up to 1.390 (average of 1.321) during inference when compared to the traditional rectangular form employed in modern CNN architectures. Similarly, when evaluated on the CIFAR-100, our method achieves superior speed improvements of up to a factor of 1.375 (average of 1.299) during training and up to 1.387 (average of 1.300) during inference. Most importantly, given the modular aspect of our approach, the proposed method can be applied to any existing convolution-based DL model without design changes.
HENASY: Learning to Assemble Scene-Entities for Egocentric Video-Language Model
Venue: arXiv cs.CV 2024
Year: 2024

Current video-language models (VLMs) rely extensively on instance-level alignment between video and language modalities, which presents two major limitations: (1) visual reasoning disobeys the natural perception that humans do in first-person perspective, leading to a lack of reasoning interpretation; and (2) learning is limited in capturing inherent fine-grained relationships between two modalities. In this paper, we take an inspiration from human perception and explore a compositional approach for egocentric video representation. We introduce HENASY (Hierarchical ENtities ASsemblY), which includes a spatiotemporal token grouping mechanism to explicitly assemble dynamically evolving scene entities through time and model their relationship for video representation. By leveraging compositional structure understanding, HENASY possesses strong interpretability via visual grounding with free-form text queries. We further explore a suite of multi-grained contrastive losses to facilitate entity-centric understandings. This comprises three alignment types: video-narration, noun-entity, verb-entities alignments. Our method demonstrates strong interpretability in both quantitative and qualitative experiments; while maintaining competitive performances on five downstream tasks via zero-shot transfer or as video/text representation, including video/text retrieval, action recognition, multi-choice query, natural language query, and moments query.
Toward 6G Optical Fronthaul: A Survey on Enabling Technologies and Research Perspectives
Venue: arXiv cs.NI 2024
Year: 2024

The anticipated launch of the Sixth Generation (6G) of mobile technology by 2030 will mark a significant milestone in the evolution of wireless communication, ushering in a new era with advancements in technology and applications. 6G is expected to deliver ultra-high data rates and almost instantaneous communications, with three-dimensional coverage for everything, everywhere, and at any time. In the 6G Radio Access Networks (RANs) architecture, the Fronthaul connects geographically distributed Remote Units (RUs) to Distributed/Digital Units (DUs)pool. Among all possible solutions for implementing 6G fronthaul, optical technologies will remain crucial in supporting the 6G fronthaul, as they offer high-speed, low-latency, and reliable transmission capabilities to meet the 6G strict requirements. This survey provides an explanation of the 5G and future 6G optical fronthaul concept and presents a comprehensive overview of the current state of the art and future research directions in 6G optical fronthaul, highlighting the key technologies and research perspectives fundamental in designing fronthaul networks for 5G and future 6G. Additionally, it examines the benefits and drawbacks of each optical technology and its potential applications in 6G fronthaul networks. This paper aims to serve as a comprehensive resource for researchers and industry professionals about the current state and future prospects of 6G optical fronthaul technologies, facilitating the development of robust and efficient wireless networks of the future.
CASE: Efficient Curricular Data Pre-training for Building Assistive Psychology Expert Models
Venue: arXiv cs.CL 2024
Year: 2024

The limited availability of psychologists necessitates efficient identification of individuals requiring urgent mental healthcare. This study explores the use of Natural Language Processing (NLP) pipelines to analyze text data from online mental health forums used for consultations. By analyzing forum posts, these pipelines can flag users who may require immediate professional attention. A crucial challenge in this domain is data privacy and scarcity. To address this, we propose utilizing readily available curricular texts used in institutes specializing in mental health for pre-training the NLP pipelines. This helps us mimic the training process of a psychologist. Our work presents CASE-BERT that flags potential mental health disorders based on forum text. CASE-BERT demonstrates superior performance compared to existing methods, achieving an f1 score of 0.91 for Depression and 0.88 for Anxiety, two of the most commonly reported mental health disorders. Our code and data are publicly available.
Optimal bounds for $\ell_p$ sensitivity sampling via $\ell_2$ augmentation
Venue: arXiv cs.DS 2024
Year: 2024

Data subsampling is one of the most natural methods to approximate a massively large data set by a small representative proxy. In particular, sensitivity sampling received a lot of attention, which samples points proportional to an individual importance measure called sensitivity. This framework reduces in very general settings the size of data to roughly the VC dimension $d$ times the total sensitivity $\mathfrak S$ while providing strong $(1\pm\varepsilon)$ guarantees on the quality of approximation. The recent work of Woodruff & Yasuda (2023c) improved substantially over the general $\tilde O(\varepsilon^{-2}\mathfrak Sd)$ bound for the important problem of $\ell_p$ subspace embeddings to $\tilde O(\varepsilon^{-2}\mathfrak S^{2/p})$ for $p\in[1,2]$. Their result was subsumed by an earlier $\tilde O(\varepsilon^{-2}\mathfrak Sd^{1-p/2})$ bound which was implicitly given in the work of Chen & Derezinski (2021). We show that their result is tight when sampling according to plain $\ell_p$ sensitivities. We observe that by augmenting the $\ell_p$ sensitivities by $\ell_2$ sensitivities, we obtain better bounds improving over the aforementioned results to optimal linear $\tilde O(\varepsilon^{-2}(\mathfrak S+d)) = \tilde O(\varepsilon^{-2}d)$ sampling complexity for all $p \in [1,2]$. In particular, this resolves an open question of Woodruff & Yasuda (2023c) in the affirmative for $p \in [1,2]$ and brings sensitivity subsampling into the regime that was previously only known to be possible using Lewis weights (Cohen & Peng, 2015). As an application of our main result, we also obtain an $\tilde O(\varepsilon^{-2}μd)$ sensitivity sampling bound for logistic regression, where $μ$ is a natural complexity measure for this problem. This improves over the previous $\tilde O(\varepsilon^{-2}μ^2 d)$ bound of Mai et al. (2021) which was based on Lewis weights subsampling.
Turnstile $\ell_p$ leverage score sampling with applications
Venue: arXiv cs.DS 2024
Year: 2024

The turnstile data stream model offers the most flexible framework where data can be manipulated dynamically, i.e., rows, columns, and even single entries of an input matrix can be added, deleted, or updated multiple times in a data stream. We develop a novel algorithm for sampling rows $a_i$ of a matrix $A\in\mathbb{R}^{n\times d}$, proportional to their $\ell_p$ norm, when $A$ is presented in a turnstile data stream. Our algorithm not only returns the set of sampled row indexes, it also returns slightly perturbed rows $\tilde{a}_i \approx a_i$, and approximates their sampling probabilities up to $\varepsilon$ relative error. When combined with preconditioning techniques, our algorithm extends to $\ell_p$ leverage score sampling over turnstile data streams. With these properties in place, it allows us to simulate subsampling constructions of coresets for important regression problems to operate over turnstile data streams with very little overhead compared to their respective off-line subsampling algorithms. For logistic regression, our framework yields the first algorithm that achieves a $(1+\varepsilon)$ approximation and works in a turnstile data stream using polynomial sketch/subsample size, improving over $O(1)$ approximations, or $\exp(1/\varepsilon)$ sketch size of previous work. We compare experimentally to plain oblivious sketching and plain leverage score sampling algorithms for $\ell_p$ and logistic regression.
Location Privacy in B5G/6G: Systematization of Knowledge
Venue: arXiv cs.NI 2024
Year: 2024

As we transition into the era of B5G/6G networks, the promise of seamless, high-speed connectivity brings unprecedented opportunities and challenges. Among the most critical concerns is the preservation of location privacy, given the enhanced precision and pervasive connectivity of these advanced networks. This paper systematically reviews the state of knowledge on location privacy in B5G/6G networks, highlighting the architectural advancements and infrastructural complexities that contribute to increased privacy risks. The urgency of studying these technologies is underscored by the rapid adoption of B5G/6G and the growing sophistication of location tracking methods. We evaluate current and emerging privacy-preserving mechanisms, exploring the implications of sophisticated tracking methods and the challenges posed by the complex network infrastructures. Our findings reveal the effectiveness of various mitigation strategies and emphasize the important role of physical layer security. Additionally, we propose innovative approaches, including decentralized authentication systems and the potential of satellite communications, to enhance location privacy. By addressing these challenges, this paper provides a comprehensive perspective on preserving user privacy in the rapidly evolving landscape of modern communication networks.
Did I Vet You Before? Assessing the Chrome Web Store Vetting Process through Browser Extension Similarity
Venue: arXiv cs.CR 2024
Year: 2024

Web browsers, particularly Google Chrome and other Chromium-based browsers, have grown in popularity over the past decade, with browser extensions becoming an integral part of their ecosystem. These extensions can customize and enhance the user experience, providing functionality that ranges from ad blockers to, more recently, AI assistants. Given the ever-increasing importance of web browsers, distribution marketplaces for extensions play a key role in keeping users safe by vetting submissions that display abusive or malicious behavior. In this paper, we characterize the prevalence of malware and other infringing extensions in the Chrome Web Store (CWS), the largest distribution platform for this type of software. To do so, we introduce SimExt, a novel methodology for detecting similarly behaving extensions that leverages static and dynamic analysis, Natural Language Processing (NLP) and vector embeddings. Our study reveals significant gaps in the CWS vetting process, as 86% of infringing extensions are extremely similar to previously vetted items, and these extensions take months or even years to be removed. By characterizing the top kinds of infringing extension, we find that 83% are New Tab Extensions (NTEs) and raise some concerns about the consistency of the vetting labels assigned by CWS analysts. Our study also reveals that only 1% of malware extensions flagged by the CWS are detected as malicious by anti-malware engines, indicating a concerning gap between the threat landscape seen by CWS moderators and the detection capabilities of the threat intelligence community.
SpikeMM: Flexi-Magnification of High-Speed Micro-Motions
Venue: arXiv cs.CV 2024
Year: 2024

The amplification of high-speed micro-motions holds significant promise, with applications spanning fault detection in fast-paced industrial environments to refining precision in medical procedures. However, conventional motion magnification algorithms often encounter challenges in high-speed scenarios due to low sampling rates or motion blur. In recent years, spike cameras have emerged as a superior alternative for visual tasks in such environments, owing to their unique capability to capture temporal and spatial frequency domains with exceptional fidelity. Unlike conventional cameras, which operate at fixed, low frequencies, spike cameras emulate the functionality of the retina, asynchronously capturing photon changes at each pixel position using spike streams. This innovative approach comprehensively records temporal and spatial visual information, rendering it particularly suitable for magnifying high-speed micro-motions.This paper introduces SpikeMM, a pioneering spike-based algorithm tailored specifically for high-speed motion magnification. SpikeMM integrates multi-level information extraction, spatial upsampling, and motion magnification modules, offering a self-supervised approach adaptable to a wide range of scenarios. Notably, SpikeMM facilitates seamless integration with high-performance super-resolution and motion magnification algorithms. We substantiate the efficacy of SpikeMM through rigorous validation using scenes captured by spike cameras, showcasing its capacity to magnify motions in real-world high-frequency settings.
Computer Simulation of DNA Computing-Based Boolean Matrix Multiplication
Venue: arXiv cs.ET 2024
Year: 2024

DNA computing is an unconventional approach to computing that harnesses the parallelism and information storage capabilities of DNA molecules. It has emerged as a promising field with potential applications in solving a variety of computationally complex problems. This paper explores a DNA computing algorithm for Boolean matrix multiplication proposed by Nobuyuki et al. (2006) using a computer simulation, inspired by similar work done in the past by Obront (2021) for the DNA computing algorithm developed by Adleman (1994) for solving the Hamiltonian path problem. We develop a Python program to simulate the logical operations involved in the DNA-based Boolean matrix multiplication algorithm. The simulation replicates the key steps of the algorithm, including DNA sequence generation and hybridization, without imitating the physical behaviour of the DNA molecules. It is intended to serve as a basic prototype for larger, more comprehensive DNA computing simulators that can be used as educational or research tools in the future. Through this work, we aim to contribute to the understanding of DNA-based computing paradigms and their potential advantages and trade-offs compared to conventional computing systems, paving the way for future research and advancements in this emerging field.
Optimistic Rates for Learning from Label Proportions
Venue: arXiv cs.LG 2024
Year: 2024

We consider a weakly supervised learning problem called Learning from Label Proportions (LLP), where examples are grouped into ``bags'' and only the average label within each bag is revealed to the learner. We study various learning rules for LLP that achieve PAC learning guarantees for classification loss. We establish that the classical Empirical Proportional Risk Minimization (EPRM) learning rule (Yu et al., 2014) achieves fast rates under realizability, but EPRM and similar proportion matching learning rules can fail in the agnostic setting. We also show that (1) a debiased proportional square loss, as well as (2) a recently proposed EasyLLP learning rule (Busa-Fekete et al., 2023) both achieve ``optimistic rates'' (Panchenko, 2002); in both the realizable and agnostic settings, their sample complexity is optimal (up to log factors) in terms of $ε, δ$, and VC dimension.
Cyberwar Strategy and Tactics: An Analysis of Cyber Goals, Strategies, Tactics, and Techniques
Venue: arXiv cs.CR 2024
Year: 2024

Cyberwar strategy and tactics today are primitive and ad-hoc, resulting in an ineffective and reactive cyber fighting force. A Cyberwar Playbook is an encoding of knowledge on how to effectively handle a variety of cyberwar situations. It takes a troubleshooting approach and defines the cyber tactics, techniques and procedures one may employ to counter or avert cyber-based attacks. It provides focus and clarity in time of chaos allowing a clear path of response to be chosen.
2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Video Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff. Due to its wide application in many downstream tasks such as video understanding, video editing, and autonomous driving. In order to deal with the task of video panoptic segmentation in the wild, we propose a robust integrated video panoptic segmentation solution. We use DVIS++ framework as our baseline to generate the initial masks. Then,we add an additional image semantic segmentation model to further improve the performance of semantic classes.Finally, our method achieves state-of-the-art performance with a VPQ score of 56.36 and 57.12 in the development and test phases, respectively, and ultimately ranked 2nd in the VPS track of the PVUW Challenge at CVPR2024.
Adaptive boosting with dynamic weight adjustment
Venue: arXiv cs.LG 2024
Year: 2024

Adaptive Boosting with Dynamic Weight Adjustment is an enhancement of the traditional Adaptive boosting commonly known as AdaBoost, a powerful ensemble learning technique. Adaptive Boosting with Dynamic Weight Adjustment technique improves the efficiency and accuracy by dynamically updating the weights of the instances based on prediction error where the weights are updated in proportion to the error rather than updating weights uniformly as we do in traditional Adaboost. Adaptive Boosting with Dynamic Weight Adjustment performs better than Adaptive Boosting as it can handle more complex data relations, allowing our model to handle imbalances and noise better, leading to more accurate and balanced predictions. The proposed model provides a more flexible and effective approach for boosting, particularly in challenging classification tasks.
The Firefighter Algorithm: A Hybrid Metaheuristic for Optimization Problems
Venue: arXiv cs.NE 2024
Year: 2024

This paper presents the Firefighter Optimization (FFO) algorithm as a new hybrid metaheuristic for optimization problems. This algorithm stems inspiration from the collaborative strategies often deployed by firefighters in firefighting activities. To evaluate the performance of FFO, extensive experiments were conducted, wherein the FFO was examined against 13 commonly used optimization algorithms, namely, the Ant Colony Optimization (ACO), Bat Algorithm (BA), Biogeography-Based Optimization (BBO), Flower Pollination Algorithm (FPA), Genetic Algorithm (GA), Grey Wolf Optimizer (GWO), Harmony Search (HS), Particle Swarm Optimization (PSO), Simulated Annealing (SA), Tabu Search (TS), and Whale Optimization Algorithm (WOA), and across 24 benchmark functions of various dimensions and complexities. The results demonstrate that FFO achieves comparative performance and, in some scenarios, outperforms commonly adopted optimization algorithms in terms of the obtained fitness, time taken for exaction, and research space covered per unit of time.