OpenRaiser/Intern-Atlas huggingface dataset:
Maximizing the practical achievability of quantum annealing attacks on factorization-based cryptography
Venue: arXiv cs.CR 2024
Year: 2024

This work focuses on quantum methods for cryptanalysis of schemes based on the integer factorization problem and the discrete logarithm problem. We demonstrate how to practically solve the largest instances of the factorization problem by improving an approach that combines quantum and classical computations, assuming the use of the best publicly available special-class quantum computer: the quantum annealer. We achieve new computational experiment results by solving the largest instance of the factorization problem ever announced as solved using quantum annealing, with a size of 29 bits. The core idea of the improved approach is to leverage known sub-exponential classical method to break the problem down into many smaller computations and perform the most critical ones on a quantum computer. This approach does not reduce the complexity class, but it assesses the pragmatic capabilities of an attacker. It also marks a step forward in the development of hybrid methods, which in practice may surpass classical methods in terms of efficiency sooner than purely quantum computations will.
On the Rigour of Scientific Writing: Criteria, Analysis, and Insights
Venue: arXiv cs.CL 2024
Year: 2024

Rigour is crucial for scientific research as it ensures the reproducibility and validity of results and findings. Despite its importance, little work exists on modelling rigour computationally, and there is a lack of analysis on whether these criteria can effectively signal or measure the rigour of scientific papers in practice. In this paper, we introduce a bottom-up, data-driven framework to automatically identify and define rigour criteria and assess their relevance in scientific writing. Our framework includes rigour keyword extraction, detailed rigour definition generation, and salient criteria identification. Furthermore, our framework is domain-agnostic and can be tailored to the evaluation of scientific rigour for different areas, accommodating the distinct salient criteria across fields. We conducted comprehensive experiments based on datasets collected from two high impact venues for Machine Learning and NLP (i.e., ICLR and ACL) to demonstrate the effectiveness of our framework in modelling rigour. In addition, we analyse linguistic patterns of rigour, revealing that framing certainty is crucial for enhancing the perception of scientific rigour, while suggestion certainty and probability uncertainty diminish it.
A Meta-Complexity Characterization of Quantum Cryptography
Venue: arXiv cs.CR 2024
Year: 2024

We prove the first meta-complexity characterization of a quantum cryptographic primitive. We show that one-way puzzles exist if and only if there is some quantum samplable distribution of binary strings over which it is hard to approximate Kolmogorov complexity. Therefore, we characterize one-way puzzles by the average-case hardness of a uncomputable problem. This brings to the quantum setting a recent line of work that characterizes classical cryptography with the average-case hardness of a meta-complexity problem, initiated by Liu and Pass. Moreover, since the average-case hardness of Kolmogorov complexity over classically polynomial-time samplable distributions characterizes one-way functions, this result poses one-way puzzles as a natural generalization of one-way functions to the quantum setting. Furthermore, our equivalence goes through probability estimation, giving us the additional equivalence that one-way puzzles exist if and only if there is a quantum samplable distribution over which probability estimation is hard. We also observe that the oracle worlds of defined by Kretschmer et. al. rule out any relativizing characterization of one-way puzzles by the hardness of a problem in NP or QMA, which means that it may not be possible with current techniques to characterize one-way puzzles with another meta-complexity problem.
Fast State Restoration in LLM Serving with HCache
Venue: arXiv cs.DC 2024
Year: 2024

The growing complexity of LLM usage today, e.g., multi-round conversation and retrieval-augmented generation (RAG), makes contextual states (i.e., KV cache) reusable across user requests. Given the capacity constraints of GPU memory, only a limited number of contexts can be cached on GPU for reusing. Existing inference systems typically evict part of the KV cache and restore it by recomputing it from the original tokens or offloading it to host storage for later retrieval, both of which introduce substantial computational or I/O overheads. We propose HCache, a novel LLM state restoration method. Its key idea is to restore LLM states from intermediate activations and thus utilize computational and I/O resources with low overhead. We enhance HCache with two techniques, including i) a bubble-free restoration scheduler that integrates resource-complementary methods to optimize the balance between computation and IO tasks; and ii) a chunk-based storage manager to address the layout mismatch issue (i.e., layer-before-token saving versus token-before-layer restoration). Our evaluations, conducted using real-world tasks, show that HCache reduces the TTFT by up to 1.93X compared to KV offload while consuming 1.92-2.40X less storage space; compared to token recomputation, HCache achieves up to 5.73X reduction in TTFT.
A Semantic Model for Physical Layer Deception
Venue: arXiv cs.IT 2024
Year: 2024

Physical layer deception (PLD) is a novel security mechanism that combines physical layer security (PLS) with deception technologies to actively defend against eavesdroppers. In this paper, we establish a novel semantic model for PLD that evaluates its performance in terms of semantic distortion. By analyzing semantic distortion at varying levels of knowledge on the receiver's part regarding the key, we derive the receiver's optimal decryption strategy, and consequently, the transmitter's optimal deception strategy. The proposed semantic model provides a more generic understanding of the PLD approach independent from coding or multiplexing schemes, and allows for efficient real-time adaptation to fading channels.
HE-Nav: A High-Performance and Efficient Navigation System for Aerial-Ground Robots in Cluttered Environments
Venue: arXiv cs.RO 2024
Year: 2024

Existing AGR navigation systems have advanced in lightly occluded scenarios (e.g., buildings) by employing 3D semantic scene completion networks for voxel occupancy prediction and constructing Euclidean Signed Distance Field (ESDF) maps for collision-free path planning. However, these systems exhibit suboptimal performance and efficiency in cluttered environments with severe occlusions (e.g., dense forests or tall walls), due to limitations arising from perception networks' low prediction accuracy and path planners' high computational overhead. In this paper, we present HE-Nav, the first high-performance and efficient navigation system tailored for AGRs operating in cluttered environments. The perception module utilizes a lightweight semantic scene completion network (LBSCNet), guided by a bird's eye view (BEV) feature fusion and enhanced by an exquisitely designed SCB-Fusion module and attention mechanism. This enables real-time and efficient obstacle prediction in cluttered areas, generating a complete local map. Building upon this completed map, our novel AG-Planner employs the energy-efficient kinodynamic A* search algorithm to guarantee planning is energy-saving. Subsequent trajectory optimization processes yield safe, smooth, dynamically feasible and ESDF-free aerial-ground hybrid paths. Extensive experiments demonstrate that HE-Nav achieved 7x energy savings in real-world situations while maintaining planning success rates of 98% in simulation scenarios. Code and video are available on our project page: https://jmwang0117.github.io/HE-Nav/.
Enhancing Job Interview Preparation Through Immersive Experiences Using Photorealistic, AI-powered Metahuman Avatars
Venue: arXiv cs.HC 2024
Year: 2024

This study will investigate the user experience while interacting with highly photorealistic virtual job interviewer avatars in Virtual Reality (VR), Augmented Reality (AR), and on a 2D screen. Having a precise speech recognition mechanism, our virtual character performs a mock-up software engineering job interview to adequately immerse the user in a life-like scenario. To evaluate the efficiency of our system, we measure factors such as the provoked level of anxiety, social presence, self-esteem, and intrinsic motivation. This research is a work in progress with a prospective within-subject user study including approximately 40 participants. All users will engage with three job interview conditions (VR, AR, and desktop) and provide their feedback. Additionally, users' bio-physical responses will be collected using a biosensor to measure the level of anxiety during the job interview.
A Seesaw Model Attack Algorithm for Distributed Learning
Venue: arXiv cs.DC 2024
Year: 2024

We investigate the Byzantine attack problem within the context of model training in distributed learning systems. While ensuring the convergence of current model training processes, common solvers (e.g. SGD, Adam, RMSProp, etc.) can be easily compromised by malicious nodes in these systems. Consequently, the training process may either converge slowly or even diverge. To develop effective secure distributed learning solvers, it is crucial to first examine attack methods to assess the robustness of these solvers. In this work, we contribute to the design of attack strategies by initially highlighting the limitations of finite-norm attacks. We then introduce the seesaw attack, which has been demonstrated to be more effective than the finite-norm attack. Through numerical experiments, we evaluate the efficacy of the seesaw attack across various gradient aggregation rules.
Unlocking Potential: Integrating Multihop, CRC, and GRAND for Wireless 5G-Beyond/6G Networks
Venue: arXiv cs.IT 2024
Year: 2024

As future wireless networks move towards millimeter wave (mmWave) and terahertz (THz) frequencies for 6G, multihop transmission using Integrated Access Backhaul (IABs) and Network-Controlled Repeaters (NCRs) will be highly essential to overcome coverage limitations. This paper examines the use of Guessing Random Additive Noise (GRAND) decoding for multihop transmissions in 3GPP networks. We explore two scenarios: one where only the destination uses GRAND decoding, and another where both relays and the destination leverage it. Interestingly, in the latter scenario, the Bit Error Rate (BER) curves for all hop counts intersect at a specific Signal-to-Noise Ratio (SNR), which we term the GRAND barrier. This finding offers valuable insights for future research and 3GPP standard development. Simulations confirm the effectiveness of GRAND in improving communication speed and quality, contributing to the robustness and interconnectivity of future wireless systems, particularly relevant for the migration towards mmWave and THz bands in 6G networks. Finally, we investigate the integration of multihop transmission, CRC detection, and GRAND decoding within 3GPP networks, demonstrating their potential to overcome coverage limitations and enhance overall network performance.
Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates
Venue: arXiv cs.CL 2024
Year: 2024

Fine-tuning large language models (LLMs) on instruction datasets is a common way to improve their generative capabilities. However, instruction datasets can be expensive and time-consuming to manually curate, and while LLM-generated data is less labor-intensive, it may violate user privacy agreements or terms of service of LLM providers. Therefore, we seek a way of constructing instruction datasets with samples that are not generated by humans or LLMs but still improve LLM generative capabilities. In this work, we introduce Cookbook, a framework that programmatically generates training data consisting of simple patterns over random tokens, resulting in a scalable, cost-effective approach that avoids legal and privacy issues. First, Cookbook uses a template -- a data generating Python function -- to produce training data that encourages the model to learn an explicit pattern-based rule that corresponds to a desired task. We find that fine-tuning on Cookbook-generated data is able to improve performance on its corresponding task by up to 52.7 accuracy points. Second, since instruction datasets improve performance on multiple downstream tasks simultaneously, Cookbook algorithmically learns how to mix data from various templates to optimize performance on multiple tasks. On the standard multi-task GPT4ALL evaluation suite, Mistral-7B fine-tuned using a Cookbook-generated dataset attains the best accuracy on average compared to other 7B parameter instruction-tuned models and is the best performing model on 3 out of 8 tasks. Finally, we analyze when and why Cookbook improves performance and present a metric that allows us to verify that the improvement is largely explained by the model's generations adhering better to template rules.
SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning
Venue: arXiv cs.LG 2024
Year: 2024

We introduce a novel anchor-free contrastive learning (AFCL) method leveraging our proposed Similarity-Orthogonality (SimO) loss. Our approach minimizes a semi-metric discriminative loss function that simultaneously optimizes two key objectives: reducing the distance and orthogonality between embeddings of similar inputs while maximizing these metrics for dissimilar inputs, facilitating more fine-grained contrastive learning. The AFCL method, powered by SimO loss, creates a fiber bundle topological structure in the embedding space, forming class-specific, internally cohesive yet orthogonal neighborhoods. We validate the efficacy of our method on the CIFAR-10 dataset, providing visualizations that demonstrate the impact of SimO loss on the embedding space. Our results illustrate the formation of distinct, orthogonal class neighborhoods, showcasing the method's ability to create well-structured embeddings that balance class separation with intra-class variability. This work opens new avenues for understanding and leveraging the geometric properties of learned representations in various machine learning tasks.
DiffuseReg: Denoising Diffusion Model for Obtaining Deformation Fields in Unsupervised Deformable Image Registration
Venue: arXiv cs.CV 2024
Year: 2024

Deformable image registration aims to precisely align medical images from different modalities or times. Traditional deep learning methods, while effective, often lack interpretability, real-time observability and adjustment capacity during registration inference. Denoising diffusion models present an alternative by reformulating registration as iterative image denoising. However, existing diffusion registration approaches do not fully harness capabilities, neglecting the critical sampling phase that enables continuous observability during the inference. Hence, we introduce DiffuseReg, an innovative diffusion-based method that denoises deformation fields instead of images for improved transparency. We also propose a novel denoising network upon Swin Transformer, which better integrates moving and fixed images with diffusion time step throughout the denoising process. Furthermore, we enhance control over the denoising registration process with a novel similarity consistency regularization. Experiments on ACDC datasets demonstrate DiffuseReg outperforms existing diffusion registration methods by 1.32 in Dice score. The sampling process in DiffuseReg enables real-time output observability and adjustment unmatched by previous deep models.
GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting
Venue: arXiv cs.CV 2024
Year: 2024

Diffusion-based 2D virtual try-on (VTON) techniques have recently demonstrated strong performance, while the development of 3D VTON has largely lagged behind. Despite recent advances in text-guided 3D scene editing, integrating 2D VTON into these pipelines to achieve vivid 3D VTON remains challenging. The reasons are twofold. First, text prompts cannot provide sufficient details in describing clothing. Second, 2D VTON results generated from different viewpoints of the same 3D scene lack coherence and spatial relationships, hence frequently leading to appearance inconsistencies and geometric distortions. To resolve these problems, we introduce an image-prompted 3D VTON method (dubbed GS-VTON) which, by leveraging 3D Gaussian Splatting (3DGS) as the 3D representation, enables the transfer of pre-trained knowledge from 2D VTON models to 3D while improving cross-view consistency. (1) Specifically, we propose a personalized diffusion model that utilizes low-rank adaptation (LoRA) fine-tuning to incorporate personalized information into pre-trained 2D VTON models. To achieve effective LoRA training, we introduce a reference-driven image editing approach that enables the simultaneous editing of multi-view images while ensuring consistency. (2) Furthermore, we propose a persona-aware 3DGS editing framework to facilitate effective editing while maintaining consistent cross-view appearance and high-quality 3D geometry. (3) Additionally, we have established a new 3D VTON benchmark, 3D-VTONBench, which facilitates comprehensive qualitative and quantitative 3D VTON evaluations. Through extensive experiments and comparative analyses with existing methods, the proposed \OM has demonstrated superior fidelity and advanced editing capabilities, affirming its effectiveness for 3D VTON.
Psychometrics for Hypnopaedia-Aware Machinery via Chaotic Projection of Artificial Mental Imagery
Venue: arXiv cs.CR 2024
Year: 2024

Neural backdoors represent insidious cybersecurity loopholes that render learning machinery vulnerable to unauthorised manipulations, potentially enabling the weaponisation of artificial intelligence with catastrophic consequences. A backdoor attack involves the clandestine infiltration of a trigger during the learning process, metaphorically analogous to hypnopaedia, where ideas are implanted into a subject's subconscious mind under the state of hypnosis or unconsciousness. When activated by a sensory stimulus, the trigger evokes conditioned reflex that directs a machine to mount a predetermined response. In this study, we propose a cybernetic framework for constant surveillance of backdoors threats, driven by the dynamic nature of untrustworthy data sources. We develop a self-aware unlearning mechanism to autonomously detach a machine's behaviour from the backdoor trigger. Through reverse engineering and statistical inference, we detect deceptive patterns and estimate the likelihood of backdoor infection. We employ model inversion to elicit artificial mental imagery, using stochastic processes to disrupt optimisation pathways and avoid convergent but potentially flawed patterns. This is followed by hypothesis analysis, which estimates the likelihood of each potentially malicious pattern being the true trigger and infers the probability of infection. The primary objective of this study is to maintain a stable state of equilibrium between knowledge fidelity and backdoor vulnerability.
CaLMFlow: Volterra Flow Matching using Causal Language Models
Venue: arXiv cs.LG 2024
Year: 2024

We introduce CaLMFlow (Causal Language Models for Flow Matching), a novel framework that casts flow matching as a Volterra integral equation (VIE), leveraging the power of large language models (LLMs) for continuous data generation. CaLMFlow enables the direct application of LLMs to learn complex flows by formulating flow matching as a sequence modeling task, bridging discrete language modeling and continuous generative modeling. Our method implements tokenization across space and time, thereby solving a VIE over these domains. This approach enables efficient handling of high-dimensional data and outperforms ODE solver-dependent methods like conditional flow matching (CFM). We demonstrate CaLMFlow's effectiveness on synthetic and real-world data, including single-cell perturbation response prediction, showcasing its ability to incorporate textual context and generalize to unseen conditions. Our results highlight LLM-driven flow matching as a promising paradigm in generative modeling, offering improved scalability, flexibility, and context-awareness.
Cyber Risk Taxonomies: Statistical Analysis of Cybersecurity Risk Classifications
Venue: arXiv cs.CR 2024
Year: 2024

Cyber risk classifications are widely used in the modeling of cyber event distributions, yet their effectiveness in out of sample forecasting performance remains underexplored. In this paper, we analyse the most commonly used classifications and argue in favour of switching the attention from goodness-of-fit and in-sample predictive performance, to focusing on the out-of sample forecasting performance. We use a rolling window analysis, to compare cyber risk distribution forecasts via threshold weighted scoring functions. Our results indicate that business motivated cyber risk classifications appear to be too restrictive and not flexible enough to capture the heterogeneity of cyber risk events. We investigate how dynamic and impact-based cyber risk classifiers seem to be better suited in forecasting future cyber risk losses than the other considered classifications. These findings suggest that cyber risk types provide limited forecasting ability concerning cyber event severity distribution, and cyber insurance ratemakers should utilize cyber risk types only when modeling the cyber event frequency distribution. Our study offers valuable insights for decision-makers and policymakers alike, contributing to the advancement of scientific knowledge in the field of cyber risk management.
Over-the-Air Federated Learning in Cell-Free MIMO with Long-term Power Constraint
Venue: arXiv cs.LG 2024
Year: 2024

Wireless networks supporting artificial intelligence have gained significant attention, with Over-the-Air Federated Learning emerging as a key application due to its unique transmission and distributed computing characteristics. This paper derives error bounds for Over-the-Air Federated Learning in a Cell-free MIMO system and formulates an optimization problem to minimize optimality gap via joint optimization of power control and beamforming. We introduce the MOP-LOFPC algorithm, which employs Lyapunov optimization to decouple long-term constraints across rounds while requiring only causal channel state information. Experimental results demonstrate that MOP-LOFPC achieves a better and more flexible trade-off between the model's training loss and adherence to long-term power constraints compared to existing baselines.
A Predictive and Optimization Approach for Enhanced Urban Mobility Using Spatiotemporal Data
Venue: arXiv cs.LG 2024
Year: 2024

In modern urban centers, effective transportation management poses a significant challenge, with traffic jams and inconsistent travel durations greatly affecting commuters and logistics operations. This study introduces a novel method for enhancing urban mobility by combining machine learning algorithms with live traffic information. We developed predictive models for journey time and congestion analysis using data from New York City's yellow taxi trips. The research employed a spatiotemporal analysis framework to identify traffic trends and implemented real-time route optimization using the GraphHopper API. This system determines the most efficient paths based on current conditions, adapting to changes in traffic flow. The methodology utilizes Spark MLlib for predictive modeling and Spark Streaming for processing data in real-time. By integrating historical data analysis with current traffic inputs, our system shows notable enhancements in both travel time forecasts and route optimization, demonstrating its potential for widespread application in major urban areas. This research contributes to ongoing efforts aimed at reducing urban congestion and improving transportation efficiency through advanced data-driven methods.