OpenRaiser/Intern-Atlas huggingface dataset:
Uncertainty-Aware Decision-Making and Planning for Autonomous Forced Merging
Venue: arXiv cs.RO 2024
Year: 2024

In this paper, we develop an uncertainty-aware decision-making and motion-planning method for an autonomous ego vehicle in forced merging scenarios, considering the motion uncertainty of surrounding vehicles. The method dynamically captures the uncertainty of surrounding vehicles by online estimation of their acceleration bounds, enabling a reactive but rapid understanding of the uncertainty characteristics of the surrounding vehicles. By leveraging these estimated bounds, a non-conservative forward occupancy of surrounding vehicles is predicted over a horizon, which is incorporated in both the decision-making process and the motion-planning strategy, to enhance the resilience and safety of the planned reference trajectory. The method successfully fulfills the tasks in challenging forced merging scenarios, and the properties are illustrated by comparison with several alternative approaches.
Props for Machine-Learning Security
Venue: arXiv cs.CR 2024
Year: 2024

We propose protected pipelines or props for short, a new approach for authenticated, privacy-preserving access to deep-web data for machine learning (ML). By permitting secure use of vast sources of deep-web data, props address the systemic bottleneck of limited high-quality training data in ML development. Props also enable privacy-preserving and trustworthy forms of inference, allowing for safe use of sensitive data in ML applications. Props are practically realizable today by leveraging privacy-preserving oracle systems initially developed for blockchain applications.
On Sparsest Cut and Conductance in Directed Polymatroidal Networks
Venue: arXiv cs.DS 2024
Year: 2024

We consider algorithms and spectral bounds for sparsest cut and conductance in directed polymatrodal networks. This is motivated by recent work on submodular hypergraphs \cite{Yoshida19,LiM18,ChenOT23,Veldt23} and previous work on multicommodity flows and cuts in polymatrodial networks \cite{ChekuriKRV15}. We obtain three results. First, we obtain an $O(\sqrt{\log n})$-approximation for sparsest cut and point out how this generalizes the result in \cite{ChenOT23}. Second, we consider the symmetric version of conductance and obtain an $O(\sqrt{OPT \log r})$ approximation where $r$ is the maximum degree and we point out how this generalizes previous work on vertex expansion in graphs. Third, we prove a non-constructive Cheeger like inequality that generalizes previous work on hypergraphs. We provide a unified treatment via line-embeddings which were shown to be effective for submodular cuts in \cite{ChekuriKRV15}.
Making Urban Art Accessible: Current Art Access Techniques, Design Considerations, and the Role of AI
Venue: arXiv cs.HC 2024
Year: 2024

Public artwork, from vibrant wall murals to captivating sculptures, can enhance the aesthetic of urban spaces, foster a sense of community and cultural identity, and help attract visitors. Despite its benefits, most public art is visual, making it often inaccessible to blind and low vision (BLV) people. In this workshop paper, we first draw on art literature to help define the space of public art, identify key differences with curated art shown in museums or galleries, and discuss implications for accessibility. We then enumerate how existing art accessibility techniques may (or may not) transfer to urban art spaces. We close by presenting future research directions and reflecting on the growing role of AI in making art accessible.
R^3AG: First Workshop on Refined and Reliable Retrieval Augmented Generation
Venue: arXiv cs.IR 2024
Year: 2024

Retrieval-augmented generation (RAG) has gained wide attention as the key component to improve generative models with external knowledge augmentation from information retrieval. It has shown great prominence in enhancing the functionality and performance of large language model (LLM)-based applications. However, with the comprehensive application of RAG, more and more problems and limitations have been identified, thus urgently requiring further fundamental exploration to improve current RAG frameworks. This workshop aims to explore in depth how to conduct refined and reliable RAG for downstream AI tasks. To this end, we propose to organize the first R3AG workshop at SIGIR-AP 2024 to call for participants to re-examine and formulate the basic principles and practical implementation of refined and reliable RAG. The workshop serves as a platform for both academia and industry researchers to conduct discussions, share insights, and foster research to build the next generation of RAG systems. Participants will engage in discussions and presentations focusing on fundamental challenges, cutting-edge research, and potential pathways to improve RAG. At the end of the workshop, we aim to have a clearer understanding of how to improve the reliability and applicability of RAG with more robust information retrieval and language generation.
ReACKed QUICer: Measuring the Performance of Instant Acknowledgments in QUIC Handshakes
Venue: arXiv cs.NI 2024
Year: 2024

In this paper, we present a detailed performance analysis of QUIC instant ACK, a standard-compliant approach to reduce waiting times during the QUIC connection setup in common CDN deployments. To understand the root causes of the performance properties, we combine numerical analysis and the emulation of eight QUIC implementations using the QUIC Interop Runner. Our experiments comprehensively cover packet loss and non-loss scenarios, different round trip times, and TLS certificate sizes. To clarify instant ACK deployments in the wild, we conduct active measurements of 1M popular domain names. For almost all domain names under control of Cloudflare, Cloudflare uses instant ACK, which in fact improves performance. We also find, however, that instant ACK may lead to unnecessary retransmissions or longer waiting times under some network conditions, raising awareness of drawbacks of instant ACK in the future.
Zero-Trust Network Access (ZTNA)
Venue: arXiv cs.CR 2024
Year: 2024

Zero-Trust Network Access (ZTNA) marks a significant shift in network security by adopting a "never trust, always verify" approach. This work provides an in-depth analysis of ZTNA, offering a comprehensive framework for understanding its principles, architectures, and applications. We discuss its role in securing modern, complex network environments, which include cloud platforms, Internet of Things (IoT) devices, and hybrid enterprise networks. Our objective is to create a key resource for researchers and practitioners by reviewing critical methodologies, analyzing current implementations, and highlighting open challenges and research directions.
Plastic Learning with Deep Fourier Features
Venue: arXiv cs.LG 2024
Year: 2024

Deep neural networks can struggle to learn continually in the face of non-stationarity. This phenomenon is known as loss of plasticity. In this paper, we identify underlying principles that lead to plastic algorithms. In particular, we provide theoretical results showing that linear function approximation, as well as a special case of deep linear networks, do not suffer from loss of plasticity. We then propose deep Fourier features, which are the concatenation of a sine and cosine in every layer, and we show that this combination provides a dynamic balance between the trainability obtained through linearity and the effectiveness obtained through the nonlinearity of neural networks. Deep networks composed entirely of deep Fourier features are highly trainable and sustain their trainability over the course of learning. Our empirical results show that continual learning performance can be drastically improved by replacing ReLU activations with deep Fourier features. These results hold for different continual learning scenarios (e.g., label noise, class incremental learning, pixel permutations) on all major supervised learning datasets used for continual learning research, such as CIFAR10, CIFAR100, and tiny-ImageNet.
A Comparative Study of Multiple Deep Learning Algorithms for Efficient Localization of Bone Joints in the Upper Limbs of Human Body
Venue: arXiv cs.CV 2024
Year: 2024

This paper addresses the medical imaging problem of joint detection in the upper limbs, viz. elbow, shoulder, wrist and finger joints. Localization of joints from X-Ray and Computerized Tomography (CT) scans is an essential step for the assessment of various bone-related medical conditions like Osteoarthritis, Rheumatoid Arthritis, and can even be used for automated bone fracture detection. Automated joint localization also detects the corresponding bones and can serve as input to deep learning-based models used for the computerized diagnosis of the aforementioned medical disorders. This in-creases the accuracy of prediction and aids the radiologists with analyzing the scans, which is quite a complex and exhausting task. This paper provides a detailed comparative study between diverse Deep Learning (DL) models - YOLOv3, YOLOv7, EfficientDet and CenterNet in multiple bone joint detections in the upper limbs of the human body. The research analyses the performance of different DL models, mathematically, graphically and visually. These models are trained and tested on a portion of the openly available MURA (musculoskeletal radiographs) dataset. The study found that the best Mean Average Precision (mAP at 0.5:0.95) values of YOLOv3, YOLOv7, EfficientDet and CenterNet are 35.3, 48.3, 46.5 and 45.9 respectively. Besides, it has been found YOLOv7 performed the best for accurately predicting the bounding boxes while YOLOv3 performed the worst in the Visual Analysis test. Code available at https://github.com/Sohambasu07/BoneJointsLocalization
NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

The performance of neural networks improves when more parameters are used. However, the model sizes are constrained by the available on-device memory during training and inference. Although applying techniques like quantization can alleviate the constraint, they suffer from performance degradation. In this work, we introduce NeuZip, a new weight compression scheme based on the entropy of floating-point numbers in neural networks. With NeuZip, we are able to achieve memory-efficient training and inference without sacrificing performance. Notably, we significantly reduce the memory footprint of training a Llama-3 8B model from 31GB to less than 16GB, while keeping the training dynamics fully unchanged. In inference, our method can reduce memory usage by more than half while maintaining near-lossless performance. Our code is publicly available.
Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradient
Venue: arXiv cs.LG 2024
Year: 2024

The development of sophisticated models for video-to-video synthesis has been facilitated by recent advances in deep reinforcement learning and generative adversarial networks (GANs). In this paper, we propose RL-V2V-GAN, a new deep neural network approach based on reinforcement learning for unsupervised conditional video-to-video synthesis. While preserving the unique style of the source video domain, our approach aims to learn a mapping from a source video domain to a target video domain. We train the model using policy gradient and employ ConvLSTM layers to capture the spatial and temporal information by designing a fine-grained GAN architecture and incorporating spatio-temporal adversarial goals. The adversarial losses aid in content translation while preserving style. Unlike traditional video-to-video synthesis methods requiring paired inputs, our proposed approach is more general because it does not require paired inputs. Thus, when dealing with limited videos in the target domain, i.e., few-shot learning, it is particularly effective. Our experiments show that RL-V2V-GAN can produce temporally coherent video results. These results highlight the potential of our approach for further advances in video-to-video synthesis.
Comparative Simulation of Phishing Attacks on a Critical Information Infrastructure Organization: An Empirical Study
Venue: arXiv cs.CR 2024
Year: 2024

Nowadays, cybersecurity is crucial. Therefore, cybersecurity awareness should be a concern for businesses, particularly critical infrastructure organizations. The results of this study, using simulated phishing attacks, indicate that in the first attempt, workers of a Thai railway firm received a phony email purporting to inform recipients of a special deal from a reputable retailer of IT equipment. The findings showed that 10.9% of the 735 workers fell for the scam. This demonstrates a good level of awareness regarding cyber dangers. The workers who were duped by the initial attack received awareness training. Next, a second attempt was carried out. This time, the strategy was for the workers to change their passwords through an email notification from the fake IT staff. According to the findings, 1.4% of the workers fell victim to both attacks (different email content), and a further 8.0% of the workers who did not fall victim to the first attack were deceived. Furthermore, after the statistical analysis, it was confirmed that there is a difference in the relationship between the workers and the two phishing attack simulations using different content. As a result, this study has demonstrated that different types of content can affect levels of awareness.
A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation
Venue: arXiv cs.CL 2024
Year: 2024

Recently, research on open domain dialogue systems have attracted extensive interests of academic and industrial researchers. The goal of an open domain dialogue system is to imitate humans in conversations. Previous works on single turn conversation generation have greatly promoted the research of open domain dialogue systems. However, understanding multiple single turn conversations is not equal to the understanding of multi turn dialogue due to the coherent and context dependent properties of human dialogue. Therefore, in open domain multi turn dialogue generation, it is essential to modeling the contextual semantics of the dialogue history, rather than only according to the last utterance. Previous research had verified the effectiveness of the hierarchical recurrent encoder-decoder framework on open domain multi turn dialogue generation. However, using RNN-based model to hierarchically encoding the utterances to obtain the representation of dialogue history still face the problem of a vanishing gradient. To address this issue, in this paper, we proposed a static and dynamic attention-based approach to model the dialogue history and then generate open domain multi turn dialogue responses. Experimental results on Ubuntu and Opensubtitles datasets verify the effectiveness of the proposed static and dynamic attention-based approach on automatic and human evaluation metrics in various experimental settings. Meanwhile, we also empirically verify the performance of combining the static and dynamic attentions on open domain multi turn dialogue generation.
zGAN: An Outlier-focused Generative Adversarial Network For Realistic Synthetic Data Generation
Venue: arXiv cs.LG 2024
Year: 2024

The phenomenon of "black swans" has posed a fundamental challenge to performance of classical machine learning models. The perceived rise in frequency of outlier conditions, especially in post-pandemic environment, has necessitated exploration of synthetic data as a complement to real data in model training. This article provides a general overview and experimental investigation of the zGAN model architecture developed for the purpose of generating synthetic tabular data with outlier characteristics. The model is put to test in binary classification environments and shows promising results on realistic synthetic data generation, as well as uplift capabilities vis-à-vis model performance. A distinctive feature of zGAN is its enhanced correlation capability between features in the generated data, replicating correlations of features in real training data. Furthermore, crucial is the ability of zGAN to generate outliers based on covariance of real data or synthetically generated covariances. This approach to outlier generation enables modeling of complex economic events and augmentation of outliers for tasks such as training predictive models and detecting, processing or removing outliers. Experiments and comparative analyses as part of this study were conducted on both private (credit risk in financial services) and public datasets.
Origami crawlers: exploring a single origami vertex for complex path navigation
Venue: arXiv cs.RO 2024
Year: 2024

The ancient art of origami, traditionally used to transform simple sheets into intricate objects, also holds potential for diverse engineering applications, such as shape morphing and robotics. In this study, we demonstrate that one of the most basic origami structures (i.e., a rigid, foldable degree-four vertex) can be engineered to create a crawler capable of navigating complex paths using only a single input. Through a combination of experimental studies and modeling, we show that modifying the geometry of a degree four vertex enables sheets to move either in a straight line or turn. Furthermore, we illustrate how leveraging the nonlinearities in folding allows the design of crawlers that can switch between moving straight and turning. Remarkably, these crawling modes can be controlled by adjusting the range of the actuation folding angle. Our study opens avenues for simple machines that can follow intricate trajectories with minimal actuation.
ADLM -- stega: A Universal Adaptive Token Selection Algorithm for Improving Steganographic Text Quality via Information Entropy
Venue: arXiv cs.CR 2024
Year: 2024

In the context of widespread global information sharing, information security and privacy protection have become focal points. Steganographic systems enhance information security by embedding confidential information into public carriers; however, existing generative text steganography methods face challenges in handling the long-tail distribution of candidate word pools, which impacts the imperceptibility of steganographic information. This paper proposes a quality control theory for steganographic text generation based on information entropy constraints, exploring the relationship between the imperceptibility of steganographic texts and information entropy. By controlling the information entropy of the candidate word pool within a specific range, we optimize the imperceptibility of the steganographic text. We establish upper and lower bounds for information entropy and introduce an adaptive truncation method to balance semantic coherence and lexical diversity. Experimental results demonstrate that reasonably controlling the candidate pool size and information entropy thresholds significantly enhances the quality and detection resistance of steganographic texts, showcasing broad application potential in the field of natural language processing.
CODES: Benchmarking Coupled ODE Surrogates
Venue: arXiv cs.LG 2024
Year: 2024

We introduce CODES, a benchmark for comprehensive evaluation of surrogate architectures for coupled ODE systems. Besides standard metrics like mean squared error (MSE) and inference time, CODES provides insights into surrogate behaviour across multiple dimensions like interpolation, extrapolation, sparse data, uncertainty quantification and gradient correlation. The benchmark emphasizes usability through features such as integrated parallel training, a web-based configuration generator, and pre-implemented baseline models and datasets. Extensive documentation ensures sustainability and provides the foundation for collaborative improvement. By offering a fair and multi-faceted comparison, CODES helps researchers select the most suitable surrogate for their specific dataset and application while deepening our understanding of surrogate learning behaviour.
Parameterized Approximation for Capacitated $d$-Hitting Set with Hard Capacities
Venue: arXiv cs.DS 2024
Year: 2024

The \textsc{Capacitated $d$-Hitting Set} problem involves a universe $U$ with a capacity function $\mathsf{cap}: U \rightarrow \mathbb{N}$ and a collection $\mathcal{A}$ of subsets of $U$, each of size at most $d$. The goal is to find a minimum subset $S \subseteq U$ and an assignment $ϕ: \mathcal{A} \rightarrow S$ such that for every $A \in \mathcal{A}$, $ϕ(A) \in A$, and for each $x \in U$, $|ϕ^{-1}(x)| \leq \mathsf{cap}(x)$. For $d=2$, this is known as \textsc{Capacitated Vertex Cover}. In the weighted variant, each element of $U$ has a positive integer weight, with the objective of finding a minimum-weight capacitated hitting set. Chuzhoy and Naor [SICOMP 2006] provided a factor-3 approximation for \textsc{Capacitated Vertex Cover} and showed that the weighted case lacks an $o(\log n)$-approximation unless $P=NP$. Kao and Wong [SODA 2017] later independently achieved a $d$-approximation for \textsc{Capacitated $d$-Hitting Set}, with no $d - ε$ improvements possible under the Unique Games Conjecture. Our main result is a parameterized approximation algorithm with runtime $\left(\frac{k}ε\right)^k 2^{k^{O(kd)}}(|U|+|\mathcal{A}|)^{O(1)}$ that either concludes no solution of size $\leq k$ exists or finds $S$ of size $\leq 4/3 \cdot k$ and weight at most $2+ε$ times the minimum weight for solutions of size $\leq k$. We further show that no FPT-approximation with factor $c > 1$ exists for unweighted \textsc{Capacitated $d$-Hitting Set} with $d \geq 3$, nor with factor $2 - ε$ for the weighted version, assuming the Exponential Time Hypothesis. These results extend to \textsc{Capacitated Vertex Cover} in multigraphs. Additionally, a variant of multi-dimensional \textsc{Knapsack} is shown hard to FPT-approximate within $2 - ε$.
Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems
Venue: arXiv cs.LG 2024
Year: 2024

Deep learning techniques have shown promise in many domain applications. This paper proposes a novel deep reservoir computing framework, termed deep recurrent stochastic configuration network (DeepRSCN) for modelling nonlinear dynamic systems. DeepRSCNs are incrementally constructed, with all reservoir nodes directly linked to the final output. The random parameters are assigned in the light of a supervisory mechanism, ensuring the universal approximation property of the built model. The output weights are updated online using the projection algorithm to handle the unknown dynamics. Given a set of training samples, DeepRSCNs can quickly generate learning representations, which consist of random basis functions with cascaded input and readout weights. Experimental results over a time series prediction, a nonlinear system identification problem, and two industrial data predictive analyses demonstrate that the proposed DeepRSCN outperforms the single-layer network in terms of modelling efficiency, learning capability, and generalization performance.