OpenRaiser/Intern-Atlas huggingface dataset:
Fuzzy Fault Trees Formalized
Venue: arXiv cs.AI 2024
Year: 2024

Fault tree analysis is a vital method of assessing safety risks. It helps to identify potential causes of accidents, assess their likelihood and severity, and suggest preventive measures. Quantitative analysis of fault trees is often done via the dependability metrics that compute the system's failure behaviour over time. However, the lack of precise data is a major obstacle to quantitative analysis, and so to reliability analysis. Fuzzy logic is a popular framework for dealing with ambiguous values and has applications in many domains. A number of fuzzy approaches have been proposed to fault tree analysis, but -- to the best of our knowledge -- none of them provide rigorous definitions or algorithms for computing fuzzy unreliability values. In this paper, we define a rigorous framework for fuzzy unreliability values. In addition, we provide a bottom-up algorithm to efficiently calculate fuzzy reliability for a system. The algorithm incorporates the concept of $α$-cuts method. That is, performing binary algebraic operations on intervals on horizontally discretised $α$-cut representations of fuzzy numbers. The method preserves the nonlinearity of fuzzy unreliability. Finally, we illustrate the results obtained from two case studies.
Multi-Objective Optimization Using Adaptive Distributed Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

The Intelligent Transportation System (ITS) environment is known to be dynamic and distributed, where participants (vehicle users, operators, etc.) have multiple, changing and possibly conflicting objectives. Although Reinforcement Learning (RL) algorithms are commonly applied to optimize ITS applications such as resource management and offloading, most RL algorithms focus on single objectives. In many situations, converting a multi-objective problem into a single-objective one is impossible, intractable or insufficient, making such RL algorithms inapplicable. We propose a multi-objective, multi-agent reinforcement learning (MARL) algorithm with high learning efficiency and low computational requirements, which automatically triggers adaptive few-shot learning in a dynamic, distributed and noisy environment with sparse and delayed reward. We test our algorithm in an ITS environment with edge cloud computing. Empirical results show that the algorithm is quick to adapt to new environments and performs better in all individual and system metrics compared to the state-of-the-art benchmark. Our algorithm also addresses various practical concerns with its modularized and asynchronous online training method. In addition to the cloud simulation, we test our algorithm on a single-board computer and show that it can make inference in 6 milliseconds.
Timed Strategies for Real-Time Rewrite Theories
Venue: arXiv cs.LO 2024
Year: 2024

In this paper we propose a language for conveniently defining a wide range of execution strategies for real-time rewrite theories, and provide Maude-strategy-implemented versions of most Real-Time Maude analysis methods, albeit with user-defined discrete and timed strategies. We also identify a new time sampling strategy that should provide both efficient and exhaustive analysis for many distributed real-time systems. We exemplify the use of our language and its analyses on a simple round trip time protocol, and compare the performance of standard Maude search with our strategy-implemented reachability analyses on the CASH scheduling algorithm benchmark.
On sampling diluted Spin Glasses using Glauber dynamics
Venue: arXiv cs.DM 2024
Year: 2024

Spin-glasses are Gibbs distributions that have been studied in CS for many decades. Recently, they have gained renewed attention as they emerge naturally in learning, inference, optimisation etc. We consider the Edwards-Anderson (EA) spin-glass distribution at inverse temperature $β$ when the underlying graph is an instance of $G(n,d/n)$. This is the random graph on $n$ vertices where each edge appears independently with probability $d/n$ and $d=Θ(1)$. We study the problem of approximate sampling from this distribution using Glauber dynamics. For a range of $β$ that depends on $d$ and for typical instances of the EA model on $G(n,d/n)$, we show that the corresponding Glauber dynamics exhibits mixing time $O(n^{2+\frac{3}{\log^2 d}})$. The range of $β$ for which we obtain our rapid-mixing results correspond to the expected influence being $<1/d$; we conjecture that this is the best possible. Unlike the mean-field spin-glasses, where the problem has been studied before, the diluted case has not. We utilise the well-known path-coupling technique. In the standard Glauber dynamics on $G(n,d/n)$, one has to deal with the so-called effect of high degree vertices. Here, rather than considering degrees, it is more natural to use a different measure on the vertices called aggregate influence. We build on the block-construction approach proposed by [Dyer et al. 2006] to circumvent the problem of high-degree vertices. Specifically, we first establish rapid mixing for an appropriately defined block-dynamics. We design this dynamics such that vertices of large aggregate influence are placed deep inside their blocks. Then, we obtain rapid mixing for the Glauber dynamics utilising a comparison argument.
Collision-Free Platooning of Mobile Robots through a Set-Theoretic Predictive Control Approach
Venue: arXiv cs.RO 2024
Year: 2024

This paper proposes a control solution to achieve collision-free platooning control of input-constrained mobile robots. The platooning policy is based on a leader-follower approach where the leader tracks a reference trajectory while followers track the leader's pose with an inter-agent delay. First, the leader and the follower kinematic models are feedback linearized and the platoon's error dynamics and input constraints characterized. Then, a set-theoretic model predictive control strategy is proposed to address the platooning trajectory tracking control problem. An ad-hoc collision avoidance policy is also proposed to guarantee collision avoidance amongst the agents. Finally, the effectiveness of the proposed control architecture is validated through experiments performed on a formation of Khepera IV differential drive robots
PathM3: A Multimodal Multi-Task Multiple Instance Learning Framework for Whole Slide Image Classification and Captioning
Venue: arXiv cs.CV 2024
Year: 2024

In the field of computational histopathology, both whole slide images (WSIs) and diagnostic captions provide valuable insights for making diagnostic decisions. However, aligning WSIs with diagnostic captions presents a significant challenge. This difficulty arises from two main factors: 1) Gigapixel WSIs are unsuitable for direct input into deep learning models, and the redundancy and correlation among the patches demand more attention; and 2) Authentic WSI diagnostic captions are extremely limited, making it difficult to train an effective model. To overcome these obstacles, we present PathM3, a multimodal, multi-task, multiple instance learning (MIL) framework for WSI classification and captioning. PathM3 adapts a query-based transformer to effectively align WSIs with diagnostic captions. Given that histopathology visual patterns are redundantly distributed across WSIs, we aggregate each patch feature with MIL method that considers the correlations among instances. Furthermore, our PathM3 overcomes data scarcity in WSI-level captions by leveraging limited WSI diagnostic caption data in the manner of multi-task joint learning. Extensive experiments with improved classification accuracy and caption generation demonstrate the effectiveness of our method on both WSI classification and captioning task.
Improved bass model using sales proportional average for one condition of mono peak curves
Venue: arXiv cs.CE 2024
Year: 2024

"This study provides a modified Bass model to deal with trend curves for basic issues of relevance to individuals from all over the world, for which we collected 16 data sets from 2004 to 2022 and that are available on Google servers as "google trends". It was discovered that the Bass model did not forecast well for curves that have a mono peak with a sharp decrease to some level then have semi-stable with small decrement sales for a long time, thus a new parameter based on r1 and r2 (ratios of average sales) was introduced, which improved the model's prediction ability and provided better results. The model was also applied to a data set taken from the Kaggle website about a subscriber digital product offering for financial services that include newsletters, webinars, and investment recommendations. The data contain 508932 data points about the products sold during 2016-2022. Compared to the traditional Bass model, the modified model showed better results in dealing with this condition, as the expected curve shape was closer to real sales, and the sum of squares error (SSE) value was reduced to a ratio ranging between (36.35-79.3%). Therefore, the improved model can be relied upon in these conditions."
VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework
Venue: arXiv cs.CV 2024
Year: 2024

With the emergence of large language models (LLMs) and vision foundation models, how to combine the intelligence and capacity of these open-sourced or API-available models to achieve open-world visual perception remains an open question. In this paper, we introduce VisionGPT to consolidate and automate the integration of state-of-the-art foundation models, thereby facilitating vision-language understanding and the development of vision-oriented AI. VisionGPT builds upon a generalized multimodal framework that distinguishes itself through three key features: (1) utilizing LLMs (e.g., LLaMA-2) as the pivot to break down users' requests into detailed action proposals to call suitable foundation models; (2) integrating multi-source outputs from foundation models automatically and generating comprehensive responses for users; (3) adaptable to a wide range of applications such as text-conditioned image understanding/generation/editing and visual question answering. This paper outlines the architecture and capabilities of VisionGPT, demonstrating its potential to revolutionize the field of computer vision through enhanced efficiency, versatility, and generalization, and performance. Our code and models will be made publicly available. Keywords: VisionGPT, Open-world visual perception, Vision-language understanding, Large language model, and Foundation model
Towards a theory of model distillation
Venue: arXiv cs.LG 2024
Year: 2024

Distillation is the task of replacing a complicated machine learning model with a simpler model that approximates the original [BCNM06,HVD15]. Despite many practical applications, basic questions about the extent to which models can be distilled, and the runtime and amount of data needed to distill, remain largely open. To study these questions, we initiate a general theory of distillation, defining PAC-distillation in an analogous way to PAC-learning [Val84]. As applications of this theory: (1) we propose new algorithms to extract the knowledge stored in the trained weights of neural networks -- we show how to efficiently distill neural networks into succinct, explicit decision tree representations when possible by using the ``linear representation hypothesis''; and (2) we prove that distillation can be much cheaper than learning from scratch, and make progress on characterizing its complexity.
Distribution and Depth-Aware Transformers for 3D Human Mesh Recovery
Venue: arXiv cs.CV 2024
Year: 2024

Precise Human Mesh Recovery (HMR) with in-the-wild data is a formidable challenge and is often hindered by depth ambiguities and reduced precision. Existing works resort to either pose priors or multi-modal data such as multi-view or point cloud information, though their methods often overlook the valuable scene-depth information inherently present in a single image. Moreover, achieving robust HMR for out-of-distribution (OOD) data is exceedingly challenging due to inherent variations in pose, shape and depth. Consequently, understanding the underlying distribution becomes a vital subproblem in modeling human forms. Motivated by the need for unambiguous and robust human modeling, we introduce Distribution and depth-aware human mesh recovery (D2A-HMR), an end-to-end transformer architecture meticulously designed to minimize the disparity between distributions and incorporate scene-depth leveraging prior depth information. Our approach demonstrates superior performance in handling OOD data in certain scenarios while consistently achieving competitive results against state-of-the-art HMR methods on controlled datasets.
All-pay Auction Based Profit Maximization in End-to-End Computation Offloading System
Venue: arXiv cs.GT 2024
Year: 2024

Pricing is an important issue in mobile edge computing. How to appropriately determine the bid of end user (EU) is an incentive factor for edge cloud (EC) to offer service. In this letter, we propose an equilibrium pricing scheme based on the all-pay auction model in end-to-end collaboration environment, wherein all EUs can acquire the service at a lower price than the own value of the required resource. In addition, we propose a set allocation algorithm to divide all the bidders into different sets according to the price, and the EUs in each set get the service, which averts the case of getting no service due to the low price. Extensive simulation results demonstrate that the proposed scheme can effectively maximize the total profit of the edge offloading system, and guarantee all EUs can access the service.
Generalized Relevance Learning Grassmann Quantization
Venue: arXiv cs.CV 2024
Year: 2024

Due to advancements in digital cameras, it is easy to gather multiple images (or videos) from an object under different conditions. Therefore, image-set classification has attracted more attention, and different solutions were proposed to model them. A popular way to model image sets is subspaces, which form a manifold called the Grassmann manifold. In this contribution, we extend the application of Generalized Relevance Learning Vector Quantization to deal with Grassmann manifold. The proposed model returns a set of prototype subspaces and a relevance vector. While prototypes model typical behaviours within classes, the relevance factors specify the most discriminative principal vectors (or images) for the classification task. They both provide insights into the model's decisions by highlighting influential images and pixels for predictions. Moreover, due to learning prototypes, the model complexity of the new method during inference is independent of dataset size, unlike previous works. We applied it to several recognition tasks including handwritten digit recognition, face recognition, activity recognition, and object recognition. Experiments demonstrate that it outperforms previous works with lower complexity and can successfully model the variation, such as handwritten style or lighting conditions. Moreover, the presence of relevances makes the model robust to the selection of subspaces' dimensionality.
SAM-Lightening: A Lightweight Segment Anything Model with Dilated Flash Attention to Achieve 30 times Acceleration
Venue: arXiv cs.CV 2024
Year: 2024

Segment Anything Model (SAM) has garnered significant attention in segmentation tasks due to their zero-shot generalization ability. However, a broader application of SAMs to real-world practice has been restricted by their low inference speed and high computational memory demands, which mainly stem from the attention mechanism. Existing work concentrated on optimizing the encoder, yet has not adequately addressed the inefficiency of the attention mechanism itself, even when distilled to a smaller model, which thus leaves space for further improvement. In response, we introduce SAM-Lightening, a variant of SAM, that features a re-engineered attention mechanism, termed Dilated Flash Attention. It not only facilitates higher parallelism, enhancing processing efficiency but also retains compatibility with the existing FlashAttention. Correspondingly, we propose a progressive distillation to enable an efficient knowledge transfer from the vanilla SAM without costly training from scratch. Experiments on COCO and LVIS reveal that SAM-Lightening significantly outperforms the state-of-the-art methods in both run-time efficiency and segmentation accuracy. Specifically, it can achieve an inference speed of 7 milliseconds (ms) per image, for images of size 1024*1024 pixels, which is 30.1 times faster than the vanilla SAM and 2.1 times than the state-of-the-art. Moreover, it takes only 244MB memory, which is 3.5\% of the vanilla SAM. The code and weights are available at https://anonymous.4open.science/r/SAM-LIGHTENING-BC25/.
Noise Dimension of GAN: An Image Compression Perspective
Venue: arXiv cs.CV 2024
Year: 2024

Generative adversial network (GAN) is a type of generative model that maps a high-dimensional noise to samples in target distribution. However, the dimension of noise required in GAN is not well understood. Previous approaches view GAN as a mapping from a continuous distribution to another continous distribution. In this paper, we propose to view GAN as a discrete sampler instead. From this perspective, we build a connection between the minimum noise required and the bits to losslessly compress the images. Furthermore, to understand the behaviour of GAN when noise dimension is limited, we propose divergence-entropy trade-off. This trade-off depicts the best divergence we can achieve when noise is limited. And as rate distortion trade-off, it can be numerically solved when source distribution is known. Finally, we verifies our theory with experiments on image generation.
Reverse em-problem based on Bregman divergence and its application to classical and quantum information theory
Venue: arXiv cs.IT 2024
Year: 2024

The recent paper (IEEE Trans. IT 69, 1680) introduced an analytical method for calculating the channel capacity without the need for iteration. This method has certain limitations that restrict its applicability. Furthermore, the paper does not provide an explanation as to why the channel capacity can be solved analytically in this particular case. In order to broaden the scope of this method and address its limitations, we turn our attention to the reverse em-problem, proposed by Toyota (Information Geometry, 3, 1355 (2020)). This reverse em-problem involves iteratively applying the inverse map of the em iteration to calculate the channel capacity, which represents the maximum mutual information. However, several open problems remained unresolved in Toyota's work. To overcome these challenges, we formulate the reverse em-problem based on Bregman divergence and provide solutions to these open problems. Building upon these results, we transform the reverse em-problem into em-problems and derive a non-iterative formula for the reverse em-problem. This formula can be viewed as a generalization of the aforementioned analytical calculation method. Importantly, this derivation sheds light on the information geometrical structure underlying this special case. By effectively addressing the limitations of the previous analytical method and providing a deeper understanding of the underlying information geometrical structure, our work significantly expands the applicability of the proposed method for calculating the channel capacity without iteration.
Gun Culture in Fringe Social Media
Venue: arXiv cs.SI 2024
Year: 2024

The increasing frequency of mass shootings in the United States has, unfortunately, become a norm. While the issue of gun control in the US involves complex legal concerns, there are also societal issues at play. One such social issue is so-called "gun culture," i.e., a general set of beliefs and actions related to gun ownership. However relatively little is known about gun culture, and even less is known when it comes to fringe online communities. This is especially worrying considering the aforementioned rise in mass shootings and numerous instances of shooters being radicalized online. To address this gap, we explore gun culture on /k/, 4chan's weapons board. More specifically, using a variety of quantitative techniques, we examine over 4M posts on /k/ and position their discussion within the larger body of theoretical understanding of gun culture. Among other things, our findings suggest that gun culture on /k/ covers a relatively diverse set of topics (with a particular focus on legal discussion), some of which are signals of fetishism.
To Label or Not to Label: Hybrid Active Learning for Neural Machine Translation
Venue: arXiv cs.CL 2024
Year: 2024

Active learning (AL) techniques reduce labeling costs for training neural machine translation (NMT) models by selecting smaller representative subsets from unlabeled data for annotation. Diversity sampling techniques select heterogeneous instances, while uncertainty sampling methods select instances with the highest model uncertainty. Both approaches have limitations - diversity methods may extract varied but trivial examples, while uncertainty sampling can yield repetitive, uninformative instances. To bridge this gap, we propose Hybrid Uncertainty and Diversity Sampling (HUDS), an AL strategy for domain adaptation in NMT that combines uncertainty and diversity for sentence selection. HUDS computes uncertainty scores for unlabeled sentences and subsequently stratifies them. It then clusters sentence embeddings within each stratum and computes diversity scores by distance to the centroid. A weighted hybrid score that combines uncertainty and diversity is then used to select the top instances for annotation in each AL iteration. Experiments on multi-domain German-English and French-English datasets demonstrate the better performance of HUDS over other strong AL baselines. We analyze the sentence selection with HUDS and show that it prioritizes diverse instances having high model uncertainty for annotation in early AL iterations.
Whittle Index Based User Association in Dense Millimeter Wave Networks
Venue: arXiv cs.NI 2024
Year: 2024

We address the problem of user association in a dense millimeter wave (mmWave) network, in which each arriving user brings a file containing a random number of packets and each time slot is divided into multiple mini-slots. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. Using a technique introduced by Whittle, we relax the hard per-stage constraint that each arriving user must be associated with exactly one mmWave base station (mBS) to a long-term constraint and then use the Lagrangian multiplier technique to convert the problem into an unconstrained problem. This decouples the process governing the system into separate Markov Decision Processes at different mBSs. We prove that the problem is Whittle indexable, present a scheme for computing the Whittle indices of different mBSs, and propose an association scheme under which, each arriving user is associated with the mBS with the smallest value of the Whittle index. Using extensive simulations, we show that the proposed Whittle index based scheme outperforms several user association schemes proposed in prior work in terms of various performance metrics such as average cost, delay, throughput, and Jain's fairness index.