OpenRaiser/Intern-Atlas huggingface dataset:
DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLM
Venue: arXiv cs.CV 2024
Year: 2024

Visual language tracking (VLT) has emerged as a cutting-edge research area, harnessing linguistic data to enhance algorithms with multi-modal inputs and broadening the scope of traditional single object tracking (SOT) to encompass video understanding applications. Despite this, most VLT benchmarks still depend on succinct, human-annotated text descriptions for each video. These descriptions often fall short in capturing the nuances of video content dynamics and lack stylistic variety in language, constrained by their uniform level of detail and a fixed annotation frequency. As a result, algorithms tend to default to a "memorize the answer" strategy, diverging from the core objective of achieving a deeper understanding of video content. Fortunately, the emergence of large language models (LLMs) has enabled the generation of diverse text. This work utilizes LLMs to generate varied semantic annotations (in terms of text lengths and granularities) for representative SOT benchmarks, thereby establishing a novel multi-modal benchmark. Specifically, we (1) propose a new visual language tracking benchmark with diverse texts, named DTVLT, based on five prominent VLT and SOT benchmarks, including three sub-tasks: short-term tracking, long-term tracking, and global instance tracking. (2) We offer four granularity texts in our benchmark, considering the extent and density of semantic information. We expect this multi-granular generation strategy to foster a favorable environment for VLT and video understanding research. (3) We conduct comprehensive experimental analyses on DTVLT, evaluating the impact of diverse text on tracking performance and hope the identified performance bottlenecks of existing algorithms can support further research in VLT and video understanding. The proposed benchmark, experimental results and toolkit will be released gradually on http://videocube.aitestunion.com/.
Efficient learning of differential network in multi-source non-paranormal graphical models
Venue: arXiv cs.LG 2024
Year: 2024

This paper addresses learning of sparse structural changes or differential network between two classes of non-paranormal graphical models. We assume a multi-source and heterogeneous dataset is available for each class, where the covariance matrices are identical for all non-paranormal graphical models. The differential network, which are encoded by the difference precision matrix, can then be decoded by optimizing a lasso penalized D-trace loss function. To this aim, an efficient approach is proposed that outputs the exact solution path, outperforming the previous methods that only sample from the solution path in pre-selected regularization parameters. Notably, our proposed method has low computational complexity, especially when the differential network are sparse. Our simulations on synthetic data demonstrate a superior performance for our strategy in terms of speed and accuracy compared to an existing method. Moreover, our strategy in combining datasets from multiple sources is shown to be very effective in inferring differential network in real-world problems. This is backed by our experimental results on drug resistance in tumor cancers. In the latter case, our strategy outputs important genes for drug resistance which are already confirmed by various independent studies.
Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment
Venue: arXiv cs.CV 2024
Year: 2024

Image quality assessment (IQA) serves as the golden standard for all models' performance in nearly all computer vision fields. However, it still suffers from poor out-of-distribution generalization ability and expensive training costs. To address these problems, we propose Dog-IQA, a standard-guided zero-shot mix-grained IQA method, which is training-free and utilizes the exceptional prior knowledge of multimodal large language models (MLLMs). To obtain accurate IQA scores, namely scores consistent with humans, we design an MLLM-based inference pipeline that imitates human experts. In detail, Dog-IQA applies two techniques. First, Dog-IQA objectively scores with specific standards that utilize MLLM's behavior pattern and minimize the influence of subjective factors. Second, Dog-IQA comprehensively takes local semantic objects and the whole image as input and aggregates their scores, leveraging local and global information. Our proposed Dog-IQA achieves state-of-the-art (SOTA) performance compared with training-free methods, and competitive performance compared with training-based methods in cross-dataset scenarios. Our code will be available at https://github.com/Kai-Liu001/Dog-IQA.
Minimax Group Fairness in Strategic Classification
Venue: arXiv cs.LG 2024
Year: 2024

In strategic classification, agents manipulate their features, at a cost, to receive a positive classification outcome from the learner's classifier. The goal of the learner in such settings is to learn a classifier that is robust to strategic manipulations. While the majority of works in this domain consider accuracy as the primary objective of the learner, in this work, we consider learning objectives that have group fairness guarantees in addition to accuracy guarantees. We work with the minimax group fairness notion that asks for minimizing the maximal group error rate across population groups. We formalize a fairness-aware Stackelberg game between a population of agents consisting of several groups, with each group having its own cost function, and a learner in the agnostic PAC setting in which the learner is working with a hypothesis class H. When the cost functions of the agents are separable, we show the existence of an efficient algorithm that finds an approximately optimal deterministic classifier for the learner when the number of groups is small. This algorithm remains efficient, both statistically and computationally, even when H is the set of all classifiers. We then consider cost functions that are not necessarily separable and show the existence of oracle-efficient algorithms that find approximately optimal randomized classifiers for the learner when H has finite strategic VC dimension. These algorithms work under the assumption that the learner is fully transparent: the learner draws a classifier from its distribution (randomized classifier) before the agents respond by manipulating their feature vectors. We highlight the effectiveness of such transparency in developing oracle-efficient algorithms. We conclude with verifying the efficacy of our algorithms on real data by conducting an experimental analysis.
HiFiSeg: High-Frequency Information Enhanced Polyp Segmentation with Global-Local Vision Transformer
Venue: arXiv cs.CV 2024
Year: 2024

Numerous studies have demonstrated the strong performance of Vision Transformer (ViT)-based methods across various computer vision tasks. However, ViT models often struggle to effectively capture high-frequency components in images, which are crucial for detecting small targets and preserving edge details, especially in complex scenarios. This limitation is particularly challenging in colon polyp segmentation, where polyps exhibit significant variability in structure, texture, and shape. High-frequency information, such as boundary details, is essential for achieving precise semantic segmentation in this context. To address these challenges, we propose HiFiSeg, a novel network for colon polyp segmentation that enhances high-frequency information processing through a global-local vision transformer framework. HiFiSeg leverages the pyramid vision transformer (PVT) as its encoder and introduces two key modules: the global-local interaction module (GLIM) and the selective aggregation module (SAM). GLIM employs a parallel structure to fuse global and local information at multiple scales, effectively capturing fine-grained features. SAM selectively integrates boundary details from low-level features with semantic information from high-level features, significantly improving the model's ability to accurately detect and segment polyps. Extensive experiments on five widely recognized benchmark datasets demonstrate the effectiveness of HiFiSeg for polyp segmentation. Notably, the mDice scores on the challenging CVC-ColonDB and ETIS datasets reached 0.826 and 0.822, respectively, underscoring the superior performance of HiFiSeg in handling the specific complexities of this task.
An Improved Variational Method for Image Denoising
Venue: arXiv cs.CV 2024
Year: 2024

The total variation (TV) method is an image denoising technique that aims to reduce noise by minimizing the total variation of the image, which measures the variation in pixel intensities. The TV method has been widely applied in image processing and computer vision for its ability to preserve edges and enhance image quality. In this paper, we propose an improved TV model for image denoising and the associated numerical algorithm to carry out the procedure, which is particularly effective in removing several types of noises and their combinations. Our improved model admits a unique solution and the associated numerical algorithm guarantees the convergence. Numerical experiments are demonstrated to show improved effectiveness and denoising quality compared to other TV models. Such encouraging results further enhance the utility of the TV method in image processing.
Preparing for Super-Reactivity: Early Fault-Detection in the Development of Exceedingly Complex Reactive Systems
Venue: arXiv cs.SE 2024
Year: 2024

We introduce the term Super-Reactive Systems to refer to reactive systems whose construction and behavior are complex, constantly changing and evolving, and heavily interwoven with other systems and the physical world. Finding hidden faults in such systems early in planning and development is critical for human safety, the environment, society and the economy. However, the complexity of the system and its interactions and the absence of adequate technical details pose a great obstacle. We propose an architecture for models and tools to overcome such barriers and enable simulation, systematic analysis, and fault detection and handling, early in the development of super-reactive systems. The approach is facilitated by the inference and abstraction capabilities and the power and knowledge afforded by large language models and associated AI tools. It is based on: (i) deferred, just-in-time interpretation of model elements that are stored in natural language form, and (ii) early capture of tacit interdependencies among seemingly orthogonal requirements.
Measuring and Improving Persuasiveness of Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

LLMs are increasingly being used in workflows involving generating content to be consumed by humans (e.g., marketing) and also in directly interacting with humans (e.g., through chatbots). The development of such systems that are capable of generating verifiably persuasive messages presents both opportunities and challenges for society. On the one hand, such systems could positively impact domains like advertising and social good, such as addressing drug addiction, and on the other, they could be misused for spreading misinformation and shaping political opinions. To channel LLMs' impact on society, we need to develop systems to measure and benchmark their persuasiveness. With this motivation, we introduce PersuasionBench and PersuasionArena, the first large-scale benchmark and arena containing a battery of tasks to measure the persuasion ability of generative models automatically. We investigate to what extent LLMs know and leverage linguistic patterns that can help them generate more persuasive language. Our findings indicate that the persuasiveness of LLMs correlates positively with model size, but smaller models can also be made to have a higher persuasiveness than much larger models. Notably, targeted training using synthetic and natural datasets significantly enhances smaller models' persuasive capabilities, challenging scale-dependent assumptions. Our findings carry key implications for both model developers and policymakers. For instance, while the EU AI Act and California's SB-1047 aim to regulate AI models based on the number of floating point operations, we demonstrate that simple metrics like this alone fail to capture the full scope of AI's societal impact. We invite the community to explore and contribute to PersuasionArena and PersuasionBench, available at https://bit.ly/measure-persuasion, to advance our understanding of AI-driven persuasion and its societal implications.
Examining Language Modeling Assumptions Using an Annotated Literary Dialect Corpus
Venue: arXiv cs.CL 2024
Year: 2024

We present a dataset of 19th century American literary orthovariant tokens with a novel layer of human-annotated dialect group tags designed to serve as the basis for computational experiments exploring literarily meaningful orthographic variation. We perform an initial broad set of experiments over this dataset using both token (BERT) and character (CANINE)-level contextual language models. We find indications that the "dialect effect" produced by intentional orthographic variation employs multiple linguistic channels, and that these channels are able to be surfaced to varied degrees given particular language modelling assumptions. Specifically, we find evidence showing that choice of tokenization scheme meaningfully impact the type of orthographic information a model is able to surface.
Grounding Large Language Models In Embodied Environment With Imperfect World Models
Venue: arXiv cs.CL 2024
Year: 2024

Despite a widespread success in various applications, large language models (LLMs) often stumble when tackling basic physical reasoning or executing robotics tasks, due to a lack of direct experience with the physical nuances of the real world. To address these issues, we propose a Grounding Large language model with Imperfect world MOdel (GLIMO), which utilizes proxy world models such as simulators to collect and synthesize trining data. GLIMO incorporates an LLM agent-based data generator to automatically create high-quality and diverse instruction datasets. The generator includes an iterative self-refining module for temporally consistent experience sampling, a diverse set of question-answering instruction seeds, and a retrieval-augmented generation module for reflecting on prior experiences. Comprehensive experiments show that our approach improve the performance of strong open-source LLMs like LLaMA-3 with a performance boost of 2.04 $\times$, 1.54 $\times$, and 1.82 $\times$ across three different benchmarks, respectively. The performance is able to compete with or surpass their larger counterparts such as GPT-4.
An Online Automatic Modulation Classification Scheme Based on Isolation Distributional Kernel
Venue: arXiv cs.LG 2024
Year: 2024

Automatic Modulation Classification (AMC), as a crucial technique in modern non-cooperative communication networks, plays a key role in various civil and military applications. However, existing AMC methods usually are complicated and can work in batch mode only due to their high computational complexity. This paper introduces a new online AMC scheme based on Isolation Distributional Kernel. Our method stands out in two aspects. Firstly, it is the first proposal to represent baseband signals using a distributional kernel. Secondly, it introduces a pioneering AMC technique that works well in online settings under realistic time-varying channel conditions. Through extensive experiments in online settings, we demonstrate the effectiveness of the proposed classifier. Our results indicate that the proposed approach outperforms existing baseline models, including two state-of-the-art deep learning classifiers. Moreover, it distinguishes itself as the first online classifier for AMC with linear time complexity, which marks a significant efficiency boost for real-time applications.
Complex-valued convolutional neural network classification of hand gesture from radar images
Venue: arXiv cs.CV 2024
Year: 2024

Hand gesture recognition systems have yielded many exciting advancements in the last decade and become more popular in HCI (human-computer interaction) with several application areas, which spans from safety and security applications to automotive field. Various deep neural network architectures have already been inspected for hand gesture recognition systems, including multi-layer perceptron (MLP), convolutional neural network (CNN), recurrent neural network (RNN) and a cascade of the last two architectures known as CNN-RNN. However, a major problem still exists, which is most of the existing ML algorithms are designed and developed the building blocks and techniques for real-valued (RV). Researchers applied various RV techniques on the complex-valued (CV) radar images, such as converting a CV optimisation problem into a RV one, by splitting the complex numbers into their real and imaginary parts. However, the major disadvantage of this method is that the resulting algorithm will double the network dimensions. Recent work on RNNs and other fundamental theoretical analysis suggest that CV numbers have a richer representational capacity, but due to the absence of the building blocks required to design such models, the performance of CV networks are marginalised. In this report, we propose a fully CV-CNN, including all building blocks, forward and backward operations, and derivatives all in complex domain. We explore our proposed classification model on two sets of CV hand gesture radar images in comparison with the equivalent RV model. In chapter five, we propose a CV-forward residual network, for the purpose of binary classification of the two sets of CV hand gesture radar datasets and compare its performance with our proposed CV-CNN and a baseline CV-forward CNN.
Enhancing ICT Literacy and Sustainable Practices in the Hospitality Industry: Insights from Mnquma Municipality
Venue: arXiv cs.CY 2024
Year: 2024

The leisure and hospitality industry is a significant driver of the global economy, with the adoption of new technologies transforming service delivery and customer experience. Despite the transformative potential and benefits associated with adopting technology, there remains a low level of adoption in rural areas, particularly among small-scale players. This study explores the role of ICT literacy and sustainable practices in influencing ICT adoption among small-scale players in the hospitality industry in rural Eastern Cape Province, South Africa, specifically focusing on Mnquma Municipality. The study employs a non-probability sampling and purposive technique, utilising a case study research design within a positivist paradigm. A random sample of 21 small-scale players (BnBs, guest houses, and non-serviced accommodations) was selected, and data were collected through a face-to-face interview and questionnaire featuring closed-ended questions. The data were analysed using descriptive statistics and the Kruskal-Wallis H Test to examine differences in ICT usage levels. The test yielded a Kruskal-Wallis H of 2.57 with a p-value of 0.277. The findings reveal that businesses with more educated workforces demonstrate higher ICT adoption levels. Moreover, key factors such as ICT literacy, awareness of sustainable practices, access to ICT resources, and contextual challenges significantly impact ICT adoption. Recommendations include integrating ICT literacy and sustainability education into training programs and developing targeted policies and support mechanisms to enhance ICT integration.
Dynamic Road Management in the Era of CAV
Venue: arXiv cs.NI 2024
Year: 2024

Traffic management and on-road safety have been a concern for the transportation authorities and the engineering communities for many years. Most of the implemented technologies for intelligent highways focus on safety measures and increased driver awareness, and expect a centralized management for the vehicular traffic flow. Leveraging recent advances in wireless communication, researchers have proposed solutions based on vehicle-to-vehicle (V2V) and vehicle-to-Infrastructure (V2I) communication in order to detect traffic jams and better disseminate data from on-road and on-vehicle sensors. Moreover, the development of connected autonomous vehicles (CAV) have motivated a paradigm shift in how traffic will be managed. Overall, these major technological advances have motivated the notion of dynamic traffic management (DTM), where smart road reconfiguration capabilities, e.g., dynamic lane reversal, adaptive traffic light timing, etc. will be exploited in real-time to improve traffic flow and adapt to unexpected incidents. This chapter discusses what the challenges in realizing DTM are and covers how CAV has revolutionized traffic management. Moreover, we highlight the issues for handling human-driven vehicles while roads are transitioning to CAV only traffic. Particularly, we articulate a new vision for inter-vehicle communication and assessment of road conditions, and promote a novel system for traffic management. Vehicle to on-road sensors as well as inter-vehicle connectivity will be enabled through the use of handheld devices such as smartphones. This not only enables real-time data sharing but also expedites the adoption of DTM without awaiting the dominant presence of autonomous vehicle on the road. ...
Travel experience in public transport: Experience sampling and cardiac activity data for spatial analysis
Venue: arXiv cs.HC 2024
Year: 2024

The transportation sector has the potential to enable a greener future if aligned with increasing mobility needs. Making public transport an attractive alternative to individual transportation requires real-world data to investigate reasons and indicators of positive and negative travel experiences. These experiences manifest not only in subjective evaluations but also in physiological reactions like cardiac activity. We present a geo-referenced dataset where participants wore electrocardiograms and reported real-time stress, satisfaction, events, and emotions while traveling by tram, train, and bus. An interactive experience map helps to visually explore the data, with benchmark analyses identifying significant stress hot spots and satisfaction cold spots during journeys. Events and emotions in these spots highlight positive and negative travel experiences in an ecologically valid setting. Data on age and self-identified gender provide insights into differences between user groups. Despite including only 44 participants, the dataset offers a valuable foundation for transportation researchers and mobility providers to combine qualitative and quantitative methods for identifying public transportation users' needs.
Bipolar fuzzy relation equations systems based on the product t-norm
Venue: arXiv cs.AI 2024
Year: 2024

Bipolar fuzzy relation equations arise as a generalization of fuzzy relation equations considering unknown variables together with their logical connective negations. The occurrence of a variable and the occurrence of its negation simultaneously can give very useful information for certain frameworks where the human reasoning plays a key role. Hence, the resolution of bipolar fuzzy relation equations systems is a research topic of great interest. This paper focuses on the study of bipolar fuzzy relation equations systems based on the max-product t-norm composition. Specifically, the solvability and the algebraic structure of the set of solutions of these bipolar equations systems will be studied, including the case in which such systems are composed of equations whose independent term be equal to zero. As a consequence, this paper complements the contribution carried out by the authors on the solvability of bipolar max-product fuzzy relation equations.