OpenRaiser/Intern-Atlas huggingface dataset:
Goal-Oriented Semantic Communication for Wireless Visual Question Answering
Venue: arXiv cs.CV 2024
Year: 2024

The rapid progress of artificial intelligence (AI) and computer vision (CV) has facilitated the development of computation-intensive applications like Visual Question Answering (VQA), which integrates visual perception and natural language processing to generate answers. To overcome the limitations of traditional VQA constrained by local computation resources, edge computing has been incorporated to provide extra computation capability at the edge side. Meanwhile, this brings new communication challenges between the local and edge, including limited bandwidth, channel noise, and multipath effects, which degrade VQA performance and user quality of experience (QoE), particularly during the transmission of large high-resolution images. To overcome these bottlenecks, we propose a goal-oriented semantic communication (GSC) framework that focuses on effectively extracting and transmitting semantic information most relevant to the VQA goals, improving the answering accuracy and enhancing the effectiveness and efficiency. The objective is to maximize the answering accuracy, and we propose a bounding box (BBox)-based image semantic extraction and ranking approach to prioritize the semantic information based on the goal of questions. We then extend it by incorporating a scene graphs (SG)-based approach to handle questions with complex relationships. Experimental results demonstrate that our GSC framework improves answering accuracy by up to 49% under AWGN channels and 59% under Rayleigh channels while reducing total latency by up to 65% compared to traditional bit-oriented transmission.
Modeling and Simulation of a Multi Robot System Architecture
Venue: arXiv cs.AI 2024
Year: 2024

A Multi Robot System (MRS) is the infrastructure of an intelligent cyberphysical system, where the robots understand the need of the human, and hence cooperate together to fulfill this need. Modeling an MRS is a crucial aspect of designing the proper system architecture, because this model can be used to simulate and measure the performance of the proposed architecture. However, an MRS solution architecture modeling is a very difficult problem, as it contains many dependent behaviors that dynamically change due to the current status of the overall system. In this paper, we introduce a general purpose MRS case study, where the humans initiate requests that are achieved by the available robots. These requests require different plans that use the current capabilities of the available robots. After proposing an architecture that defines the solution components, three steps are followed. First is modeling these components via Business Process Model and Notation (BPMN) language. BPMN provides a graphical notation to precisely represent the behaviors of every component, which is an essential need to model the solution. Second is to simulate these components behaviors and interaction in form of software agents. Java Agent DEvelopment (JADE) middleware has been used to develop and simulate the proposed model. JADE is based on a reactive agent approach, therefore it can dynamically represent the interaction among the solution components. Finally is to analyze the performance of the solution by defining a number of quantitative measurements, which can be obtained while simulating the system model in JADE middleware, therefore the solution can be analyzed and compared to another architecture.
Fast and Robust Information Spreading in the Noisy PULL Model
Venue: arXiv cs.DC 2024
Year: 2024

Understanding how information can efficiently spread in distributed systems under noisy communications is a fundamental question in both biological research and artificial system design. When agents are able to control whom they interact with, noise can often be mitigated through redundancy or other coding techniques, but it may have fundamentally different consequences on well-mixed systems. Specifically, Boczkowski et al. (2018) considered the noisy $\mathcal{PULL}(h)$ model, where each message can be viewed as any other message with probability $δ$. The authors proved that in this model, the basic task of propagating a bit value from a single source to the whole population requires $Ω(\frac{nδ}{h(1-δ|Σ|)^2})$ (parallel) rounds. The current work shows that the aforementioned lower bound is almost tight. In particular, when each agent observes all other agents in each round, which relates to scenarios where each agent senses the system's average tendency, information spreading can reliably be achieved in $\mathcal{O}(\log n)$ time, assuming constant noise. We present two simple and highly efficient protocols, thus suggesting their applicability to real-life scenarios. Notably, they also work in the presence of multiple conflicting sources and efficiently converge to their plurality opinion. The first protocol we present uses 1-bit messages but relies on a simultaneous wake-up assumption. By increasing the message size to 2 bits and removing the speedup in the information spreading time that may result from having multiple sources, we also present a simple and highly efficient self-stabilizing protocol that avoids the simultaneous wake-up requirement. Overall, our results demonstrate how, under stochastic communication, increasing the sample size can compensate for the lack of communication structure by linearly accelerating information spreading time.
Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Models
Venue: arXiv cs.CV 2024
Year: 2024

Instruction tuning constitutes a prevalent technique for tailoring Large Vision Language Models (LVLMs) to meet individual task requirements. To date, most of the existing approaches are confined to single-task adaptation, whereas the requirements in real-world scenarios are inherently varied and continually evolving. Thus an ideal LVLM should sustain continual instruction tuning in the face of stream-task distributions (i.e., different domains, emerging capabilities, and new datasets) while minimizing the forgetting of previously acquired knowledge. To achieve this, we propose a new benchmark for COntinuAl inStruction Tuning on LVLMs (COAST), which encompasses the aforementioned domain-incremental, capability-incremental, and dataset-incremental configurations. In terms of methodology, we propose Continual LLaVA, a rehearsal-free method tailored for continual instruction tuning in LVLMs. To circumvent the additional overhead associated with experience replay, we freeze LVLMs and construct the dual increment embeddings for each input instruction to facilitate parameter-efficient tuning. Specifically, the increment embeddings can be decomposed into two principal components: 1) intrinsic increment embeddings to encode task-specific characteristics. To achieve this, we set up a low-rank pool containing candidate embeddings, from which we select the relevant ones based on their similarity with the user instructions; 2) contextual increment embeddings to investigate the inter-dependencies across tasks. In this regard, the low-rank embeddings chosen in the previous tasks are aggregated via learnable weighted sum to provide complementary hints. Extensive experiments indicate that the proposed Continual LLaVA outperforms previous methods by significantly reducing the forgetting during the continual instruction tuning process.
The Intersectionality Problem for Algorithmic Fairness
Venue: arXiv cs.LG 2024
Year: 2024

A yet unmet challenge in algorithmic fairness is the problem of intersectionality, that is, achieving fairness across the intersection of multiple groups -- and verifying that such fairness has been attained. Because intersectional groups tend to be small, verifying whether a model is fair raises statistical as well as moral-methodological challenges. This paper (1) elucidates the problem of intersectionality in algorithmic fairness, (2) develops desiderata to clarify the challenges underlying the problem and guide the search for potential solutions, (3) illustrates the desiderata and potential solutions by sketching a proposal using simple hypothesis testing, and (4) evaluates, partly empirically, this proposal against the proposed desiderata.
Real-Time Detection for Small UAVs: Combining YOLO and Multi-frame Motion Analysis
Venue: arXiv cs.CV 2024
Year: 2024

Unmanned Aerial Vehicle (UAV) detection technology plays a critical role in mitigating security risks and safeguarding privacy in both military and civilian applications. However, traditional detection methods face significant challenges in identifying UAV targets with extremely small pixels at long distances. To address this issue, we propose the Global-Local YOLO-Motion (GL-YOMO) detection algorithm, which combines You Only Look Once (YOLO) object detection with multi-frame motion detection techniques, markedly enhancing the accuracy and stability of small UAV target detection. The YOLO detection algorithm is optimized through multi-scale feature fusion and attention mechanisms, while the integration of the Ghost module further improves efficiency. Additionally, a motion detection approach based on template matching is being developed to augment detection capabilities for minute UAV targets. The system utilizes a global-local collaborative detection strategy to achieve high precision and efficiency. Experimental results on a self-constructed fixed-wing UAV dataset demonstrate that the GL-YOMO algorithm significantly enhances detection accuracy and stability, underscoring its potential in UAV detection applications.
TileTracker: Tracking Based Vector HD Mapping using Top-Down Road Images
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we propose a tracking-based HD mapping algorithm for top-down road images, referred to as tile images. While HD maps traditionally rely on perspective camera images, our approach shows that tile images can also be effectively utilized, offering valuable contributions to this research area as it can be start of a new path in HD mapping algorithms. We modified the BEVFormer layers to generate BEV masks from tile images, which are then used by the model to generate divider and boundary lines. Our model was tested with both color and intensity images, and we present quantitative and qualitative results to demonstrate its performance.
Taming the Beast of User-Programmed Transactions on Blockchains: A Declarative Transaction Approach
Venue: arXiv cs.CR 2024
Year: 2024

Blockchains are being positioned as the "technology of trust" that can be used to mediate transactions between non-trusting parties without the need for a central authority. They support transaction types that are native to the blockchain platform or user-defined via user programs called smart contracts. Despite the significant flexibility in transaction programmability that smart contracts offer, they pose several usability, robustness, and performance challenges. This paper proposes an alternative transaction framework that incorporates more primitives into the native set of transaction types (reducing the likelihood of requiring user-defined transaction programs often). The framework is based on the concept of declarative blockchain transactions whose strength lies in the fact that it addresses several of the limitations of smart contracts simultaneously. A formal and implementation framework is presented, and a subset of commonly occurring transaction behaviors are modeled and implemented as use cases, using an open-source blockchain database, BigchchainDB, as the implementation context. A performance study comparing the declarative transaction approach to equivalent smart contract transaction models reveals several advantages of the proposed approach.
Towards Context-Aware Adaptation in Extended Reality: A Design Space for XR Interfaces and an Adaptive Placement Strategy
Venue: arXiv cs.HC 2024
Year: 2024

By converting the entire 3D space around the user into a screen, Extended Reality (XR) can ameliorate traditional displays' space limitations and facilitate the consumption of multiple pieces of information at a time. However, if designed inappropriately, these XR interfaces can overwhelm the user and complicate information access. In this work, we explored the design dimensions that can be adapted to enable suitable presentation and interaction within an XR interface. To investigate a specific use case of context-aware adaptations within our proposed design space, we concentrated on the spatial layout of the XR content and investigated non-adaptive and adaptive placement strategies. In this paper, we (1) present a comprehensive design space for XR interfaces, (2) propose Environment-referenced, an adaptive placement strategy that uses a relevant intermediary from the environment within a Hybrid Frame of Reference (FoR) for each XR object, and (3) evaluate the effectiveness of this adaptive placement strategy and a non-adaptive Body-Fixed placement strategy in four contextual scenarios varying in terms of social setting and user mobility in the environment. The performance of these placement strategies from our within-subjects user study emphasized the importance of intermediaries' relevance to the user's focus. These findings underscore the importance of context-aware interfaces, indicating that the appropriate use of an adaptive content placement strategy in a context can significantly improve task efficiency, accuracy, and usability.
Estimating the Number and Locations of Boundaries in Reverberant Environments with Deep Learning
Venue: arXiv cs.SD 2024
Year: 2024

Underwater acoustic environment estimation is a challenging but important task for remote sensing scenarios. Current estimation methods require high signal strength and a solution to the fragile echo labeling problem to be effective. In previous publications, we proposed a general deep learning-based method for two-dimensional environment estimation which outperformed the state-of-the-art, both in simulation and in real-life experimental settings. A limitation of this method was that some prior information had to be provided by the user on the number and locations of the reflective boundaries, and that its neural networks had to be re-trained accordingly for different environments. Utilizing more advanced neural network and time delay estimation techniques, the proposed improved method no longer requires prior knowledge the number of boundaries or their locations, and is able to estimate two-dimensional environments with one or two boundaries. Future work will extend the proposed method to more boundaries and larger-scale environments.
Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning
Venue: arXiv cs.LG 2024
Year: 2024

Machine unlearning--enabling a trained model to forget specific data--is crucial for addressing biased data and adhering to privacy regulations like the General Data Protection Regulation (GDPR)'s "right to be forgotten". Recent works have paid little attention to privacy concerns, leaving the data intended for forgetting vulnerable to membership inference attacks. Moreover, they often come with high computational overhead. In this work, we propose Pseudo-Probability Unlearning (PPU), a novel method that enables models to forget data efficiently and in a privacy-preserving manner. Our method replaces the final-layer output probabilities of the neural network with pseudo-probabilities for the data to be forgotten. These pseudo-probabilities follow either a uniform distribution or align with the model's overall distribution, enhancing privacy and reducing risk of membership inference attacks. Our optimization strategy further refines the predictive probability distributions and updates the model's weights accordingly, ensuring effective forgetting with minimal impact on the model's overall performance. Through comprehensive experiments on multiple benchmarks, our method achieves over 20% improvements in forgetting error compared to the state-of-the-art. Additionally, our method enhances privacy by preventing the forgotten set from being inferred to around random guesses.
M-CELS: Counterfactual Explanation for Multivariate Time Series Data Guided by Learned Saliency Maps
Venue: arXiv cs.LG 2024
Year: 2024

Over the past decade, multivariate time series classification has received great attention. Machine learning (ML) models for multivariate time series classification have made significant strides and achieved impressive success in a wide range of applications and tasks. The challenge of many state-of-the-art ML models is a lack of transparency and interpretability. In this work, we introduce M-CELS, a counterfactual explanation model designed to enhance interpretability in multidimensional time series classification tasks. Our experimental validation involves comparing M-CELS with leading state-of-the-art baselines, utilizing seven real-world time-series datasets from the UEA repository. The results demonstrate the superior performance of M-CELS in terms of validity, proximity, and sparsity, reinforcing its effectiveness in providing transparent insights into the decisions of machine learning models applied to multivariate time series data.
Linear-Time Algorithms for k-Edge-Connected Components, k-Lean Tree Decompositions, and More
Venue: arXiv cs.DS 2024
Year: 2024

We present $k^{O(k^2)} m$ time algorithms for various problems about decomposing a given undirected graph by edge cuts or vertex separators of size $<k$ into parts that are ``well-connected'' with respect to cuts or separators of size $<k$; here, $m$ is the total number of vertices and edges of the graph. As an application of our results, we obtain for every fixed $k$ a linear-time algorithm for computing the $k$-edge-connected components of a given graph, solving a long-standing open problem. More generally, we obtain a $k^{O(k^2)} m$ time algorithm for computing a $k$-Gomory-Hu tree of a given graph, which is a structure representing pairwise minimum cuts of size $<k$. Our main technical result, from which the other results follow, is a $k^{O(k^2)} m$ time algorithm for computing a $k$-lean tree decomposition of a given graph. This is a tree decomposition with adhesion size $<k$ that captures the existence of separators of size $<k$ between subsets of its bags. A $k$-lean tree decomposition is also an unbreakable tree decomposition with optimal unbreakability parameters for the adhesion size bound $k$. As further applications, we obtain $k^{O(k^2)} m$ time algorithms for $k$-vertex connectivity and for element connectivity $k$-Gomory-Hu tree. All of our algorithms are deterministic. Our techniques are inspired by the tenth paper of the Graph Minors series of Robertson and Seymour and by Bodlaender's parameterized linear-time algorithm for treewidth.
Visually Analyze SHAP Plots to Diagnose Misclassifications in ML-based Intrusion Detection
Venue: arXiv cs.CR 2024
Year: 2024

Intrusion detection has been a commonly adopted detective security measures to safeguard systems and networks from various threats. A robust intrusion detection system (IDS) can essentially mitigate threats by providing alerts. In networks based IDS, typically we deal with cyber threats like distributed denial of service (DDoS), spoofing, reconnaissance, brute-force, botnets, and so on. In order to detect these threats various machine learning (ML) and deep learning (DL) models have been proposed. However, one of the key challenges with these predictive approaches is the presence of false positive (FP) and false negative (FN) instances. This FPs and FNs within any black-box intrusion detection system (IDS) make the decision-making task of an analyst further complicated. In this paper, we propose an explainable artificial intelligence (XAI) based visual analysis approach using overlapping SHAP plots that presents the feature explanation to identify potential false positive and false negatives in IDS. Our approach can further provide guidance to security analysts for effective decision-making. We present case study with multiple publicly available network traffic datasets to showcase the efficacy of our approach for identifying false positive and false negative instances. Our use-case scenarios provide clear guidance for analysts on how to use the visual analysis approach for reliable course-of-actions against such threats.
JPEC: A Novel Graph Neural Network for Competitor Retrieval in Financial Knowledge Graphs
Venue: arXiv cs.IR 2024
Year: 2024

Knowledge graphs have gained popularity for their ability to organize and analyze complex data effectively. When combined with graph embedding techniques, such as graph neural networks (GNNs), knowledge graphs become a potent tool in providing valuable insights. This study explores the application of graph embedding in identifying competitors from a financial knowledge graph. Existing state-of-the-art(SOTA) models face challenges due to the unique attributes of our knowledge graph, including directed and undirected relationships, attributed nodes, and minimal annotated competitor connections. To address these challenges, we propose a novel graph embedding model, JPEC(JPMorgan Proximity Embedding for Competitor Detection), which utilizes graph neural network to learn from both first-order and second-order node proximity together with vital features for competitor retrieval. JPEC had outperformed most existing models in extensive experiments, showcasing its effectiveness in competitor retrieval.
Energy Efficient and Balanced Task Assignment Strategy for Multi-UAV Patrol Inspection System in Mobile Edge Computing Network
Venue: arXiv cs.NI 2024
Year: 2024

This paper considers a patrol inspection scenario where multiple unmanned aerial vehicles (UAVs) are adopted to traverse multiple predetermined cruise points for data collection. The UAVs are connected to cellular networks and they would offload the collected data to the ground base stations (GBSs) for data processing within the constrained duration. This paper proposes a balanced task assignment strategy among patrol UAVs and an energy-efficient trajectory design method. Through jointly optimizing the cruise point assignment, communication scheduling, computational allocation, and UAV trajectory, a novel solution can be obtained to balance the multiple UAVs' task completion time and minimize the total energy consumption. Firstly, we propose a novel clustering method that considers geometry topology, communication rate, and offload volume; it can determine each UAV's cruise points and balance the UAVs' patrol task. Secondly, a hybrid Time-Energy traveling salesman problem is formulated to analyze the cruise point traversal sequence, and the energy-efficient UAV trajectory can be designed by adopting the successive convex approximation (SCA) technique and block coordinate descent (BCD) scheme. The numerical results demonstrate that the proposed balanced task assignment strategy can efficiently balance the multiple UAVs' tasks. Moreover, the min-max task completion time and total energy consumption performance of the proposed solution outperform that of the current conventional approach.