OpenRaiser/Intern-Atlas huggingface dataset:
Prototype Optimization with Neural ODE for Few-Shot Learning
Venue: arXiv cs.CV 2024
Year: 2024

Few-Shot Learning (FSL) is a challenging task, which aims to recognize novel classes with few examples. Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then performing class prediction via a cosine classifier with mean-based prototypes. Nevertheless, due to the data scarcity, the mean-based prototypes are usually biased. In this paper, we attempt to diminish the prototype bias by regarding it as a prototype optimization problem. To this end, we propose a novel prototype optimization framework to rectify prototypes, i.e., introducing a meta-optimizer to optimize prototypes. Although the existing meta-optimizers can also be adapted to our framework, they all overlook a crucial gradient bias issue, i.e., the mean-based gradient estimation is also biased on sparse data. To address this issue, in this paper, we regard the gradient and its flow as meta-knowledge and then propose a novel Neural Ordinary Differential Equation (ODE)-based meta-optimizer to optimize prototypes, called MetaNODE. Although MetaNODE has shown superior performance, it suffers from a huge computational burden. To further improve its computation efficiency, we conduct a detailed analysis on MetaNODE and then design an effective and efficient MetaNODE extension version (called E2MetaNODE). It consists of two novel modules: E2GradNet and E2Solver, which aim to estimate accurate gradient flows and solve optimal prototypes in an effective and efficient manner, respectively. Extensive experiments show that 1) our methods achieve superior performance over previous FSL methods and 2) our E2MetaNODE significantly improves computation efficiency meanwhile without performance degradation.
KDC-MAE: Knowledge Distilled Contrastive Mask Auto-Encoder
Venue: arXiv cs.CV 2024
Year: 2024

In this work, we attempted to extend the thought and showcase a way forward for the Self-supervised Learning (SSL) learning paradigm by combining contrastive learning, self-distillation (knowledge distillation) and masked data modelling, the three major SSL frameworks, to learn a joint and coordinated representation. The proposed technique of SSL learns by the collaborative power of different learning objectives of SSL. Hence to jointly learn the different SSL objectives we proposed a new SSL architecture KDC-MAE, a complementary masking strategy to learn the modular correspondence, and a weighted way to combine them coordinately. Experimental results conclude that the contrastive masking correspondence along with the KD learning objective has lent a hand to performing better learning for multiple modalities over multiple tasks.
Enhancing UX Research Activities Using GenAI -- Potential Applications and Challenges
Venue: arXiv cs.HC 2024
Year: 2024

User Experience (UX) Research covers various methods for gathering the users' subjective impressions of a product. For this, practitioners face different activities and tasks related to the research process. This includes processing a large amount of data based on qualitative and quantitative data. However, this can be very laborious in practice. Thus, the application of GenAI can support UX research activities. This paper provides a practical perspective on this topic. Based on previous studies, we present different use cases indicating the potential of GenAI in UX research. Moreover, we provide insights into an exploratory study using GenAI along an entire UX research process. Results show that Large Language Models (LLMs) are useful for various tasks. Thus, the research activities can be carried out more efficiently. However, the researcher should always review results to ensure quality. In summary, we want to express the potential of GenAI enhancing UX research
Could Humans Outshine AI in Visual Data Analysis?
Venue: arXiv cs.HC 2024
Year: 2024

People often use visualizations not only to explore a dataset but also to draw generalizable conclusions about underlying models or phenomena. While previous research has viewed deviations from rational analysis as problematic, we hypothesize that human reliance on non-normative heuristics may be advantageous in certain situations. In this study, we investigate scenarios where human intuition might outperform idealized statistical rationality. Our experiment assesses participants' accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings show that, while participants generally demonstrated lower accuracy than statistical models, they often outperformed Bayesian agents, particularly when dealing with extreme samples. These results suggest that, even when deviating from rationality, human gut reactions to visualizations can provide an advantage. Our findings offer insights into how analyst intuition and statistical models can be integrated to improve inference and decision-making, with important implications for the design of visual analytics tools.
Physics-Guided Detector for SAR Airplanes
Venue: arXiv cs.CV 2024
Year: 2024

The disperse structure distributions (discreteness) and variant scattering characteristics (variability) of SAR airplane targets lead to special challenges of object detection and recognition. The current deep learning-based detectors encounter challenges in distinguishing fine-grained SAR airplanes against complex backgrounds. To address it, we propose a novel physics-guided detector (PGD) learning paradigm for SAR airplanes that comprehensively investigate their discreteness and variability to improve the detection performance. It is a general learning paradigm that can be extended to different existing deep learning-based detectors with "backbone-neck-head" architectures. The main contributions of PGD include the physics-guided self-supervised learning, feature enhancement, and instance perception, denoted as PGSSL, PGFE, and PGIP, respectively. PGSSL aims to construct a self-supervised learning task based on a wide range of SAR airplane targets that encodes the prior knowledge of various discrete structure distributions into the embedded space. Then, PGFE enhances the multi-scale feature representation of a detector, guided by the physics-aware information learned from PGSSL. PGIP is constructed at the detection head to learn the refined and dominant scattering point of each SAR airplane instance, thus alleviating the interference from the complex background. We propose two implementations, denoted as PGD and PGD-Lite, and apply them to various existing detectors with different backbones and detection heads. The experiments demonstrate the flexibility and effectiveness of the proposed PGD, which can improve existing detectors on SAR airplane detection with fine-grained classification task (an improvement of 3.1\% mAP most), and achieve the state-of-the-art performance (90.7\% mAP) on SAR-AIRcraft-1.0 dataset. The project is open-source at \url{https://github.com/XAI4SAR/PGD}.
Emergence of Implicit World Models from Mortal Agents
Venue: arXiv cs.NE 2024
Year: 2024

We discuss the possibility of world models and active exploration as emergent properties of open-ended behavior optimization in autonomous agents. In discussing the source of the open-endedness of living things, we start from the perspective of biological systems as understood by the mechanistic approach of theoretical biology and artificial life. From this perspective, we discuss the potential of homeostasis in particular as an open-ended objective for autonomous agents and as a general, integrative extrinsic motivation. We then discuss the possibility of implicitly acquiring a world model and active exploration through the internal dynamics of a network, and a hypothetical architecture for this, by combining meta-reinforcement learning, which assumes domain adaptation as a system that achieves robust homeostasis.
DGTR: Distributed Gaussian Turbo-Reconstruction for Sparse-View Vast Scenes
Venue: arXiv cs.CV 2024
Year: 2024

Novel-view synthesis (NVS) approaches play a critical role in vast scene reconstruction. However, these methods rely heavily on dense image inputs and prolonged training times, making them unsuitable where computational resources are limited. Additionally, few-shot methods often struggle with poor reconstruction quality in vast environments. This paper presents DGTR, a novel distributed framework for efficient Gaussian reconstruction for sparse-view vast scenes. Our approach divides the scene into regions, processed independently by drones with sparse image inputs. Using a feed-forward Gaussian model, we predict high-quality Gaussian primitives, followed by a global alignment algorithm to ensure geometric consistency. Synthetic views and depth priors are incorporated to further enhance training, while a distillation-based model aggregation mechanism enables efficient reconstruction. Our method achieves high-quality large-scale scene reconstruction and novel-view synthesis in significantly reduced training times, outperforming existing approaches in both speed and scalability. We demonstrate the effectiveness of our framework on vast aerial scenes, achieving high-quality results within minutes. Code will released on our [https://3d-aigc.github.io/DGTR].
Leveraging NFTs for Spectrum Securitization in 6G Networks
Venue: arXiv cs.CE 2024
Year: 2024

Dynamic Spectrum Sharing can enhance spectrum resource utilization by promoting the dynamic distribution of spectrum resources. However, to effectively implement dynamic spectrum resource allocation, certain mechanisms are needed to incentivize primary users to proactively share their spectrum resources. This paper, based on the ERC404 standard and integrating Non-Fungible Token and Fungible Token technologies, proposes a spectrum securitization model to incentivize spectrum resource sharing and implements it on the Ethereum test net.
Do LLMs Understand Ambiguity in Text? A Case Study in Open-world Question Answering
Venue: arXiv cs.CL 2024
Year: 2024

Ambiguity in natural language poses significant challenges to Large Language Models (LLMs) used for open-domain question answering. LLMs often struggle with the inherent uncertainties of human communication, leading to misinterpretations, miscommunications, hallucinations, and biased responses. This significantly weakens their ability to be used for tasks like fact-checking, question answering, feature extraction, and sentiment analysis. Using open-domain question answering as a test case, we compare off-the-shelf and few-shot LLM performance, focusing on measuring the impact of explicit disambiguation strategies. We demonstrate how simple, training-free, token-level disambiguation methods may be effectively used to improve LLM performance for ambiguous question answering tasks. We empirically show our findings and discuss best practices and broader impacts regarding ambiguity in LLMs.
Human-AI Co-Creativity: Exploring Synergies Across Levels of Creative Collaboration
Venue: arXiv cs.HC 2024
Year: 2024

Human-AI co-creativity represents a transformative shift in how humans and generative AI tools collaborate in creative processes. This chapter explores the synergies between human ingenuity and AI capabilities across four levels of interaction: Digital Pen, AI Task Specialist, AI Assistant, and AI Co-Creator. While earlier digital tools primarily facilitated creativity, generative AI systems now contribute actively, demonstrating autonomous creativity in producing novel and valuable outcomes. Empirical evidence from mathematics showcases how AI can extend human creative potential, from computational problem-solving to co-creative partnerships yielding breakthroughs in longstanding challenges. By analyzing these collaborations, the chapter highlights AI's potential to enhance human creativity without replacing it, underscoring the importance of balancing AI's contributions with human oversight and contextual understanding. This integration pushes the boundaries of creative achievements, emphasizing the need for human-centered AI systems that foster collaboration while preserving the unique qualities of human creativity.
Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination
Venue: arXiv cs.CV 2024
Year: 2024

Multimodal large language models (MLLMs) have advanced the integration of visual and linguistic modalities, establishing themselves as the dominant paradigm for visual-language tasks. Current approaches like chain of thought (CoT) reasoning have augmented the cognitive capabilities of large language models (LLMs), yet their adaptation to MLLMs is hindered by heightened risks of hallucination in cross-modality comprehension. In this paper, we find that the thinking while looking paradigm in current multimodal CoT approaches--where reasoning chains are generated alongside visual input--fails to mitigate hallucinations caused by misleading images. To address these limitations, we propose the Visual Inference Chain (VIC) framework, a novel approach that constructs reasoning chains using textual context alone before introducing visual input, effectively reducing cross-modal biases and enhancing multimodal reasoning accuracy. Comprehensive evaluations demonstrate that VIC significantly improves zero-shot performance across various vision-related tasks, mitigating hallucinations while refining the reasoning capabilities of MLLMs. Our code repository can be found at https://github.com/Terry-Xu-666/visual_inference_chain.
STREAM: A Universal State-Space Model for Sparse Geometric Data
Venue: arXiv cs.CV 2024
Year: 2024

Handling sparse and unstructured geometric data, such as point clouds or event-based vision, is a pressing challenge in the field of machine vision. Recently, sequence models such as Transformers and state-space models entered the domain of geometric data. These methods require specialized preprocessing to create a sequential view of a set of points. Furthermore, prior works involving sequence models iterate geometric data with either uniform or learned step sizes, implicitly relying on the model to infer the underlying geometric structure. In this work, we propose to encode geometric structure explicitly into the parameterization of a state-space model. State-space models are based on linear dynamics governed by a one-dimensional variable such as time or a spatial coordinate. We exploit this dynamic variable to inject relative differences of coordinates into the step size of the state-space model. The resulting geometric operation computes interactions between all pairs of N points in O(N) steps. Our model deploys the Mamba selective state-space model with a modified CUDA kernel to efficiently map sparse geometric data to modern hardware. The resulting sequence model, which we call STREAM, achieves competitive results on a range of benchmarks from point-cloud classification to event-based vision and audio classification. STREAM demonstrates a powerful inductive bias for sparse geometric data by improving the PointMamba baseline when trained from scratch on the ModelNet40 and ScanObjectNN point cloud analysis datasets. It further achieves, for the first time, 100% test accuracy on all 11 classes of the DVS128 Gestures dataset.
Securing Satellite Link Segment: A Secure-by-Component Design
Venue: arXiv cs.CR 2024
Year: 2024

The rapid evolution of communication technologies, compounded by recent geopolitical events such as the Viasat cyberattack in February 2022, has highlighted the urgent need for fast and reliable satellite missions for military and civil security operations. Consequently, this paper examines two Earth observation (EO) missions: one utilizing a single low Earth orbit (LEO) satellite and another through a network of LEO satellites, employing a secure-by-component design strategy. This approach begins by defining the scope of technical security engineering, decomposing the system into components and data flows, and enumerating attack surfaces. Then it proceeds by identifying threats to low-level components, applying secure-by-design principles, redesigning components into secure blocks in alignment with the Space Attack Research & Tactic Analysis (SPARTA) framework, and crafting shall statements to refactor the system design, with a particular focus on improving the security of the link segment.
M3D: Dual-Stream Selective State Spaces and Depth-Driven Framework for High-Fidelity Single-View 3D Reconstruction
Venue: arXiv cs.CV 2024
Year: 2024

The precise reconstruction of 3D objects from a single RGB image in complex scenes presents a critical challenge in virtual reality, autonomous driving, and robotics. Existing neural implicit 3D representation methods face significant difficulties in balancing the extraction of global and local features, particularly in diverse and complex environments, leading to insufficient reconstruction precision and quality. We propose M3D, a novel single-view 3D reconstruction framework, to tackle these challenges. This framework adopts a dual-stream feature extraction strategy based on Selective State Spaces to effectively balance the extraction of global and local features, thereby improving scene comprehension and representation precision. Additionally, a parallel branch extracts depth information, effectively integrating visual and geometric features to enhance reconstruction quality and preserve intricate details. Experimental results indicate that the fusion of multi-scale features with depth information via the dual-branch feature extraction significantly boosts geometric consistency and fidelity, achieving state-of-the-art reconstruction performance.
Optimizing Airline Reservation Systems with Edge-Enabled Microservices: A Framework for Real-Time Data Processing and Enhanced User Responsiveness
Venue: arXiv cs.SE 2024
Year: 2024

The growing complexity of the operations of airline reservations requires a smart solution for the adoption of novel approaches to the development of quick, efficient, and adaptive reservation systems. This paper outlines in detail a conceptual framework for the implementation of edge computing microservices in order to address the shortcomings of traditional centralized architectures. Specifically, as edge computing allows for certain activities such as seat inventory checks, booking processes and even confirmation to be done nearer to the user, thus lessening the overall response time and improving the performance of the system. In addition, the framework value should include achieving the high performance of the system such as low latency, high throughput and higher user experience. The major design components include deployed distributed computing microservices orchestrated by Kubernetes, real-time message processing system with Kafka and its elastic scaling. Other operational components include Prometheus and Grafana, which are used to monitor and manage resources, ensuring that all operational processes are optimized. Although this research focuses on a design and theoretical scheming of the framework, its use is foreseen to be more advantageous in facilitating a transform in the provision of services in the airline industry by improving customers' satisfaction, providing infrastructure which is cheap to install and efficiently supporting technology changes such as artificial intelligence and internet of things embedded systems. This research addresses the increasing demand for new technologies with modern well-distributed and real-time-centric systems and also provides a basis for future case implementation and testing. As such, the proposed architecture offers a market-ready, extensible solution to the problems posed by existing airline reservation systems .
Identifying patterns of proprioception and target matching acuity in healthy humans
Venue: arXiv cs.RO 2024
Year: 2024

Traditional approaches to measurement in upper-limb therapy have gaps that electronic sensing and recording can help fill. We highlight shortcomings in current kinematic recording devices, and we introduce a wrist sensing device that performs multimodal sensing during single-axis rotation. Our goal is to characterize normative kinesthetic perception and real-world performance as a multimodal sensory "fingerprint" that can serve as a reference point for identifying deficit in persons affected by stroke, and then as a jumping point for later neuroscientific interrogation. We present an experiment involving psychophysical measurements of passive stimuli discrimination, matching adjustment acuity, and ADL performance in 11 neurologically-intact persons. We found that passive velocity sense and active position sense of healthy controls, measured by velocity discrimination and position matching respectively, correlated in rank with each other, but other score comparisons of acuity or task performance had no statistically significant correlations. We also found that participants differed in acuity between passive and active velocity sense, which supports current understanding about muscle spindle activation being modulated by conscious motor command. The potential for our null correlation results to reveal dissociable aspects of deficit is discussed, as well as implications for future neuroscientific study with more kinematic measures and larger datasets.
Neurosymbolic Graph Enrichment for Grounded World Models
Venue: arXiv cs.AI 2024
Year: 2024

The development of artificial intelligence systems capable of understanding and reasoning about complex real-world scenarios is a significant challenge. In this work we present a novel approach to enhance and exploit LLM reactive capability to address complex problems and interpret deeply contextual real-world meaning. We introduce a method and a tool for creating a multimodal, knowledge-augmented formal representation of meaning that combines the strengths of large language models with structured semantic representations. Our method begins with an image input, utilizing state-of-the-art large language models to generate a natural language description. This description is then transformed into an Abstract Meaning Representation (AMR) graph, which is formalized and enriched with logical design patterns, and layered semantics derived from linguistic and factual knowledge bases. The resulting graph is then fed back into the LLM to be extended with implicit knowledge activated by complex heuristic learning, including semantic implicatures, moral values, embodied cognition, and metaphorical representations. By bridging the gap between unstructured language models and formal semantic structures, our method opens new avenues for tackling intricate problems in natural language understanding and reasoning.
ISAC Super-Resolution Receivers: The Effect of Different Dictionary Matrices
Venue: arXiv cs.IT 2024
Year: 2024

This paper presents an off-the-grid estimator for ISAC systems using lifted atomic norm minimization (LANM). The main challenge in the ISAC systems is the unknown nature of both transmitted signals and radar-communication channels. We use a known dictionary to encode transmit signals and show that LANM can localize radar targets and decode communication symbols when the number of observations is proportional to the system's degrees of freedom and the coherence of the dictionary matrix. We reformulate LANM using a dual method and solve it with semidefinite relaxation (SDR) for different dictionary matrices to reduce the number of observations required at the receiver. Simulations demonstrate that the proposed LANM accurately estimates communication data and target parameters under varying complexity by selecting different dictionary matrices.