OpenRaiser/Intern-Atlas huggingface dataset:
MiTREE: Multi-input Transformer Ecoregion Encoder for Species Distribution Modelling
Venue: arXiv cs.CV 2024
Year: 2024

Climate change poses an extreme threat to biodiversity, making it imperative to efficiently model the geographical range of different species. The availability of large-scale remote sensing images and environmental data has facilitated the use of machine learning in Species Distribution Models (SDMs), which aim to predict the presence of a species at any given location. Traditional SDMs, reliant on expert observation, are labor-intensive, but advancements in remote sensing and citizen science data have facilitated machine learning approaches to SDM development. However, these models often struggle with leveraging spatial relationships between different inputs -- for instance, learning how climate data should inform the data present in satellite imagery -- without upsampling or distorting the original inputs. Additionally, location information and ecological characteristics at a location play a crucial role in predicting species distribution models, but these aspects have not yet been incorporated into state-of-the-art approaches. In this work, we introduce MiTREE: a multi-input Vision-Transformer-based model with an ecoregion encoder. MiTREE computes spatial cross-modal relationships without upsampling as well as integrates location and ecological context. We evaluate our model on the SatBird Summer and Winter datasets, the goal of which is to predict bird species encounter rates, and we find that our approach improves upon state-of-the-art baselines.
GeoMatch++: Morphology Conditioned Geometry Matching for Multi-Embodiment Grasping
Venue: arXiv cs.RO 2024
Year: 2024

Despite recent progress on multi-finger dexterous grasping, current methods focus on single grippers and unseen objects, and even the ones that explore cross-embodiment, often fail to generalize well to unseen end-effectors. This work addresses the problem of dexterous grasping generalization to unseen end-effectors via a unified policy that learns correlation between gripper morphology and object geometry. Robot morphology contains rich information representing how joints and links connect and move with respect to each other and thus, we leverage it through attention to learn better end-effector geometry features. Our experiments show an average of 9.64% increase in grasp success rate across 3 out-of-domain end-effectors compared to previous methods.
Tempus Core: Area-Power Efficient Temporal-Unary Convolution Core for Low-Precision Edge DLAs
Venue: arXiv cs.AR 2024
Year: 2024

The increasing complexity of deep neural networks (DNNs) poses significant challenges for edge inference deployment due to resource and power constraints of edge devices. Recent works on unary-based matrix multiplication hardware aim to leverage data sparsity and low-precision values to enhance hardware efficiency. However, the adoption and integration of such unary hardware into commercial deep learning accelerators (DLA) remain limited due to processing element (PE) array dataflow differences. This work presents Tempus Core, a convolution core with highly scalable unary-based PE array comprising of tub (temporal-unary-binary) multipliers that seamlessly integrates with the NVDLA (NVIDIA's open-source DLA for accelerating CNNs) while maintaining dataflow compliance and boosting hardware efficiency. Analysis across various datapath granularities shows that for INT8 precision in 45nm CMOS, Tempus Core's PE cell unit (PCU) yields 59.3% and 15.3% reductions in area and power consumption, respectively, over NVDLA's CMAC unit. Considering a 16x16 PE array in Tempus Core, area and power improves by 75% and 62%, respectively, while delivering 5x and 4x iso-area throughput improvements for INT8 and INT4 precisions. Post-place and route analysis of Tempus Core's PCU shows that the 16x4 PE array for INT4 precision in 45nm CMOS requires only 0.017 mm^2 die area and consumes only 6.2mW of total power. We demonstrate that area-power efficient unary-based hardware can be seamlessly integrated into conventional DLAs, paving the path for efficient unary hardware for edge AI inference.
Channel-Aware Optimal Transport: A Theoretical Framework for Generative Communication
Venue: arXiv cs.IT 2024
Year: 2024

Optimal transport has numerous applications, particularly in machine learning tasks involving generative models. In practice, the transportation process often encounters an information bottleneck, typically arising from the conversion of a communication channel into a rate-limited bit pipeline using error correction codes. While this conversion enables a channel-oblivious approach to optimal transport, it fails to fully exploit the available degrees of freedom. Motivated by the emerging paradigm of generative communication, this paper examines the problem of channel-aware optimal transport, where a block of i.i.d. random variables is transmitted through a memoryless channel to generate another block of i.i.d. random variables with a prescribed marginal distribution such that the end-to-end distortion is minimized. With unlimited common randomness available to the encoder and decoder, the source-channel separation architecture is shown to be asymptotically optimal as the blocklength approaches infinity. On the other hand, in the absence of common randomness, the source-channel separation architecture is generally suboptimal. For this scenario, a hybrid coding scheme is proposed, which partially retains the generative capabilities of the given channel while enabling reliable transmission of digital information. It is demonstrated that the proposed hybrid coding scheme can outperform both separation-based and uncoded schemes.
FFCG: Effective and Fast Family Column Generation for Solving Large-Scale Linear Program
Venue: arXiv cs.LG 2024
Year: 2024

Column Generation (CG) is an effective and iterative algorithm to solve large-scale linear programs (LP). During each CG iteration, new columns are added to improve the solution of the LP. Typically, CG greedily selects one column with the most negative reduced cost, which can be improved by adding more columns at once. However, selecting all columns with negative reduced costs would lead to the addition of redundant columns that do not improve the objective value. Therefore, selecting the appropriate columns to add is still an open problem and previous machine-learning-based approaches for CG only add a constant quantity of columns per iteration due to the state-space explosion problem. To address this, we propose Fast Family Column Generation (FFCG) -- a novel reinforcement-learning-based CG that selects a variable number of columns as needed in an iteration. Specifically, we formulate the column selection problem in CG as an MDP and design a reward metric that balances both the convergence speed and the number of redundant columns. In our experiments, FFCG converges faster on the common benchmarks and reduces the number of CG iterations by 77.1% for Cutting Stock Problem (CSP) and 84.8% for Vehicle Routing Problem with Time Windows (VRPTW), and a 71.4% reduction in computing time for CSP and 84.0% for VRPTW on average compared to several state-of-the-art baselines.
Integrating Artificial Open Generative Artificial Intelligence into Software Supply Chain Security
Venue: arXiv cs.CR 2024
Year: 2024

While new technologies emerge, human errors always looming. Software supply chain is increasingly complex and intertwined, the security of a service has become paramount to ensuring the integrity of products, safeguarding data privacy, and maintaining operational continuity. In this work, we conducted experiments on the promising open Large Language Models (LLMs) into two main software security challenges: source code language errors and deprecated code, with a focus on their potential to replace conventional static and dynamic security scanners that rely on predefined rules and patterns. Our findings suggest that while LLMs present some unexpected results, they also encounter significant limitations, particularly in memory complexity and the management of new and unfamiliar data patterns. Despite these challenges, the proactive application of LLMs, coupled with extensive security databases and continuous updates, holds the potential to fortify Software Supply Chain (SSC) processes against emerging threats.
How Can Haptic Feedback Assist People with Blind and Low Vision (BLV): A Systematic Literature Review
Venue: arXiv cs.HC 2024
Year: 2024

People who are blind or have low vision (BLV) encounter numerous challenges in their daily lives and work. To support them, various haptic assistive tools have been developed. Despite these advancements, the effective utilization of these tools -- including the optimal haptic feedback and on-body stimulation positions for different tasks along with their limitations -- remains poorly understood. Recognizing these gaps, we conducted a systematic literature review spanning two decades (2004-2024) to evaluate the development of haptic assistive tools within the HCI community. Our findings reveal that these tools are primarily used for understanding graphical information, providing guidance and navigation, and facilitating education and training, among other life and work tasks. We identified three main limitations: hardware limitations, functionality limitations, and UX and evaluation methods limitations. Based on these insights, we discuss potential research avenues and offer suggestions for enhancing the effectiveness of future haptic assistive technologies.
SILC-EFSA: Self-aware In-context Learning Correction for Entity-level Financial Sentiment Analysis
Venue: arXiv cs.CL 2024
Year: 2024

In recent years, fine-grained sentiment analysis in finance has gained significant attention, but the scarcity of entity-level datasets remains a key challenge. To address this, we have constructed the largest English and Chinese financial entity-level sentiment analysis datasets to date. Building on this foundation, we propose a novel two-stage sentiment analysis approach called Self-aware In-context Learning Correction (SILC). The first stage involves fine-tuning a base large language model to generate pseudo-labeled data specific to our task. In the second stage, we train a correction model using a GNN-based example retriever, which is informed by the pseudo-labeled data. This two-stage strategy has allowed us to achieve state-of-the-art performance on the newly constructed datasets, advancing the field of financial sentiment analysis. In a case study, we demonstrate the enhanced practical utility of our data and methods in monitoring the cryptocurrency market. Our datasets and code are available at https://github.com/NLP-Bin/SILC-EFSA.
Generating Editable Head Avatars with 3D Gaussian GANs
Venue: arXiv cs.CV 2024
Year: 2024

Generating animatable and editable 3D head avatars is essential for various applications in computer vision and graphics. Traditional 3D-aware generative adversarial networks (GANs), often using implicit fields like Neural Radiance Fields (NeRF), achieve photorealistic and view-consistent 3D head synthesis. However, these methods face limitations in deformation flexibility and editability, hindering the creation of lifelike and easily modifiable 3D heads. We propose a novel approach that enhances the editability and animation control of 3D head avatars by incorporating 3D Gaussian Splatting (3DGS) as an explicit 3D representation. This method enables easier illumination control and improved editability. Central to our approach is the Editable Gaussian Head (EG-Head) model, which combines a 3D Morphable Model (3DMM) with texture maps, allowing precise expression control and flexible texture editing for accurate animation while preserving identity. To capture complex non-facial geometries like hair, we use an auxiliary set of 3DGS and tri-plane features. Extensive experiments demonstrate that our approach delivers high-quality 3D-aware synthesis with state-of-the-art controllability. Our code and models are available at https://github.com/liguohao96/EGG3D.
VINEVI: A Virtualized Network Vision Architecture for Smart Monitoring of Heterogeneous Applications and Infrastructures
Venue: arXiv cs.NI 2024
Year: 2024

Monitoring heterogeneous infrastructures and applications is essential to cope with user requirements properly, but it still lacks enhancements. The well-known state-of-the-art methods and tools do not support seamless monitoring of bare-metal, low-cost infrastructures, neither hosted nor virtualized services with fine-grained details. This work proposes VIrtualized NEtwork VIsion architecture (VINEVI), an intelligent method for seamless monitoring heterogeneous infrastructures and applications. The VINEVI architecture advances state of the art with a node-embedded traffic classification agent placing physical and virtualized infrastructures enabling real-time traffic classification. VINEVI combines this real-time traffic classification with well-known tools such as Prometheus and Victoria Metrics to monitor the entire stack from the hardware to the virtualized applications. Experimental results showcased that VINEVI architecture allowed seamless heterogeneous infrastructure monitoring with a higher level of detail beyond literature. Also, our node-embedded real-time Internet traffic classifier evolved with flexibility the methods with monitoring heterogeneous infrastructures seamlessly.
Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
Venue: arXiv cs.CV 2024
Year: 2024

Video Question Answering (Video QA) is a challenging video understanding task that requires models to comprehend entire videos, identify the most relevant information based on contextual cues from a given question, and reason accurately to provide answers. Recent advancements in Multimodal Large Language Models (MLLMs) have transformed video QA by leveraging their exceptional commonsense reasoning capabilities. This progress is largely driven by the effective alignment between visual data and the language space of MLLMs. However, for video QA, an additional space-time alignment poses a considerable challenge for extracting question-relevant information across frames. In this work, we investigate diverse temporal modeling techniques to integrate with MLLMs, aiming to achieve question-guided temporal modeling that leverages pre-trained visual and textual alignment in MLLMs. We propose T-Former, a novel temporal modeling method that creates a question-guided temporal bridge between frame-wise visual perception and the reasoning capabilities of LLMs. Our evaluation across multiple video QA benchmarks demonstrates that T-Former competes favorably with existing temporal modeling approaches and aligns with recent advancements in video QA.
Protecting Cryptographic Libraries against Side-Channel and Code-Reuse Attacks
Venue: arXiv cs.CR 2024
Year: 2024

Cryptographic libraries, an essential part of cybersecurity, are shown to be susceptible to different types of attacks, including side-channel and memory-corruption attacks. In this article, we examine popular cryptographic libraries in terms of the security measures they implement, pinpoint security vulnerabilities, and suggest security improvements in their development process.
A novel framework for MCDM based on Z numbers and soft likelihood function
Venue: arXiv cs.AI 2024
Year: 2024

The optimization on the structure of process of information management under uncertain environment has attracted lots of attention from researchers around the world. Nevertheless, how to obtain accurate and rational evaluation from assessments produced by experts is still an open problem. Specially, intuitionistic fuzzy set provides an effective solution in handling indeterminate information. And Yager proposes a novel method for fusion of probabilistic evidence to handle uncertain and conflicting information lately which is called soft likelihood function. This paper devises a novel framework of soft likelihood function based on information volume of fuzzy membership and credibility measure for extracting truly useful and valuable information from uncertainty. An application is provided to verify the validity and correctness of the proposed framework. Besides, the comparisons with other existing methods further demonstrate the superiority of the novel framework of soft likelihood function.
Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones
Venue: arXiv cs.LG 2024
Year: 2024

Early Exiting (EE) is a promising technique for speeding up inference by adaptively allocating compute resources to data points based on their difficulty. The approach enables predictions to exit at earlier layers for simpler samples while reserving more computation for challenging ones. In this study, we first present a novel perspective on the EE approach, showing that larger models deployed with EE can achieve higher performance than smaller models while maintaining similar computational costs. As existing EE approaches rely on confidence estimation at each exit point, we further study the impact of overconfidence on the controllability of the compute-performance trade-off. We introduce Performance Control Early Exiting (PCEE), a method that enables accuracy thresholding by basing decisions not on a data point's confidence but on the average accuracy of samples with similar confidence levels from a held-out validation set. In our experiments, we show that PCEE offers a simple yet computationally efficient approach that provides better control over performance than standard confidence-based approaches, and allows us to scale up model sizes to yield performance gain while reducing the computational cost.
Sparse recovery from quadratic equations, part II: hardness and incoherence
Venue: arXiv cs.IT 2024
Year: 2024

We study the square root bottleneck in the recovery of sparse vectors from quadratic equations. It is acknowledged that a sparse vector $ \mathbf x_0\in \mathbb{R}^n$, $\| \mathbf x_0\|_0 = k$ can in theory be recovered from as few as $O(k)$ generic quadratic equations but no polynomial time algorithm is known for this task unless $m = Ω(k^2)$. This bottleneck was in fact shown in previous work to be essentially related to the initialization of descent algorithms. Starting such algorithms sufficiently close to the planted signal is known to imply convergence to this signal. In this paper, we show that as soon as $m\gtrsim μ_0^{-2}k \vee μ_0^{-4}$ (up to log factors) where $μ_0 = \| \mathbf x_0\|_\infty/\| \mathbf x_0\|_2$, it is possible to recover a $k$-sparse vector $ \mathbf x_0\in \mathbb{R}^n$ from $m$ quadratic equations of the form $\langle \mathbf A_i, \mathbf x \mathbf x^\intercal\rangle = \langle \mathbf A_i, \mathbf x_0 \mathbf x_0^\intercal\rangle + \varepsilon_i $ by minimizing the classical empirical loss. The proof idea carries over to the phase retrieval setting for which it provides an original initialization that matches the current optimal sample complexity (see e.g. [Cai 2023]). In the maximally incoherent regime $μ_0^{-2}=k$, and for $m=o(k^2)$ we provide evidence for topological hardness by showing that a property known as the Overlap Gap Property (OGP), which originated in spin glass theory and is conjectured to be indicative of algorithmic intractability when optimizing over random structures, holds for a particular level of overparametrization. The key ingredient of the proof is a lower bound on the tail of chi-squared random variables which follows from the theory of moderate deviations.
Semi-Supervised Learning from Small Annotated Data and Large Unlabeled Data for Fine-grained PICO Entity Recognition
Venue: arXiv cs.CL 2024
Year: 2024

Objective: Extracting PICO elements -- Participants, Intervention, Comparison, and Outcomes -- from clinical trial literature is essential for clinical evidence retrieval, appraisal, and synthesis. Existing approaches do not distinguish the attributes of PICO entities. This study aims to develop a named entity recognition (NER) model to extract PICO entities with fine granularities. Materials and Methods: Using a corpus of 2,511 abstracts with PICO mentions from 4 public datasets, we developed a semi-supervised method to facilitate the training of a NER model, FinePICO, by combining limited annotated data of PICO entities and abundant unlabeled data. For evaluation, we divided the entire dataset into two subsets: a smaller group with annotations and a larger group without annotations. We then established the theoretical lower and upper performance bounds based on the performance of supervised learning models trained solely on the small, annotated subset and on the entire set with complete annotations, respectively. Finally, we evaluated FinePICO on both the smaller annotated subset and the larger, initially unannotated subset. We measured the performance of FinePICO using precision, recall, and F1. Results: Our method achieved precision/recall/F1 of 0.567/0.636/0.60, respectively, using a small set of annotated samples, outperforming the baseline model (F1: 0.437) by more than 16\%. The model demonstrates generalizability to a different PICO framework and to another corpus, which consistently outperforms the benchmark in diverse experimental settings (p-value \textless0.001). Conclusion: This study contributes a generalizable and effective semi-supervised approach to named entity recognition leveraging large unlabeled data together with small, annotated data. It also initially supports fine-grained PICO extraction.
A Matrix Logic Approach to Efficient Frequent Itemset Discovery in Large Data Sets
Venue: arXiv cs.DB 2024
Year: 2024

This paper proposes a frequent itemset mining algorithm based on the Boolean matrix method, aiming to solve the storage and computational bottlenecks of traditional frequent pattern mining algorithms in high-dimensional and large-scale transaction databases. By representing the itemsets in the transaction database as Boolean matrices, the algorithm uses Boolean logic operations such as AND and OR to efficiently calculate the support of the itemsets, avoiding the generation and storage of a large number of candidates itemsets in traditional algorithms. The algorithm recursively mines frequent itemsets through matrix operations and can flexibly adapt to different data scales and support thresholds. In the experiment, the public Groceries dataset was selected, and the running efficiency test and frequent itemset mining effect test were designed to evaluate the algorithm's performance indicators such as running time, memory usage, and number of frequent itemsets under different transaction numbers and support thresholds. The experimental results show that the algorithm can efficiently mine a large number of frequent itemsets when the support threshold is low, and focus on strong association rules with high support when the threshold is high. In addition, the changing trends of running time and memory usage show that the Boolean matrix method can still maintain good running efficiency when the number of transactions increases significantly and has high scalability and robustness. Future research can improve memory optimization and matrix block operations, and combine distributed computing and deep learning models to further enhance the algorithm's applicability and real-time processing capabilities in ultra-large-scale data environments. The algorithm has broad application potential and development prospects in the fields of market analysis, recommendation systems, and network security.