OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging Uncertainty Estimates To Improve Classifier Performance
Venue: arXiv cs.LG 2023
Year: 2023

Binary classification involves predicting the label of an instance based on whether the model score for the positive class exceeds a threshold chosen based on the application requirements (e.g., maximizing recall for a precision bound). However, model scores are often not aligned with the true positivity rate. This is especially true when the training involves a differential sampling across classes or there is distributional drift between train and test settings. In this paper, we provide theoretical analysis and empirical evidence of the dependence of model score estimation bias on both uncertainty and score itself. Further, we formulate the decision boundary selection in terms of both model score and uncertainty, prove that it is NP-hard, and present algorithms based on dynamic programming and isotonic regression. Evaluation of the proposed algorithms on three real-world datasets yield 25%-40% gain in recall at high precision bounds over the traditional approach of using model score alone, highlighting the benefits of leveraging uncertainty.
Assessing the Threat Level of Software Supply Chains with the Log Model
Venue: arXiv cs.CR 2023
Year: 2023

The use of free and open source software (FOSS) components in all software systems is estimated to be above 90%. With such high usage and because of the heterogeneity of FOSS tools, repositories, developers and ecosystem, the level of complexity of managing software development has also increased. This has amplified both the attack surface for malicious actors and the difficulty of making sure that the software products are free from threats. The rise of security incidents involving high profile attacks is evidence that there is still much to be done to safeguard software products and the FOSS supply chain. Software Composition Analysis (SCA) tools and the study of attack trees help with improving security. However, they still lack the ability to comprehensively address how interactions within the software supply chain may impact security. This work presents a novel approach of assessing threat levels in FOSS supply chains with the log model. This model provides information capture and threat propagation analysis that not only account for security risks that may be caused by attacks and the usage of vulnerable software, but also how they interact with the other elements to affect the threat level for any element in the model.
A Zero-Forcing Approach for the RIS-Aided MIMO Broadcast Channel
Venue: arXiv cs.IT 2023
Year: 2023

We present efficient algorithms for the sum-spectral efficiency (SE) maximization of the multi-user reconfigurable intelligent surface (RIS)-aided multiple-input multiple-output (MIMO) broadcast channel based on a zero-forcing approach. These methods conduct a user allocation for which the computation is independent of the number of elements at the RIS, that is usually large. Specifically, two algorithms are given that exploit the line-of-sight (LOS) structure between the base station (BS) and the RIS. Simulations show superior SE performance compared to other linear precoding algorithms but with lower complexity.
Practical cross-sensor color constancy using a dual-mapping strategy
Venue: arXiv cs.CV 2023
Year: 2023

Deep Neural Networks (DNNs) have been widely used for illumination estimation, which is time-consuming and requires sensor-specific data collection. Our proposed method uses a dual-mapping strategy and only requires a simple white point from a test sensor under a D65 condition. This allows us to derive a mapping matrix, enabling the reconstructions of image data and illuminants. In the second mapping phase, we transform the re-constructed image data into sparse features, which are then optimized with a lightweight multi-layer perceptron (MLP) model using the re-constructed illuminants as ground truths. This approach effectively reduces sensor discrepancies and delivers performance on par with leading cross-sensor methods. It only requires a small amount of memory (~0.003 MB), and takes ~1 hour training on an RTX3070Ti GPU. More importantly, the method can be implemented very fast, with ~0.3 ms and ~1 ms on a GPU or CPU respectively, and is not sensitive to the input image resolution. Therefore, it offers a practical solution to the great challenges of data recollection that is faced by the industry.
Simultaneous Robot-World and Hand-Eye Calibration
Venue: arXiv cs.RO 2023
Year: 2023

Recently, Zhuang, Roth, \& Sudhakar [1] proposed a method that allows simultaneous computation of the rigid transformations from world frame to robot base frame and from hand frame to camera frame. Their method attempts to solve a homogeneous matrix equation of the form AX=ZB. They use quaternions to derive explicit linear solutions for X and Z. In this short paper, we present two new solutions that attempt to solve the homogeneous matrix equation mentioned above: (i) a closed-form method which uses quaternion algebra and a positive quadratic error function associated with this representation and (ii) a method based on non-linear constrained minimization and which simultaneously solves for rotations and translations. These results may be useful to other problems that can be formulated in the same mathematical form. We perform a sensitivity analysis for both our two methods and the linear method developed by Zhuang et al. This analysis allows the comparison of the three methods. In the light of this comparison the non-linear optimization method, which solves for rotations and translations simultaneously, seems to be the most stable one with respect to noise and to measurement errors.
LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
Venue: arXiv cs.CV 2023
Year: 2023

Multimodal Large Language Models (MLLMs) have endowed LLMs with the ability to perceive and understand multi-modal signals. However, most of the existing MLLMs mainly adopt vision encoders pretrained on coarsely aligned image-text pairs, leading to insufficient extraction and reasoning of visual knowledge. To address this issue, we devise a dual-Level vIsual knOwledge eNhanced Multimodal Large Language Model (LION), which empowers the MLLM by injecting visual knowledge in two levels. 1) Progressive incorporation of fine-grained spatial-aware visual knowledge. We design a vision aggregator cooperated with region-level vision-language (VL) tasks to incorporate fine-grained spatial-aware visual knowledge into the MLLM. To alleviate the conflict between image-level and region-level VL tasks during incorporation, we devise a dedicated stage-wise instruction-tuning strategy with mixture-of-adapters. This progressive incorporation scheme contributes to the mutual promotion between these two kinds of VL tasks. 2) Soft prompting of high-level semantic visual evidence. We facilitate the MLLM with high-level semantic visual evidence by leveraging diverse image tags. To mitigate the potential influence caused by imperfect predicted tags, we propose a soft prompting method by embedding a learnable token into the tailored text instruction. Comprehensive experiments on several multi-modal benchmarks demonstrate the superiority of our model (e.g., improvement of 5% accuracy on VSR and 3% CIDEr on TextCaps over InstructBLIP, 5% accuracy on RefCOCOg over Kosmos-2).
Generating Valid and Natural Adversarial Examples with Large Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Deep learning-based natural language processing (NLP) models, particularly pre-trained language models (PLMs), have been revealed to be vulnerable to adversarial attacks. However, the adversarial examples generated by many mainstream word-level adversarial attack models are neither valid nor natural, leading to the loss of semantic maintenance, grammaticality, and human imperceptibility. Based on the exceptional capacity of language understanding and generation of large language models (LLMs), we propose LLM-Attack, which aims at generating both valid and natural adversarial examples with LLMs. The method consists of two stages: word importance ranking (which searches for the most vulnerable words) and word synonym replacement (which substitutes them with their synonyms obtained from LLMs). Experimental results on the Movie Review (MR), IMDB, and Yelp Review Polarity datasets against the baseline adversarial attack models illustrate the effectiveness of LLM-Attack, and it outperforms the baselines in human and GPT-4 evaluation by a significant margin. The model can generate adversarial examples that are typically valid and natural, with the preservation of semantic meaning, grammaticality, and human imperceptibility.
Secure Data Transmission over Insecure Radio Channel in Wireless of Things (WoT) Network
Venue: arXiv cs.CR 2023
Year: 2023

Potential capacity of processors is enhancing rapidly which leads to the increase of computational ability of the adversary. As a result, the required key size for conventional encryption techniques is growing everyday for complex unbreakable security communication systems. The Public Key Cryptography (PKC) techniques which use larger keys cannot be fitted in tiny resource constrained Wireless of Things (WoT) devices. Some Symmetric Key Cryptosystems (SKC) use smaller keys, which can be fitted in the tiny devices. But in large networks where the number of nodes is in the order of 103, the memory constraint does not allow the system to do so. The existing secure data communication in insecure medium uses various conventional encryption methods like Public Key Cryptography (PKC) and Symmetric Key Cryptosystems (SKC). Generally, modern encryption methods need huge processing power, memory and time. Also in some cases, Key Pre-distribution System (KPS) is used among different communicating devices. With the growing need for larger key size in the conventional secure communication system, the existing resources in the communicating devices suffer from resource starvation. Hence, the need of a novel mechanism for secure communication is inevitable. But the existing secure communication mechanisms like PKC, SKC or KPS do not ensure elimination of resource starvation issue in tiny devices during communication. In these existing conventional mechanisms, the plain text is generally converted into cipher text with greater size than the plain text at the device level, which leads to resource starvation. At the time of transmission, the cipher text at the device end requires more bandwidth than the plain text which puts bandwidth overhead on the broadcast channel (BC).
Look into the Mirror: Evolving Self-Dual Bent Boolean Functions
Venue: arXiv cs.NE 2023
Year: 2023

Bent Boolean functions are important objects in cryptography and coding theory, and there are several general approaches for constructing such functions. Metaheuristics proved to be a strong choice as they can provide many bent functions, even when the size of the Boolean function is large (e.g., more than 20 inputs). While bent Boolean functions represent only a small part of all Boolean functions, there are several subclasses of bent functions providing specific properties and challenges. One of the most interesting subclasses comprises (anti-)self-dual bent Boolean functions. This paper provides a detailed experimentation with evolutionary algorithms with the goal of evolving (anti-)self-dual bent Boolean functions. We experiment with two encodings and two fitness functions to directly evolve self-dual bent Boolean functions. Our experiments consider Boolean functions with sizes of up to 16 inputs, and we successfully construct self-dual bent functions for each dimension. Moreover, when comparing with the evolution of bent Boolean functions, we notice that the difficulty for evolutionary algorithms is rather similar. Finally, we also tried evolving secondary constructions for self-dual bent functions, but this direction provided no successful results.
SniffyArt: The Dataset of Smelling Persons
Venue: arXiv cs.CV 2023
Year: 2023

Smell gestures play a crucial role in the investigation of past smells in the visual arts yet their automated recognition poses significant challenges. This paper introduces the SniffyArt dataset, consisting of 1941 individuals represented in 441 historical artworks. Each person is annotated with a tightly fitting bounding box, 17 pose keypoints, and a gesture label. By integrating these annotations, the dataset enables the development of hybrid classification approaches for smell gesture recognition. The datasets high-quality human pose estimation keypoints are achieved through the merging of five separate sets of keypoint annotations per person. The paper also presents a baseline analysis, evaluating the performance of representative algorithms for detection, keypoint estimation, and classification tasks, showcasing the potential of combining keypoint estimation with smell gesture classification. The SniffyArt dataset lays a solid foundation for future research and the exploration of multi-task approaches leveraging pose keypoints and person boxes to advance human gesture and olfactory dimension analysis in historical artworks.
Real-Time Surface-to-Air Missile Engagement Zone Prediction Using Simulation and Machine Learning
Venue: arXiv cs.LG 2023
Year: 2023

Surface-to-Air Missiles (SAMs) are crucial in modern air defense systems. A critical aspect of their effectiveness is the Engagement Zone (EZ), the spatial region within which a SAM can effectively engage and neutralize a target. Notably, the EZ is intrinsically related to the missile's maximum range; it defines the furthest distance at which a missile can intercept a target. The accurate computation of this EZ is essential but challenging due to the dynamic and complex factors involved, which often lead to high computational costs and extended processing times when using conventional simulation methods. In light of these challenges, our study investigates the potential of machine learning techniques, proposing an approach that integrates machine learning with a custom-designed simulation tool to train supervised algorithms. We leverage a comprehensive dataset of pre-computed SAM EZ simulations, enabling our model to accurately predict the SAM EZ for new input parameters. It accelerates SAM EZ simulations, enhances air defense strategic planning, and provides real-time insights, improving SAM system performance. The study also includes a comparative analysis of machine learning algorithms, illuminating their capabilities and performance metrics and suggesting areas for future research, highlighting the transformative potential of machine learning in SAM EZ simulations.
Use of Augmented Reality in Human Wayfinding: A Systematic Review
Venue: arXiv cs.HC 2023
Year: 2023

Augmented reality technology has emerged as a promising solution to assist with wayfinding difficulties, bridging the gap between obtaining navigational assistance and maintaining an awareness of one's real-world surroundings. This article presents a systematic review of research literature related to AR navigation technologies. An in-depth analysis of 65 salient studies was conducted, addressing four main research topics: 1) current state-of-the-art of AR navigational assistance technologies, 2) user experiences with these technologies, 3) the effect of AR on human wayfinding performance, and 4) impacts of AR on human navigational cognition. Notably, studies demonstrate that AR can decrease cognitive load and improve cognitive map development, in contrast to traditional guidance modalities. However, findings regarding wayfinding performance and user experience were mixed. Some studies suggest little impact of AR on improving outdoor navigational performance, and certain information modalities may be distracting and ineffective. This article discusses these nuances in detail, supporting the conclusion that AR holds great potential in enhancing wayfinding by providing enriched navigational cues, interactive experiences, and improved situational awareness.
Coded Computing for Fault-Tolerant Parallel QR Decomposition
Venue: arXiv cs.DC 2023
Year: 2023

QR decomposition is an essential operation for solving linear equations and obtaining least-squares solutions. In high-performance computing systems, large-scale parallel QR decomposition often faces node faults. We address this issue by proposing a fault-tolerant algorithm that incorporates `coded computing' into the parallel Gram-Schmidt method, commonly used for QR decomposition. Coded computing introduces error-correcting codes into computational processes to enhance resilience against intermediate failures. While traditional coding strategies cannot preserve the orthogonality of $Q$, recent work has proven a post-orthogonalization condition that allows low-cost restoration of the degraded orthogonality. In this paper, we construct a checksum-generator matrix for multiple-node failures that satisfies the post-orthogonalization condition and prove that our code satisfies the maximum-distance separable (MDS) property with high probability. Furthermore, we consider in-node checksum storage setting where checksums are stored in original nodes. We obtain the minimal number of checksums required to be resilient to any $f$ failures under the in-node checksum storage, and also propose an in-node systematic MDS coding strategy that achieves the lower bound. Extensive experiments validate our theories and showcase the negligible overhead of our coded computing framework for fault-tolerant QR decomposition.
FinanceBench: A New Benchmark for Financial Question Answering
Venue: arXiv cs.CL 2023
Year: 2023

FinanceBench is a first-of-its-kind test suite for evaluating the performance of LLMs on open book financial question answering (QA). It comprises 10,231 questions about publicly traded companies, with corresponding answers and evidence strings. The questions in FinanceBench are ecologically valid and cover a diverse set of scenarios. They are intended to be clear-cut and straightforward to answer to serve as a minimum performance standard. We test 16 state of the art model configurations (including GPT-4-Turbo, Llama2 and Claude2, with vector stores and long context prompts) on a sample of 150 cases from FinanceBench, and manually review their answers (n=2,400). The cases are available open-source. We show that existing LLMs have clear limitations for financial QA. Notably, GPT-4-Turbo used with a retrieval system incorrectly answered or refused to answer 81% of questions. While augmentation techniques such as using longer context window to feed in relevant evidence improve performance, they are unrealistic for enterprise settings due to increased latency and cannot support larger financial documents. We find that all models examined exhibit weaknesses, such as hallucinations, that limit their suitability for use by enterprises.
Correlated Attention in Transformers for Multivariate Time Series
Venue: arXiv cs.LG 2023
Year: 2023

Multivariate time series (MTS) analysis prevails in real-world applications such as finance, climate science and healthcare. The various self-attention mechanisms, the backbone of the state-of-the-art Transformer-based models, efficiently discover the temporal dependencies, yet cannot well capture the intricate cross-correlation between different features of MTS data, which inherently stems from complex dynamical systems in practice. To this end, we propose a novel correlated attention mechanism, which not only efficiently captures feature-wise dependencies, but can also be seamlessly integrated within the encoder blocks of existing well-known Transformers to gain efficiency improvement. In particular, correlated attention operates across feature channels to compute cross-covariance matrices between queries and keys with different lag values, and selectively aggregate representations at the sub-series level. This architecture facilitates automated discovery and representation learning of not only instantaneous but also lagged cross-correlations, while inherently capturing time series auto-correlation. When combined with prevalent Transformer baselines, correlated attention mechanism constitutes a better alternative for encoder-only architectures, which are suitable for a wide range of tasks including imputation, anomaly detection and classification. Extensive experiments on the aforementioned tasks consistently underscore the advantages of correlated attention mechanism in enhancing base Transformer models, and demonstrate our state-of-the-art results in imputation, anomaly detection and classification.
Adaptive Training Distributions with Scalable Online Bilevel Optimization
Venue: arXiv cs.LG 2023
Year: 2023

Large neural networks pretrained on web-scale corpora are central to modern machine learning. In this paradigm, the distribution of the large, heterogeneous pretraining data rarely matches that of the application domain. This work considers modifying the pretraining distribution in the case where one has a small sample of data reflecting the targeted test conditions. We propose an algorithm motivated by a recent formulation of this setting as an online, bilevel optimization problem. With scalability in mind, our algorithm prioritizes computing gradients at training points which are likely to most improve the loss on the targeted distribution. Empirically, we show that in some cases this approach is beneficial over existing strategies from the domain adaptation literature but may not succeed in other cases. We propose a simple test to evaluate when our approach can be expected to work well and point towards further research to address current limitations.
Rate-Independent Gradient Crystal Plasticity Theory -- Robust Algorithmic Formulations based on Incremental Energy Minimization
Venue: arXiv cs.CE 2023
Year: 2023

Numerically robust algorithmic formulations suitable for rate-independent crystal plasticity are presented. They cover classic local models as well as gradient-enhanced theories in which the gradients of the plastic slips are incorporated by means of the micromorphic approach. The elaborated algorithmic formulations rely on the underlying variational structure of (associative) crystal plasticity. To be more precise and in line with so-called variational constitutive updates or incremental energy minimization principles, an incrementally defined energy derived from the underlying time-continuous constitutive model represents the starting point of the novel numerically robust algorithmic formulations. This incrementally defined potential allows to compute all variables jointly as minimizers of this energy. While such discrete variational constitutive updates are not new in general, they are considered here in order to employ powerful techniques from non-linear constrained optimization theory in order to compute robustly the aforementioned minimizers. The analyzed prototype models are based on (1) nonlinear complementarity problem (NCP) functions as well as on (2) the augmented Lagrangian formulation. Numerical experiments show the numerical robustness of the resulting algorithmic formulations. Furthermore, it is shown that the novel algorithmic ideas can also be integrated into classic, non-variational, return-mapping schemes.
The ImmersaDesk3 -- Experiences With A Flat Panel Display for Virtual Reality
Venue: arXiv cs.HC 2023
Year: 2023

In this paper we discuss the design and implementation of a plasma display panel for a wide field of view desktop virtual reality environment. Present commercial plasma displays are not designed with virtual reality in mind, leading to several problems in generating stereo imagery and obtaining good tracking information. Although we developed solutions for a number of these problems, the limitations of the system preclude its current use in practical applications, and point to issues that must be resolved for flat panel displays to be useful for VR.