OpenRaiser/Intern-Atlas huggingface dataset:
Selective Encryption using Segmentation Mask with Chaotic Henon Map for Multidimensional Medical Images
Venue: arXiv cs.CR 2024
Year: 2024

A user-centric design and resource optimization should be at the center of any technology or innovation. The user-centric perspective gives the developer the opportunity to develop with task-based optimization. The user in the medical image field is a medical professional who analyzes the medical images and gives their diagnosis results to the patient. This scheme, having the medical professional user's perspective, innovates in the area of Medical Image storage and security. The architecture is designed with three main segments, namely: Segmentation, Storage, and Retrieval. This architecture was designed owing to the fact that the number of retrieval operations done by medical professionals was toweringly higher when compared to the storage operations done for some handful number of times for a particular medical image. This gives room for our innovation to segment out the medically indispensable part of the medical image, encrypt it, and store it. By encrypting the vital parts of the image using a strong encryption algorithm like the chaotic Henon map, we are able to keep the security intact. Now retrieving the medical image demands only the computationally less stressing decryption of the segmented region of interest. The decryption of the segmented region of interest results in the full recovery of the medical image which can be viewed on demand by the medical professionals for various diagnosis purposes. In this scheme, we were able to achieve a retrieval speed improvement of around 47% when compared to a full image encryption of brain medical CT images.
A Survey on Temporal Knowledge Graph: Representation Learning and Applications
Venue: arXiv cs.CL 2024
Year: 2024

Knowledge graphs have garnered significant research attention and are widely used to enhance downstream applications. However, most current studies mainly focus on static knowledge graphs, whose facts do not change with time, and disregard their dynamic evolution over time. As a result, temporal knowledge graphs have attracted more attention because a large amount of structured knowledge exists only within a specific period. Knowledge graph representation learning aims to learn low-dimensional vector embeddings for entities and relations in a knowledge graph. The representation learning of temporal knowledge graphs incorporates time information into the standard knowledge graph framework and can model the dynamics of entities and relations over time. In this paper, we conduct a comprehensive survey of temporal knowledge graph representation learning and its applications. We begin with an introduction to the definitions, datasets, and evaluation metrics for temporal knowledge graph representation learning. Next, we propose a taxonomy based on the core technologies of temporal knowledge graph representation learning methods, and provide an in-depth analysis of different methods in each category. Finally, we present various downstream applications related to the temporal knowledge graphs. In the end, we conclude the paper and have an outlook on the future research directions in this area.
Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
Venue: arXiv cs.CR 2024
Year: 2024

Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP), offering transformative capabilities in understanding and generating human-like text. However, with their rising prominence, the security and vulnerability aspects of these models have garnered significant attention. This paper presents a comprehensive survey of the various forms of attacks targeting LLMs, discussing the nature and mechanisms of these attacks, their potential impacts, and current defense strategies. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation. The paper also explores the effectiveness of different attack methodologies, the resilience of LLMs against these attacks, and the implications for model integrity and user trust. By examining the latest research, we provide insights into the current landscape of LLM vulnerabilities and defense mechanisms. Our objective is to offer a nuanced understanding of LLM attacks, foster awareness within the AI community, and inspire robust solutions to mitigate these risks in future developments.
Topic Modeling Analysis of Aviation Accident Reports: A Comparative Study between LDA and NMF Models
Venue: arXiv cs.CL 2024
Year: 2024

Aviation safety is paramount in the modern world, with a continuous commitment to reducing accidents and improving safety standards. Central to this endeavor is the analysis of aviation accident reports, rich textual resources that hold insights into the causes and contributing factors behind aviation mishaps. This paper compares two prominent topic modeling techniques, Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), in the context of aviation accident report analysis. The study leverages the National Transportation Safety Board (NTSB) Dataset with the primary objective of automating and streamlining the process of identifying latent themes and patterns within accident reports. The Coherence Value (C_v) metric was used to evaluate the quality of generated topics. LDA demonstrates higher topic coherence, indicating stronger semantic relevance among words within topics. At the same time, NMF excelled in producing distinct and granular topics, enabling a more focused analysis of specific aspects of aviation accidents.
Online Training of Large Language Models: Learn while chatting
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models(LLMs) have dramatically revolutionized the field of Natural Language Processing(NLP), offering remarkable capabilities that have garnered widespread usage. However, existing interaction paradigms between LLMs and users are constrained by either inflexibility, limitations in customization, or a lack of persistent learning. This inflexibility is particularly evident as users, especially those without programming skills, have restricted avenues to enhance or personalize the model. Existing frameworks further complicate the model training and deployment process due to their computational inefficiencies and lack of user-friendly interfaces. To overcome these challenges, this paper introduces a novel interaction paradigm-'Online Training using External Interactions'-that merges the benefits of persistent, real-time model updates with the flexibility for individual customization through external interactions such as AI agents or online/offline knowledge bases.
UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining
Venue: arXiv cs.CV 2024
Year: 2024

Tables convey factual and quantitative data with implicit conventions created by humans that are often challenging for machines to parse. Prior work on table recognition (TR) has mainly centered around complex task-specific combinations of available inputs and tools. We present UniTable, a training framework that unifies both the training paradigm and training objective of TR. Its training paradigm combines the simplicity of purely pixel-level inputs with the effectiveness and scalability empowered by self-supervised pretraining from diverse unannotated tabular images. Our framework unifies the training objectives of all three TR tasks - extracting table structure, cell content, and cell bounding box - into a unified task-agnostic training objective: language modeling. Extensive quantitative and qualitative analyses highlight UniTable's state-of-the-art (SOTA) performance on four of the largest TR datasets. UniTable's table parsing capability has surpassed both existing TR methods and general large vision-language models, e.g., GPT-4o, GPT-4-turbo with vision, and LLaVA. Our code is publicly available at https://github.com/poloclub/unitable, featuring a Jupyter Notebook that includes the complete inference pipeline, fine-tuned across multiple TR datasets, supporting all three TR tasks.
Self-Supervision in Time for Satellite Images(S3-TSS): A novel method of SSL technique in Satellite images
Venue: arXiv cs.AI 2024
Year: 2024

With the limited availability of labeled data with various atmospheric conditions in remote sensing images, it seems useful to work with self-supervised algorithms. Few pretext-based algorithms, including from rotation, spatial context and jigsaw puzzles are not appropriate for satellite images. Often, satellite images have a higher temporal frequency. So, the temporal dimension of remote sensing data provides natural augmentation without requiring us to create artificial augmentation of images. Here, we propose S3-TSS, a novel method of self-supervised learning technique that leverages natural augmentation occurring in temporal dimension. We compare our results with current state-of-the-art methods and also perform various experiments. We observed that our method was able to perform better than baseline SeCo in four downstream datasets. Code for our work can be found here: https://github.com/hewanshrestha/Why-Self-Supervision-in-Time
ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
Venue: arXiv cs.IR 2024
Year: 2024

Applications increasingly leverage mixed-modality data, and must jointly search over vector data, such as embedded images, text and video, as well as structured data, such as attributes and keywords. Proposed methods for this hybrid search setting either suffer from poor performance or support a severely restricted set of search predicates (e.g., only small sets of equality predicates), making them impractical for many applications. To address this, we present ACORN, an approach for performant and predicate-agnostic hybrid search. ACORN builds on Hierarchical Navigable Small Worlds (HNSW), a state-of-the-art graph-based approximate nearest neighbor index, and can be implemented efficiently by extending existing HNSW libraries. ACORN introduces the idea of predicate subgraph traversal to emulate a theoretically ideal, but impractical, hybrid search strategy. ACORN's predicate-agnostic construction algorithm is designed to enable this effective search strategy, while supporting a wide array of predicate sets and query semantics. We systematically evaluate ACORN on both prior benchmark datasets, with simple, low-cardinality predicate sets, and complex multi-modal datasets not supported by prior methods. We show that ACORN achieves state-of-the-art performance on all datasets, outperforming prior methods with 2-1,000x higher throughput at a fixed recall.
Improved Lower Bound for Differentially Private Facility Location
Venue: arXiv cs.DS 2024
Year: 2024

We consider the differentially private (DP) facility location problem in the so called super-set output setting proposed by Gupta et al. [SODA 2010]. The current best known expected approximation ratio for an $ε$-DP algorithm is $O\left(\frac{\log n}{\sqrtε}\right)$ due to Cohen-Addad et al. [AISTATS 2022] where $n$ denote the size of the metric space, meanwhile the best known lower bound is $Ω(1/\sqrtε)$ [NeurIPS 2019]. In this short note, we give a lower bound of $\tildeΩ\left(\min\left\{\log n, \sqrt{\frac{\log n}ε}\right\}\right)$ on the expected approximation ratio of any $ε$-DP algorithm, which is the first evidence that the approximation ratio has to grow with the size of the metric space.
Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering
Venue: arXiv cs.CL 2024
Year: 2024

In this paper, we propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios, along with clinician-approved reasoned answers. Additionally, we implement a prompt driven by Chain of Thought (CoT) reasoning, CLINICR, to mirror the prospective process of incremental reasoning, reaching a correct response to medical questions. We empirically demonstrate how CLINICR outperforms the state-of-the-art 5-shot CoT-based prompt (Liévin et al., 2022). We also present an approach that mirrors real-life clinical practice by first exploring multiple differential diagnoses through MCQ-CLINICR and subsequently narrowing down to a final diagnosis using MCQ-ELIMINATIVE. Finally, emphasizing the importance of response verification in medical settings, we utilize a reward model mechanism, replacing the elimination process performed by MCQ-ELIMINATIVE.
A Clique-Based Separator for Intersection Graphs of Geodesic Disks in $\mathbb{R}^2$
Venue: arXiv cs.CG 2024
Year: 2024

Let $d$ be a (well-behaved) shortest-path metric defined on a path-connected subset of $\mathbb{R}^2$ and let $\mathcal{D}=\{D_1,\ldots,D_n\}$ be a set of geodesic disks with respect to the metric $d$. We prove that $\mathcal{G}^{\times}(\mathcal{D})$, the intersection graph of the disks in $\mathcal{D}$, has a clique-based separator consisting of $O(n^{3/4+\varepsilon})$ cliques. This significantly extends the class of objects whose intersection graphs have small clique-based separators. Our clique-based separator yields an algorithm for $q$-COLORING that runs in time $2^{O(n^{3/4+\varepsilon})}$, assuming the boundaries of the disks $D_i$ can be computed in polynomial time. We also use our clique-based separator to obtain a simple, efficient, and almost exact distance oracle for intersection graphs of geodesic disks. Our distance oracle uses $O(n^{7/4+\varepsilon})$ storage and can report the hop distance between any two nodes in $\mathcal{G}^{\times}(\mathcal{D})$ in $O(n^{3/4+\varepsilon})$ time, up to an additive error of one. So far, distance oracles with an additive error of one that use subquadratic storage and sublinear query time were not known for such general graph classes.
Improving the Equation of Exchange for Cryptoasset Valuation Using Empirical Data
Venue: arXiv cs.CE 2024
Year: 2024

In the evolving domain of cryptocurrency markets, accurate token valuation remains a critical aspect influencing investment decisions and policy development. Whilst the prevailing equation of exchange pricing model offers a quantitative valuation approach based on the interplay between token price, transaction volume, supply, and either velocity or holding time, it exhibits intrinsic shortcomings. Specifically, the model may not consistently delineate the relationship between average token velocity and holding time. This paper aims to refine this equation, enhancing the depth of insight into token valuation methodologies.
Designing Human-Machine Interactions in the Automated City: Methodologies, Considerations, Principles
Venue: arXiv cs.HC 2024
Year: 2024

Technological progress paves the way to ever-increasing opportunities for automating city services. This spans from already existing concepts, such as automated shuttles at airports, to more speculative applications, such as fully autonomous delivery robots. As these services are being automated, it is critical that this process is underpinned by a human-centred perspective. This chapter provides a framework for future research and practice in this emerging domain. It draws on research from the field of human-computer interaction and introduces a number of methodologies that can be used to structure the process of designing interactions between people and automated urban applications. Based on research case studies, the chapter discusses specific elements that need to be considered when designing human-machine interactions in an urban environment. The chapter further proposes a model for designing automated urban applications and a set of principles to guide their prototyping and deployment.
Know Your Audience: The benefits and pitfalls of generating plain language summaries beyond the "general" audience
Venue: arXiv cs.HC 2024
Year: 2024

Language models (LMs) show promise as tools for communicating science to the general public by simplifying and summarizing complex language. Because models can be prompted to generate text for a specific audience (e.g., college-educated adults), LMs might be used to create multiple versions of plain language summaries for people with different familiarities of scientific topics. However, it is not clear what the benefits and pitfalls of adaptive plain language are. When is simplifying necessary, what are the costs in doing so, and do these costs differ for readers with different background knowledge? Through three within-subjects studies in which we surface summaries for different envisioned audiences to participants of different backgrounds, we found that while simpler text led to the best reading experience for readers with little to no familiarity in a topic, high familiarity readers tended to ignore certain details in overly plain summaries (e.g., study limitations). Our work provides methods and guidance on ways of adapting plain language summaries beyond the single "general" audience.
Profile of Vulnerability Remediations in Dependencies Using Graph Analysis
Venue: arXiv cs.SE 2024
Year: 2024

This research introduces graph analysis methods and a modified Graph Attention Convolutional Neural Network (GAT) to the critical challenge of open source package vulnerability remediation by analyzing control flow graphs to profile breaking changes in applications occurring from dependency upgrades intended to remediate vulnerabilities. Our approach uniquely applies node centrality metrics -- degree, norm, and closeness centrality -- to the GAT model, enabling a detailed examination of package code interactions with a focus on identifying and understanding vulnerable nodes, and when dependency package upgrades will interfere with application workflow. The study's application on a varied dataset reveals an unexpected limited inter-connectivity of vulnerabilities in core code, thus challenging established notions in software security. The results demonstrate the effectiveness of the enhanced GAT model in offering nuanced insights into the relational dynamics of code vulnerabilities, proving its potential in advancing cybersecurity measures. This approach not only aids in the strategic mitigation of vulnerabilities but also lays the groundwork for the development of sophisticated, sustainable monitoring systems for the evaluation of work effort for vulnerability remediation resulting from open source software. The insights gained from this study mark a significant advancement in the field of package vulnerability analysis and cybersecurity.
Debiased Multimodal Understanding for Human Language Sequences
Venue: arXiv cs.AI 2024
Year: 2024

Human multimodal language understanding (MLU) is an indispensable component of expression analysis (e.g., sentiment or humor) from heterogeneous modalities, including visual postures, linguistic contents, and acoustic behaviours. Existing works invariably focus on designing sophisticated structures or fusion strategies to achieve impressive improvements. Unfortunately, they all suffer from the subject variation problem due to data distribution discrepancies among subjects. Concretely, MLU models are easily misled by distinct subjects with different expression customs and characteristics in the training data to learn subject-specific spurious correlations, limiting performance and generalizability across new subjects. Motivated by this observation, we introduce a recapitulative causal graph to formulate the MLU procedure and analyze the confounding effect of subjects. Then, we propose SuCI, a simple yet effective causal intervention module to disentangle the impact of subjects acting as unobserved confounders and achieve model training via true causal effects. As a plug-and-play component, SuCI can be widely applied to most methods that seek unbiased predictions. Comprehensive experiments on several MLU benchmarks clearly show the effectiveness of the proposed module.
Data-Dependent LSH for the Earth Mover's Distance
Venue: arXiv cs.DS 2024
Year: 2024

We give new data-dependent locality sensitive hashing schemes (LSH) for the Earth Mover's Distance ($\mathsf{EMD}$), and as a result, improve the best approximation for nearest neighbor search under $\mathsf{EMD}$ by a quadratic factor. Here, the metric $\mathsf{EMD}_s(\mathbb{R}^d,\ell_p)$ consists of sets of $s$ vectors in $\mathbb{R}^d$, and for any two sets $x,y$ of $s$ vectors the distance $\mathsf{EMD}(x,y)$ is the minimum cost of a perfect matching between $x,y$, where the cost of matching two vectors is their $\ell_p$ distance. Previously, Andoni, Indyk, and Krauthgamer gave a (data-independent) locality-sensitive hashing scheme for $\mathsf{EMD}_s(\mathbb{R}^d,\ell_p)$ when $p \in [1,2]$ with approximation $O(\log^2 s)$. By being data-dependent, we improve the approximation to $\tilde{O}(\log s)$. Our main technical contribution is to show that for any distribution $μ$ supported on the metric $\mathsf{EMD}_s(\mathbb{R}^d, \ell_p)$, there exists a data-dependent LSH for dense regions of $μ$ which achieves approximation $\tilde{O}(\log s)$, and that the data-independent LSH actually achieves a $\tilde{O}(\log s)$-approximation outside of those dense regions. Finally, we show how to "glue" together these two hashing schemes without any additional loss in the approximation. Beyond nearest neighbor search, our data-dependent LSH also gives optimal (distributional) sketches for the Earth Mover's Distance. By known sketching lower bounds, this implies that our LSH is optimal (up to $\mathrm{poly}(\log \log s)$ factors) among those that collide close points with constant probability.
Are Human Conversations Special? A Large Language Model Perspective
Venue: arXiv cs.CL 2024
Year: 2024

This study analyzes changes in the attention mechanisms of large language models (LLMs) when used to understand natural conversations between humans (human-human). We analyze three use cases of LLMs: interactions over web content, code, and mathematical texts. By analyzing attention distance, dispersion, and interdependency across these domains, we highlight the unique challenges posed by conversational data. Notably, conversations require nuanced handling of long-term contextual relationships and exhibit higher complexity through their attention patterns. Our findings reveal that while language models exhibit domain-specific attention behaviors, there is a significant gap in their ability to specialize in human conversations. Through detailed attention entropy analysis and t-SNE visualizations, we demonstrate the need for models trained with a diverse array of high-quality conversational data to enhance understanding and generation of human-like dialogue. This research highlights the importance of domain specialization in language models and suggests pathways for future advancement in modeling human conversational nuances.
Efficient Calculations for Inverse of $k$-diagonal Circulant Matrices and Cyclic Banded Matrices
Venue: arXiv cs.MS 2024
Year: 2024

$k$-diagonal circulant matrices and cyclic banded matrices are widely used in numerical simulations and signal processing of circular linear systems. Algorithms that directly involve or specify linear or quadratic complexity for the inverses of these two types of matrices are rare. We find that the inverse of a $k$-diagonal circulant matrix can be uniquely determined by a recursive formula, which can be derived within $O(k^3 \log n+k^4)$. Similarly for the inverse of a cyclic banded matrix, its inverse can be uniquely determined by a series of recursive formulas, with the initial terms of these recursions computable within $O(k^3 n+k^5)$. The additional costs for solving the complete inverses of these two types of matrices are $kn$ and $kn^2$. Our calculations enable rapid representation with most processes defined by explicit formulas. Additionally, most algorithms for inverting $k$-diagonal circulant matrices rely on the Fast Fourier Transform, which is not applicable to finite fields, while our algorithms can be applied to computations in finite fields.