OpenRaiser/Intern-Atlas huggingface dataset:
Trained Latent Space Navigation to Prevent Lack of Photorealism in Generated Images on Style-based Models
Venue: arXiv cs.CV 2023
Year: 2023

Recent studies on StyleGAN variants show promising performances for various generation tasks. In these models, latent codes have traditionally been manipulated and searched for the desired images. However, this approach sometimes suffers from a lack of photorealism in generated images due to a lack of knowledge about the geometry of the trained latent space. In this paper, we show a simple unsupervised method that provides well-trained local latent subspace, enabling latent code navigation while preserving the photorealism of the generated images. Specifically, the method identifies densely mapped latent spaces and restricts latent manipulations within the local latent subspace. Experimental results demonstrate that images generated within the local latent subspace maintain photorealism even when the latent codes are significantly and repeatedly manipulated. Moreover, experiments show that the method can be applied to latent code optimization for various types of style-based models. Our empirical evidence of the method will benefit applications in style-based models.
Easier Said Than Done: The Failure of Top-Level Cybersecurity Advice for Consumer IoT Devices
Venue: arXiv cs.CR 2023
Year: 2023

Consumer IoT devices are generally assumed to lack adequate default security, thus requiring user action. However, it may not be immediately clear to users what action to take and how. This uncertainty begs the question of what the minimum is that the user-base can reliably be asked to do as a prompt to secure their devices. To explore this question, we analyze security actions advocated at a national level and how these connect to user materials for a range of specific devices. We identify four pieces of converging advice across three nation-level initiatives. We then assess the extent to which these pieces of advice are aligned with instruction materials for 40 different IoT devices across five device classes (including device manuals and manufacturer websites). We expose a disconnect between the advice and the device materials. A stunning finding is that there is not a single assessed device to which all four top pieces of converging advice can be applied. At best, the supporting materials for 36 of the 40 devices provide sufficient information to apply just two of the four pieces of advice, typically the installation and enabling of (auto)updates. As something of a contradiction, it is necessary for a non-expert user to assess whether expert advice applies to a device. This risks additional user burden and proxy changes being made without the proposed security benefits. We propose recommendations, including that governments and researchers alike should declare their own working models of IoT devices when considering the user view.
Distilling Influences to Mitigate Prediction Churn in Graph Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

Models with similar performances exhibit significant disagreement in the predictions of individual samples, referred to as prediction churn. Our work explores this phenomenon in graph neural networks by investigating differences between models differing only in their initializations in their utilized features for predictions. We propose a novel metric called Influence Difference (ID) to quantify the variation in reasons used by nodes across models by comparing their influence distribution. Additionally, we consider the differences between nodes with a stable and an unstable prediction, positing that both equally utilize different reasons and thus provide a meaningful gradient signal to closely match two models even when the predictions for nodes are similar. Based on our analysis, we propose to minimize this ID in Knowledge Distillation, a domain where a new model should closely match an established one. As an efficient approximation, we introduce DropDistillation (DD) that matches the output for a graph perturbed by edge deletions. Our empirical evaluation of six benchmark datasets for node classification validates the differences in utilized features. DD outperforms previous methods regarding prediction stability and overall performance in all considered Knowledge Distillation experiments.
A Novel IoT Trust Model Leveraging Fully Distributed Behavioral Fingerprinting and Secure Delegation
Venue: arXiv cs.CR 2023
Year: 2023

With the number of connected smart devices expected to constantly grow in the next years, Internet of Things (IoT) solutions are experimenting a booming demand to make data collection and processing easier. The ability of IoT appliances to provide pervasive and better support to everyday tasks, in most cases transparently to humans, is also achieved through the high degree of autonomy of such devices. However, the higher the number of new capabilities and services provided in an autonomous way, the wider the attack surface that exposes users to data hacking and lost. In this scenario, many critical challenges arise also because IoT devices have heterogeneous computational capabilities (i.e., in the same network there might be simple sensors/actuators as well as more complex and smart nodes). In this paper, we try to provide a contribution in this setting, tackling the non-trivial issues of equipping smart things with a strategy to evaluate, also through their neighbors, the trustworthiness of an object in the network before interacting with it. To do so, we design a novel and fully distributed trust model exploiting devices' behavioral fingerprints, a distributed consensus mechanism and the Blockchain technology. Beyond the detailed description of our framework, we also illustrate the security model associated with it and the tests carried out to evaluate its correctness and performance.
Multi-task Learning with 3D-Aware Regularization
Venue: arXiv cs.CV 2023
Year: 2023

Deep neural networks have become a standard building block for designing models that can perform multiple dense computer vision tasks such as depth estimation and semantic segmentation thanks to their ability to capture complex correlations in high dimensional feature space across tasks. However, the cross-task correlations that are learned in the unstructured feature space can be extremely noisy and susceptible to overfitting, consequently hurting performance. We propose to address this problem by introducing a structured 3D-aware regularizer which interfaces multiple tasks through the projection of features extracted from an image encoder to a shared 3D feature space and decodes them into their task output space through differentiable rendering. We show that the proposed method is architecture agnostic and can be plugged into various prior multi-task backbones to improve their performance; as we evidence using standard benchmarks NYUv2 and PASCAL-Context.
TSO Games -- On the decidability of safety games under the total store order semantics
Venue: arXiv cs.GT 2023
Year: 2023

We consider an extension of the classical Total Store Order (TSO) semantics by expanding it to turn-based 2-player safety games. During her turn, a player can select any of the communicating processes and perform its next transition. We consider different formulations of the safety game problem depending on whether one player or both of them transfer messages from the process buffers to the shared memory. We give the complete decidability picture for all the possible alternatives.
Counterfactual Causality for Reachability and Safety based on Distance Functions
Venue: arXiv cs.LO 2023
Year: 2023

Investigations of causality in operational systems aim at providing human-understandable explanations of why a system behaves as it does. There is, in particular, a demand to explain what went wrong on a given counterexample execution that shows that a system does not satisfy a given specification. To this end, this paper investigates a notion of counterfactual causality in transition systems based on Stalnaker's and Lewis' semantics of counterfactuals in terms of most similar possible worlds and introduces a novel corresponding notion of counterfactual causality in two-player games. Using distance functions between paths in transition systems, this notion defines whether reaching a certain set of states is a cause for the violation of a reachability or safety property. Similarly, using distance functions between memoryless strategies in reachability and safety games, it is defined whether reaching a set of states is a cause for the fact that a given strategy for the player under investigation is losing. The contribution of the paper is two-fold: In transition systems, it is shown that counterfactual causality can be checked in polynomial time for three prominent distance functions between paths. In two-player games, the introduced notion of counterfactual causality is shown to be checkable in polynomial time for two natural distance functions between memoryless strategies. Further, a notion of explanation that can be extracted from a counterfactual cause and that pinpoints changes to be made to the given strategy in order to transform it into a winning strategy is defined. For the two distance functions under consideration, the problem to decide whether such an explanation imposes only minimal necessary changes to the given strategy with respect to the used distance function turns out to be coNP-complete and not to be solvable in polynomial time if P is not equal to NP, respectively.
Finding a reconfiguration sequence between longest increasing subsequences
Venue: arXiv cs.DS 2023
Year: 2023

In this note, we consider the problem of finding a step-by-step transformation between two longest increasing subsequences in a sequence, namely Longest Increasing Subsequence Reconfiguration. We give a polynomial-time algorithm for deciding whether there is a reconfiguration sequence between two longest increasing subsequences in a sequence. This implies that Independent Set Reconfiguration and Token Sliding are polynomial-time solvable on permutation graphs, provided that the input two independent sets are largest among all independent sets in the input graph. We also consider a special case, where the underlying permutation graph of an input sequence is bipartite. In this case, we give a polynomial-time algorithm for finding a shortest reconfiguration sequence (if it exists).
On Fulfilling the Exigent Need for Automating and Modernizing Logistics Infrastructure in India: Enabling AI-based Integration, Digitalization, and Smart Automation of Industrial Parks and Robotic Warehouses
Venue: arXiv cs.RO 2023
Year: 2023

To stay competitive, the Low- or Middle-Income Countries (LMICs) need to embrace Industry 4.0 and Logistics 4.0. This requires government-level interventions and policy-making to incentivize quality product solutions and drive innovation in traditionally resistant economic sectors. In this position paper, we support the establishment of Smart Industrial Parks (SIPs) with a focus on enhancing operational efficiencies and bringing together MSMEs and startups targeting niche clientele with innovative Industry 4.0 solutions. SIPs along with the phased deployment of well-planned robotic automation technologies shall enable bringing down India's untenable logistics costs. Toward the successful execution of SIPs, we are required to implement the efficient allocation of manufacturing resources and capabilities within SIPs. Thus, we emphasize the importance of efficient resource utilization, collaboration, and technology adoption in industrial parks to promote industrial development and economic growth. We advocate the use of a cloud-based cyber-physical system for real-time data access and analysis in SIPs. Such centralized cloud-based monitoring of factory floors, warehouses, and industrial units using IoT infrastructure shall improve decision-making, efficiency, and safety. Digital Twins (DTs), which are cyber-replicas of physical systems, could play a significant role in enabling simulation, optimization, and real-time monitoring of smart manufacturing and distributed manufacturing systems. However, there are several challenges involved in implementing DTs in distributed manufacturing systems, such as defining data schemas and collaboration protocols, ensuring interoperability, the need for effective authentication technology, distributed machine learning models, and scalability to manage multiple DTs.
Computing Functions by Teams of Deterministic Finite Automata
Venue: arXiv cs.DC 2023
Year: 2023

We consider the task of computing functions $f: \mathbb{N}^k\to \mathbb{N}$, where $ \mathbb{N}$ is the set of natural numbers, by finite teams of agents modelled as deterministic finite automata. The computation is carried out in a distributed way, using the {\em discrete half-line}, which is the infinite graph with one node of degree 1 (called the root) and infinitely many nodes of degree 2. The node at distance $j$ from the root represents the integer $j$. We say that a team $\mathcal{A}^f$ of automata computes a function $f$, if in the beginning of the computation all automata from $\mathcal{A}^f$ are located at the arguments $x_1,\dots,x_k$ of the function $f$, in groups $\mathcal{A}^f _j$ at $x_j$, and at the end, all automata of the team gather at $f(x_1,\dots,x_k)$ and transit to a special state $STOP$. At each step of the computation, an automaton $a$ can ``see'' states of all automata colocated at the same node: the set of these states forms an input of $a$. Our main result shows that, for every primitive recursive function, there exists a finite team of automata that computes this function. We prove this by showing that basic primitive recursive functions can be computed by teams of automata, and that functions resulting from the operations of composition and of primitive recursion can be computed by teams of automata, provided that the ingredient functions of these operations can be computed by teams of automata. We also observe that cooperation between automata is necessary: even some very simple functions $f: \mathbb{N}\to \mathbb{N}$ cannot be computed by a single automaton.
Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives
Venue: arXiv cs.CR 2023
Year: 2023

This paper provides a systematic analysis of the opportunities, challenges, and potential solutions of harnessing Large Language Models (LLMs) such as GPT-4 to dig out vulnerabilities within smart contracts based on our ongoing research. For the task of smart contract vulnerability detection, achieving practical usability hinges on identifying as many true vulnerabilities as possible while minimizing the number of false positives. Nonetheless, our empirical study reveals contradictory yet interesting findings: generating more answers with higher randomness largely boosts the likelihood of producing a correct answer but inevitably leads to a higher number of false positives. To mitigate this tension, we propose an adversarial framework dubbed GPTLens that breaks the conventional one-stage detection into two synergistic stages $-$ generation and discrimination, for progressive detection and refinement, wherein the LLM plays dual roles, i.e., auditor and critic, respectively. The goal of auditor is to yield a broad spectrum of vulnerabilities with the hope of encompassing the correct answer, whereas the goal of critic that evaluates the validity of identified vulnerabilities is to minimize the number of false positives. Experimental results and illustrative examples demonstrate that auditor and critic work together harmoniously to yield pronounced improvements over the conventional one-stage detection. GPTLens is intuitive, strategic, and entirely LLM-driven without relying on specialist expertise in smart contracts, showcasing its methodical generality and potential to detect a broad spectrum of vulnerabilities. Our code is available at: https://github.com/git-disl/GPTLens.
Graph-Theoretic Bézier Curve Optimization over Safe Corridors for Safe and Smooth Motion Planning
Venue: arXiv cs.RO 2023
Year: 2023

As a parametric motion representation, Bézier curves have significant applications in polynomial trajectory optimization for safe and smooth motion planning of various robotic systems, including flying drones, autonomous vehicles, and robotic manipulators. An essential component of Bézier curve optimization is the optimization objective, as it significantly influences the resulting robot motion. Standard physical optimization objectives, such as minimizing total velocity, acceleration, jerk, and snap, are known to yield quadratic optimization of Bézier curve control points. In this paper, we present a unifying graph-theoretic perspective for defining and understanding Bézier curve optimization objectives using a consensus distance of Bézier control points derived based on their interaction graph Laplacian. In addition to demonstrating how standard physical optimization objectives define a consensus distance between Bézier control points, we also introduce geometric and statistical optimization objectives as alternative consensus distances, constructed using finite differencing and differential variance. To compare these optimization objectives, we apply Bézier curve optimization over convex polygonal safe corridors that are automatically constructed around a maximal-clearance minimal-length reference path. We provide an explicit analytical formulation for quadratic optimization of Bézier curves using Bézier matrix operations. We conclude that the norm and variance of the finite differences of Bézier control points lead to simpler and more intuitive interaction graphs and optimization objectives compared to Bézier derivative norms, despite having similar robot motion profiles.
Reducing Hyperexponential Functions over Monomial Extensions
Venue: arXiv cs.SC 2023
Year: 2023

We extend the shell and kernel reductions for hyperexponential functions over the field of rational functions to a monomial extension. Both of the reductions are incorporated into one algorithm. As an application, we present an additive decomposition in rationally hyperexponential towers. The decomposition yields an alternative algorithm for computing elementary integrals over such towers. The alternative can find some elementary integrals that are unevaluated by the integrators in the latest versions of Maple and Mathematica.
appjsonify: An Academic Paper PDF-to-JSON Conversion Toolkit
Venue: arXiv cs.CL 2023
Year: 2023

We present appjsonify, a Python-based PDF-to-JSON conversion toolkit for academic papers. It parses a PDF file using several visual-based document layout analysis models and rule-based text processing approaches. appjsonify is a flexible tool that allows users to easily configure the processing pipeline to handle a specific format of a paper they wish to process. We are publicly releasing appjsonify as an easy-to-install toolkit available via PyPI and GitHub.
Sequential Data Generation with Groupwise Diffusion Process
Venue: arXiv cs.CV 2023
Year: 2023

We present the Groupwise Diffusion Model (GDM), which divides data into multiple groups and diffuses one group at one time interval in the forward diffusion process. GDM generates data sequentially from one group at one time interval, leading to several interesting properties. First, as an extension of diffusion models, GDM generalizes certain forms of autoregressive models and cascaded diffusion models. As a unified framework, GDM allows us to investigate design choices that have been overlooked in previous works, such as data-grouping strategy and order of generation. Furthermore, since one group of the initial noise affects only a certain group of the generated data, latent space now possesses group-wise interpretable meaning. We can further extend GDM to the frequency domain where the forward process sequentially diffuses each group of frequency components. Dividing the frequency bands of the data as groups allows the latent variables to become a hierarchical representation where individual groups encode data at different levels of abstraction. We demonstrate several applications of such representation including disentanglement of semantic attributes, image editing, and generating variations.
Attention Sorting Combats Recency Bias In Long Context Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Current language models often fail to incorporate long contexts efficiently during generation. We show that a major contributor to this issue are attention priors that are likely learned during pre-training: relevant information located earlier in context is attended to less on average. Yet even when models fail to use the information from a relevant document in their response, they still pay preferential attention to that document compared to an irrelevant document at the same position. We leverage this fact to introduce ``attention sorting'': perform one step of decoding, sort documents by the attention they receive (highest attention going last), repeat the process, generate the answer with the newly sorted context. We find that attention sorting improves performance of long context models. Our findings highlight some challenges in using off-the-shelf language models for retrieval augmented generation.
Building Flexible, Scalable, and Machine Learning-ready Multimodal Oncology Datasets
Venue: arXiv cs.LG 2023
Year: 2023

The advancements in data acquisition, storage, and processing techniques have resulted in the rapid growth of heterogeneous medical data. Integrating radiological scans, histopathology images, and molecular information with clinical data is essential for developing a holistic understanding of the disease and optimizing treatment. The need for integrating data from multiple sources is further pronounced in complex diseases such as cancer for enabling precision medicine and personalized treatments. This work proposes Multimodal Integration of Oncology Data System (MINDS) - a flexible, scalable, and cost-effective metadata framework for efficiently fusing disparate data from public sources such as the Cancer Research Data Commons (CRDC) into an interconnected, patient-centric framework. MINDS offers an interface for exploring relationships across data types and building cohorts for developing large-scale multimodal machine learning models. By harmonizing multimodal data, MINDS aims to potentially empower researchers with greater analytical ability to uncover diagnostic and prognostic insights and enable evidence-based personalized care. MINDS tracks granular end-to-end data provenance, ensuring reproducibility and transparency. The cloud-native architecture of MINDS can handle exponential data growth in a secure, cost-optimized manner while ensuring substantial storage optimization, replication avoidance, and dynamic access capabilities. Auto-scaling, access controls, and other mechanisms guarantee pipelines' scalability and security. MINDS overcomes the limitations of existing biomedical data silos via an interoperable metadata-driven approach that represents a pivotal step toward the future of oncology data integration.