OpenRaiser/Intern-Atlas huggingface dataset:
RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models
Venue: arXiv cs.CV 2023
Year: 2023

Recent advancements in diffusion-based models have demonstrated significant success in generating images from text. However, video editing models have not yet reached the same level of visual quality and user control. To address this, we introduce RAVE, a zero-shot video editing method that leverages pre-trained text-to-image diffusion models without additional training. RAVE takes an input video and a text prompt to produce high-quality videos while preserving the original motion and semantic structure. It employs a novel noise shuffling strategy, leveraging spatio-temporal interactions between frames, to produce temporally consistent videos faster than existing methods. It is also efficient in terms of memory requirements, allowing it to handle longer videos. RAVE is capable of a wide range of edits, from local attribute modifications to shape transformations. In order to demonstrate the versatility of RAVE, we create a comprehensive video evaluation dataset ranging from object-focused scenes to complex human activities like dancing and typing, and dynamic scenes featuring swimming fish and boats. Our qualitative and quantitative experiments highlight the effectiveness of RAVE in diverse video editing scenarios compared to existing methods. Our code, dataset and videos can be found in https://rave-video.github.io.
Algorithms for the Global Domination Problem
Venue: arXiv cs.DM 2023
Year: 2023

A dominating set D in a graph G is a subset of its vertices such that every vertex of the graph which does not belong to set D is adjacent to at least one vertex from set D. A set of vertices of graph G is a global dominating set if it is a dominating set for both, graph G and its complement. The objective is to find a global dominating set with the minimum cardinality. The problem is known to be NP-hard. Neither exact nor approximation algorithm existed . We propose two exact solution methods, one of them being based on an integer linear program (ILP) formulation, three heuristic algorithms and a special purification procedure that further reduces the size of a global dominated set delivered by any of our heuristic algorithms. We show that the problem remains NP-hard for restricted types of graphs and specify some families of graphs for which the heuristics guarantee the optimality. The second exact algorithm turned out to be about twice faster than ILP for graphs with more than 230 vertices and up to 1080 vertices, which were the largest benchmark instances that were solved optimally. The heuristics were tested for the existing 2284 benchmark problem instances with up to 14000 vertices and delivered solutions for the largest instances in less than one minute. Remarkably, for about 52% of the 1000 instances with the obtained optimal solutions, at least one of the heuristics generated an optimal solution, where the average approximation error for the remaining instances was 1.07%.
Visual Geometry Grounded Deep Structure From Motion
Venue: arXiv cs.CV 2023
Year: 2023

Structure-from-motion (SfM) is a long-standing problem in the computer vision community, which aims to reconstruct the camera poses and 3D structure of a scene from a set of unconstrained 2D images. Classical frameworks solve this problem in an incremental manner by detecting and matching keypoints, registering images, triangulating 3D points, and conducting bundle adjustment. Recent research efforts have predominantly revolved around harnessing the power of deep learning techniques to enhance specific elements (e.g., keypoint matching), but are still based on the original, non-differentiable pipeline. Instead, we propose a new deep pipeline VGGSfM, where each component is fully differentiable and thus can be trained in an end-to-end manner. To this end, we introduce new mechanisms and simplifications. First, we build on recent advances in deep 2D point tracking to extract reliable pixel-accurate tracks, which eliminates the need for chaining pairwise matches. Furthermore, we recover all cameras simultaneously based on the image and track features instead of gradually registering cameras. Finally, we optimise the cameras and triangulate 3D points via a differentiable bundle adjustment layer. We attain state-of-the-art performance on three popular datasets, CO3D, IMC Phototourism, and ETH3D.
Harnessing LSTM for Nonlinear Ship Deck Motion Prediction in UAV Autonomous Landing amidst High Sea States
Venue: arXiv cs.RO 2023
Year: 2023

Autonomous landing of UAVs in high sea states requires the UAV to land exclusively during the ship deck's "rest period," coinciding with minimal movement. Given this scenario, determining the ship's "rest period" based on its movement patterns becomes a fundamental prerequisite for addressing this challenge. This study employs the Long Short-Term Memory (LSTM) neural network to predict the ship's motion across three dimensions: longi-tudinal, transverse, and vertical waves. In the absence of actual ship data under high sea states, this paper employs a composite sine wave model to simulate ship deck motion. Through this approach, a highly accurate model is established, exhibiting promising outcomes within various stochastic sine wave combination models.
Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects
Venue: arXiv cs.AI 2023
Year: 2023

The complexity of psychological principles underscore a significant societal challenge, given the vast social implications of psychological problems. Bridging the gap between understanding these principles and their actual clinical and real-world applications demands rigorous exploration and adept implementation. In recent times, the swift advancement of highly adaptive and reusable artificial intelligence (AI) models has emerged as a promising way to unlock unprecedented capabilities in the realm of psychology. This paper emphasizes the importance of performance validation for these large-scale AI models, emphasizing the need to offer a comprehensive assessment of their verification from diverse perspectives. Moreover, we review the cutting-edge advancements and practical implementations of these expansive models in psychology, highlighting pivotal work spanning areas such as social media analytics, clinical nursing insights, vigilant community monitoring, and the nuanced exploration of psychological theories. Based on our review, we project an acceleration in the progress of psychological fields, driven by these large-scale AI models. These future generalist AI models harbor the potential to substantially curtail labor costs and alleviate social stress. However, this forward momentum will not be without its set of challenges, especially when considering the paradigm changes and upgrades required for medical instrumentation and related applications.
Automated SELinux RBAC Policy Verification Using SMT
Venue: arXiv cs.CR 2023
Year: 2023

Security-Enhanced Linux (SELinux) is a Linux kernel module that allows for a role-based access control (RBAC) mechanism. It provides a fine-grained security framework enabling system administrators to define security policies at the system and application level. Whilst SELinux offers robust security features through a customisable, powerful RBAC model, its manual policy management is prone to error, leaving the system vulnerable to accidental misconfigurations or loopholes. We present a tool to automate the conversion of SELinux policies into satisfiability modulo theories (SMT), enabling the verification of the intended security configurations using automated theorem proving. Our tool is capable of flagging common policy misconfigurations by asserting consistency between supplied RBAC policies and the intended specification by the user in SMT. RBAC policies are inherently complicated to verify entirely. We envision that the automated tool presented here can be further extended to identify an even broader range of policy misconfigurations, relieving the burden of managing convoluted policies on system administrators.
On Sarcasm Detection with OpenAI GPT-based Models
Venue: arXiv cs.CL 2023
Year: 2023

Sarcasm is a form of irony that requires readers or listeners to interpret its intended meaning by considering context and social cues. Machine learning classification models have long had difficulty detecting sarcasm due to its social complexity and contradictory nature. This paper explores the applications of the Generative Pretrained Transformer (GPT) models, including GPT-3, InstructGPT, GPT-3.5, and GPT-4, in detecting sarcasm in natural language. It tests fine-tuned and zero-shot models of different sizes and releases. The GPT models were tested on the political and balanced (pol-bal) portion of the popular Self-Annotated Reddit Corpus (SARC 2.0) sarcasm dataset. In the fine-tuning case, the largest fine-tuned GPT-3 model achieves accuracy and $F_1$-score of 0.81, outperforming prior models. In the zero-shot case, one of GPT-4 models yields an accuracy of 0.70 and $F_1$-score of 0.75. Other models score lower. Additionally, a model's performance may improve or deteriorate with each release, highlighting the need to reassess performance after each release.
VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment
Venue: arXiv cs.CV 2023
Year: 2023

We present a 3D-aware one-shot head reenactment method based on a fully volumetric neural disentanglement framework for source appearance and driver expressions. Our method is real-time and produces high-fidelity and view-consistent output, suitable for 3D teleconferencing systems based on holographic displays. Existing cutting-edge 3D-aware reenactment methods often use neural radiance fields or 3D meshes to produce view-consistent appearance encoding, but, at the same time, they rely on linear face models, such as 3DMM, to achieve its disentanglement with facial expressions. As a result, their reenactment results often exhibit identity leakage from the driver or have unnatural expressions. To address these problems, we propose a neural self-supervised disentanglement approach that lifts both the source image and driver video frame into a shared 3D volumetric representation based on tri-planes. This representation can then be freely manipulated with expression tri-planes extracted from the driving images and rendered from an arbitrary view using neural radiance fields. We achieve this disentanglement via self-supervised learning on a large in-the-wild video dataset. We further introduce a highly effective fine-tuning approach to improve the generalizability of the 3D lifting using the same real-world data. We demonstrate state-of-the-art performance on a wide range of datasets, and also showcase high-quality 3D-aware head reenactment on highly challenging and diverse subjects, including non-frontal head poses and complex expressions for both source and driver.
NeuSD: Surface Completion with Multi-View Text-to-Image Diffusion
Venue: arXiv cs.CV 2023
Year: 2023

We present a novel method for 3D surface reconstruction from multiple images where only a part of the object of interest is captured. Our approach builds on two recent developments: surface reconstruction using neural radiance fields for the reconstruction of the visible parts of the surface, and guidance of pre-trained 2D diffusion models in the form of Score Distillation Sampling (SDS) to complete the shape in unobserved regions in a plausible manner. We introduce three components. First, we suggest employing normal maps as a pure geometric representation for SDS instead of color renderings which are entangled with the appearance information. Second, we introduce the freezing of the SDS noise during training which results in more coherent gradients and better convergence. Third, we propose Multi-View SDS as a way to condition the generation of the non-observable part of the surface without fine-tuning or making changes to the underlying 2D Stable Diffusion model. We evaluate our approach on the BlendedMVS dataset demonstrating significant qualitative and quantitative improvements over competing methods.
Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games
Venue: arXiv cs.CL 2023
Year: 2023

In this work, we introduce a self-supervised behavior cloning transformer for text games, which are challenging benchmarks for multi-step reasoning in virtual environments. Traditionally, Behavior Cloning Transformers excel in such tasks but rely on supervised training data. Our approach auto-generates training data by exploring trajectories (defined by common macro-action sequences) that lead to reward within the games, while determining the generality and utility of these trajectories by rapidly training small models then evaluating their performance on unseen development games. Through empirical analysis, we show our method consistently uncovers generalizable training data, achieving about 90\% performance of supervised systems across three benchmark text games.
Model-based Digital Twins of Medicine Dispensers for Healthcare IoT Applications
Venue: arXiv cs.SE 2023
Year: 2023

Healthcare applications with the Internet of Things (IoT) are often safety-critical, thus, require extensive testing. Such applications are often connected to smart medical devices from various vendors. System-level testing of such applications requires test infrastructures physically integrating medical devices, which is time and monetary-wise expensive. Moreover, applications continuously evolve, e.g., introducing new devices and users and updating software. Nevertheless, a test infrastructure enabling testing with a few devices is insufficient for testing healthcare IoT systems, hence compromising their dependability. In this paper, we propose a model-based approach for the creation and operation of digital twins (DTs) of medicine dispensers as a replacement for physical devices to support the automated testing of IoT applications at scale. We evaluate our approach with an industrial IoT system with medicine dispensers in the context of Oslo City and its industrial partners, providing healthcare services to its residents. We study the fidelity of DTs in terms of their functional similarities with their physical counterparts: medicine dispensers. Results show that the DTs behave more than 92% similar to the physical medicine dispensers, providing a faithful replacement for the dispenser.
Optimal Multi-Pass Lower Bounds for MST in Dynamic Streams
Venue: arXiv cs.DS 2023
Year: 2023

The seminal work of Ahn, Guha, and McGregor in 2012 introduced the graph sketching technique and used it to present the first streaming algorithms for various graph problems over dynamic streams with both insertions and deletions of edges. This includes algorithms for cut sparsification, spanners, matchings, and minimum spanning trees (MSTs). These results have since been improved or generalized in various directions, leading to a vastly rich host of efficient algorithms for processing dynamic graph streams. A curious omission from the list of improvements has been the MST problem. The best algorithm for this problem remains the original AGM algorithm that for every integer $p \geq 1$, uses $n^{1+O(1/p)}$ space in $p$ passes on $n$-vertex graphs, and thus achieves the desired semi-streaming space of $\tilde{O}(n)$ at a relatively high cost of $O(\frac{\log{n}}{\log\log{n}})$ passes. On the other hand, no lower bounds beyond a folklore one-pass lower bound is known for this problem. We provide a simple explanation for this lack of improvements: The AGM algorithm for MSTs is optimal for the entire range of its number of passes! We prove that even for the simplest decision version of the problem -- deciding whether the weight of MSTs is at least a given threshold or not -- any $p$-pass dynamic streaming algorithm requires $n^{1+Ω(1/p)}$ space. This implies that semi-streaming algorithms do need $Ω(\frac{\log{n}}{\log\log{n}})$ passes. Our result relies on proving new multi-round communication complexity lower bounds for a variant of the universal relation problem that has been instrumental in proving prior lower bounds for single-pass dynamic streaming algorithms. The proof also involves proving new composition theorems in communication complexity, including majority lemmas and multi-party XOR lemmas, via information complexity approaches.
How to guess a gradient
Venue: arXiv cs.LG 2023
Year: 2023

How much can you say about the gradient of a neural network without computing a loss or knowing the label? This may sound like a strange question: surely the answer is "very little." However, in this paper, we show that gradients are more structured than previously thought. Gradients lie in a predictable low-dimensional subspace which depends on the network architecture and incoming features. Exploiting this structure can significantly improve gradient-free optimization schemes based on directional derivatives, which have struggled to scale beyond small networks trained on toy datasets. We study how to narrow the gap in optimization performance between methods that calculate exact gradients and those that use directional derivatives. Furthermore, we highlight new challenges in overcoming the large gap between optimizing with exact gradients and guessing the gradients.
The Internet of Responsibilities-Connecting Human Responsibilities using Big Data and Blockchain
Venue: arXiv cs.CY 2023
Year: 2023

Accountability in the workplace is critically important and remains a challenging problem, especially with respect to workplace safety management. In this paper, we introduce a novel notion, the Internet of Responsibilities, for accountability management. Our method sorts through the list of responsibilities with respect to hazardous positions. The positions are interconnected using directed acyclic graphs (DAGs) indicating the hierarchy of responsibilities in the organization. In addition, the system detects and collects responsibilities, and represents risk areas in terms of the positions of the responsibility nodes. Finally, an automatic reminder and assignment system is used to enforce a strict responsibility control without human intervention. Using blockchain technology, we further extend our system with the capability to store, recover and encrypt responsibility data. We show that through the application of the Internet of Responsibility network model driven by Big Data, enterprise and government agencies can attain a highly secured and safe workplace. Therefore, our model offers a combination of interconnected responsibilities, accountability, monitoring, and safety which is crucial for the protection of employees and the success of organizations.
DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions
Venue: arXiv cs.CR 2023
Year: 2023

With the advancement of Large Language Models (LLMs), significant progress has been made in code generation, enabling LLMs to transform natural language into programming code. These Code LLMs have been widely accepted by massive users and organizations. However, a dangerous nature is hidden in the code, which is the existence of fatal vulnerabilities. While some LLM providers have attempted to address these issues by aligning with human guidance, these efforts fall short of making Code LLMs practical and robust. Without a deep understanding of the performance of the LLMs under the practical worst cases, it would be concerning to apply them to various real-world applications. In this paper, we answer the critical issue: Are existing Code LLMs immune to generating vulnerable code? If not, what is the possible maximum severity of this issue in practical deployment scenarios? In this paper, we introduce DeceptPrompt, a novel algorithm that can generate adversarial natural language instructions that drive the Code LLMs to generate functionality correct code with vulnerabilities. DeceptPrompt is achieved through a systematic evolution-based algorithm with a fine grain loss design. The unique advantage of DeceptPrompt enables us to find natural prefix/suffix with totally benign and non-directional semantic meaning, meanwhile, having great power in inducing the Code LLMs to generate vulnerable code. This feature can enable us to conduct the almost-worstcase red-teaming on these LLMs in a real scenario, where users are using natural language. Our extensive experiments and analyses on DeceptPrompt not only validate the effectiveness of our approach but also shed light on the huge weakness of LLMs in the code generation task. When applying the optimized prefix/suffix, the attack success rate (ASR) will improve by average 50% compared with no prefix/suffix applying.
Reinforcement Learning Based Dynamic Power Control for UAV Mobility Management
Venue: arXiv cs.IT 2023
Year: 2023

Modern communication systems need to fulfill multiple and often conflicting objectives at the same time. In particular, new applications require high reliability while operating at low transmit powers. Moreover, reliability constraints may vary over time depending on the current state of the system. One solution to address this problem is to use joint transmissions from a number of base stations (BSs) to meet the reliability requirements. However, this approach is inefficient when considering the overall total transmit power. In this work, we propose a reinforcement learning-based power allocation scheme for an unmanned aerial vehicle (UAV) communication system with varying communication reliability requirements. In particular, the proposed scheme aims to minimize the total transmit power of all BSs while achieving an outage probability that is less than a tolerated threshold. This threshold varies over time, e.g., when the UAV enters a critical zone with high-reliability requirements. Our results show that the proposed learning scheme uses dynamic power allocation to meet varying reliability requirements, thus effectively conserving energy.
MetaDetect: Metamorphic Testing Based Anomaly Detection for Multi-UAV Wireless Networks
Venue: arXiv cs.NI 2023
Year: 2023

The reliability of wireless Ad Hoc Networks (WANET) communication is much lower than wired networks. WANET will be impacted by node overload, routing protocol, weather, obstacle blockage, and many other factors, all those anomalies cannot be avoided. Accurate prediction of the network entirely stopping in advance is essential after people could do networking re-routing or changing to different bands. In the present study, there are two primary goals. Firstly, design anomaly events detection patterns based on Metamorphic Testing (MT) methodology. Secondly, compare the performance of evaluation metrics, such as Transfer Rate, Occupancy rate, and the Number of packets received. Compared to other studies, the most significant advantage of mathematical interpretability, as well as not requiring dependence on physical environmental information, only relies on the networking physical layer and Mac layer data. The analysis of the results demonstrates that the proposed MT detection method is helpful for automatically identifying incidents/accident events on WANET. The physical layer transfer Rate metric could get the best performance.