OpenRaiser/Intern-Atlas huggingface dataset:
FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems
Venue: arXiv cs.CL 2023
Year: 2023

Response ranking in dialogues plays a crucial role in retrieval-based conversational systems. In a multi-turn dialogue, to capture the gist of a conversation, contextual information serves as essential knowledge to achieve this goal. In this paper, we present a flexible neural framework that can integrate contextual information from multiple channels. Specifically for the current task, our approach is to provide two information channels in parallel, Fusing Conversation history and domain knowledge extracted from Candidate provenance (FCC), where candidate responses are curated, as contextual information to improve the performance of multi-turn dialogue response ranking. The proposed approach can be generalized as a module to incorporate miscellaneous contextual features for other context-oriented tasks. We evaluate our model on the MSDialog dataset widely used for evaluating conversational response ranking tasks. Our experimental results show that our framework significantly outperforms the previous state-of-the-art models, improving Recall@1 by 7% and MAP by 4%. Furthermore, we conduct ablation studies to evaluate the contributions of each information channel, and of the framework components, to the overall ranking performance, providing additional insights and directions for further improvements.
Sector Bounds for Vertical Cable Force Error in Cable-Suspended Load Transportation System
Venue: arXiv cs.RO 2023
Year: 2023

This article studies the collaborative transportation of a cable-suspended pipe by two quadrotors. A force-coordination control scheme is proposed, where a force-consensus term is introduced to average the load distribution between the quadrotors. Since thrust uncertainty and cable force are coupled together in the acceleration channel, disturbance observer can only obtain the lumped disturbance estimate. Under the quasi-static condition, a disturbance separation strategy is developed to remove the thrust uncertainty estimate for precise cable force estimation. The stability of the overall system is analyzed using Lyapunov theory. Both numerical simulations and indoor experiments using heterogeneous quadrotors validate the effectiveness of thrust uncertainty separation and force-consensus algorithm.
Human-Robot Interaction in Retinal Surgery: A Comparative Study of Serial and Parallel Cooperative Robots
Venue: arXiv cs.RO 2023
Year: 2023

Cooperative robots for intraocular surgery allow surgeons to perform vitreoretinal surgery with high precision and stability. Several robot structural designs have shown capabilities to perform these surgeries. This research investigates the comparative performance of a serial and parallel cooperative-controlled robot in completing a retinal vessel-following task, with a focus on human-robot interaction performance and user experience. Our results indicate that despite differences in robot structure and interaction forces and torques, the two robots exhibited similar levels of performance in terms of general robot-to-patient interaction and average operating time. These findings have implications for the development and implementation of surgical robotics, suggesting that both serial and parallel cooperative-controlled robots can be effective for vitreoretinal surgery tasks.
Reusing Deep Neural Network Models through Model Re-engineering
Venue: arXiv cs.SE 2023
Year: 2023

Training deep neural network (DNN) models, which has become an important task in today's software development, is often costly in terms of computational resources and time. With the inspiration of software reuse, building DNN models through reusing existing ones has gained increasing attention recently. Prior approaches to DNN model reuse have two main limitations: 1) reusing the entire model, while only a small part of the model's functionalities (labels) are required, would cause much overhead (e.g., computational and time costs for inference), and 2) model reuse would inherit the defects and weaknesses of the reused model, and hence put the new system under threats of security attack. To solve the above problem, we propose SeaM, a tool that re-engineers a trained DNN model to improve its reusability. Specifically, given a target problem and a trained model, SeaM utilizes a gradient-based search method to search for the model's weights that are relevant to the target problem. The re-engineered model that only retains the relevant weights is then reused to solve the target problem. Evaluation results on widely-used models show that the re-engineered models produced by SeaM only contain 10.11% weights of the original models, resulting 42.41% reduction in terms of inference time. For the target problem, the re-engineered models even outperform the original models in classification accuracy by 5.85%. Moreover, reusing the re-engineered models inherits an average of 57% fewer defects than reusing the entire model. We believe our approach to reducing reuse overhead and defect inheritance is one important step forward for practical model reuse.
Recover Triggered States: Protect Model Against Backdoor Attack in Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

A backdoor attack allows a malicious user to manipulate the environment or corrupt the training data, thus inserting a backdoor into the trained agent. Such attacks compromise the RL system's reliability, leading to potentially catastrophic results in various key fields. In contrast, relatively limited research has investigated effective defenses against backdoor attacks in RL. This paper proposes the Recovery Triggered States (RTS) method, a novel approach that effectively protects the victim agents from backdoor attacks. RTS involves building a surrogate network to approximate the dynamics model. Developers can then recover the environment from the triggered state to a clean state, thereby preventing attackers from activating backdoors hidden in the agent by presenting the trigger. When training the surrogate to predict states, we incorporate agent action information to reduce the discrepancy between the actions taken by the agent on predicted states and the actions taken on real states. RTS is the first approach to defend against backdoor attacks in a single-agent setting. Our results show that using RTS, the cumulative reward only decreased by 1.41% under the backdoor attack.
Two Variants of Bezout Subresultants for Several Univariate Polynomials
Venue: arXiv cs.SC 2023
Year: 2023

In this paper, we develop two variants of Bezout subresultant formulas for several polynomials, i.e., hybrid Bezout subresultant polynomial and non-homogeneous Bezout subresultant polynomial. Rather than simply extending the variants of Bezout subresultant formulas developed by Diaz-Toca and Gonzalez-Vega in 2004 for two polynomials to arbitrary number of polynomials, we propose a new approach to formulating two variants of the Bezout-type subresultant polynomials for a set of univariate polynomials. Experimental results show that the Bezout-type subresultant formulas behave better than other known formulas when used to compute multi-polynomial subresultants, among which the non-homogeneous Bezout-type formula shows the best performance.
Vision Transformers with Mixed-Resolution Tokenization
Venue: arXiv cs.CV 2023
Year: 2023

Vision Transformer models process input images by dividing them into a spatially regular grid of equal-size patches. Conversely, Transformers were originally introduced over natural language sequences, where each token represents a subword - a chunk of raw data of arbitrary size. In this work, we apply this approach to Vision Transformers by introducing a novel image tokenization scheme, replacing the standard uniform grid with a mixed-resolution sequence of tokens, where each token represents a patch of arbitrary size. Using the Quadtree algorithm and a novel saliency scorer, we construct a patch mosaic where low-saliency areas of the image are processed in low resolution, routing more of the model's capacity to important image regions. Using the same architecture as vanilla ViTs, our Quadformer models achieve substantial accuracy gains on image classification when controlling for the computational budget. Code and models are publicly available at https://github.com/TomerRonen34/mixed-resolution-vit .
Cost and Reliability Aware Scheduling of Workflows Across Multiple Clouds with Security Constraints
Venue: arXiv cs.DC 2023
Year: 2023

Many real-world scientific workflows can be represented by a Directed Acyclic Graph (DAG), where each node represents a task and a directed edge signifies a dependency between two tasks. Due to the increasing computational resource requirements of these workflows, they are deployed on multi-cloud systems for execution. In this paper, we propose a scheduling algorithm that allocates resources to the tasks present in the workflow using an efficient list-scheduling approach based on the parameters cost, processing time, and reliability. Next, for a given a task-resource mapping, we propose a cipher assignment algorithm that assigns security services to edges responsible for transferring data in time-optimal manner subject to a given security constraint. The proposed algorithms have been analyzed to understand their time and space requirements. We implement the proposed scheduling and cipher assignment algorithm and experimented with two real-world scientific workflows namely Epigenomics and Cybershake. We compare the performance of the proposed scheduling algorithm with the state-of-art evolutionary methods. We observe that our method outperforms the state-of-art methods always in terms of cost and reliability, and is inferior in terms of makespan in some cases.
Multi-view reconstruction of bullet time effect based on improved NSFF model
Venue: arXiv cs.CV 2023
Year: 2023

Bullet time is a type of visual effect commonly used in film, television and games that makes time seem to slow down or stop while still preserving dynamic details in the scene. It usually requires multiple sets of cameras to move slowly with the subject and is synthesized using post-production techniques, which is costly and one-time. The dynamic scene perspective reconstruction technology based on neural rendering field can be used to solve this requirement, but most of the current methods are poor in reconstruction accuracy due to the blurred input image and overfitting of dynamic and static regions. Based on the NSFF algorithm, this paper reconstructed the common time special effects scenes in movies and television from a new perspective. To improve the accuracy of the reconstructed images, fuzzy kernel was added to the network for reconstruction and analysis of the fuzzy process, and the clear perspective after analysis was input into the NSFF to improve the accuracy. By using the optical flow prediction information to suppress the dynamic network timely, the network is forced to improve the reconstruction effect of dynamic and static networks independently, and the ability to understand and reconstruct dynamic and static scenes is improved. To solve the overfitting problem of dynamic and static scenes, a new dynamic and static cross entropy loss is designed. Experimental results show that compared with original NSFF and other new perspective reconstruction algorithms of dynamic scenes, the improved NSFF-RFCT improves the reconstruction accuracy and enhances the understanding ability of dynamic and static scenes.
Physics-informed machine learning for moving load problems
Venue: arXiv cs.LG 2023
Year: 2023

This paper presents a new approach to simulate forward and inverse problems of moving loads using physics-informed machine learning (PIML). Physics-informed neural networks (PINNs) utilize the underlying physics of moving load problems and aim to predict the deflection of beams and the magnitude of the loads. The mathematical representation of the moving load considered in this work involves a Dirac delta function, to capture the effect of the load moving across the structure. Approximating the Dirac delta function with PINNs is challenging because of its instantaneous change of output at a single point, causing difficulty in the convergence of the loss function. We propose to approximate the Dirac delta function with a Gaussian function. The incorporated Gaussian function physical equations are used in the physics-informed neural architecture to simulate beam deflections and to predict the magnitude of the load. Numerical results show that PIML is an effective method for simulating the forward and inverse problems for the considered model of a moving load.
Scalable Attestation of Virtualized Execution Environments in Hybrid- and Multi-Cloud
Venue: arXiv cs.CR 2023
Year: 2023

Existing attestation mechanisms lack scalability and support for heterogeneous virtual execution environments (VEEs), such as virtual machines and containers executed inside or outside hardware isolation on different vendors' hardware in clouds managed by various organizations. To overcome these limitations, hardware vendors and cloud providers implement proprietary mechanisms (Intel DCAP, Amazon NitroTPM, Google Titan) to support their offerings. However, due to their plurality, the attestation becomes cumbersome because it increases maintenance and integration costs and reduces portability required in hybrid- and multi-cloud deployments. We introduce WAWEL, a framework that enables scalable attestation of heterogeneous VEEs. WAWEL can be plugged into existing hardware-specific attestation mechanisms, offering a unified interface. WAWEL supports the widely adopted trusted platform module (TPM) attestation standard. We implemented a prototype and integrated it with three different VEEs. It supports runtime integrity attestation with Linux integrity measurement architecture (IMA) and legacy applications requiring zero-code changes. The evaluation demonstrated that the WAWEL prototype achieves very good performance and scalability despite the indirections between the VEE and hardware root of trust.
ASC: Adaptive Skill Coordination for Robotic Mobile Manipulation
Venue: arXiv cs.RO 2023
Year: 2023

We present Adaptive Skill Coordination (ASC) -- an approach for accomplishing long-horizon tasks like mobile pick-and-place (i.e., navigating to an object, picking it, navigating to another location, and placing it). ASC consists of three components -- (1) a library of basic visuomotor skills (navigation, pick, place), (2) a skill coordination policy that chooses which skill to use when, and (3) a corrective policy that adapts pre-trained skills in out-of-distribution states. All components of ASC rely only on onboard visual and proprioceptive sensing, without requiring detailed maps with obstacle layouts or precise object locations, easing real-world deployment. We train ASC in simulated indoor environments, and deploy it zero-shot (without any real-world experience or fine-tuning) on the Boston Dynamics Spot robot in eight novel real-world environments (one apartment, one lab, two microkitchens, two lounges, one office space, one outdoor courtyard). In rigorous quantitative comparisons in two environments, ASC achieves near-perfect performance (59/60 episodes, or 98%), while sequentially executing skills succeeds in only 44/60 (73%) episodes. Extensive perturbation experiments show that ASC is robust to hand-off errors, changes in the environment layout, dynamic obstacles (e.g., people), and unexpected disturbances. Supplementary videos at adaptiveskillcoordination.github.io.
Near-Field Channel Estimation for Extremely Large-Scale Reconfigurable Intelligent Surface (XL-RIS)-Aided Wideband mmWave Systems
Venue: arXiv cs.IT 2023
Year: 2023

Near-field communications present new opportunities over near-field channels, however, the spherical wavefront propagation makes near-field signal processing challenging. In this context, this paper proposes efficient near-field channel estimation methods for wideband MIMO mmWave systems with the aid of extremely large-scale reconfigurable intelligent surfaces (XL-RIS). For the wideband signals reflected by the analog RIS, we characterize their near-field beam squint effect in both angle and distance domains. Based on the mathematical analysis of the near-field beam patterns over all frequencies, a wideband spherical-domain dictionary is constructed by minimizing the coherence of two arbitrary beams. In light of this, we formulate a two-dimensional compressive sensing problem to recover the channel parameter based on the spherical-domain sparsity of mmWave channels. To this end, we present a correlation coefficient-based atom matching method within our proposed multi-frequency parallelizable subspace recovery framework for efficient solutions. Additionally, we propose a two-dimensional oracle estimator as a benchmark and derive its lower bound across all subcarriers. Our findings emphasize the significance of system hyperparameters and the sensing matrix of each subcarrier in determining the accuracy of the estimation. Finally, numerical results show that our proposed method achieves considerable performance compared with the lower bound and has a time complexity linear to the number of RIS elements.
Querying Large Language Models with SQL
Venue: arXiv cs.DB 2023
Year: 2023

In many use-cases, information is stored in text but not available in structured data. However, extracting data from natural language text to precisely fit a schema, and thus enable querying, is a challenging task. With the rise of pre-trained Large Language Models (LLMs), there is now an effective solution to store and use information extracted from massive corpora of text documents. Thus, we envision the use of SQL queries to cover a broad range of data that is not captured by traditional databases by tapping the information in LLMs. To ground this vision, we present Galois, a prototype based on a traditional database architecture, but with new physical operators for querying the underlying LLM. The main idea is to execute some operators of the the query plan with prompts that retrieve data from the LLM. For a large class of SQL queries, querying LLMs returns well structured relations, with encouraging qualitative results. Preliminary experimental results make pre-trained LLMs a promising addition to the field of database systems, introducing a new direction for hybrid query processing. However, we pinpoint several research challenges that must be addressed to build a DBMS that exploits LLMs. While some of these challenges necessitate integrating concepts from the NLP literature, others offer novel research avenues for the DB community.
Graph Mining for Cybersecurity: A Survey
Venue: arXiv cs.CR 2023
Year: 2023

The explosive growth of cyber attacks nowadays, such as malware, spam, and intrusions, caused severe consequences on society. Securing cyberspace has become an utmost concern for organizations and governments. Traditional Machine Learning (ML) based methods are extensively used in detecting cyber threats, but they hardly model the correlations between real-world cyber entities. In recent years, with the proliferation of graph mining techniques, many researchers investigated these techniques for capturing correlations between cyber entities and achieving high performance. It is imperative to summarize existing graph-based cybersecurity solutions to provide a guide for future studies. Therefore, as a key contribution of this paper, we provide a comprehensive review of graph mining for cybersecurity, including an overview of cybersecurity tasks, the typical graph mining techniques, and the general process of applying them to cybersecurity, as well as various solutions for different cybersecurity tasks. For each task, we probe into relevant methods and highlight the graph types, graph approaches, and task levels in their modeling. Furthermore, we collect open datasets and toolkits for graph-based cybersecurity. Finally, we outlook the potential directions of this field for future research.
Robust Ellipsoid Fitting Using Axial Distance and Combination
Venue: arXiv cs.CV 2023
Year: 2023

In random sample consensus (RANSAC), the problem of ellipsoid fitting can be formulated as a problem of minimization of point-to-model distance, which is realized by maximizing model score. Hence, the performance of ellipsoid fitting is affected by distance metric. In this paper, we proposed a novel distance metric called the axial distance, which is converted from the algebraic distance by introducing a scaling factor to solve nongeometric problems of the algebraic distance. There is complementarity between the axial distance and Sampson distance because their combination is a stricter metric when calculating the model score of sample consensus and the weight of the weighted least squares (WLS) fitting. Subsequently, a novel sample-consensus-based ellipsoid fitting method is proposed by using the combination between the axial distance and Sampson distance (CAS). We compare the proposed method with several representative fitting methods through experiments on synthetic and real datasets. The results show that the proposed method has a higher robustness against outliers, consistently high accuracy, and a speed close to that of the method based on sample consensus.
Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning
Venue: arXiv cs.LG 2023
Year: 2023

Many sequential decision-making problems that are currently automated, such as those in manufacturing or recommender systems, operate in an environment where there is either little uncertainty, or zero risk of catastrophe. As companies and researchers attempt to deploy autonomous systems in less constrained environments, it is increasingly important that we endow sequential decision-making algorithms with the ability to reason about uncertainty and risk. In this thesis, we will address both planning and reinforcement learning (RL) approaches to sequential decision-making. In the planning setting, it is assumed that a model of the environment is provided, and a policy is optimised within that model. Reinforcement learning relies upon extensive random exploration, and therefore usually requires a simulator in which to perform training. In many real-world domains, it is impossible to construct a perfectly accurate model or simulator. Therefore, the performance of any policy is inevitably uncertain due to the incomplete knowledge about the environment. Furthermore, in stochastic domains, the outcome of any given run is also uncertain due to the inherent randomness of the environment. These two sources of uncertainty are usually classified as epistemic, and aleatoric uncertainty, respectively. The over-arching goal of this thesis is to contribute to developing algorithms that mitigate both sources of uncertainty in sequential decision-making problems. We make a number of contributions towards this goal, with a focus on model-based algorithms...