OpenRaiser/Intern-Atlas huggingface dataset:
Sparsity exploitation via discovering graphical models in multi-variate time-series forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Graph neural networks (GNNs) have been widely applied in multi-variate time-series forecasting (MTSF) tasks because of their capability in capturing the correlations among different time-series. These graph-based learning approaches improve the forecasting performance by discovering and understanding the underlying graph structures, which represent the data correlation. When the explicit prior graph structures are not available, most existing works cannot guarantee the sparsity of the generated graphs that make the overall model computational expensive and less interpretable. In this work, we propose a decoupled training method, which includes a graph generating module and a GNNs forecasting module. First, we use Graphical Lasso (or GraphLASSO) to directly exploit the sparsity pattern from data to build graph structures in both static and time-varying cases. Second, we fit these graph structures and the input data into a Graph Convolutional Recurrent Network (GCRN) to train a forecasting model. The experimental results on three real-world datasets show that our novel approach has competitive performance against existing state-of-the-art forecasting algorithms while providing sparse, meaningful and explainable graph structures and reducing training time by approximately 40%. Our PyTorch implementation is publicly available at https://github.com/HySonLab/GraphLASSO
Evaluation of AI-Supported Input Methods in Augmented Reality Environment
Venue: arXiv cs.HC 2023
Year: 2023

Augmented Reality (AR) solutions are providing tools that could improve applications in the medical and industrial fields. Augmentation can provide additional information in training, visualization, and work scenarios, to increase efficiency, reliability, and safety, while improving communication with other devices and systems on the network. Unfortunately, tasks in these fields often require both hands to execute, reducing the variety of input methods suitable to control AR applications. People with certain physical disabilities, where they are not able to use their hands, are also negatively impacted when using these devices. The goal of this work is to provide novel hand-free interfacing methods, using AR technology, in association with AI support approaches to produce an improved Human-Computer interaction solution.
ItyFuzz: Snapshot-Based Fuzzer for Smart Contract
Venue: arXiv cs.CR 2023
Year: 2023

Smart contracts are critical financial instruments, and their security is of utmost importance. However, smart contract programs are difficult to fuzz due to the persistent blockchain state behind all transactions. Mutating sequences of transactions are complex and often lead to a suboptimal exploration for both input and program spaces. In this paper, we introduce a novel snapshot-based fuzzer ItyFuzz for testing smart contracts. In ItyFuzz, instead of storing sequences of transactions and mutating from them, we snapshot states and singleton transactions. To explore interesting states, ItyFuzz introduces a dataflow waypoint mechanism to identify states with more potential momentum. ItyFuzz also incorporates comparison waypoints to prune the space of states. By maintaining snapshots of the states, ItyFuzz can synthesize concrete exploits like reentrancy attacks quickly. Because ItyFuzz has second-level response time to test a smart contract, it can be used for on-chain testing, which has many benefits compared to local development testing. Finally, we evaluate ItyFuzz on real-world smart contracts and some hacked on-chain DeFi projects. ItyFuzz outperforms existing fuzzers in terms of instructional coverage and can find and generate realistic exploits for on-chain projects quickly.
Generate Anything Anywhere in Any Scene
Venue: arXiv cs.CV 2023
Year: 2023

Text-to-image diffusion models have attracted considerable interest due to their wide applicability across diverse fields. However, challenges persist in creating controllable models for personalized object generation. In this paper, we first identify the entanglement issues in existing personalized generative models, and then propose a straightforward and efficient data augmentation training strategy that guides the diffusion model to focus solely on object identity. By inserting the plug-and-play adapter layers from a pre-trained controllable diffusion model, our model obtains the ability to control the location and size of each generated personalized object. During inference, we propose a regionally-guided sampling technique to maintain the quality and fidelity of the generated images. Our method achieves comparable or superior fidelity for personalized objects, yielding a robust, versatile, and controllable text-to-image diffusion model that is capable of generating realistic and personalized images. Our approach demonstrates significant potential for various applications, such as those in art, entertainment, and advertising design.
FANET Experiment: Real-Time Surveillance Applications Connected to Image Processing System
Venue: arXiv cs.CV 2023
Year: 2023

The major goal of this paper is to use image enhancement techniques for enhancing and extracting data in FANET applications to improve the efficiency of surveillance. The proposed conceptual system design can improve the likelihood of FANET operations in oil pipeline surveillance, and sports and media coverage with the ultimate goal of providing efficient services to those who are interested. The system architecture model is based on current scientific principles and developing technologies. A FANET, which is capable of gathering image data from video-enabled drones, and an image processing system that permits data collection and analysis are the two primary components of the system. Based on the image processing technique, a proof of concept for efficient data extraction and enhancement in FANET situations and possible services is illustrated.
Empowering NLG: Offline Reinforcement Learning for Informal Summarization in Online Domains
Venue: arXiv cs.CL 2023
Year: 2023

Our research introduces an innovative Natural Language Generation (NLG) approach that aims to optimize user experience and alleviate the workload of human customer support agents. Our primary objective is to generate informal summaries for online articles and posts using an offline reinforcement learning technique. In our study, we compare our proposed method with existing approaches to text generation and provide a comprehensive overview of our architectural design, which incorporates crawling, reinforcement learning, and text generation modules. By presenting this original approach, our paper makes a valuable contribution to the field of NLG by offering a fresh perspective on generating natural language summaries for online content. Through the implementation of Empowering NLG, we are able to generate higher-quality replies in the online domain. The experimental results demonstrate a significant improvement in the average "like" score, increasing from 0.09954378 to 0.5000152. This advancement has the potential to enhance the efficiency and effectiveness of customer support services and elevate the overall user experience when consuming online content.
Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis
Venue: arXiv cs.CL 2023
Year: 2023

Generative Adversarial Networks (GAN) is a model for data synthesis, which creates plausible data through the competition of generator and discriminator. Although GAN application to image synthesis is extensively studied, it has inherent limitations to natural language generation. Because natural language is composed of discrete tokens, a generator has difficulty updating its gradient through backpropagation; therefore, most text-GAN studies generate sentences starting with a random token based on a reward system. Thus, the generators of previous studies are pre-trained in an autoregressive way before adversarial training, causing data memorization that synthesized sentences reproduce the training data. In this paper, we synthesize sentences using a framework similar to the original GAN. More specifically, we propose Text Embedding Space Generative Adversarial Networks (TESGAN) which generate continuous text embedding spaces instead of discrete tokens to solve the gradient backpropagation problem. Furthermore, TESGAN conducts unsupervised learning which does not directly refer to the text of the training data to overcome the data memorization issue. By adopting this novel method, TESGAN can synthesize new sentences, showing the potential of unsupervised learning for text synthesis. We expect to see extended research combining Large Language Models with a new perspective of viewing text as an continuous space.
Deep Reinforcement Learning for Privacy-Preserving Task Offloading in Integrated Satellite-Terrestrial Networks
Venue: arXiv cs.CR 2023
Year: 2023

Satellite communication networks have attracted widespread attention for seamless network coverage and collaborative computing. In satellite-terrestrial networks, ground users can offload computing tasks to visible satellites that with strong computational capabilities. Existing solutions on satellite-assisted task computing generally focused on system performance optimization such as task completion time and energy consumption. However, due to the high-speed mobility pattern and unreliable communication channels, existing methods still suffer from serious privacy leakages. In this paper, we present an integrated satellite-terrestrial network to enable satellite-assisted task offloading under dynamic mobility nature. We also propose a privacy-preserving task offloading scheme to bridge the gap between offloading performance and privacy leakage. In particular, we balance two offloading privacy, called the usage pattern privacy and the location privacy, with different offloading targets (e.g., completion time, energy consumption, and communication reliability). Finally, we formulate it into a joint optimization problem, and introduce a deep reinforcement learning-based privacy-preserving algorithm for an optimal offloading policy. Experimental results show that our proposed algorithm outperforms other benchmark algorithms in terms of completion time, energy consumption, privacy-preserving level, and communication reliability. We hope this work could provide improved solutions for privacy-persevering task offloading in satellite-assisted edge computing.
An Intelligent Mechanism for Monitoring and Detecting Intrusions in IoT Devices
Venue: arXiv cs.DC 2023
Year: 2023

The current amount of IoT devices and their limitations has come to serve as a motivation for malicious entities to take advantage of such devices and use them for their own gain. To protect against cyberattacks in IoT devices, Machine Learning techniques can be applied to Intrusion Detection Systems. Moreover, privacy related issues associated with centralized approaches can be mitigated through Federated Learning. This work proposes a Host-based Intrusion Detection Systems that leverages Federated Learning and Multi-Layer Perceptron neural networks to detected cyberattacks on IoT devices with high accuracy and enhancing data privacy protection.
Optimal Testing and Containment Strategies for Universities in Mexico amid COVID-19
Venue: arXiv cs.GT 2023
Year: 2023

This work sets out a testing and containment framework developed for reopening universities in Mexico following the lockdown due to COVID-19. We treat diagnostic testing as a resource allocation problem and develop a testing allocation mechanism and practical web application to assist educational institutions in making the most of limited testing resources. In addition to the technical results and tools, we also provide a reflection on our current experience of running a pilot of our framework within the Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM), a leading private university in Mexico, as well as on our broader experience bridging research with academic policy in the Mexican context.
Improved Algorithms for Online Rent Minimization Problem Under Unit-Size Jobs
Venue: arXiv cs.DS 2023
Year: 2023

We consider the Online Rent Minimization problem, where online jobs with release times, deadlines, and processing times must be scheduled on machines that can be rented for a fixed length period of $T$. The objective is to minimize the number of machine rents. This problem generalizes the Online Machine Minimization problem where machines can be rented for an infinite period, and both problems have an asymptotically optimal competitive ratio of $O(\log(p_{\max}/p_{\min}))$ for general processing times, where $p_{\max}$ and $p_{\min}$ are the maximum and minimum processing times respectively. However, for small values of $p_{\max}/p_{\min}$, a better competitive ratio can be achieved by assuming unit-size jobs. Under this assumption, Devanur et al. (2014) gave an optimal $e$-competitive algorithm for Online Machine Minimization, and Chen and Zhang (2022) gave a $(3e+7)\approx 15.16$-competitive algorithm for Online Rent Minimization. In this paper, we significantly improve the competitive ratio of the Online Rent Minimization problem under unit size to $6$, by using a clean oracle-based online algorithm framework.
The power of motifs as inductive bias for learning molecular distributions
Venue: arXiv cs.LG 2023
Year: 2023

Machine learning for molecules holds great potential for efficiently exploring the vast chemical space and thus streamlining the drug discovery process by facilitating the design of new therapeutic molecules. Deep generative models have shown promising results for molecule generation, but the benefits of specific inductive biases for learning distributions over small graphs are unclear. Our study aims to investigate the impact of subgraph structures and vocabulary design on distribution learning, using small drug molecules as a case study. To this end, we introduce Subcover, a new subgraph-based fragmentation scheme, and evaluate it through a two-step variational auto-encoder. Our results show that Subcover's improved identification of chemically meaningful subgraphs leads to a relative improvement of the FCD score by 30%, outperforming previous methods. Our findings highlight the potential of Subcover to enhance the performance and scalability of existing methods, contributing to the advancement of drug discovery.
Subgraph Stationary Hardware-Software Inference Co-Design
Venue: arXiv cs.DC 2023
Year: 2023

A growing number of applications depend on Machine Learning (ML) functionality and benefits from both higher quality ML predictions and better timeliness (latency) at the same time. A growing body of research in computer architecture, ML, and systems software literature focuses on reaching better latency-accuracy tradeoffs for ML models. Efforts include compression, quantization, pruning, early-exit models, mixed DNN precision, as well as ML inference accelerator designs that minimize latency and energy, while preserving delivered accuracy. All of them, however, yield improvements for a single static point in the latency-accuracy tradeoff space. We make a case for applications that operate in dynamically changing deployment scenarios, where no single static point is optimal. We draw on a recently proposed weight-shared SuperNet mechanism to enable serving a stream of queries that uses (activates) different SubNets within this weight-shared construct. This creates an opportunity to exploit the inherent temporal locality with our proposed SubGraph Stationary (SGS) optimization. We take a hardware-software co-design approach with a real implementation of SGS in SushiAccel and the implementation of a software scheduler SushiSched controlling which SubNets to serve and what to cache in real-time. Combined, they are vertically integrated into SUSHI-an inference serving stack. For the stream of queries, SUSHI yields up to 25% improvement in latency, 0.98% increase in served accuracy. SUSHI can achieve up to 78.7% off-chip energy savings.
Training-Free Neural Matte Extraction for Visual Effects
Venue: arXiv cs.CV 2023
Year: 2023

Alpha matting is widely used in video conferencing as well as in movies, television, and social media sites. Deep learning approaches to the matte extraction problem are well suited to video conferencing due to the consistent subject matter (front-facing humans), however training-based approaches are somewhat pointless for entertainment videos where varied subjects (spaceships, monsters, etc.) may appear only a few times in a single movie -- if a method of creating ground truth for training exists, just use that method to produce the desired mattes. We introduce a training-free high quality neural matte extraction approach that specifically targets the assumptions of visual effects production. Our approach is based on the deep image prior, which optimizes a deep neural network to fit a single image, thereby providing a deep encoding of the particular image. We make use of the representations in the penultimate layer to interpolate coarse and incomplete "trimap" constraints. Videos processed with this approach are temporally consistent. The algorithm is both very simple and surprisingly effective.
Citations as Queries: Source Attribution Using Language Models as Rerankers
Venue: arXiv cs.CL 2023
Year: 2023

This paper explores new methods for locating the sources used to write a text, by fine-tuning a variety of language models to rerank candidate sources. After retrieving candidates sources using a baseline BM25 retrieval model, a variety of reranking methods are tested to see how effective they are at the task of source attribution. We conduct experiments on two datasets, English Wikipedia and medieval Arabic historical writing, and employ a variety of retrieval and generation based reranking models. In particular, we seek to understand how the degree of supervision required affects the performance of various reranking models. We find that semisupervised methods can be nearly as effective as fully supervised methods while avoiding potentially costly span-level annotation of the target and source documents.
What Could a Social Mediator Robot Do? Lessons from Real-World Mediation Scenarios
Venue: arXiv cs.RO 2023
Year: 2023

The use of social robots as instruments for social mediation has been gaining traction in the field of Human-Robot Interaction (HRI). So far, the design of such robots and their behaviors is often driven by technological platforms and experimental setups in controlled laboratory environments. To address complex social relationships in the real world, it is crucial to consider the actual needs and consequences of the situations found therein. This includes understanding when a mediator is necessary, what specific role such a robot could play, and how it moderates human social dynamics. In this paper, we discuss six relevant roles for robotic mediators that we identified by investigating a collection of videos showing realistic group situations. We further discuss mediation behaviors and target measures to evaluate the success of such interventions. We hope that our findings can inspire future research on robot-assisted social mediation by highlighting a wider set of mediation applications than those found in prior studies. Specifically, we aim to inform the categorization and selection of interaction scenarios that reflect real situations, where a mediation robot can have a positive and meaningful impact on group dynamics.
TransDetector: A Transformer-Based Detector for Underwater Acoustic Differential OFDM Communications
Venue: arXiv cs.IT 2023
Year: 2023

Inter-carrier interference (ICI) and noise mitigation is crucial for precise signal detection in underwater acoustic (UWA) differential orthogonal frequency division multiplexing (DOFDM) communication systems. In this paper, we adopt the Transformer to design a detector, referred to as the TransDetector, which can dramatically mitigate ICI implicitly and noise explicitly, even without requiring any pilot. Compared with the standard Transformer, we come up with three creative designs. Firstly, we break the inner-encoder computation paradigm of the multi-head attention (MHA) in the standard Transformer, and design a brand new inter-encoder attention mechanism, referred to as the interactive MHA, which can significantly improve the performance, as well as accelerate the convergence rate. Secondly, to reduce the noise component attached to the received signal, we design an auto-perception denoising structure, which allows the network to learn the noise distribution in received signals. Thirdly, to better match the characteristics of DOFDM signals and selectively focus on the data at specified locations, we propose a trapezoidal positional encoding (PE), instead of adopting the original sine-cosine PE in the Transformer. Experimental results on both the realistic underwater channel and the simulation channel show that the TransDetector outperforms the classical $\mathcal{X}$-FFT algorithms and the DNNDetector in terms of the BER and the MSE. For example, the BER achieved by the TransDetector is reduced by $27.21\%$ and $12.50\%$ when the signal-to-noise ratio $(\text{SNR})=0$~dB and by $47.44\%$ and $33.49\%$ when $\text{SNR}=20$~dB against the PS-FFT and the DNNDetector based on the realistic channel, respectively.
Topological Data Analysis Guided Segment Anything Model Prompt Optimization for Zero-Shot Segmentation in Biological Imaging
Venue: arXiv cs.CV 2023
Year: 2023

Emerging foundation models in machine learning are models trained on vast amounts of data that have been shown to generalize well to new tasks. Often these models can be prompted with multi-modal inputs that range from natural language descriptions over images to point clouds. In this paper, we propose topological data analysis (TDA) guided prompt optimization for the Segment Anything Model (SAM) and show preliminary results in the biological image segmentation domain. Our approach replaces the standard grid search approach that is used in the original implementation and finds point locations based on their topological significance. Our results show that the TDA optimized point cloud is much better suited for finding small objects and massively reduces computational complexity despite the extra step in scenarios which require many segmentations.
Knowledge Base Completion for Long-Tail Entities
Venue: arXiv cs.CL 2023
Year: 2023

Despite their impressive scale, knowledge bases (KBs), such as Wikidata, still contain significant gaps. Language models (LMs) have been proposed as a source for filling these gaps. However, prior works have focused on prominent entities with rich coverage by LMs, neglecting the crucial case of long-tail entities. In this paper, we present a novel method for LM-based-KB completion that is specifically geared for facts about long-tail entities. The method leverages two different LMs in two stages: for candidate retrieval and for candidate verification and disambiguation. To evaluate our method and various baselines, we introduce a novel dataset, called MALT, rooted in Wikidata. Our method outperforms all baselines in F1, with major gains especially in recall.
TTSWING: a Dataset for Table Tennis Swing Analysis
Venue: arXiv cs.LG 2023
Year: 2023

We introduce TTSWING, a novel dataset designed for table tennis swing analysis. This dataset comprises comprehensive swing information obtained through 9-axis sensors integrated into custom-made racket grips, accompanied by anonymized demographic data of the players. We detail the data collection and annotation procedures. Furthermore, we conduct pilot studies utilizing diverse machine learning models for swing analysis. TTSWING holds tremendous potential to facilitate innovative research in table tennis analysis and is a valuable resource for the scientific community. We release the dataset and experimental codes at https://github.com/DEPhantom/TTSWING.