OpenRaiser/Intern-Atlas huggingface dataset:
Mover: Mask and Recovery based Facial Part Consistency Aware Method for Deepfake Video Detection
Venue: arXiv cs.MM 2023
Year: 2023

Deepfake techniques have been widely used for malicious purposes, prompting extensive research interest in developing Deepfake detection methods. Deepfake manipulations typically involve tampering with facial parts, which can result in inconsistencies across different parts of the face. For instance, Deepfake techniques may change smiling lips to an upset lip, while the eyes remain smiling. Existing detection methods depend on specific indicators of forgery, which tend to disappear as the forgery patterns are improved. To address the limitation, we propose Mover, a new Deepfake detection model that exploits unspecific facial part inconsistencies, which are inevitable weaknesses of Deepfake videos. Mover randomly masks regions of interest (ROIs) and recovers faces to learn unspecific features, which makes it difficult for fake faces to be recovered, while real faces can be easily recovered. Specifically, given a real face image, we first pretrain a masked autoencoder to learn facial part consistency by dividing faces into three parts and randomly masking ROIs, which are then recovered based on the unmasked facial parts. Furthermore, to maximize the discrepancy between real and fake videos, we propose a novel model with dual networks that utilize the pretrained encoder and masked autoencoder, respectively. 1) The pretrained encoder is finetuned for capturing the encoding of inconsistent information in the given video. 2) The pretrained masked autoencoder is utilized for mapping faces and distinguishing real and fake videos. Our extensive experiments on standard benchmarks demonstrate that Mover is highly effective.
Survey of Code Search Based on Deep Learning
Venue: arXiv cs.SE 2023
Year: 2023

Code writing is repetitive and predictable, inspiring us to develop various code intelligence techniques. This survey focuses on code search, that is, to retrieve code that matches a given query by effectively capturing the semantic similarity between the query and code. Deep learning, being able to extract complex semantics information, has achieved great success in this field. Recently, various deep learning methods, such as graph neural networks and pretraining models, have been applied to code search with significant progress. Deep learning is now the leading paradigm for code search. In this survey, we provide a comprehensive overview of deep learning-based code search. We review the existing deep learning-based code search framework which maps query/code to vectors and measures their similarity. Furthermore, we propose a new taxonomy to illustrate the state-of-the-art deep learning-based code search in a three-steps process: query semantics modeling, code semantics modeling, and matching modeling which involves the deep learning model training. Finally, we suggest potential avenues for future research in this promising field.
Transaction Confirmation in Coded Blockchain
Venue: arXiv cs.IT 2023
Year: 2023

As blockchains continue to seek to scale to a larger number of nodes, the communication complexity of protocols has become a significant priority as the network can quickly become overburdened. Several schemes have attempted to address this, one of which uses coded computation to lighten the load. Here we seek to address one issue with all such coded blockchain schemes known to the authors: transaction confirmation. In a coded blockchain, only the leader has access to the uncoded block, while the nodes receive encoded data that makes it effectively impossible for them to identify which transactions were included in the block. As a result, a Byzantine leader might choose not to notify a sender or receiver of a transaction that the transaction went into the block, and even with an honest leader, they would not be able to produce a proof of a transaction's inclusion. To address this, we have constructed a protocol to send the nodes enough information so that a client sending or receiving a transaction is guaranteed to not only be notified but also to receive a proof of that transaction's inclusion in the block. Crucially, we do this without substantially increasing the bit complexity of the original coded blockchain protocol.
DMNR: Unsupervised De-noising of Point Clouds Corrupted by Airborne Particles
Venue: arXiv cs.CV 2023
Year: 2023

LiDAR sensors are critical for autonomous driving and robotics applications due to their ability to provide accurate range measurements and their robustness to lighting conditions. However, airborne particles, such as fog, rain, snow, and dust, will degrade its performance and it is inevitable to encounter these inclement environmental conditions outdoors. It would be a straightforward approach to remove them by supervised semantic segmentation. But annotating these particles point wisely is too laborious. To address this problem and enhance the perception under inclement conditions, we develop two dynamic filtering methods called Dynamic Multi-threshold Noise Removal (DMNR) and DMNR-H by accurate analysis of the position distribution and intensity characteristics of noisy points and clean points on publicly available WADS and DENSE datasets. Both DMNR and DMNR-H outperform state-of-the-art unsupervised methods by a significant margin on the two datasets and are slightly better than supervised deep learning-based methods. Furthermore, our methods are more robust to different LiDAR sensors and airborne particles, such as snow and fog.
Safe motion planning with environment uncertainty
Venue: arXiv cs.RO 2023
Year: 2023

We present an approach for safe motion planning under robot state and environment (obstacle and landmark location) uncertainties. To this end, we first develop an approach that accounts for the landmark uncertainties during robot localization. Existing planning approaches assume that the landmark locations are well known or are known with little uncertainty. However, this might not be true in practice. Noisy sensors and imperfect motions compound to the errors originating from the estimate of environment features. Moreover, possible occlusions and dynamic objects in the environment render imperfect landmark estimation. Consequently, not considering this uncertainty can wrongly localize the robot, leading to inefficient plans. Our approach thus incorporates the landmark uncertainty within the Bayes filter estimation framework. We also analyze the effect of considering this uncertainty and delineate the conditions under which it can be ignored. Second, we extend the state-of-the-art by computing an exact expression for the collision probability under Gaussian distributed robot motion, perception and obstacle location uncertainties. We formulate the collision probability process as a quadratic form in random variables. Under Gaussian distribution assumptions, an exact expression for collision probability is thus obtained which is computable in real-time. In contrast, existing approaches approximate the collision probability using upper-bounds that can lead to overly conservative estimate and thereby suboptimal plans. We demonstrate and evaluate our approach using a theoretical example and simulations. We also present a comparison of our approach to different state-of-the-art methods.
Stochastic Chemical Reaction Networks for MAP Detection in Cellular Receivers
Venue: arXiv cs.ET 2023
Year: 2023

In order to fully exploit the potential of molecular communication (MC) for intra-body communication, practically implementable cellular receivers are an important long-term goal. A variety of receiver architectures based on chemical reaction networks (CRNs) and gene-regulatory networks (GRNs) has been introduced in the literature, because cells use these concepts to perform computations in nature. However, practical feasibility is still limited by stochastic fluctuations of chemical reactions and long computation times in GRNs. Therefore, in this paper, we propose two receiver designs based on stochastic CRNs, i.e., CRNs that perform computations by exploiting the intrinsic fluctuations of chemical reactions with very low molecule counts. The first CRN builds on a recent result from chemistry that showed how Boltzmann machines (BMs), a commonly used machine learning model, can be implemented with CRNs. We show that BMs with optimal parameter values and their CRN implementations can act as maximum-a-posteriori (MAP) detectors. Furthermore, we show that BMs can be efficiently trained from simulation data to achieve close-to-MAP performance. While this approach yields a fixed CRN once deployed, our second approach based on a manually designed CRN can be trained with pilot symbols even within the cell and thus adapt to changing channel conditions. We extend the literature by showing that practical robust detectors can achieve close-to-MAP performance even without explicit channel knowledge.
Scalable orthogonal delay-division multiplexed OEO artificial neural network trained for TI-ADC equalization
Venue: arXiv cs.ET 2023
Year: 2023

We propose a new signaling scheme for on-chip optical-electrical-optical artificial neural networks that utilizes orthogonal delay-division multiplexing and pilot-tone based self-homodyne detection. This scheme offers a more efficient scaling of the optical power budget with increasing network complexity. Our simulations, based on a 220 nm SOI silicon photonics technology, suggest that the network can support 31 x 31 neurons, with 961 links and freely programmable weights, using a single 500 mW optical comb and an SNR of 21.3 dB per neuron. Moreover, it features a low sensitivity to temperature fluctuations, ensuring that it can be operated outside of a laboratory environment. We demonstrate the network's effectiveness in nonlinear equalization tasks by training it to equalize a time-interleaved ADC architecture, achieving an ENOB over 4 over the entire 75 GHz ADC bandwidth. We anticipate that this network architecture will enable broadband and low latency nonlinear signal processing in practical settings such as ultra-broadband data converters and real-time control systems.
DNN Verification, Reachability, and the Exponential Function Problem
Venue: arXiv cs.LO 2023
Year: 2023

Deep neural networks (DNNs) are increasingly being deployed to perform safety-critical tasks. The opacity of DNNs, which prevents humans from reasoning about them, presents new safety and security challenges. To address these challenges, the verification community has begun developing techniques for rigorously analyzing DNNs, with numerous verification algorithms proposed in recent years. While a significant amount of work has gone into developing these verification algorithms, little work has been devoted to rigorously studying the computability and complexity of the underlying theoretical problems. Here, we seek to contribute to the bridging of this gap. We focus on two kinds of DNNs: those that employ piecewise-linear activation functions (e.g., ReLU), and those that employ piecewise-smooth activation functions (e.g., Sigmoids). We prove the two following theorems: 1) The decidability of verifying DNNs with a particular set of piecewise-smooth activation functions is equivalent to a well-known, open problem formulated by Tarski; and 2) The DNN verification problem for any quantifier-free linear arithmetic specification can be reduced to the DNN reachability problem, whose approximation is NP-complete. These results answer two fundamental questions about the computability and complexity of DNN verification, and the ways it is affected by the network's activation functions and error tolerance; and could help guide future efforts in developing DNN verification tools.
Pebble guided Treasure Hunt in Plane
Venue: arXiv cs.DC 2023
Year: 2023

We study the problem of treasure hunt in a Euclidean plane by a mobile agent with the guidance of pebbles. The initial position of the agent and position of the treasure are modeled as special points in the Euclidean plane. The treasure is situated at a distance at most $D>0$ from the initial position of the agent. The agent has a perfect compass, but an adversary controls the speed of the agent. Hence, the agent can not measure how much distance it traveled for a given time. The agent can find the treasure only when it reaches the exact position of the treasure. The cost of the treasure hunt is defined as the total distance traveled by the agent before it finds the treasure. The agent has no prior knowledge of the position of the treasure or the value of $D$. An Oracle, which knows the treasure's position and the agent's initial location, places some pebbles to guide the agent towards the treasure. Once decided to move along some specified angular direction, the agent can decide to change its direction only when it encounters a pebble or a special point. We ask the following central question in this paper: ``For given $k \ge 0$, What is cheapest treasure hunt algorithm if at most $k$ pebbles are placed by the Oracle?" We show that for $k=1$, there does not exist any treasure hunt algorithm that finds the treasure with finite cost. We show the existence of an algorithm with cost $O(D)$ for $k=2$. For $k>8$ we have designed an algorithm that uses $k$ many pebbles to find the treasure with cost $O(k^{2}) + D(\sinθ' + \cosθ')$, where $θ'=\fracπ{2^{k-8}}$. The second result shows the existence of an algorithm with cost arbitrarily close to $D$ for sufficiently large values of $D$.
Access-Redundancy Tradeoffs in Quantized Linear Computations
Venue: arXiv cs.IT 2023
Year: 2023

Linear real-valued computations over distributed datasets are common in many applications, most notably as part of machine learning inference. In particular, linear computations that are quantized, i.e., where the coefficients are restricted to a predetermined set of values (such as $\pm 1$), have gained increasing interest lately due to their role in efficient, robust, or private machine learning models. Given a dataset to store in a distributed system, we wish to encode it so that all such computations could be conducted by accessing a small number of servers, called the access parameter of the system. Doing so relieves the remaining servers to execute other tasks. Minimizing the access parameter gives rise to an access-redundancy tradeoff, where a smaller access parameter requires more redundancy in the system, and vice versa. In this paper, we study this tradeoff and provide several explicit low-access schemes for $\{\pm1\}$ quantized linear computations based on covering codes in a novel way. While the connection to covering codes has been observed in the past, our results strictly outperform the state-of-the-art for two-valued linear computations. We further show that the same storage scheme can be used to retrieve any linear combination with two distinct coefficients -- regardless of what those coefficients are -- with the same access parameter. This universality result is then extended to all possible quantizations with any number of values; while the storage remains identical, the access parameter increases according to a new additive-combinatorics property we call coefficient complexity. We then turn to study the coefficient complexity -- we characterize the complexity of small sets of coefficients, provide bounds, and identify coefficient sets having the highest and lowest complexity.
XMI-ICU: Explainable Machine Learning Model for Pseudo-Dynamic Prediction of Mortality in the ICU for Heart Attack Patients
Venue: arXiv cs.LG 2023
Year: 2023

Heart attack remain one of the greatest contributors to mortality in the United States and globally. Patients admitted to the intensive care unit (ICU) with diagnosed heart attack (myocardial infarction or MI) are at higher risk of death. In this study, we use two retrospective cohorts extracted from the eICU and MIMIC-IV databases, to develop a novel pseudo-dynamic machine learning framework for mortality prediction in the ICU with interpretability and clinical risk analysis. The method provides accurate prediction for ICU patients up to 24 hours before the event and provide time-resolved interpretability results. The performance of the framework relying on extreme gradient boosting was evaluated on a held-out test set from eICU, and externally validated on the MIMIC-IV cohort using the most important features identified by time-resolved Shapley values achieving AUCs of 91.0 (balanced accuracy of 82.3) for 6-hour prediction of mortality respectively. We show that our framework successfully leverages time-series physiological measurements by translating them into stacked static prediction problems to be robustly predictive through time in the ICU stay and can offer clinical insight from time-resolved interpretability
NeRF2: Neural Radio-Frequency Radiance Fields
Venue: arXiv cs.NI 2023
Year: 2023

Although Maxwell discovered the physical laws of electromagnetic waves 160 years ago, how to precisely model the propagation of an RF signal in an electrically large and complex environment remains a long-standing problem. The difficulty is in the complex interactions between the RF signal and the obstacles (e.g., reflection, diffraction, etc.). Inspired by the great success of using a neural network to describe the optical field in computer vision, we propose a neural radio-frequency radiance field, NeRF$^\textbf{2}$, which represents a continuous volumetric scene function that makes sense of an RF signal's propagation. Particularly, after training with a few signal measurements, NeRF$^\textbf{2}$ can tell how/what signal is received at any position when it knows the position of a transmitter. As a physical-layer neural network, NeRF$^\textbf{2}$ can take advantage of the learned statistic model plus the physical model of ray tracing to generate a synthetic dataset that meets the training demands of application-layer artificial neural networks (ANNs). Thus, we can boost the performance of ANNs by the proposed turbo-learning, which mixes the true and synthetic datasets to intensify the training. Our experiment results show that turbo-learning can enhance performance with an approximate 50% increase. We also demonstrate the power of NeRF$^\textbf{2}$ in the field of indoor localization and 5G MIMO.
Average Awake Complexity of MIS and Matching
Venue: arXiv cs.DS 2023
Year: 2023

Chatterjee, Gmyr, and Pandurangan [PODC 2020] recently introduced the notion of awake complexity for distributed algorithms, which measures the number of rounds in which a node is awake. In the other rounds, the node is sleeping and performs no computation or communication. Measuring the number of awake rounds can be of significance in many settings of distributed computing, e.g., in sensor networks where energy consumption is of concern. In that paper, Chatterjee et al. provide an elegant randomized algorithm for the Maximal Independent Set (MIS) problem that achieves an $O(1)$ node-averaged awake complexity. That is, the average awake time among the nodes is $O(1)$ rounds. However, to achieve that, the algorithm sacrifices the more standard round complexity measure from the well-known $O(\log n)$ bound of MIS, due to Luby [STOC'85], to $O(\log^{3.41} n)$ rounds. Our first contribution is to present a simple randomized distributed MIS algorithm that, with high probability, has $O(1)$ node-averaged awake complexity and $O(\log n)$ worst-case round complexity. Our second, and more technical contribution, is to show algorithms with the same $O(1)$ node-averaged awake complexity and $O(\log n)$ worst-case round complexity for $(1+\varepsilon)$-approximation of maximum matching and $(2+\varepsilon)$-approximation of minimum vertex cover, where $\varepsilon$ denotes an arbitrary small positive constant.
StarCoder: may the source be with you!
Venue: arXiv cs.CL 2023
Year: 2023

The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15.5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention. StarCoderBase is trained on 1 trillion tokens sourced from The Stack, a large collection of permissively licensed GitHub repositories with inspection tools and an opt-out process. We fine-tuned StarCoderBase on 35B Python tokens, resulting in the creation of StarCoder. We perform the most comprehensive evaluation of Code LLMs to date and show that StarCoderBase outperforms every open Code LLM that supports multiple programming languages and matches or outperforms the OpenAI code-cushman-001 model. Furthermore, StarCoder outperforms every model that is fine-tuned on Python, can be prompted to achieve 40\% pass@1 on HumanEval, and still retains its performance on other programming languages. We take several important steps towards a safe open-access model release, including an improved PII redaction pipeline and a novel attribution tracing tool, and make the StarCoder models publicly available under a more commercially viable version of the Open Responsible AI Model license.
Conflict Analysis and Resolution of Safety and Security Boundary Conditions for Industrial Control Systems
Venue: arXiv cs.CR 2023
Year: 2023

Safety and security are the two most important properties of industrial control systems (ICS), and their integration is necessary to ensure that safety goals do not undermine security goals and vice versa. Sometimes, safety and security co-engineering leads to conflicting requirements or violations capable of impacting the normal behavior of the system. Identification, analysis, and resolution of conflicts arising from safety and security co-engineering is a major challenge, an under-researched area in safety-critical systems(ICS). This paper presents an STPA-SafeSec-CDCL approach that addresses the challenge. Our proposed methodology combines the STPA-SafeSec approach for safety and security analysis and the Conflict-Driven Clause Learning (CDCL) approach for the identification, analysis, and resolution of conflicts where conflicting constraints are encoded in satisfiability (SAT) problems. We apply our framework to the Tennessee Eastman Plant process model, a chemical process model developed specifically for the study of industrial control processes, to demonstrate how to use the proposed method. Our methodology goes beyond the requirement analysis phase and can be applied to the early stages of system design and development to increase system reliability, robustness, and resilience.
Learning in a Single Domain for Non-Stationary Multi-Texture Synthesis
Venue: arXiv cs.CV 2023
Year: 2023

This paper aims for a new generation task: non-stationary multi-texture synthesis, which unifies synthesizing multiple non-stationary textures in a single model. Most non-stationary textures have large scale variance and can hardly be synthesized through one model. To combat this, we propose a multi-scale generator to capture structural patterns of various scales and effectively synthesize textures with a minor cost. However, it is still hard to handle textures of different categories with different texture patterns. Therefore, we present a category-specific training strategy to focus on learning texture pattern of a specific domain. Interestingly, once trained, our model is able to produce multi-pattern generations with dynamic variations without the need to finetune the model for different styles. Moreover, an objective evaluation metric is designed for evaluating the quality of texture expansion and global structure consistency. To our knowledge, ours is the first scheme for this challenging task, including model, training, and evaluation. Experimental results demonstrate the proposed method achieves superior performance and time efficiency. The code will be available after the publication.