OpenRaiser/Intern-Atlas huggingface dataset:
The Theory of an Arbitrary Higher $λ$-Model
Venue: arXiv cs.LO 2023
Year: 2023

One takes advantage of some basic properties of every homotopic $λ$-model (e.g.\ extensional Kan complex) to explore the higher $βη$-conversions, which would correspond to proofs of equality between terms of a theory of equality of any extensional Kan complex. Besides, Identity types based on computational paths are adapted to a type-free theory with higher $λ$-terms, whose equality rules would be contained in the theory of any $λ$-homotopic model.
Finding the Minimum Norm and Center Density of Cyclic Lattices via Nonlinear Systems
Venue: arXiv cs.IT 2023
Year: 2023

Lattices with a circulant generator matrix represent a subclass of cyclic lattices. This subclass can be described by a basis containing a vector and its circular shifts. In this paper, we present certain conditions under which the norm expression of an arbitrary vector of this type of lattice is substantially simplified, and then investigate some of the lattices obtained under these conditions. We exhibit systems of nonlinear equations whose solutions yield lattices as dense as $D_n$ in odd dimensions. As far as even dimensions, we obtain lattices denser than $A_n$ as long as $n \in 2\mathbb{Z} \backslash 4\mathbb{Z}$.
LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model
Venue: arXiv cs.LG 2023
Year: 2023

The system log generated in a computer system refers to large-scale data that are collected simultaneously and used as the basic data for determining errors, intrusion and abnormal behaviors. The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention, which is a critical problem in the industry. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template using a parser. Particularly, a template corresponding to a specific event should be defined in advance for all the log data using which the information within the log key may get lost. In this study, we propose LAnoBERT, a parser free system log anomaly detection method that uses the BERT model, exhibiting excellent natural language processing performance. The proposed method, LAnoBERT, learns the model through masked language modeling, which is a BERT-based pre-training method, and proceeds with unsupervised learning-based anomaly detection using the masked language modeling loss function per log key during the test process. In addition, we also propose an efficient inference process to establish a practically applicable pipeline to the actual system. Experiments on three well-known log datasets, i.e., HDFS, BGL, and Thunderbird, show that not only did LAnoBERT yield a higher anomaly detection performance compared to unsupervised learning-based benchmark models, but also it resulted in a comparable performance with supervised learning-based benchmark models.
Joint Delay and Phase Precoding Under True-Time Delay Constraints for THz Massive MIMO
Venue: arXiv cs.IT 2023
Year: 2023

A new approach is presented to the problem of compensating the beam squint effect arising in wideband terahertz (THz) hybrid massive multiple-input multiple-output (MIMO) systems, based on the joint optimization of the phase shifter (PS) and true-time delay (TTD) values under per-TTD device time delay constraints. Unlike the prior approaches, the new approach does not require the unbounded time delay assumption; the range of time delay values that a TTD device can produce is strictly limited in our approach. Instead of focusing on the design of TTD values, we jointly optimize both the TTD and PS values to effectively cope with the practical time delay constraints. Simulation results that illustrate the performance benefits of the new method for the beam squint compensation are presented. Through simulations and analysis, we show that our approach is a generalization of the prior TTD-based precoding approaches.
Consequences of Optimality
Venue: arXiv cs.CY 2023
Year: 2023

Rationality is often related to optimal decision making. Humans are known to be bounded rational agents. However, recent advances in computing, and other scientific and technical fields along with large amount of data have led to a feeling that this could result in extending the limits of bounded rationality in humans through augmented machine intelligence. In this paper, results from a computational model show that as more agents reach global optimality, faster with enhanced computing, etc., solving the same problem independently, this leads to accelerated "tragedy of the commons" due to quicker resource consumption. Thus, bounded rationality could be seen as blessing in disguise (providing diversity to solutions for the same problem) from sustainability standpoint.
Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Skeleton-based action recognition is widely used in varied areas, e.g., surveillance and human-machine interaction. Existing models are mainly learned in a supervised manner, thus heavily depending on large-scale labeled data which could be infeasible when labels are prohibitively expensive. In this paper, we propose a novel Contrast-Reconstruction Representation Learning network (CRRL) that simultaneously captures postures and motion dynamics for unsupervised skeleton-based action recognition. It mainly consists of three parts: Sequence Reconstructor, Contrastive Motion Learner, and Information Fuser. The Sequence Reconstructor learns representation from skeleton coordinate sequence via reconstruction, thus the learned representation tends to focus on trivial postural coordinates and be hesitant in motion learning. To enhance the learning of motions, the Contrastive Motion Learner performs contrastive learning between the representations learned from coordinate sequence and additional velocity sequence, respectively. Finally, in the Information Fuser, we explore varied strategies to combine the Sequence Reconstructor and Contrastive Motion Learner, and propose to capture postures and motions simultaneously via a knowledge-distillation based fusion strategy that transfers the motion learning from the Contrastive Motion Learner to the Sequence Reconstructor. Experimental results on several benchmarks, i.e., NTU RGB+D 60, NTU RGB+D 120, CMU mocap, and NW-UCLA, demonstrate the promise of the proposed CRRL method by far outperforming state-of-the-art approaches.
Bacteria-inspired robotic propulsion from bundling of soft helical filaments at low Reynolds number
Venue: arXiv cs.RO 2023
Year: 2023

The bundling of flagella is known to create a "run" phase, where the bacteria moves in a nearly straight line rather than making changes in direction. Historically, mechanical explanations for the bundling phenomenon intrigued many researchers, and significant advances were made in physical models and experimental methods. Contributing to the field of research, we present a bacteria-inspired centimeter-scale soft robotic hardware platform and a computational framework for a physically plausible simulation model of the multi-flagellated robot under low Reynolds number (~0.1). The fluid-structure interaction simulation couples the Discrete Elastic Rods algorithm with the method of Regularized Stokeslet Segments. Contact between two flagella is handled by a penalty-based method. We present a comparison between our experimental and simulation results and verify that the simulation tool can capture the essential physics of this problem. Preliminary findings on robustness to buckling provided by the bundling phenomenon and the efficiency of a multi-flagellated soft robot are compared with the single-flagellated counterparts. Observations were made on the coupling between geometry and elasticity, which manifests itself in the propulsion of the robot by nonlinear dependency on the rotational speed of the flagella.
A Close Look at Few-shot Real Image Super-resolution from the Distortion Relation Perspective
Venue: arXiv cs.CV 2023
Year: 2023

Collecting amounts of distorted/clean image pairs in the real world is non-trivial, which seriously limits the practical applications of these supervised learning-based methods on real-world image super-resolution (RealSR). Previous works usually address this problem by leveraging unsupervised learning-based technologies to alleviate the dependency on paired training samples. However, these methods typically suffer from unsatisfactory texture synthesis due to the lack of supervision of clean images. To overcome this problem, we are the first to have a close look at the under-explored direction for RealSR, i.e., few-shot real-world image super-resolution, which aims to tackle the challenging RealSR problem with few-shot distorted/clean image pairs. Under this brand-new scenario, we propose Distortion Relation guided Transfer Learning (DRTL) for the few-shot RealSR by transferring the rich restoration knowledge from auxiliary distortions (i.e., synthetic distortions) to the target RealSR under the guidance of distortion relation. Concretely, DRTL builds a knowledge graph to capture the distortion relation between auxiliary distortions and target distortion (i.e., real distortions in RealSR). Based on the distortion relation, DRTL adopts a gradient reweighting strategy to guide the knowledge transfer process between auxiliary distortions and target distortions. In this way, DRTL could quickly learn the most relevant knowledge from the synthetic distortions for the target distortion. We instantiate DRTL with two commonly-used transfer learning paradigms, including pre-training and meta-learning pipelines, to realize a distortion relation-aware Few-shot RealSR. Extensive experiments on multiple benchmarks and thorough ablation studies demonstrate the effectiveness of our DRTL.
Data Fusion Challenges Privacy: What Can Privacy Regulation Do?
Venue: arXiv cs.CY 2023
Year: 2023

This paper focuses on some shortcomings in current privacy and data protection regulations' ability to adequately address the ramifications of AI-driven data processing practices, in particular where data sets are combined and processed by AI systems. We raise attention to two regulatory anomalies related to two fundamental assumptions underlying traditional privacy and data protection approaches: (1) Only Personally Identifiable Information (PII) and Personal Data (PD) require privacy protection: Privacy and data protection regulations are only triggered with respect to PII/PD, but not anonymous data. This is not only problematic because determining whether data falls in the former or latter category is no longer straightforward, but also because privacy risks associated with data processing may exist whether or not an individual can be identified. (2) Given sufficient information provided in a transparent and understandable manner, individuals are able to adequately assess the privacy implications of their actions and protect their privacy interests: However, relying on human privacy expectations fails to address important privacy threats, because those expectations are at odds with the actual privacy implications of data processing practices, as most people lack the necessary technical literacy to understand the sophisticated technologies at play, and to correctly assess their privacy implications. To tackle these anomalies we recommend regulatory reform in two directions: (1) Abolishing the distinction between personal and anonymized data for the purposes of triggering the application of privacy and data protection regulations and (2) developing methods to prioritize regulatory intervention based on the level of privacy risk posed by individual data processing actions.
Complexity and Approximability of Edge-Vertex Domination in UDG
Venue: arXiv cs.CG 2023
Year: 2023

Given an undirected graph $G=(V,E)$, a vertex $v\in V$ is edge-vertex (ev) dominated by an edge $e\in E$ if $v$ is either incident to $e$ or incident to an adjacent edge of $e$. A set $S^{ev}\subseteq E$ is an edge-vertex dominating set (referred to as \textit{ev}-dominating set and in short as \textit{EVDS}) of $G$ if every vertex of $G$ is \textit{ev}-dominated by at least one edge of $S^{ev}$. The minimum cardinality of an \textit{ev}-dominating set is the \textit{ev}-domination number. The edge-vertex dominating set problem is to find a minimum \textit{ev}-domination number. In this paper, we prove that the \textit{ev}-dominating set problem is {\tt NP-hard} on unit disk graphs. We also prove that this problem admits a polynomial-time approximation scheme on unit disk graphs. Finally, we give a simple 5-factor linear-time approximation algorithm.
Is Causal Reasoning Harder than Probabilistic Reasoning?
Venue: arXiv cs.LO 2023
Year: 2023

Many tasks in statistical and causal inference can be construed as problems of \emph{entailment} in a suitable formal language. We ask whether those problems are more difficult, from a computational perspective, for \emph{causal} probabilistic languages than for pure probabilistic (or "associational") languages. Despite several senses in which causal reasoning is indeed more complex -- both expressively and inferentially -- we show that causal entailment (or satisfiability) problems can be systematically and robustly reduced to purely probabilistic problems. Thus there is no jump in computational complexity. Along the way we answer several open problems concerning the complexity of well known probability logics, in particular demonstrating the $\exists\mathbb{R}$-completeness of a polynomial probability calculus, as well as a seemingly much simpler system, the logic of comparative conditional probability.
The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration
Venue: arXiv cs.CV 2023
Year: 2023

In spite of the dominant performances of deep neural networks, recent works have shown that they are poorly calibrated, resulting in over-confident predictions. Miscalibration can be exacerbated by overfitting due to the minimization of the cross-entropy during training, as it promotes the predicted softmax probabilities to match the one-hot label assignments. This yields a pre-softmax activation of the correct class that is significantly larger than the remaining activations. Recent evidence from the literature suggests that loss functions that embed implicit or explicit maximization of the entropy of predictions yield state-of-the-art calibration performances. We provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses. Specifically, these losses could be viewed as approximations of a linear penalty (or a Lagrangian) imposing equality constraints on logit distances. This points to an important limitation of such underlying equality constraints, whose ensuing gradients constantly push towards a non-informative solution, which might prevent from reaching the best compromise between the discriminative performance and calibration of the model during gradient-based optimization. Following our observations, we propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances. Comprehensive experiments on a variety of image classification, semantic segmentation and NLP benchmarks demonstrate that our method sets novel state-of-the-art results on these tasks in terms of network calibration, without affecting the discriminative performance. The code is available at https://github.com/by-liu/MbLS .
Robust and Provably Monotonic Networks
Venue: arXiv cs.LG 2023
Year: 2023

The Lipschitz constant of the map between the input and output space represented by a neural network is a natural metric for assessing the robustness of the model. We present a new method to constrain the Lipschitz constant of dense deep learning models that can also be generalized to other architectures. The method relies on a simple weight normalization scheme during training that ensures the Lipschitz constant of every layer is below an upper limit specified by the analyst. A simple monotonic residual connection can then be used to make the model monotonic in any subset of its inputs, which is useful in scenarios where domain knowledge dictates such dependence. Examples can be found in algorithmic fairness requirements or, as presented here, in the classification of the decays of subatomic particles produced at the CERN Large Hadron Collider. Our normalization is minimally constraining and allows the underlying architecture to maintain higher expressiveness compared to other techniques which aim to either control the Lipschitz constant of the model or ensure its monotonicity. We show how the algorithm was used to train a powerful, robust, and interpretable discriminator for heavy-flavor-quark decays, which has been adopted for use as the primary data-selection algorithm in the LHCb real-time data-processing system in the current LHC data-taking period known as Run 3. In addition, our algorithm has also achieved state-of-the-art performance on benchmarks in medicine, finance, and other applications.
Compiling Universal Probabilistic Programming Languages with Efficient Parallel Sequential Monte Carlo Inference
Venue: arXiv cs.PL 2023
Year: 2023

Probabilistic programming languages (PPLs) allow users to encode arbitrary inference problems, and PPL implementations provide general-purpose automatic inference for these problems. However, constructing inference implementations that are efficient enough is challenging for many real-world problems. Often, this is due to PPLs not fully exploiting available parallelization and optimization opportunities. For example, handling probabilistic checkpoints in PPLs through continuation-passing style transformations or non-preemptive multitasking -- as is done in many popular PPLs -- often disallows compilation to low-level languages required for high-performance platforms such as GPUs. To solve the checkpoint problem, we introduce the concept of PPL control-flow graphs (PCFGs) -- a simple and efficient approach to checkpoints in low-level languages. We use this approach to implement RootPPL: a low-level PPL built on CUDA and C++ with OpenMP, providing highly efficient and massively parallel SMC inference. We also introduce a general method of compiling universal high-level PPLs to PCFGs and illustrate its application when compiling Miking CorePPL -- a high-level universal PPL -- to RootPPL. The approach is the first to compile a universal PPL to GPUs with SMC inference. We evaluate RootPPL and the CorePPL compiler through a set of real-world experiments in the domains of phylogenetics and epidemiology, demonstrating up to 6x speedups over state-of-the-art PPLs implementing SMC inference.
Newton methods based convolution neural networks using parallel processing
Venue: arXiv cs.LG 2023
Year: 2023

Training of convolutional neural networks is a high dimensional and a non-convex optimization problem. At present, it is inefficient in situations where parametric learning rates can not be confidently set. Some past works have introduced Newton methods for training deep neural networks. Newton methods for convolutional neural networks involve complicated operations. Finding the Hessian matrix in second-order methods becomes very complex as we mainly use the finite differences method with the image data. Newton methods for convolutional neural networks deals with this by using the sub-sampled Hessian Newton methods. In this paper, we have used the complete data instead of the sub-sampled methods that only handle partial data at a time. Further, we have used parallel processing instead of serial processing in mini-batch computations. The results obtained using parallel processing in this study, outperform the time taken by the previous approach.
Axiomatic Analysis of Medial Centrality Measures
Venue: arXiv cs.SI 2023
Year: 2023

We perform the first axiomatic analysis of medial centrality measures. These measures, also called betweenness-like centralities, assess the role of a node in connecting others in the network. We focus on a setting with one target node and several source nodes. We consider three classic medial centrality measures adapted to this setting: Betweenness Centrality, Stress Centrality and Random Walk Betweenness Centrality. While Betweenness and Stress Centralities assume that the information in the network follows shortest paths, Random Walk Betweenness Centrality assumes it moves randomly along the edges. We develop the first axiomatic characterizations of all three measures. Our analysis shows that Random Walk Betweenness, while conceptually different, shares several common properties with classic Betweenness and Stress Centralities.
Tracking Patches for Open Source Software Vulnerabilities
Venue: arXiv cs.SE 2023
Year: 2023

Open source software (OSS) vulnerabilities threaten the security of software systems that use OSS. Vulnerability databases provide valuable information (e.g., vulnerable version and patch) to mitigate OSS vulnerabilities. There arises a growing concern about the information quality of vulnerability databases. However, it is unclear what the quality of patches in existing vulnerability databases is; and existing manual or heuristic-based approaches for patch tracking are either too expensive or too specific to apply to all OSS vulnerabilities.
CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection
Venue: arXiv cs.CV 2023
Year: 2023

Most of the existing bi-modal (RGB-D and RGB-T) salient object detection methods utilize the convolution operation and construct complex interweave fusion structures to achieve cross-modal information integration. The inherent local connectivity of the convolution operation constrains the performance of the convolution-based methods to a ceiling. In this work, we rethink these tasks from the perspective of global information alignment and transformation. Specifically, the proposed \underline{c}ross-mod\underline{a}l \underline{v}iew-mixed transform\underline{er} (CAVER) cascades several cross-modal integration units to construct a top-down transformer-based information propagation path. CAVER treats the multi-scale and multi-modal feature integration as a sequence-to-sequence context propagation and update process built on a novel view-mixed attention mechanism. Besides, considering the quadratic complexity w.r.t. the number of input tokens, we design a parameter-free patch-wise token re-embedding strategy to simplify operations. Extensive experimental results on RGB-D and RGB-T SOD datasets demonstrate that such a simple two-stream encoder-decoder framework can surpass recent state-of-the-art methods when it is equipped with the proposed components. Code and pretrained models will be available at \href{https://github.com/lartpang/CAVER}{the link}.
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone
Venue: arXiv cs.SD 2023
Year: 2023

YourTTS brings the power of a multilingual approach to the task of zero-shot multi-speaker TTS. Our method builds upon the VITS model and adds several novel modifications for zero-shot multi-speaker and multilingual training. We achieved state-of-the-art (SOTA) results in zero-shot multi-speaker TTS and results comparable to SOTA in zero-shot voice conversion on the VCTK dataset. Additionally, our approach achieves promising results in a target language with a single-speaker dataset, opening possibilities for zero-shot multi-speaker TTS and zero-shot voice conversion systems in low-resource languages. Finally, it is possible to fine-tune the YourTTS model with less than 1 minute of speech and achieve state-of-the-art results in voice similarity and with reasonable quality. This is important to allow synthesis for speakers with a very different voice or recording characteristics from those seen during training.