OpenRaiser/Intern-Atlas huggingface dataset:
Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight
Venue: arXiv cs.CV 2023
Year: 2023

This paper strives to recognize activities in the dark, as well as in the day. We first establish that state-of-the-art activity recognizers are effective during the day, but not trustworthy in the dark. The main causes are the limited availability of labeled dark videos to learn from, as well as the distribution shift towards the lower color contrast at test-time. To compensate for the lack of labeled dark videos, we introduce a pseudo-supervised learning scheme, which utilizes easy to obtain unlabeled and task-irrelevant dark videos to improve an activity recognizer in low light. As the lower color contrast results in visual information loss, we further propose to incorporate the complementary activity information within audio, which is invariant to illumination. Since the usefulness of audio and visual features differs depending on the amount of illumination, we introduce our `darkness-adaptive' audio-visual recognizer. Experiments on EPIC-Kitchens, Kinetics-Sound, and Charades demonstrate our proposals are superior to image enhancement, domain adaptation and alternative audio-visual fusion methods, and can even improve robustness to local darkness caused by occlusions. Project page: https://xiaobai1217.github.io/Day2Dark/
Graph Convolutional Neural Networks with Diverse Negative Samples via Decomposed Determinant Point Processes
Venue: arXiv cs.LG 2023
Year: 2023

Graph convolutional networks (GCNs) have achieved great success in graph representation learning by extracting high-level features from nodes and their topology. Since GCNs generally follow a message-passing mechanism, each node aggregates information from its first-order neighbour to update its representation. As a result, the representations of nodes with edges between them should be positively correlated and thus can be considered positive samples. However, there are more non-neighbour nodes in the whole graph, which provide diverse and useful information for the representation update. Two non-adjacent nodes usually have different representations, which can be seen as negative samples. Besides the node representations, the structural information of the graph is also crucial for learning. In this paper, we used quality-diversity decomposition in determinant point processes (DPP) to obtain diverse negative samples. When defining a distribution on diverse subsets of all non-neighbouring nodes, we incorporate both graph structure information and node representations. Since the DPP sampling process requires matrix eigenvalue decomposition, we propose a new shortest-path-base method to improve computational efficiency. Finally, we incorporate the obtained negative samples into the graph convolution operation. The ideas are evaluated empirically in experiments on node classification tasks. These experiments show that the newly proposed methods not only improve the overall performance of standard representation learning but also significantly alleviate over-smoothing problems.
Review of medical data analysis based on spiking neural networks
Venue: arXiv cs.NE 2023
Year: 2023

Medical data mainly includes various types of biomedical signals and medical images, which can be used by professional doctors to make judgments on patients' health conditions. However, the interpretation of medical data requires a lot of human cost and there may be misjudgments, so many scholars use neural networks and deep learning to classify and study medical data, which can improve the efficiency and accuracy of doctors and detect diseases early for early diagnosis, etc. Therefore, it has a wide range of application prospects. However, traditional neural networks have disadvantages such as high energy consumption and high latency (slow computation speed). This paper presents recent research on signal classification and disease diagnosis based on a third-generation neural network, the spiking neuron network, using medical data including EEG signals, ECG signals, EMG signals and MRI images. The advantages and disadvantages of pulsed neural networks compared with traditional networks are summarized and its development orientation in the future is prospected.
Bagging is an Optimal PAC Learner
Venue: arXiv cs.LG 2023
Year: 2023

Determining the optimal sample complexity of PAC learning in the realizable setting was a central open problem in learning theory for decades. Finally, the seminal work by Hanneke (2016) gave an algorithm with a provably optimal sample complexity. His algorithm is based on a careful and structured sub-sampling of the training data and then returning a majority vote among hypotheses trained on each of the sub-samples. While being a very exciting theoretical result, it has not had much impact in practice, in part due to inefficiency, since it constructs a polynomial number of sub-samples of the training data, each of linear size. In this work, we prove the surprising result that the practical and classic heuristic bagging (a.k.a. bootstrap aggregation), due to Breiman (1996), is in fact also an optimal PAC learner. Bagging pre-dates Hanneke's algorithm by twenty years and is taught in most undergraduate machine learning courses. Moreover, we show that it only requires a logarithmic number of sub-samples to reach optimality.
Generalization of Higher Order Methods for Fast Iterative Matrix Inversion Suitable for GPU Acceleration
Venue: arXiv cs.OH 2023
Year: 2023

Recent technological developments have led to big data processing, which resulted in significant computational difficulties when solving large-scale linear systems or inverting matrices. As a result, fast approximate iterative matrix inversion methodologies via Graphical Processing Unit (GPU) acceleration has been a subject of extensive research, to find solutions where classic and direct inversion are too expensive to conduct. Some currently used methods are Neumann Series (NS), Newton iteration (NI), Chebyshev Iteration (CI), and Successive Over-Relaxation, to cite a few. In this work, we develop a new iterative algorithm based off the NS, which we named 'Nested Neumann' (NN). This new methodology generalizes higher orders of the NI (or CI), by taking advantage of a computationally free iterative update of the preconditioning matrix as a function of a given 'inception depth'. It has been mathematically demonstrated that the NN: (i) convergences given the preconditioning satisfies the spectral norm condition of the NS, (ii) has an order of rate of convergence has been shown to be equivalent to the order (inception depth plus one), and (iii) has an optimal inception depth is an inception depth of one or preferably two, depending on RAM constraints. Furthermore, we derive an explicit formula for the NN, which is applicable to massive sparse matrices, given an increase in computational cost. Importantly, the NN finds an analytic equivalancy statement between the NS and the the NN (NI, CI, and higher orders), which is of importance for mMIMO systems. Finally, the NN method is applicable positive semi-definite matrices for matrix inversion, and applicable to any linear system (sparse, non-sparse, complex, etc.).
The Multi-cluster Fluctuating Two-Ray Fading Model
Venue: arXiv cs.IT 2023
Year: 2023

We introduce a new class of fading channels, built as the superposition of two fluctuating specular components with random phases, plus a clustering of scattered waves: the Multi-cluster Fluctuating Two-Ray (MFTR) fading channel. The MFTR model emerges as a natural generalization of both the fluctuating two-ray (FTR) and the $κ$-$μ$ shadowed fading models through a more general yet equally mathematically tractable model. This generalization enables the presence of additional multipath clusters in the purely ray-based FTR model, and the convenience of the new underlying fading channel model is discussed in depth. Then, we derive all the chief probability functions of the MFTR model (e.g., probability density function (PDF), cumulative density function (CDF), and moment generation function) in closed-form, having {a mathematical complexity similar to} other fading models in the state-of-the-art. We also provide two additional analytical formulations for the PDF and the CDF: (i) in terms of a continuous mixture of $κ$-$μ$ shadowed distributions, and (ii) as an infinite discrete mixture of Gamma distributions. Such expressions enable to conduct performance analysis under MFTR fading by directly leveraging readily available results for the $κ$-$μ$ shadowed or Nakagami-$m$ cases, respectively. The performance of wireless communications systems undergoing MFTR fading is exemplified in terms of a classical benchmarking metric like the outage probability, both in exact and asymptotic forms, and the amount of fading.
Collabs: A Flexible and Performant CRDT Collaboration Framework
Venue: arXiv cs.DC 2023
Year: 2023

A collaboration framework is a distributed system that serves as the data layer for a collaborative app. Conflict-free Replicated Data Types (CRDTs) are a promising theoretical technique for implementing collaboration frameworks. However, existing frameworks are inflexible: they are often one-off implementations of research papers or only permit a restricted set of CRDT semantics, and they do not allow app-specific optimizations. Until now, there was no general framework that lets programmers mix, match, and modify CRDTs. We solve this with Collabs, a CRDT-based collaboration framework that lets programmers implement their own CRDTs, either from-scratch or by composing existing building blocks. Collabs prioritizes both semantic flexibility and performance flexibility: it allows arbitrary app-specific CRDT behaviors and optimizations, while still providing strong eventual consistency. We demonstrate Collabs's capabilities and programming model with example apps and CRDT implementations. We then show that a collaborative rich-text editor using Collabs's built-in CRDTs can scale to over 100 simultaneous users, unlike existing CRDT frameworks and Google Docs. Collabs also has lower end-to-end latency and server CPU usage than a popular Operational Transformation framework, with acceptable CRDT metadata overhead.
A Generic Methodology for the Modular Verification of Security Protocol Implementations (extended version)
Venue: arXiv cs.CR 2023
Year: 2023

Security protocols are essential building blocks of modern IT systems. Subtle flaws in their design or implementation may compromise the security of entire systems. It is, thus, important to prove the absence of such flaws through formal verification. Much existing work focuses on the verification of protocol *models*, which is not sufficient to show that their *implementations* are actually secure. Verification techniques for protocol implementations (e.g., via code generation or model extraction) typically impose severe restrictions on the used programming language and code design, which may lead to sub-optimal implementations. In this paper, we present a methodology for the modular verification of strong security properties directly on the level of the protocol implementations. Our methodology leverages state-of-the-art verification logics and tools to support a wide range of implementations and programming languages. We demonstrate its effectiveness by verifying memory safety and security of Go implementations of the Needham-Schroeder-Lowe, Diffie-Hellman key exchange, and WireGuard protocols, including forward secrecy and injective agreement for WireGuard. We also show that our methodology is agnostic to a particular language or program verifier with a prototype implementation for C.
Benchmarking AutoML algorithms on a collection of synthetic classification problems
Venue: arXiv cs.LG 2023
Year: 2023

Automated machine learning (AutoML) algorithms have grown in popularity due to their high performance and flexibility to adapt to different problems and data sets. With the increasing number of AutoML algorithms, deciding which would best suit a given problem becomes increasingly more work. Therefore, it is essential to use complex and challenging benchmarks which would be able to differentiate the AutoML algorithms from each other. This paper compares the performance of four different AutoML algorithms: Tree-based Pipeline Optimization Tool (TPOT), Auto-Sklearn, Auto-Sklearn 2, and H2O AutoML. We use the Diverse and Generative ML benchmark (DIGEN), a diverse set of synthetic datasets derived from generative functions designed to highlight the strengths and weaknesses of the performance of common machine learning algorithms. We confirm that AutoML can identify pipelines that perform well on all included datasets. Most AutoML algorithms performed similarly; however, there were some differences depending on the specific dataset and metric used.
Improved Beam Search for Hallucination Mitigation in Abstractive Summarization
Venue: arXiv cs.CL 2023
Year: 2023

Advancement in large pretrained language models has significantly improved their performance for conditional language generation tasks including summarization albeit with hallucinations. To reduce hallucinations, conventional methods proposed improving beam search or using a fact checker as a postprocessing step. In this paper, we investigate the use of the Natural Language Inference (NLI) entailment metric to detect and prevent hallucinations in summary generation. We propose an NLI-assisted beam re-ranking mechanism by computing entailment probability scores between the input context and summarization model-generated beams during saliency-enhanced greedy decoding. Moreover, a diversity metric is introduced to compare its effectiveness against vanilla beam search. Our proposed algorithm significantly outperforms vanilla beam decoding on XSum and CNN/DM datasets.
QEBVerif: Quantization Error Bound Verification of Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

To alleviate the practical constraints for deploying deep neural networks (DNNs) on edge devices, quantization is widely regarded as one promising technique. It reduces the resource requirements for computational power and storage space by quantizing the weights and/or activation tensors of a DNN into lower bit-width fixed-point numbers, resulting in quantized neural networks (QNNs). While it has been empirically shown to introduce minor accuracy loss, critical verified properties of a DNN might become invalid once quantized. Existing verification methods focus on either individual neural networks (DNNs or QNNs) or quantization error bound for partial quantization. In this work, we propose a quantization error bound verification method, named QEBVerif, where both weights and activation tensors are quantized. QEBVerif consists of two parts, i.e., a differential reachability analysis (DRA) and a mixed-integer linear programming (MILP) based verification method. DRA performs difference analysis between the DNN and its quantized counterpart layer-by-layer to compute a tight quantization error interval efficiently. If DRA fails to prove the error bound, then we encode the verification problem into an equivalent MILP problem which can be solved by off-the-shelf solvers. Thus, QEBVerif is sound, complete, and reasonably efficient. We implement QEBVerif and conduct extensive experiments, showing its effectiveness and efficiency.
An advanced YOLOv3 method for small object detection
Venue: arXiv cs.CV 2023
Year: 2023

Small object detection has important application value in the fields of autonomous driving and drone scene analysis. As one of the most advanced object detection algorithms, YOLOv3 suffers some challenges when detecting small objects, such as the problem of detection failure of small objects and occluded objects. To solve these problems, an improved YOLOv3 algorithm for small object detection is proposed. In the proposed method, the dilated convolutions mish (DCM) module is introduced into the backbone network of YOLOv3 to improve the feature expression ability by fusing the feature maps of different receptive fields. In the neck network of YOLOv3, the convolutional block attention module (CBAM) and multi-level fusion module are introduced to select the important information for small object detection in the shallow network, suppress the uncritical information, and use the fusion module to fuse the feature maps of different scales, so as to improve the detection accuracy of the algorithm. In addition, the Soft-NMS and Complete-IoU (CloU) strategies are applied to candidate frame screening, which improves the accuracy of the algorithm for the detection of occluded objects. The ablation experiment of the MS COCO2017 object detection task proves the effectiveness of several modules introduced in this paper for small object detection. The experimental results on the MS COCO2017, VOC2007, and VOC2012 datasets show that the Average Precision (AP) of this method is 16.5%, 8.71%, and 9.68% higher than that of YOLOv3, respectively.
Sparsity-Dimension Trade-Offs for Oblivious Subspace Embeddings
Venue: arXiv cs.DS 2023
Year: 2023

An oblivious subspace embedding (OSE), characterized by parameters $m,n,d,ε,δ$, is a random matrix $Π\in \mathbb{R}^{m\times n}$ such that for any $d$-dimensional subspace $T\subseteq \mathbb{R}^n$, $\Pr_Π[\forall x\in T, (1-ε)\|x\|_2 \leq \|Πx\|_2\leq (1+ε)\|x\|_2] \geq 1-δ$. When an OSE has $s\le 1/2.001ε$ nonzero entries in each column, we show it must hold that $m = Ω\left(d^2/( ε^2s^{1+O(δ)})\right)$, which is the first lower bound with multiplicative factors of $d^2$ and $1/ε$, improving on the previous $Ω\left(d^2/s^{O(δ)}\right)$ lower bound due to Li and Liu (PODS 2022). When an OSE has $s=Ω(\log(1/ε)/ε)$ nonzero entries in each column, we show it must hold that $m = Ω\left((d/ε)^{1+1/4.001εs}/s^{O(δ)}\right)$, which is the first lower bound with multiplicative factors of $d$ and $1/ε$, improving on the previous $Ω\left(d^{1+1/(16εs+4)}\right)$ lower bound due to Nelson and Nguyen (ICALP 2014). This second result is a special case of a more general trade-off among $d,ε,s,δ$ and $m$.
Active Classification of Moving Targets with Learned Control Policies
Venue: arXiv cs.RO 2023
Year: 2023

In this paper, we consider the problem where a drone has to collect semantic information to classify multiple moving targets. In particular, we address the challenge of computing control inputs that move the drone to informative viewpoints, position and orientation, when the information is extracted using a "black-box" classifier, e.g., a deep learning neural network. These algorithms typically lack of analytical relationships between the viewpoints and their associated outputs, preventing their use in information-gathering schemes. To fill this gap, we propose a novel attention-based architecture, trained via Reinforcement Learning (RL), that outputs the next viewpoint for the drone favoring the acquisition of evidence from as many unclassified targets as possible while reasoning about their movement, orientation, and occlusions. Then, we use a low-level MPC controller to move the drone to the desired viewpoint taking into account its actual dynamics. We show that our approach not only outperforms a variety of baselines but also generalizes to scenarios unseen during training. Additionally, we show that the network scales to large numbers of targets and generalizes well to different movement dynamics of the targets.
Fresh-CSMA: A Distributed Protocol for Minimizing Age of Information
Venue: arXiv cs.NI 2023
Year: 2023

We consider the design of distributed scheduling algorithms that minimize age of information in single-hop wireless networks. The centralized max-weight policy is known to be nearly optimal in this setting; hence, our goal is to design a distributed CSMA scheme that can mimic its performance. To that end, we propose a distributed protocol called Fresh-CSMA and show that in an idealized setting, Fresh-CSMA can match the scheduling decisions of the max-weight policy with high probability in each frame, and also match the theoretical performance guarantees of the max-weight policy over the entire time horizon. We then consider a more realistic setting and study the impact of protocol parameters on the probability of collisions and the overhead caused by the distributed nature of the protocol. We also consider the monitoring of Markov sources and extend our approach to CSMA protocols that incorporate Age of Incorrect Information (AoII) instead of AoI. Finally, we provide simulations that support our theoretical results and show that the performance gap between the ideal and realistic versions of Fresh-CSMA is small.
DiffTune$^+$: Hyperparameter-Free Auto-Tuning using Auto-Differentiation
Venue: arXiv cs.RO 2023
Year: 2023

Controller tuning is a vital step to ensure the controller delivers its designed performance. DiffTune has been proposed as an automatic tuning method that unrolls the dynamical system and controller into a computational graph and uses auto-differentiation to obtain the gradient for the controller's parameter update. However, DiffTune uses the vanilla gradient descent to iteratively update the parameter, in which the performance largely depends on the choice of the learning rate (as a hyperparameter). In this paper, we propose to use hyperparameter-free methods to update the controller parameters. We find the optimal parameter update by maximizing the loss reduction, where a predicted loss based on the approximated state and control is used for the maximization. Two methods are proposed to optimally update the parameters and are compared with related variants in simulations on a Dubin's car and a quadrotor. Simulation experiments show that the proposed first-order method outperforms the hyperparameter-based methods and is more robust than the second-order hyperparameter-free methods.
PØDA: Prompt-driven Zero-shot Domain Adaptation
Venue: arXiv cs.CV 2023
Year: 2023

Domain adaptation has been vastly investigated in computer vision but still requires access to target images at train time, which might be intractable in some uncommon conditions. In this paper, we propose the task of `Prompt-driven Zero-shot Domain Adaptation', where we adapt a model trained on a source domain using only a general description in natural language of the target domain, i.e., a prompt. First, we leverage a pretrained contrastive vision-language model (CLIP) to optimize affine transformations of source features, steering them towards the target text embedding while preserving their content and semantics. To achieve this, we propose Prompt-driven Instance Normalization (PIN). Second, we show that these prompt-driven augmentations can be used to perform zero-shot domain adaptation for semantic segmentation. Experiments demonstrate that our method significantly outperforms CLIP-based style transfer baselines on several datasets for the downstream task at hand, even surpassing one-shot unsupervised domain adaptation. A similar boost is observed on object detection and image classification. The code is available at https://github.com/astra-vision/PODA .
Cube Interestingness: Novelty, Relevance, Peculiarity and Surprise
Venue: arXiv cs.DB 2023
Year: 2023

In this paper, we discuss methods to assess the interestingness of a query in an environment of data cubes. We assume a hierarchical multidimensional database, storing data cubes and level hierarchies. We start with a comprehensive review of related work in the fields of studies of human behavior and computer science. We define the interestingness of a query as a vector of scores along difference dimensions, like novelty, relevance, surprise and peculiarity and complement this definition with a taxonomy of the information that can be used to assess each of these dimensions of interestingness. We provide both syntactic (result-independent) checks and extensional (result-dependent) measures and algorithms for assessing the different dimensions of interestingness in a quantitative fashion. We also report our findings on a user study that we conducted, analyzing the significance of each dimension, its evolution over time and the behavior of the study's participants.