Miracle-dz/UniFAR-FTUs huggingface dataset:
Truth-Table Net: A New Convolutional Architecture Encodable By Design Into SAT Formulas

With the expanding role of neural networks, the need for complete and sound veriﬁcation of their property has become critical. In the recent years, it was established that Binary Neural Networks (BNNs) have an equivalent representation in Boolean logic and can be formally analyzed using logical reasoning tools such as SAT solvers. However, to date, only BNNs can be transformed into a SAT formula. In this work, we introduce Truth Table Deep Convolutional Neural Networks (TTnets), a new family of SAT-encodable models featuring for the ﬁrst time real-valued weights. Furthermore, it admits, by construction, some valuable conversion features including post-tuning and tractability in the robustness veri-ﬁcation setting. The latter property leads to a more compact SAT symbolic encoding than BNNs. This enables the use of general SAT solvers, making property veriﬁcation easier. We demonstrate the value of TTnets regarding the formal robustness property: TTnets outperform the veriﬁed accuracy of all BNNs with a comparable computation time. More generally, they represent a relevant trade-off between all known complete veriﬁcation methods: TTnets achieve high veriﬁed accuracy with fast veriﬁcation time, being complete with no timeouts. We are exploring here a proof of concept of TTnets for a very important application (complete veriﬁcation of robustness) and we believe this novel real-valued network constitutes a practical response to the rising need for functional formal veriﬁcation. We postulate that TTnets can apply to various CNN-based architectures and be extended to other properties such as fairness, fault attack and exact rule extraction.
Vector Spaces of Generalized Euler Integrals

We study vector spaces associated to a family of generalized Euler integrals. Their dimension is given by the Euler characteristic of a very affine variety. Motivated by Feynman integrals from particle physics, this has been investigated using tools from homological algebra and the theory of $D$-modules. We present an overview and uncover new relations between these approaches. We also provide new algorithmic tools.
Enhanced wave localization in multifractal scattering media

In this paper we study the structural, scattering, and wave localization properties of multifractal arrays of electric point dipoles generated from multiplicative random fields with different degrees of multiscale correlations. Specifically, using the rigorous Green's matrix method, we investigate the scattering resonances and wave localization behavior of systems with $N=10^{4}$ dipoles and demonstrate an enhanced localization behavior in highly inhomogeneous multifractal structures compared to homogeneous fractals, or monofractals. We show distinctive spectral properties, such as the absence of level repulsion in the strong multiple scattering regime and power-law statistics of level spacings, which indicate a clear localization transition enhanced in non-homogeneous multifractals. Our findings unveil the importance of multifractal structural correlations in the multiple scattering regime of electric dipole arrays and provide an efficient model for the design of multiscale nanophotonic systems with enhanced light-matter coupling and localization phenomena beyond what is possible with traditional fractal systems.
Bottom Backscattering Strengths Measured in Shallow and Deep Water

Abstract : For a low-frequency (LF; 1 kHz) or mid-frequency (MF; 1 to 10 kHz) active sonar, scattering from the seafloor, coupled with propagationconditions, can severely limit the detectability of returns from features of interest. Acoustic scattering from the seabed can be a complex mix ofsurface roughness and volume heterogeneity contributions, so that reverberation levels can vary dramatically, depending on the local geology.The Naval Research Laboratory (NRL) performed LF and MF, direct-path bottom backscattering strength (BBS) measurements at 105 sites over6 experiments in 5 distinct environments from 1993 to 2005. This report presents the BBS results from these experiments, as well as empiricalfits to the MF results.
Learning Spatial-Frequency Transformer for Visual Object Tracking

Recently, some researchers have begun to adopt the Transformer to combine or replace the widely used ResNet as their new backbone network. As the Transformer captures the long-range relations between pixels well using the self-attention scheme, which complements the issues caused by the limited receptive field of CNN. Although their trackers work well in regular scenarios, they simply flatten the 2D features into a sequence to better match the Transformer. We believe these operations ignore the spatial prior of the target object, which may lead to sub-optimal results only. In addition, many works demonstrate that self-attention is actually a low-pass filter, which is independent of input features or keys/queries. That is to say, it may suppress the high-frequency component of the input features and preserve or even amplify the low-frequency information. To handle these issues, in this paper, we propose a unified Spatial-Frequency Transformer that models the Gaussian spatial Prior and High-frequency emphasis Attention (GPHA) simultaneously. To be specific, Gaussian spatial prior is generated using dual Multi-Layer Perceptrons (MLPs) and injected into the similarity matrix produced by multiplying Query and Key features in self-attention. The output will be fed into a softmax layer and then decomposed into two components, i.e., the direct and high-frequency signal. The low- and high-pass branches are rescaled and combined to achieve all-pass, therefore, the high-frequency features will be protected well in stacked self-attention layers. We further integrate the Spatial-Frequency Transformer into the Siamese tracking framework and propose a novel tracking algorithm termed SFTransT. The cross-scale fusion based SwinTransformer is adopted as the backbone, and also a multi-head cross-attention module is used to boost the interaction between search and template features. The output will be fed into the tracking head for target localization. Extensive experiments on short-term and long-term tracking benchmarks all demonstrate the effectiveness of our proposed framework. Source code will be released at https://github.com/Tchuanm/SFTransT.git.
Open Information Extraction from 2007 to 2022 - A Survey

Open information extraction is an important NLP task that targets extracting structured information from unstructured text without limitations on the relation type or the domain of the text. This survey paper covers open information extraction technologies from 2007 to 2022 with a focus on new models not covered by previous surveys. We propose a new categorization method from the source of information perspective to accommodate the development of recent OIE technologies. In addition, we summarize three major approaches based on task settings as well as current popular datasets and model evaluation metrics. Given the comprehensive review, several future directions are shown from datasets, source of information, output form, method, and evaluation metric aspects.
Unifying Visual Perception by Dispersible Points Learning

We present a conceptually simple, flexible, and universal visual perception head for variant visual tasks, e.g., classification, object detection, instance segmentation and pose estimation, and different frameworks, such as one-stage or two-stage pipelines. Our approach effectively identifies an object in an image while simultaneously generating a high-quality bounding box or contour-based segmentation mask or set of keypoints. The method, called UniHead, views different visual perception tasks as the dispersible points learning via the transformer encoder architecture. Given a fixed spatial coordinate, UniHead adaptively scatters it to different spatial points and reasons about their relations by transformer encoder. It directly outputs the final set of predictions in the form of multiple points, allowing us to perform different visual tasks in different frameworks with the same head design. We show extensive evaluations on ImageNet classification and all three tracks of the COCO suite of challenges, including object detection, instance segmentation and pose estimation. Without bells and whistles, UniHead can unify these visual tasks via a single visual head design and achieve comparable performance compared to expert models developed for each task.We hope our simple and universal UniHead will serve as a solid baseline and help promote universal visual perception research. Code and models are available at https://github.com/Sense-X/UniHead.
Rindler space derivation of dark energy

We present a miraculously short derivation of the missing dark energy density of the universe which is in absolute agreement with the most recent accurate cosmological measurements and observations. The derivation is based upon a Rindler space setting, the associated wedge horizon and Unruh temperature. That way the topological ordinary energy is found to be half of the topological Unruh fluctuation mass m(O) = φ3 multiplied with the square of the topological speed of light c2 = φ2 where = φ = 2 /( 5 + 1). This is exactly equal to the area of the spear-like hyperbolic triangular part of the Rindler wedge. The corresponding physical ordinary energy density is thus E(O) = (1/2) ( φ3)( φ2) mc2 = (φ5/2)( mc2), where φ5 is Hardy’s probability of quantum entanglement. The topological dark energy density on the other hand is equal half of the topological Kaluza-Klein five dimensional mass m(D) = 5 multiplied with c2 = φ2. This in turn is exactly equal to the circular segment part of the wedge which together with the hyperbolic triangular entangled area forms the complete Lorentzian invariant triangular area of the wedge. Consequently the physical dark energy density which is uncorrelated is given by E(D) = (1/2) (5)( φ2)( mc2) = (5 φ2 /2)( mc2) in full agreement with observation. Adding E(O) and E(D) one finds E(Einstein) = mc2.
Quickest Detection for Human-Sensor Systems Using Quantum Decision Theory

A sensor observes an underlying state of nature in noise, computes a posterior probability of this state, and provides this posterior as a recommendation to a human decision-maker. The human then makes decisions based on this recommendation. By recording these human decisions over time, how can a quickest detector detect a change in the underlying state? This paper addresses the above human-sensor interface problem. This framework generalizes classical quickest detection since the detector only has access to human decisions rather than noisy signals with known statistical properties. We utilize a Quantum Decision Theory model (widely studied in mathematical psychology) to capture peculiarities of human decision making. We provide theoretical guarantees for the quickest detector's optimal policy, such as a threshold structure, a bound on performance, robustness to variations in the decision-model parameters, and increased performance under ‘rational’ human decisions. Finally, we illustrate a numerical implementation of this quickest detector in the context of the Prisoner's Dilemma problem, in which it has been observed that Quantum Decision Theory can uniquely model empirically tested violations of the sure-thing principle.
Mere Contrastive Learning for Cross-Domain Sentiment Analysis

Cross-domain sentiment analysis aims to predict the sentiment of texts in the target domain using the model trained on the source domain to cope with the scarcity of labeled data. Previous studies are mostly cross-entropy-based methods for the task, which suffer from instability and poor generalization. In this paper, we explore contrastive learning on the cross-domain sentiment analysis task. We propose a modified contrastive objective with in-batch negative samples so that the sentence representations from the same class can be pushed close while those from the different classes become further apart in the latent space. Experiments on two widely used datasets show that our model can achieve state-of-the-art performance in both cross-domain and multi-domain sentiment analysis tasks. Meanwhile, visualizations demonstrate the effectiveness of transferring knowledge learned in the source domain to the target domain and the adversarial test verifies the robustness of our model.
Deep Neural Network Approximation of Invariant Functions through Dynamical Systems

We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions includes the much studied translation-invariant ones involving image tasks, but also encompasses many permutation-invariant functions that finds emerging applications in science and engineering. We prove sufficient conditions for universal approximation of these functions by a controlled equivariant dynamical system, which can be viewed as a general abstraction of deep residual networks with symmetry constraints. These results not only imply the universal approximation for a variety of commonly employed neural network architectures for symmetric function approximation, but also guide the design of architectures with approximation guarantees for applications involving new symmetry requirements.
ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization

We present a novel semi-supervised learning framework that intelligently leverages the consistency regularization between the model's predictions from two strongly-augmented views of an image, weighted by a confidence of pseudo-label, dubbed ConMatch. While the latest semi-supervised learning methods use weakly- and strongly-augmented views of an image to define a directional consistency loss, how to define such direction for the consistency regularization between two strongly-augmented views remains unexplored. To account for this, we present novel confidence measures for pseudo-labels from strongly-augmented views by means of weakly-augmented view as an anchor in non-parametric and parametric approaches. Especially, in parametric approach, we present, for the first time, to learn the confidence of pseudo-label within the networks, which is learned with backbone model in an end-to-end manner. In addition, we also present a stage-wise training to boost the convergence of training. When incorporated in existing semi-supervised learners, ConMatch consistently boosts the performance. We conduct experiments to demonstrate the effectiveness of our ConMatch over the latest methods and provide extensive ablation studies. Code has been made publicly available at https://github.com/JiwonCocoder/ConMatch.
LogKernel: A Threat Hunting Approach Based on Behaviour Provenance Graph and Graph Kernel Clustering

Cyber threat hunting is a proactive search process for hidden threats in an organization’s information system. It is a crucial component of active defense against advanced persistent threats (APTs). However, most of the current threat hunting methods rely on Cyber Threat Intelligence (CTI), which can find known attacks but cannot find unknown attacks that have not been disclosed by CTI. In this paper, we propose LogKernel, a threat hunting method based on graph kernel clustering which can effectively separate attack behaviour from benign activities. LogKernel first abstracts system audit logs into behaviour provenance graphs (BPGs) and then clusters graphs by embedding them into a continuous space using a graph kernel. In particular, we designed a new graph kernel clustering method based on the characteristics of BPGs, which can capture both structure information and rich label information of the BPGs. To reduce false positives, LogKernel further quantifies the threat of abnormal behaviour. We evaluate LogKernel on the malicious dataset, which includes seven simulated attack scenarios, and the DAPRA CADETS dataset, which includes four attack scenarios. The result shows that LogKernel can hunt all attack scenarios among them, and compared to the state-of-the-art methods, it can find unknown attacks.
Quality issues in Machine Learning Software Systems

Context: An increasing demand is observed in various domains to employ Machine Learning (ML) for solving complex problems. ML models are implemented as software components and deployed in Machine Learning Software Systems (MLSSs). Problem: There is a strong need for ensuring the serving quality of MLSSs. False or poor decisions of such systems can lead to malfunction of other systems, significant financial losses, or even threat to human life. The quality assurance of MLSSs is considered as a challenging task and currently is a hot research topic. Moreover, it is important to cover all various aspects of the quality in MLSSs. Objective: This paper aims to investigate the characteristics of real quality issues in MLSSs from the viewpoint of practitioners. This empirical study aims to identify a catalog of bad-practices related to poor quality in MLSSs. Method: We plan to conduct a set of interviews with practitioners/experts, believing that interviews are the best method to retrieve their experience and practices when dealing with quality issues. We expect that the catalog of issues developed at this step will also help us later to identify the severity, root causes, and possible remedy for quality issues of MLSSs, allowing us to develop efficient quality assurance tools for ML models and MLSSs.
High Dimensional Statistical Estimation Under Uniformly Dithered One-Bit Quantization

In this paper, we propose a uniformly dithered 1-bit quantization scheme for high-dimensional statistical estimation. The scheme contains truncation, dithering, and quantization as typical steps. As canonical examples, the quantization scheme is applied to the estimation problems of sparse covariance matrix estimation, sparse linear regression (i.e., compressed sensing), and matrix completion. We study both sub-Gaussian and heavy-tailed regimes, where the underlying distribution of heavy-tailed data is assumed to have bounded moments of some order. We propose new estimators based on 1-bit quantized data. In sub-Gaussian regime, our estimators achieve minimax rates up to logarithmic factors, indicating that our quantization scheme costs very little. In heavy-tailed regime, while the rates of our estimators become essentially slower, these results are either the first ones in an 1-bit quantized and heavy-tailed setting, or already improve on existing comparable results from some respect. Under the observations in our setting, the rates are almost tight in compressed sensing and matrix completion. Our 1-bit compressed sensing results feature general sensing vector that is sub-Gaussian or even heavy-tailed. We also first investigate a novel setting where both the covariate and response are quantized. In addition, our approach to 1-bit matrix completion does not rely on likelihood and represents the first method robust to pre-quantization noise with unknown distribution. Experimental results on synthetic data are presented to support our theoretical analysis.
Mechanistic framework for deriving reduced-order models in soft materials: Application to granular intrusion

Soft materials often display complex behaviors that transition through apparent solid- and ﬂuid-like regimes. While a growing number of microscale simulation methods exist for these materials, reduced-order models that encapsulate the global-scale physics are often desired to predict how external bodies interact with soft media, as occurs in diverse situations from impact and penetration problems to locomotion over natural terrains. This work proposes a sys-tematic program to develop three-dimensional reduced-order models for soft materials from a fundamental basis using continuum symmetries and rheological principles. In particular, we derive a reduced-order technique for modeling intrusion in granular media which we term three-dimensional Resistive Force Theory (3D-RFT), which is capable of accurately and quickly predicting the resistive stress distribution on arbitrary-shaped intruding bodies. Aided by a continuum description of the granular medium, a comprehensive set of spatial symmetry constraints, and a limited amount of reference data, we develop a self-consistent and accurate 3D-RFT. We verify the model capabilities in a wide range of cases and show it can be quickly recal-ibrated to different media and intruder surface types. The premises leading to 3D-RFT anticipate application to other soft materials with strongly hyperlocalized intrusion behavior.
KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems

The progress of recommender systems is hampered mainly by evaluation as it requires real-time interactions between humans and systems, which is too laborious and expensive. This issue is usually approached by utilizing the interaction history to conduct offline evaluation. However, existing datasets of user-item interactions are partially observed, leaving it unclear how and to what extent the missing interactions will influence the evaluation. To answer this question, we collect a fully-observed dataset from Kuaishou's online environment, where almost all 1,411 users have been exposed to all 3,327 items. To the best of our knowledge, this is the first real-world fully-observed data with millions of user-item interactions. With this unique dataset, we conduct a preliminary analysis of how the two factors - data density and exposure bias - affect the evaluation results of multi-round conversational recommendation. Our main discoveries are that the performance ranking of different methods varies with the two factors, and this effect can only be alleviated in certain cases by estimating missing interactions for user simulation. This demonstrates the necessity of the fully-observed dataset. We release the dataset and the pipeline implementation for evaluation at https://kuairec.com
How Many Perturbations Break This Model? Evaluating Robustness Beyond Adversarial Accuracy

Robustness to adversarial attacks is typically evaluated with adversarial accuracy. While essential, this metric does not capture all aspects of robustness and in particular leaves out the question of how many perturbations can be found for each point. In this work, we introduce an alternative approach, adversarial sparsity, which quantifies how difficult it is to find a successful perturbation given both an input point and a constraint on the direction of the perturbation. We show that sparsity provides valuable insight into neural networks in multiple ways: for instance, it illustrates important differences between current state-of-the-art robust models them that accuracy analysis does not, and suggests approaches for improving their robustness. When applying broken defenses effective against weak attacks but not strong ones, sparsity can discriminate between the totally ineffective and the partially effective defenses. Finally, with sparsity we can measure increases in robustness that do not affect accuracy: we show for example that data augmentation can by itself increase adversarial robustness, without using adversarial training.