Miracle-dz/UniFAR-FTUs huggingface dataset:
Recursive Optimization of Finite Blocklength Allocation to Mitigate Age-of-Information Outage

As an emerging metric for the timeliness of information delivery, Age-of-Information (AoI) raises a special interest in the research area of tolerance-critical communications, wherein sufficiently short blocklength is usually adopted as an essential requirement. However, the interplay between AoI and finite blocklength is scantly treated. This paper studies the occurrence of critically high AoI, i.e., AoI outage, in generic orthogonal multi-access (OMA) systems with respect to the blocklength allocation among users. A Markov Decision Process model is set up for the problem, which enables a static state analysis. Therewith a recursive optimizer to reduce AoI outage is proposed, and further enhanced by heuristic penalty functions. The burstiness of AoI outage is also analyzed to provide additional insights into this problem in the finite blocklength (FBL) regime. It is shown that, different from average AoI optimizations, a risk-sensitive approach is significantly beneficial for AoI outage optimizations, on account of the FBL regime.
Parameterized Inapproximability for Steiner Orientation by Gap Amplification

In the $k$-Steiner Orientation problem, we are given a mixed graph, that is, with both directed and undirected edges, and a set of $k$ terminal pairs. The goal is to find an orientation of the undirected edges that maximizes the number of terminal pairs for which there is a path from the source to the sink. The problem is known to be W[1]-hard when parameterized by k and hard to approximate up to some constant for FPT algorithms assuming Gap-ETH. On the other hand, no approximation factor better than $O(k)$ is known. We show that $k$-Steiner Orientation is unlikely to admit an approximation algorithm with any constant factor, even within FPT running time. To obtain this result, we construct a self-reduction via a hashing-based gap amplification technique, which turns out useful even outside of the FPT paradigm. Precisely, we rule out any approximation factor of the form $(\log k)^{o(1)}$ for FPT algorithms (assuming FPT $\ne$ W[1]) and $(\log n)^{o(1)}$ for~purely polynomial-time algorithms (assuming that the class W[1] does not admit randomized FPT algorithms). Moreover, we prove $k$-Steiner Orientation to belong to W[1], which entails W[1]-completeness of $(\log k)^{o(1)}$-approximation for $k$-Steiner Orientation This provides an example of a natural approximation task that is complete in a parameterized complexity class. Finally, we apply our technique to the maximization version of directed multicut - Max $(k,p)$-Directed Multicut - where we are given a directed graph, $k$ terminals pairs, and a budget $p$. The goal is to maximize the number of separated terminal pairs by removing $p$ edges. We present a simple proof that the problem admits no FPT approximation with factor $O(k^{\frac 1 2 - \epsilon})$ (assuming FPT $\ne$ W[1]) and no polynomial-time approximation with ratio $O(|E(G)|^{\frac 1 2 - \epsilon})$ (assuming NP $\not\subseteq$ co-RP).
Mean-Field Analysis of Two-Layer Neural Networks: Non-Asymptotic Rates and Generalization Bounds

A recent line of work in deep learning theory has utilized the mean-field analysis to demonstrate the global convergence of noisy (stochastic) gradient descent for training over-parameterized two-layer neural networks. However, existing results in the mean-field setting do not provide the convergence rate of neural network training, and the generalization error bound is largely missing. In this paper, we provide a mean-field analysis in a generalized neural tangent kernel regime, and show that noisy gradient descent with weight decay can still exhibit a "kernel-like" behavior. This implies that the training loss converges linearly up to a certain accuracy in such regime. We also establish a generalization error bound for two-layer neural networks trained by noisy gradient descent with weight decay. Our results shed light on the connection between mean field analysis and the neural tangent kernel based analysis.
Smooth Homotopy of Infinite-Dimensional $C^{\infty}$-Manifolds.

In this paper, we use homotopical algebra (or abstract homotopical methods) to study smooth homotopical problems of infinite-dimensional $C^\infty$-manifolds in convenient calculus. More precisely, we discuss the smoothing of maps, sections, principal bundles, and gauge transformations. We first introduce the notion of hereditary $C^\infty$-paracompactness along with the semiclassicality condition on a $C^\infty$-manifold, which enables us to use local convexity in local arguments. Then, we prove that for $C^\infty$-manifolds $M$ and $N$, the smooth singular complex of $C^\infty(M,N)$ is weakly equivalent to the ordinary singular complex of $\mathcal{C}^0(M,N)$ under the hereditary $C^\infty$-paracompactness and semiclassicality conditions on $M$. We next generalize this result to sections of fiber bundles over a $C^\infty$-manifold $M$ under the same conditions on $M$. Further, we establish the Dwyer-Kan equivalence between the simplicial groupoid of smooth principal $G$-bundles over $M$ and that of continuous principal $G$-bundles over $M$ for a Lie group $G$ and a $C^\infty$-manifold $M$ under the same conditions on $M$, encoding the smoothing results for principal bundles and gauge transformations. For the proofs, we fully faithfully embed the category $C^\infty$ of $C^\infty$-manifolds into the category $\mathcal{D}$ of diffeological spaces and develop the smooth homotopy theory of diffeological spaces via a homotopical algebraic study of the model category $\mathcal{D}$ and the model category $\mathcal{C}^0$ of arc-generated spaces. Then, the hereditary $C^\infty$-paracompactness and semiclassicality conditions on $M$ imply that $M$ has the smooth homotopy type of a cofibrant object in $\mathcal{D}$. This result can be regarded as a smooth refinement of the results of Milnor, Palais, and Heisey on the homotopy type of infinite-dimensional topological manifolds.
Soft Threshold Weight Reparameterization for Learnable Sparsity

Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget. Existing methods rely on uniform or heuristic non-uniform sparsity budgets which have sub-optimal layer-wise parameter allocation resulting in a) lower prediction accuracy or b) higher inference cost (FLOPs). This work proposes Soft Threshold Reparameterization (STR), a novel use of the soft-threshold operator on DNN weights. STR smoothly induces sparsity while learning pruning thresholds thereby obtaining a non-uniform sparsity budget. Our method achieves state-of-the-art accuracy for unstructured sparsity in CNNs (ResNet50 and MobileNetV1 on ImageNet-1K), and, additionally, learns non-uniform budgets that empirically reduce the FLOPs by up to 50%. Notably, STR boosts the accuracy over existing results by up to 10% in the ultra sparse (99%) regime and can also be used to induce low-rank (structured sparsity) in RNNs. In short, STR is a simple mechanism which learns effective sparsity budgets that contrast with popular heuristics. Code, pretrained models and sparsity budgets are at this https URL.
Adversarial TCAV - Robust and Effective Interpretation of Intermediate Layers in Neural Networks

Interpreting neural network decisions and the information learned in intermediate layers is still a challenge due to the opaque internal state and shared non-linear interactions. Although (Kim et al, 2017) proposed to interpret intermediate layers by quantifying its ability to distinguish a user-defined concept (from random examples), the questions of robustness (variation against the choice of random examples) and effectiveness (retrieval rate of concept images) remain. We investigate these two properties and propose improvements to make concept activations reliable for practical use. Effectiveness: If the intermediate layer has effectively learned a user-defined concept, it should be able to recall --- at the testing step --- most of the images containing the proposed concept. For instance, we observed that the recall rate of Tiger shark and Great white shark from the ImageNet dataset with "Fins" as a user-defined concept was only 18.35% for VGG16. To increase the effectiveness of concept learning, we propose A-CAV --- the Adversarial Concept Activation Vector --- this results in larger margins between user concepts and (negative) random examples. This approach improves the aforesaid recall to 76.83% for VGG16. For robustness, we define it as the ability of an intermediate layer to be consistent in its recall rate (the effectiveness) for different random seeds. We observed that TCAV has a large variance in recalling a concept across different random seeds. For example, the recall of cat images (from a layer learning the concept of tail) varies from 18% to 86% with 20.85% standard deviation on VGG16. We propose a simple and scalable modification that employs a Gram-Schmidt process to sample random noise from concepts and learn an average "concept classifier". This approach improves the aforesaid standard deviation from 20.85% to 6.4%.
Perturbation level interacting dark energy model and its consequence on late-time cosmological parameters

In the present paper, we study the capability of interacting dark energy model with pure momentum transfer in the dark sector to reconcile tensions between low redshift observations and cosmic microwave background (CMB) results. This class of interacting model with pure momentum exchange introduces modifications to the standard model in the level of perturbation. We investigate the model by comparing to observational data, including integrated Sachs-Wolfe-galaxy cross-correlation, galaxy power spectrum, $f \sigma_8$, and CMB data. It is shown that this model can alleviate the observed tension between local and global measurements of $\sigma_8$. According to our results, the best fit value of $\sigma_8$ for interacting model is $0.700$, which is lower than the one for $\Lambda$CDM model and also is consistent with low redshift observations. Furthermore, we perform a forecast analysis to find the constraints on parameters of the interacting model from future experiments.
Learning to Grasp 3D Objects using Deep Residual U-Nets

Grasp synthesis is one of the challenging tasks for any robot object manipulation task. In this paper, we present a new deep learning-based grasp synthesis approach for 3D objects. In particular, we propose an end-to-end 3D Convolutional Neural Network to predict the objects’ graspable areas. We named our approach Res-U-Net since the architecture of the network is designed based on U-Net structure and residual network-styled blocks. It devised to plan 6-DOF grasps for any desired object, be efficient to compute and use, and be robust against varying point cloud density and Gaussian noise. We have performed extensive experiments to assess the performance of the proposed approach concerning graspable part detection, grasp success rate, and robustness to varying point cloud density and Gaussian noise. Experiments validate the promising performance of the proposed architecture in all aspects. A video showing the performance of our approach in the simulation environment can be found at http://youtu.be/5_yAJCc8owo
A Scalable Evolution Strategy with Directional Gaussian Smoothing for Blackbox Optimization

We propose an improved evolution strategy (ES) using a novel nonlocal gradient operator for high-dimensional black-box optimization. Standard ES methods with $d$-dimensional Gaussian smoothing suffer from the curse of dimensionality due to the high variance of Monte Carlo (MC) based gradient estimators. To control the variance, Gaussian smoothing is usually limited in a small region, so existing ES methods lack nonlocal exploration ability required for escaping from local minima. We develop a nonlocal gradient operator with directional Gaussian smoothing (DGS) to address this challenge. The DGS conducts 1D nonlocal explorations along $d$ orthogonal directions in $\mathbb{R}^d$, each of which defines a nonlocal directional derivative as a 1D integral. We then use Gauss-Hermite quadrature, instead of MC sampling, to estimate the $d$ 1D integrals to ensure high accuracy (i.e., small variance). Our method enables effective nonlocal exploration to facilitate the global search in high-dimensional optimization. We demonstrate the superior performance of our method in three sets of examples, including benchmark functions for global optimization, and real-world science and engineering applications.
StreamChain: Rethinking Blockchain for Datacenters.

Permissioned blockchains promise secure decentralized data management in business-to-business use-cases. In contrast to Bitcoin and similar public blockchains which rely on Proof-of-Work for consensus and are deployed on thousands of geo-distributed nodes, business-to-business use-cases (such as supply chain management and banking) require significantly fewer nodes, cheaper consensus, and are often deployed in datacenter-like environments with fast networking. However, permissioned blockchains often follow the architectural thinkining behind their WAN-oriented public relatives, which results in end-to-end latencies several orders of magnitude higher than necessary. In this work, we propose StreamChain, a permissioned blockchain design that eliminates blocks in favor of processing transactions in a streaming fashion. This results in a drastically lower latency without reducing throughput or forfeiting reliability and security guarantees. To demonstrate the wide applicability of our design, we prototype StreamChain based on the Hyperledger Fabric, and show that it delivers latency two orders of magnitude lower than Fabric, while sustaining similar throughput. This performance makes StreamChain a potential alternative to traditional databases and, thanks to its streaming paradigm, enables further research around reducing latency through relying on modern hardware in datacenters.
A Spike in Performance: Training Hybrid-Spiking Neural Networks with Quantized Activation Functions

The machine learning community has become increasingly interested in the energy efficiency of neural networks. The Spiking Neural Network (SNN) is a promising approach to energy-efficient computing, since its activation levels are quantized into temporally sparse, one-bit values (i.e., "spike" events), which additionally converts the sum over weight-activity products into a simple addition of weights (one weight for each spike). However, the goal of maintaining state-of-the-art (SotA) accuracy when converting a non-spiking network into an SNN has remained an elusive challenge, primarily due to spikes having only a single bit of precision. Adopting tools from signal processing, we cast neural activation functions as quantizers with temporally-diffused error, and then train networks while smoothly interpolating between the non-spiking and spiking regimes. We apply this technique to the Legendre Memory Unit (LMU) to obtain the first known example of a hybrid SNN outperforming SotA recurrent architectures---including the LSTM, GRU, and NRU---in accuracy, while reducing activities to at most 3.74 bits on average with 1.26 significant bits multiplying each weight. We discuss how these methods can significantly improve the energy efficiency of neural networks.
The Shapley Taylor Interaction Index

The attribution problem, that is the problem of attributing a model's prediction to its base features, is well-studied. We extend the notion of attribution to also apply to feature interactions. The Shapley value is a commonly used method to attribute a model's prediction to its base features. We propose a generalization of the Shapley value called Shapley-Taylor index that attributes the model's prediction to interactions of subsets of features up to some size k. The method is analogous to how the truncated Taylor Series decomposes the function value at a certain point using its derivatives at a different point. In fact, we show that the Shapley Taylor index is equal to the Taylor Series of the multilinear extension of the set-theoretic behavior of the model. We axiomatize this method using the standard Shapley axioms -- linearity, dummy, symmetry and efficiency -- and an additional axiom that we call the interaction distribution axiom. This new axiom explicitly characterizes how interactions are distributed for a class of functions that model pure interaction. We contrast the Shapley-Taylor index against the previously proposed Shapley Interaction index (cf. [9]) from the cooperative game theory literature. We also apply the Shapley Taylor index to three models and identify interesting qualitative insights.
Time Series Alignment with Global Invariances

In this work we address the problem of comparing time series while taking into account both feature space transformation and temporal variability. The proposed framework combines a latent global transformation of the feature space with the widely used Dynamic Time Warping (DTW). The latent global transformation captures the feature invariance while the DTW (or its smooth counterpart soft-DTW) deals with the temporal shifts. We cast the problem as a joint optimization over the global transformation and the temporal alignments. The versatility of our framework allows for several variants depending on the invariance class at stake. Among our contributions we define a differentiable loss for time series and present two algorithms for the computation of time series barycenters under our new geometry. We illustrate the interest of our approach on both simulated and real world data.
Try Depth Instead of Weight Correlations: Mean-field is a Less Restrictive Assumption for Deeper Networks

We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive. We argue mathematically that full-covariance approximations only improve the ELBO if they improve the expected log-likelihood. We further show that deeper mean-field networks are able to express predictive distributions approximately equivalent to shallower full-covariance networks. We validate these observations empirically, demonstrating that deeper models decrease the divergence between diagonal- and full-covariance Gaussian fits to the true posterior.
Graph Neural Distance Metric Learning with Graph-Bert

Graph distance metric learning serves as the foundation for many graph learning problems, e.g., graph clustering, graph classification and graph matching. Existing research works on graph distance metric (or graph kernels) learning fail to maintain the basic properties of such metrics, e.g., non-negative, identity of indiscernibles, symmetry and triangle inequality, respectively. In this paper, we will introduce a new graph neural network based distance metric learning approaches, namely GB-DISTANCE (GRAPH-BERT based Neural Distance). Solely based on the attention mechanism, GB-DISTANCE can learn graph instance representations effectively based on a pre-trained GRAPH-BERT model. Different from the existing supervised/unsupervised metrics, GB-DISTANCE can be learned effectively in a semi-supervised manner. In addition, GB-DISTANCE can also maintain the distance metric basic properties mentioned above. Extensive experiments have been done on several benchmark graph datasets, and the results demonstrate that GB-DISTANCE can out-perform the existing baseline methods, especially the recent graph neural network model based graph metrics, with a significant gap in computing the graph distance.
Message Type Identification of Binary Network Protocols using Continuous Segment Similarity

Protocol reverse engineering based on traffic traces infers the behavior of unknown network protocols by analyzing observable network messages. To perform correct deduction of message semantics or behavior analysis, accurate message type identification is an essential first step. However, identifying message types is particularly difficult for binary protocols, whose structural features are hidden in their densely packed data representation. We leverage the intrinsic structural features of binary protocols and propose an accurate method for discriminating message types.Our approach uses a similarity measure with continuous value range by comparing feature vectors where vector elements correspond to the fields in a message, rather than discrete byte values. This enables a better recognition of structural patterns, which remain hidden when only exact value matches are considered. We combine Hirschberg alignment with DBSCAN as cluster algorithm to yield a novel inference mechanism. By applying novel autoconfiguration schemes, we do not require manually configured parameters for the analysis of an unknown protocol, as required by earlier approaches.Results of our evaluations show that our approach has considerable advantages in message type identification result quality and also execution performance over previous approaches.
6DoF Object Pose Estimation via Differentiable Proxy Voting Loss

Estimating a 6DOF object pose from a single image is very challenging due to occlusions or textureless appearances. Vector-field based keypoint voting has demonstrated its effectiveness and superiority on tackling those issues. However, direct regression of vector-fields neglects that the distances between pixels and keypoints also affect the deviations of hypotheses dramatically. In other words, small errors in direction vectors may generate severely deviated hypotheses when pixels are far away from a keypoint. In this paper, we aim to reduce such errors by incorporating the distances between pixels and keypoints into our objective. To this end, we develop a simple yet effective differentiable proxy voting loss (DPVL) which mimics the hypothesis selection in the voting procedure. By exploiting our voting loss, we are able to train our network in an end-to-end manner. Experiments on widely used datasets, i.e., LINEMOD and Occlusion LINEMOD, manifest that our DPVL improves pose estimation performance significantly and speeds up the training convergence.
A Dataset of Enterprise-Driven Open Source Software

We present a dataset of open source software developed mainly by enterprises rather than volunteers. This can be used to address known generalizability concerns, and, also, to perform research on open source business software development. Based on the premise that an enterprise's employees are likely to contribute to a project developed by their organization using the email account provided by it, we mine domain names associated with enterprises from open data sources as well as through white- and blacklisting, and use them through three heuristics to identify 17 264 enterprise GitHub projects. We provide these as a dataset detailing their provenance and properties. A manual evaluation of a dataset sample shows an identification accuracy of 89%. Through an exploratory data analysis we found that projects are staffed by a plurality of enterprise insiders, who appear to be pulling more than their weight, and that in a small percentage of relatively large projects development happens exclusively through enterprise insiders.