OpenRaiser/Intern-Atlas huggingface dataset:
Distilled Non-Semantic Speech Embeddings with Binary Neural Networks for Low-Resource Devices
Venue: arXiv cs.SD 2023
Year: 2023

This work introduces BRILLsson, a novel binary neural network-based representation learning model for a broad range of non-semantic speech tasks. We train the model with knowledge distillation from a large and real-valued TRILLsson model with only a fraction of the dataset used to train TRILLsson. The resulting BRILLsson models are only 2MB in size with a latency less than 8ms, making them suitable for deployment in low-resource devices such as wearables. We evaluate BRILLsson on eight benchmark tasks (including but not limited to spoken language identification, emotion recognition, health condition diagnosis, and keyword spotting), and demonstrate that our proposed ultra-light and low-latency models perform as well as large-scale models.
On the Complexity of Identifying Strongly Regular Graphs
Venue: arXiv cs.CC 2023
Year: 2023

In this paper, we show that Graph Isomorphism (GI) is not $\textsf{AC}^{0}$-reducible to several problems, including the Latin Square Isotopy problem, isomorphism testing of several families of Steiner designs, and isomorphism testing of conference graphs. As a corollary, we obtain that GI is not $\textsf{AC}^{0}$-reducible to isomorphism testing of Latin square graphs and strongly regular graphs arising from special cases of Steiner $2$-designs. We accomplish this by showing that the generator-enumeration technique for each of these problems can be implemented in $β_{2}\textsf{FOLL}$, which cannot compute Parity (Chattopadhyay, Torán, & Wagner, ACM Trans. Comp. Theory, 2013).
URANUS: Radio Frequency Tracking, Classification and Identification of Unmanned Aircraft Vehicles
Venue: arXiv cs.LG 2023
Year: 2023

Safety and security issues for Critical Infrastructures are growing as attackers adopt drones as an attack vector flying in sensitive airspaces, such as airports, military bases, city centers, and crowded places. Despite the use of UAVs for logistics, shipping recreation activities, and commercial applications, their usage poses severe concerns to operators due to the violations and the invasions of the restricted airspaces. A cost-effective and real-time framework is needed to detect the presence of drones in such cases. In this contribution, we propose an efficient radio frequency-based detection framework called URANUS. We leverage real-time data provided by the Radio Frequency/Direction Finding system, and radars in order to detect, classify and identify drones (multi-copter and fixed-wings) invading no-drone zones. We adopt a Multilayer Perceptron neural network to identify and classify UAVs in real-time, with $90$% accuracy. For the tracking task, we use a Random Forest model to predict the position of a drone with an MSE $\approx0.29$, MAE $\approx0.04$, and $R^2\approx 0.93$. Furthermore, coordinate regression is performed using Universal Transverse Mercator coordinates to ensure high accuracy. Our analysis shows that URANUS is an ideal framework for identifying, classifying, and tracking UAVs that most Critical Infrastructure operators can adopt.
DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation
Venue: arXiv cs.CV 2023
Year: 2023

One key challenge of exemplar-guided image generation lies in establishing fine-grained correspondences between input and guided images. Prior approaches, despite the promising results, have relied on either estimating dense attention to compute per-point matching, which is limited to only coarse scales due to the quadratic memory cost, or fixing the number of correspondences to achieve linear complexity, which lacks flexibility. In this paper, we propose a dynamic sparse attention based Transformer model, termed Dynamic Sparse Transformer (DynaST), to achieve fine-level matching with favorable efficiency. The heart of our approach is a novel dynamic-attention unit, dedicated to covering the variation on the optimal number of tokens one position should focus on. Specifically, DynaST leverages the multi-layer nature of Transformer structure, and performs the dynamic attention scheme in a cascaded manner to refine matching results and synthesize visually-pleasing outputs. In addition, we introduce a unified training objective for DynaST, making it a versatile reference-based image translation framework for both supervised and unsupervised scenarios. Extensive experiments on three applications, pose-guided person image generation, edge-based face synthesis, and undistorted image style transfer, demonstrate that DynaST achieves superior performance in local details, outperforming the state of the art while reducing the computational cost significantly. Our code is available at https://github.com/Huage001/DynaST
SlimSeg: Slimmable Semantic Segmentation with Boundary Supervision
Venue: arXiv cs.CV 2023
Year: 2023

Accurate semantic segmentation models typically require significant computational resources, inhibiting their use in practical applications. Recent works rely on well-crafted lightweight models to achieve fast inference. However, these models cannot flexibly adapt to varying accuracy and efficiency requirements. In this paper, we propose a simple but effective slimmable semantic segmentation (SlimSeg) method, which can be executed at different capacities during inference depending on the desired accuracy-efficiency tradeoff. More specifically, we employ parametrized channel slimming by stepwise downward knowledge distillation during training. Motivated by the observation that the differences between segmentation results of each submodel are mainly near the semantic borders, we introduce an additional boundary guided semantic segmentation loss to further improve the performance of each submodel. We show that our proposed SlimSeg with various mainstream networks can produce flexible models that provide dynamic adjustment of computational cost and better performance than independent models. Extensive experiments on semantic segmentation benchmarks, Cityscapes and CamVid, demonstrate the generalization ability of our framework.
Attention: Not Just Another Dataset for Patch-Correctness Checking
Venue: arXiv cs.SE 2023
Year: 2023

Automated Program Repair (APR) techniques have drawn wide attention from both academia and industry. Meanwhile, one main limitation with the current state-of-the-art APR tools is that patches passing all the original tests are not necessarily the correct ones wanted by developers, i.e., the plausible patch problem. To date, various Patch-Correctness Checking (PCC) techniques have been proposed to address this important issue. However, they are only evaluated on very limited datasets as the APR tools used for generating such patches can only explore a small subset of the search space of possible patches, posing serious threats to external validity to existing PCC studies. In this paper, we construct an extensive PCC dataset (the largest manually labeled PCC dataset to our knowledge) to revisit all state-of-the-art PCC techniques. More specifically, our PCC dataset includes 1,988 patches generated from the recent PraPR APR tool, which leverages highly-optimized bytecode-level patch executions and can exhaustively explore all possible plausible patches within its large predefined search space (including well-known fixing patterns from various prior APR tools). Our extensive study of representative PCC techniques on the new dataset has revealed various surprising findings and provided guidelines for future PCC research.
Finite strain porohyperelasticity: An asymptotic multiscale ALE-FSI approach supported by ANNs
Venue: arXiv cs.CE 2023
Year: 2023

The governing equations and numerical solution strategy to solve porohyperelastic problems as multiscale multiphysics media are provided in this contribution. The problem starts from formulating and non-dimensionalising a Fluid-Solid Interaction (FSI) problem using Arbitrary Lagrangian-Eulerian (ALE) technique at the pore level. The resultant ALE-FSI coupled systems of PDEs are expanded and analysed using the asymptotic homogenisation technique which yields three partially novel systems of PDEs, one governing the macroscopic/effective problem supplied by two microscale problems (fluid and solid). The latter two provide the microscopic response fields whose average value is required in real-time/online form to determine the macroscale response. This is possible efficiently by training an Artificial Neural Network (ANN) as a surrogate for the Direct Numerical Solution (DNS) of the microscale solid problem. The present methodology allows for solving finite strain (multiscale) porohyperelastic problems accurately using the direct derivative of the strain energy, for the first time. Furthermore, a simple real-time output density check is introduced to achieve an optimal and reliable training dataset from DNS. A Representative Volume Element (RVE) is adopted which is followed by performing a microscale (RVE) sensitivity analysis and a multiscale confined consolidation simulation showing the importance of employing the present method when dealing with finite strain poroelastic/porohyperelastic problems.
Semi-supervised cross-lingual speech emotion recognition
Venue: arXiv cs.SD 2023
Year: 2023

Performance in Speech Emotion Recognition (SER) on a single language has increased greatly in the last few years thanks to the use of deep learning techniques. However, cross-lingual SER remains a challenge in real-world applications due to two main factors: the first is the big gap among the source and the target domain distributions; the second factor is the major availability of unlabeled utterances in contrast to the labeled ones for the new language. Taking into account previous aspects, we propose a Semi-Supervised Learning (SSL) method for cross-lingual emotion recognition when only few labeled examples in the target domain (i.e. the new language) are available. Our method is based on a Transformer and it adapts to the new domain by exploiting a pseudo-labeling strategy on the unlabeled utterances. In particular, the use of a hard and soft pseudo-labels approach is investigated. We thoroughly evaluate the performance of the proposed method in a speaker-independent setup on both the source and the new language and show its robustness across five languages belonging to different linguistic strains. The experimental findings indicate that the unweighted accuracy is increased by an average of 40% compared to state-of-the-art methods.
Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions
Venue: arXiv cs.CV 2023
Year: 2023

Due to the scarcity of dense pixel-level semantic annotations for images recorded in adverse visual conditions, there has been a keen interest in unsupervised domain adaptation (UDA) for the semantic segmentation of such images. UDA adapts models trained on normal conditions to the target adverse-condition domains. Meanwhile, multiple datasets with driving scenes provide corresponding images of the same scenes across multiple conditions, which can serve as a form of weak supervision for domain adaptation. We propose Refign, a generic extension to self-training-based UDA methods which leverages these cross-domain correspondences. Refign consists of two steps: (1) aligning the normal-condition image to the corresponding adverse-condition image using an uncertainty-aware dense matching network, and (2) refining the adverse prediction with the normal prediction using an adaptive label correction mechanism. We design custom modules to streamline both steps and set the new state of the art for domain-adaptive semantic segmentation on several adverse-condition benchmarks, including ACDC and Dark Zurich. The approach introduces no extra training parameters, minimal computational overhead -- during training only -- and can be used as a drop-in extension to improve any given self-training-based UDA method. Code is available at https://github.com/brdav/refign.
A PoW-less Bitcoin with Certified Byzantine Consensus
Venue: arXiv cs.DC 2023
Year: 2023

Distributed Ledger Technologies (DLTs), when managed by a few trusted validators, require most but not all of the machinery available in public DLTs. In this work, we explore one possible way to profit from this state of affairs. We devise a combination of a modified Practical Byzantine Fault Tolerant (PBFT) protocol and a revised Flexible Round-Optimized Schnorr Threshold Signatures (FROST) scheme, and then we inject the resulting proof-of-authority consensus algorithm into Bitcoin (chosen for the reliability, openness, and liveliness it brings in), replacing its PoW machinery. The combined protocol may operate as a modern, safe foundation for digital payment systems and Central Bank Digital Currencies (CBDC).
Kernelization for Graph Packing Problems via Rainbow Matching
Venue: arXiv cs.DS 2023
Year: 2023

We introduce a new kernelization tool, called rainbow matching technique}, that is appropriate for the design of polynomial kernels for packing problems and their hitting counterparts. Our technique capitalizes on the powerful combinatorial results of [Graf, Harris, Haxell, SODA 2021]. We apply the rainbow matching technique on four (di)graph packing or hitting problems, namely the Triangle-Packing in Tournament problem (TPT), where we ask for a packing of $k$ directed triangles in a tournament, Directed Feedback Vertex Set in Tournament problem (FVST), where we ask for a (hitting) set of at most $k$ vertices which intersects all triangles of a tournament, the Induced 2-Path-Packing (IPP) where we ask for a packing of $k$ induced paths of length two in a graph and Induced 2-Path Hitting Set problem (IPHS), where we ask for a (hitting) set of at most $k$ vertices which intersects all induced paths of length two in a graph. The existence of a sub-quadratic kernels for these problems was proven for the first time in [Fomin, Le, Lokshtanov, Saurabh, Thomassé, Zehavi. ACM Trans. Algorithms, 2019], where they gave a kernel of $O(k^{3/2})$ vertices for the two first problems and $O(k^{5/3})$ vertices for the two last. In the same paper it was questioned whether these bounds can be (optimally) improved to linear ones. Motivated by this question, we apply the rainbow matching technique and prove that TPT and FVST admit (almost linear) kernels of $k^{1+\frac{O(1)}{\sqrt{\log{k}}}}$ vertices and that IPP and IPHS admit kernels of $O(k)$ vertices.
Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Venue: arXiv cs.LG 2023
Year: 2023

This paper provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models. We collect a total of 341 publications concerning bias mitigation for ML classifiers. These methods can be distinguished based on their intervention procedure (i.e., pre-processing, in-processing, post-processing) and the technique they apply. We investigate how existing bias mitigation methods are evaluated in the literature. In particular, we consider datasets, metrics and benchmarking. Based on the gathered insights (e.g., What is the most popular fairness metric? How many datasets are used for evaluating bias mitigation methods?), we hope to support practitioners in making informed choices when developing and evaluating new bias mitigation methods.
Computing Optimal Kernels in Two Dimensions
Venue: arXiv cs.CG 2023
Year: 2023

Let $P$ be a set of $n$ points in $\Re^2$. For a parameter $\varepsilon\in (0,1)$, a subset $C\subseteq P$ is an \emph{$\varepsilon$-kernel} of $P$ if the projection of the convex hull of $C$ approximates that of $P$ within $(1-\varepsilon)$-factor in every direction. The set $C$ is a \emph{weak $\varepsilon$-kernel} of $P$ if its directional width approximates that of $P$ in every direction. Let $\mathsf{k}_{\varepsilon}(P)$ (resp.\ $\mathsf{k}^{\mathsf{w}}_{\varepsilon}(P)$) denote the minimum-size of an $\varepsilon$-kernel (resp. weak $\varepsilon$-kernel) of $P$. We present an $O(n\mathsf{k}_{\varepsilon}(P)\log n)$-time algorithm for computing an $\varepsilon$-kernel of $P$ of size $\mathsf{k}_{\varepsilon}(P)$, and an $O(n^2\log n)$-time algorithm for computing a weak $\varepsilon$-kernel of $P$ of size ${\mathsf{k}}^{\mathsf{w}}_{\varepsilon}(P)$. We also present a fast algorithm for the Hausdorff variant of this problem. In addition, we introduce the notion of \emph{$\varepsilon$-core}, a convex polygon lying inside $\mathsf{ch}(P)$, prove that it is a good approximation of the optimal $\varepsilon$-kernel, present an efficient algorithm for computing it, and use it to compute an $\varepsilon$-kernel of small size.
Lightweight Vision Transformer with Cross Feature Attention
Venue: arXiv cs.CV 2023
Year: 2023

Recent advances in vision transformers (ViTs) have achieved great performance in visual recognition tasks. Convolutional neural networks (CNNs) exploit spatial inductive bias to learn visual representations, but these networks are spatially local. ViTs can learn global representations with their self-attention mechanism, but they are usually heavy-weight and unsuitable for mobile devices. In this paper, we propose cross feature attention (XFA) to bring down computation cost for transformers, and combine efficient mobile CNNs to form a novel efficient light-weight CNN-ViT hybrid model, XFormer, which can serve as a general-purpose backbone to learn both global and local representation. Experimental results show that XFormer outperforms numerous CNN and ViT-based models across different tasks and datasets. On ImageNet1K dataset, XFormer achieves top-1 accuracy of 78.5% with 5.5 million parameters, which is 2.2% and 6.3% more accurate than EfficientNet-B0 (CNN-based) and DeiT (ViT-based) for similar number of parameters. Our model also performs well when transferring to object detection and semantic segmentation tasks. On MS COCO dataset, XFormer exceeds MobileNetV2 by 10.5 AP (22.7 -> 33.2 AP) in YOLOv3 framework with only 6.3M parameters and 3.8G FLOPs. On Cityscapes dataset, with only a simple all-MLP decoder, XFormer achieves mIoU of 78.5 and FPS of 15.3, surpassing state-of-the-art lightweight segmentation networks.
Towards unsupervised assessment with open-source data of the accuracy of deep learning-based distributed PV mapping
Venue: arXiv cs.CV 2023
Year: 2023

Photovoltaic (PV) energy is rapidly growing and key to mitigating the energy crisis. However, distributed PV generation, which amounts to half of the PV installed capacity, is typically unavailable to transmission system operators (TSOs), making it increasingly difficult to balance the load and supply and avoid grid congestions. To assess distributed PV generation, TSOs need precise knowledge regarding the metadata of distributed PV installations. Many remote sensing-based approaches have been proposed to map these installations in recent years. However, to use these methods in industrial processes, assessing their accuracy over the mapping area, i.e., the area covered by the model during deployment, is necessary. We define the downstream task accuracy (DTA) as the accuracy over the mapping area, automatically computed using publicly available data sources and the model's outputs and expressed in an interpretable way for operators. We benchmark existing models for distributed PV mapping and show how they perform in terms of DTA. We show that the accuracy computed on the test set overestimates by about 30 percentage points the accuracy on the mapping area. Our approach paves the way for safer integration of deep-learning-based pipelines for remote PV mapping. Code is available at https://github.com/gabrielkasmi/deeppvmapper.
Stable Invariant Models via Koopman Spectra
Venue: arXiv cs.LG 2023
Year: 2023

Weight-tied models have attracted attention in the modern development of neural networks. The deep equilibrium model (DEQ) represents infinitely deep neural networks with weight-tying, and recent studies have shown the potential of this type of approach. DEQs are needed to iteratively solve root-finding problems in training and are built on the assumption that the underlying dynamics determined by the models converge to a fixed point. In this paper, we present the stable invariant model (SIM), a new class of deep models that in principle approximates DEQs under stability and extends the dynamics to more general ones converging to an invariant set (not restricted in a fixed point). The key ingredient in deriving SIMs is a representation of the dynamics with the spectra of the Koopman and Perron--Frobenius operators. This perspective approximately reveals stable dynamics with DEQs and then derives two variants of SIMs. We also propose an implementation of SIMs that can be learned in the same way as feedforward models. We illustrate the empirical performance of SIMs with experiments and demonstrate that SIMs achieve comparative or superior performance against DEQs in several learning tasks.
Securing name resolution in the IoT: DNS over CoAP
Venue: arXiv cs.NI 2023
Year: 2023

In this paper, we present the design, implementation, and analysis of DNS over CoAP (DoC), a new proposal for secure and privacy-friendly name resolution of constrained IoT devices. We implement different design choices of DoC in RIOT, an open-source operating system for the IoT, evaluate performance measures in a testbed, compare with DNS over UDP and DNS over DTLS, and validate our protocol design based on empirical DNS IoT data. Our findings indicate that plain DoC is on par with common DNS solutions for the constrained IoT but significantly outperforms when additional standard features of CoAP are used such as caching. With OSCORE, we can save more than 10 kBytes of code memory compared to DTLS, when a CoAP application is already present, and retain the end-to-end trust chain with intermediate proxies, while leveraging features such as group communication or encrypted en-route caching. We also discuss a compression scheme for very restricted links that reduces data by up to 70%.
An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System
Venue: arXiv cs.AR 2023
Year: 2023

Training machine learning (ML) algorithms is a computationally intensive process, which is frequently memory-bound due to repeatedly accessing large training datasets. As a result, processor-centric systems (e.g., CPU, GPU) suffer from costly data movement between memory units and processing units, which consumes large amounts of energy and execution cycles. Memory-centric computing systems, i.e., with processing-in-memory (PIM) capabilities, can alleviate this data movement bottleneck. Our goal is to understand the potential of modern general-purpose PIM architectures to accelerate ML training. To do so, we (1) implement several representative classic ML algorithms (namely, linear regression, logistic regression, decision tree, K-Means clustering) on a real-world general-purpose PIM architecture, (2) rigorously evaluate and characterize them in terms of accuracy, performance and scaling, and (3) compare to their counterpart implementations on CPU and GPU. Our evaluation on a real memory-centric computing system with more than 2500 PIM cores shows that general-purpose PIM architectures can greatly accelerate memory-bound ML workloads, when the necessary operations and datatypes are natively supported by PIM hardware. For example, our PIM implementation of decision tree is $27\times$ faster than a state-of-the-art CPU version on an 8-core Intel Xeon, and $1.34\times$ faster than a state-of-the-art GPU version on an NVIDIA A100. Our K-Means clustering on PIM is $2.8\times$ and $3.2\times$ than state-of-the-art CPU and GPU versions, respectively. To our knowledge, our work is the first one to evaluate ML training on a real-world PIM architecture. We conclude with key observations, takeaways, and recommendations that can inspire users of ML workloads, programmers of PIM architectures, and hardware designers & architects of future memory-centric computing systems.