OpenRaiser/Intern-Atlas huggingface dataset:
A Trustworthiness Score to Evaluate DNN Predictions
Venue: arXiv cs.LG 2023
Year: 2023

Due to the black box nature of deep neural networks (DNN), the continuous validation of DNN during operation is challenging with the absence of a human monitor. As a result this makes it difficult for developers and regulators to gain confidence in the deployment of autonomous systems employing DNN. It is critical for safety during operation to know when DNN's predictions are trustworthy or suspicious. With the absence of a human monitor, the basic approach is to use the model's output confidence score to assess if predictions are trustworthy or suspicious. However, the model's confidence score is a result of computations coming from a black box, therefore lacks transparency and makes it challenging to automatedly credit trustworthiness to predictions. We introduce the trustworthiness score (TS), a simple metric that provides a more transparent and effective way of providing confidence in DNN predictions compared to model's confidence score. The metric quantifies the trustworthiness in a prediction by checking for the existence of certain features in the predictions made by the DNN. We also use the underlying idea of the TS metric, to provide a suspiciousness score (SS) in the overall input frame to help in the detection of suspicious frames where false negatives exist. We conduct a case study using YOLOv5 on persons detection to demonstrate our method and usage of TS and SS. The case study shows that using our method consistently improves the precision of predictions compared to relying on model confidence score alone, for both 1) approving of trustworthy predictions (~20% improvement) and 2) detecting suspicious frames (~5% improvement).
Outage Performance of Fluid Antenna System (FAS)-aided Terahertz Communication Networks
Venue: arXiv cs.IT 2023
Year: 2023

Millimeter-wave networks have already been successfully rolled out in many countries and now the research direction heads toward new technologies and standards to enable Tbps rates for future sixth-generation (6G) wireless communication systems. This work studies a point-to-point terahertz (THz) communication network exploiting the concept of a fluid antenna system (FAS) over correlated alpha-mu fading channels, nicely fitting the THz communication. Furthermore, the considered system is expanded to the selection-combining-FAS (SC-FAS) and maximum-gain-combining- FAS (MGC-FAS) diversity variates at the receiver side. The proposed FAS and its diversity configuration techniques are aimed to combat the high path loss, blockages, and molecular absorption effect related to the THz band. Our contribution includes comprehensive outage probability (OP) performance analysis for the THz band given the non-diversity and diversity FAS receivers. Moreover, the derived outage probability formulas are verified via Monte Carlo simulations. Numerical results have confirmed the superior performance of the MGC-FAS scheme in terms of OP. Finally, this work justifies that a higher number of antenna ports dramatically improves the system performance, even in the presence of correlation.
Capacity-CRB Tradeoff in OFDM Integrated Sensing and Communication Systems
Venue: arXiv cs.IT 2023
Year: 2023

Integrated sensing and communication (ISAC) has emerged as a key technology for future communication systems. In this paper, we provide a general framework to reveal the fundamental tradeoff between sensing and communication in OFDM systems, where a unified ISAC waveform is exploited to perform both tasks. In particular, we define the Capacity-Bayesian Cramer Rao Bound (BCRB) region in the asymptotically case when the number of subcarriers is large. Specifically, we show that the asymptotically optimal input distribution that achieves the Pareto boundary point of the Capacity-BCRB region is Gaussian and the entire Pareto boundary can be obtained by solving a convex power allocation problem. Moreover, we characterize the structure of the sensing-optimal power allocation in the asymptotically case. Finally, numerical simulations are conducted to verify the theoretical analysis and provide useful insights.
Using deterministic tourist walk as a small-world metric on Watts-Strogatz networks
Venue: arXiv cs.SI 2023
Year: 2023

The Watts-Strogatz model (WS) has been demonstrated to effectively describe real-world networks due to its ability to reproduce the small-world properties commonly observed in a variety of systems, including social networks, computer networks, biochemical reactions, and neural networks. As the presence of small-world properties is a prevalent characteristic in many real-world networks, the measurement of "small-worldness" has become a crucial metric in the field of network science, leading to the development of various methods for its assessment over the past two decades. In contrast, the deterministic tourist walk (DTW) method has emerged as a prominent technique for texture analysis and network classification. In this paper, we propose the use of a modified version of the DTW method to classify networks into three categories: regular networks, random networks, and small-world networks. Additionally, we construct a small-world metric, denoted by the coefficient $χ$, from the DTW method. Results indicate that the proposed method demonstrates excellent performance in the task of network classification, achieving over $90\%$ accuracy. Furthermore, the results obtained using the coefficient $χ$ on real-world networks provide evidence that the proposed method effectively serves as a satisfactory small-world metric.
Raw or Cooked? Object Detection on RAW Images
Venue: arXiv cs.CV 2023
Year: 2023

Images fed to a deep neural network have in general undergone several handcrafted image signal processing (ISP) operations, all of which have been optimized to produce visually pleasing images. In this work, we investigate the hypothesis that the intermediate representation of visually pleasing images is sub-optimal for downstream computer vision tasks compared to the RAW image representation. We suggest that the operations of the ISP instead should be optimized towards the end task, by learning the parameters of the operations jointly during training. We extend previous works on this topic and propose a new learnable operation that enables an object detector to achieve superior performance when compared to both previous works and traditional RGB images. In experiments on the open PASCALRAW dataset, we empirically confirm our hypothesis.
A Semantic Modular Framework for Events Topic Modeling in Social Media
Venue: arXiv cs.LG 2023
Year: 2023

The advancement of social media contributes to the growing amount of content they share frequently. This framework provides a sophisticated place for people to report various real-life events. Detecting these events with the help of natural language processing has received researchers' attention, and various algorithms have been developed for this goal. In this paper, we propose a Semantic Modular Model (SMM) consisting of 5 different modules, namely Distributional Denoising Autoencoder, Incremental Clustering, Semantic Denoising, Defragmentation, and Ranking and Processing. The proposed model aims to (1) cluster various documents and ignore the documents that might not contribute to the identification of events, (2) identify more important and descriptive keywords. Compared to the state-of-the-art methods, the results show that the proposed model has a higher performance in identifying events with lower ranks and extracting keywords for more important events in three English Twitter datasets: FACup, SuperTuesday, and USElection. The proposed method outperformed the best reported results in the mean keyword-precision metric by 7.9\%.
Cellular Network Speech Enhancement: Removing Background and Transmission Noise
Venue: arXiv cs.SD 2023
Year: 2023

The primary objective of speech enhancement is to reduce background noise while preserving the target's speech. A common dilemma occurs when a speaker is confined to a noisy environment and receives a call with high background and transmission noise. To address this problem, the Deep Noise Suppression (DNS) Challenge focuses on removing the background noise with the next-generation deep learning models to enhance the target's speech; however, researchers fail to consider Voice Over IP (VoIP) applications their transmission noise. Focusing on Google Meet and its cellular application, our work achieves state-of-the-art performance on the Google Meet To Phone Track of the VoIP DNS Challenge. This paper demonstrates how to beat industrial performance and achieve 1.92 PESQ and 0.88 STOI, as well as superior acoustic fidelity, perceptual quality, and intelligibility in various metrics.
Autonomous Rendezvous with Non-cooperative Target Objects with Swarm Chasers and Observers
Venue: arXiv cs.RO 2023
Year: 2023

Space debris is on the rise due to the increasing demand for spacecraft for com-munication, navigation, and other applications. The Space Surveillance Network (SSN) tracks over 27,000 large pieces of debris and estimates the number of small, un-trackable fragments at over 1,00,000. To control the growth of debris, the for-mation of further debris must be reduced. Some solutions include deorbiting larger non-cooperative resident space objects (RSOs) or servicing satellites in or-bit. Both require rendezvous with RSOs, and the scale of the problem calls for autonomous missions. This paper introduces the Multipurpose Autonomous Ren-dezvous Vision-Integrated Navigation system (MARVIN) developed and tested at the ORION Facility at Florida Institution of Technology. MARVIN consists of two sub-systems: a machine vision-aided navigation system and an artificial po-tential field (APF) guidance algorithm which work together to command a swarm of chasers to safely rendezvous with the RSO. We present the MARVIN architec-ture and hardware-in-the-loop experiments demonstrating autonomous, collabo-rative swarm satellite operations successfully guiding three drones to rendezvous with a physical mockup of a non-cooperative satellite in motion.
Differentially Private Natural Language Models: Recent Advances and Future Directions
Venue: arXiv cs.CL 2023
Year: 2023

Recent developments in deep learning have led to great success in various natural language processing (NLP) tasks. However, these applications may involve data that contain sensitive information. Therefore, how to achieve good performance while also protecting the privacy of sensitive data is a crucial challenge in NLP. To preserve privacy, Differential Privacy (DP), which can prevent reconstruction attacks and protect against potential side knowledge, is becoming a de facto technique for private data analysis. In recent years, NLP in DP models (DP-NLP) has been studied from different perspectives, which deserves a comprehensive review. In this paper, we provide the first systematic review of recent advances in DP deep learning models in NLP. In particular, we first discuss some differences and additional challenges of DP-NLP compared with the standard DP deep learning. Then, we investigate some existing work on DP-NLP and present its recent developments from three aspects: gradient perturbation based methods, embedding vector perturbation based methods, and ensemble model based methods. We also discuss some challenges and future directions.
The umbilical cord of finite model theory
Venue: arXiv cs.LO 2023
Year: 2023

Model theory was born and developed as a part of mathematical logic. It has various application domains but is not beholden to any of them. A priori, the research area known as finite model theory would be just a part of model theory but didn't turn out that way. There is one application domain -- relational database management -- that finite model theory had been beholden to during a substantial early period when databases provided the motivation and were the main application target for finite model theory. Arguably, finite model theory was motivated even more by complexity theory. But the subject of this paper is how relational database theory influenced finite model theory. This is NOT a scholarly history of the subject with proper credits to all participants. My original intent was to cover just the developments that I witnessed or participated in. The need to make the story coherent forced me to cover some additional developments.
Game Theoretic Decision Making by Actively Learning Human Intentions Applied on Autonomous Driving
Venue: arXiv cs.RO 2023
Year: 2023

The ability to estimate human intentions and interact with human drivers intelligently is crucial for autonomous vehicles to successfully achieve their objectives. In this paper, we propose a game theoretic planning algorithm that models human opponents with an iterative reasoning framework and estimates human latent cognitive states through probabilistic inference and active learning. By modeling the interaction as a partially observable Markov decision process with adaptive state and action spaces, our algorithm is able to accomplish real-time lane changing tasks in a realistic driving simulator. We compare our algorithm's lane changing performance in dense traffic with a state-of-the-art autonomous lane changing algorithm to show the advantage of iterative reasoning and active learning in terms of avoiding overly conservative behaviors and achieving the driving objective successfully.
A graph isomorphism invariant based on neighborhood aggregation
Venue: arXiv cs.DS 2023
Year: 2023

This paper presents a new graph isomorphism invariant, called $\mathfrak{w}$-labeling, that can be used to design a polynomial-time algorithm for solving the graph isomorphism problem for various graph classes. For example, all non-cospectral graph pairs are distinguished by the proposed combinatorial method, furthermore, even non-isomorphic cospectral graphs can be distinguished assuming certain properties of their eigenspaces. We also investigate a refinement of the aforementioned labeling, called $\mathfrak{s}^k$-labeling, which has both theoretical and practical applications. Among others, it can be used to generate graph fingerprints, which uniquely identify all graphs in the considered databases, including all strongly regular graphs on at most 64 nodes and all graphs on at most 12 nodes. It provably identifies all trees and 3-connected planar graphs up to isomorphism, which -- as a byproduct -- gives a new isomorphism algorithm for both graph classes. The practical importance of this fingerprint lies in significantly speeding up searching in graph databases, which is a commonly required task in biological and chemical applications.
Discovering the Traces of Disinformation on Instagram in the Internet Archive
Venue: arXiv cs.DL 2023
Year: 2023

Disinformation, which is fabricated, misleading content spread with the intent to deceive others, is accumulating substantial engagements and reaching a vast audience on Instagram. However, the temporary nature of the platform and the security guidelines that remove malicious content make studying this disinformation a challenge. The only way to access removed content and banned accounts that are no longer on the live web is by searching the web archives. In this study, we set out to quantify the replayability and quality of past captures of Instagram account pages, specifically focusing on a group of anti-vaxx content creators known as the Disinformation Dozen. We found that the number of mementos listed for these users' account pages on the Internet Archive's Wayback Machine can be misleading, because a majority of the mementos are actually redirections to the Instagram login page, and of the remaining replayable mementos, many are missing post images. In fact, 96.13% of mementos from the Disinformation Dozen accounts redirect to the login page, and only 27.16% of the remaining replayable mementos contain every post image. Combined, these results reveal that merely 1.05% of mementos for the Disinformation Dozen accounts are replayable with complete post images. Furthermore, we found that the percentage of replayable mementos is decreasing over time, with a particular lack of replayable mementos for the years 2021 and 2022.
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Large-scale Pre-Trained Language Models (PTLMs) capture knowledge from massive human-written data which contains latent societal biases and toxic contents. In this paper, we leverage the primary task of PTLMs, i.e., language modeling, and propose a new metric to quantify manifested implicit representational harms in PTLMs towards 13 marginalized demographics. Using this metric, we conducted an empirical analysis of 24 widely used PTLMs. Our analysis provides insights into the correlation between the proposed metric in this work and other related metrics for representational harm. We observe that our metric correlates with most of the gender-specific metrics in the literature. Through extensive experiments, we explore the connections between PTLMs architectures and representational harms across two dimensions: depth and width of the networks. We found that prioritizing depth over width, mitigates representational harms in some PTLMs. Our code and data can be found at https://github.com/microsoft/SafeNLP.
AttMEMO : Accelerating Transformers with Memoization on Big Memory Systems
Venue: arXiv cs.PF 2023
Year: 2023

Transformer models gain popularity because of their superior inference accuracy and inference throughput. However, the transformer is computation-intensive, causing a long inference time. The existing works on transformer inference acceleration have limitations caused by either the modification of transformer architectures or the need of specialized hardware. In this paper, we identify the opportunities of using memoization to accelerate the self-attention mechanism in transformers without the above limitations. Built upon a unique observation that there is rich similarity in attention computation across inference sequences, we build a memoization database that leverages the emerging big memory system. We introduce a novel embedding technique to find semantically similar inputs to identify computation similarity. We also introduce a series of techniques such as memory mapping and selective memoization to avoid memory copy and unnecessary overhead. We enable 22% inference-latency reduction on average (up to 68%) with negligible loss in inference accuracy.
FInC Flow: Fast and Invertible $k \times k$ Convolutions for Normalizing Flows
Venue: arXiv cs.CV 2023
Year: 2023

Invertible convolutions have been an essential element for building expressive normalizing flow-based generative models since their introduction in Glow. Several attempts have been made to design invertible $k \times k$ convolutions that are efficient in training and sampling passes. Though these attempts have improved the expressivity and sampling efficiency, they severely lagged behind Glow which used only $1 \times 1$ convolutions in terms of sampling time. Also, many of the approaches mask a large number of parameters of the underlying convolution, resulting in lower expressivity on a fixed run-time budget. We propose a $k \times k$ convolutional layer and Deep Normalizing Flow architecture which i.) has a fast parallel inversion algorithm with running time O$(n k^2)$ ($n$ is height and width of the input image and k is kernel size), ii.) masks the minimal amount of learnable parameters in a layer. iii.) gives better forward pass and sampling times comparable to other $k \times k$ convolution-based models on real-world benchmarks. We provide an implementation of the proposed parallel algorithm for sampling using our invertible convolutions on GPUs. Benchmarks on CIFAR-10, ImageNet, and CelebA datasets show comparable performance to previous works regarding bits per dimension while significantly improving the sampling time.
PCBDet: An Efficient Deep Neural Network Object Detection Architecture for Automatic PCB Component Detection on the Edge
Venue: arXiv cs.CV 2023
Year: 2023

There can be numerous electronic components on a given PCB, making the task of visual inspection to detect defects very time-consuming and prone to error, especially at scale. There has thus been significant interest in automatic PCB component detection, particularly leveraging deep learning. However, deep neural networks typically require high computational resources, possibly limiting their feasibility in real-world use cases in manufacturing, which often involve high-volume and high-throughput detection with constrained edge computing resource availability. As a result of an exploration of efficient deep neural network architectures for this use case, we introduce PCBDet, an attention condenser network design that provides state-of-the-art inference throughput while achieving superior PCB component detection performance compared to other state-of-the-art efficient architecture designs. Experimental results show that PCBDet can achieve up to 2$\times$ inference speed-up on an ARM Cortex A72 processor when compared to an EfficientNet-based design while achieving $\sim$2-4\% higher mAP on the FICS-PCB benchmark dataset.
DAG-based Task Orchestration for Edge Computing
Venue: arXiv cs.DC 2023
Year: 2023

As we increase the number of personal computing devices that we carry (mobile devices, tablets, e-readers, and laptops) and these come equipped with increasing resources, there is a vast potential computation power that can be utilized from those devices. Edge computing promises to exploit these underlying computation resources closer to users to help run latency-sensitive applications such as augmented reality and video analytics. However, one key missing piece has been how to incorporate personally owned unmanaged devices into a usable edge computing system. The primary challenges arise due to the heterogeneity, lack of interference management, and unpredictable availability of such devices. In this paper we propose an orchestration framework IBDASH, which orchestrates application tasks on an edge system that comprises a mix of commercial and personal edge devices. IBDASH targets reducing both end-to-end latency of execution and probability of failure for applications that have dependency among tasks, captured by directed acyclic graphs (DAGs). IBDASH takes memory constraints of each edge device and network bandwidth into consideration. To assess the effectiveness of IBDASH, we run real application tasks on real edge devices with widely varying capabilities.We feed these measurements into a simulator that runs IBDASH at scale. Compared to three state-of-the-art edge orchestration schemes, LAVEA, Petrel, and LaTS, and two intuitive baselines, IBDASH reduces the end-to-end latency and probability of failure, by 14% and 41% on average respectively. The main takeaway from our work is that it is feasible to combine personal and commercial devices into a usable edge computing platform, one that delivers low latency and predictable and high availability.