OpenRaiser/Intern-Atlas huggingface dataset:
Salient Object Detection by LTP Texture Characterization on Opposing Color Pairs under SLICO Superpixel Constraint
Venue: arXiv cs.CV 2022
Year: 2022

The effortless detection of salient objects by humans has been the subject of research in several fields, including computer vision as it has many applications. However, salient object detection remains a challenge for many computer models dealing with color and textured images. Herein, we propose a novel and efficient strategy, through a simple model, almost without internal parameters, which generates a robust saliency map for a natural image. This strategy consists of integrating color information into local textural patterns to characterize a color micro-texture. Most models in the literature that use the color and texture features treat them separately. In our case, it is the simple, yet powerful LTP (Local Ternary Patterns) texture descriptor applied to opposing color pairs of a color space that allows us to achieve this end. Each color micro-texture is represented by vector whose components are from a superpixel obtained by SLICO (Simple Linear Iterative Clustering with zero parameter) algorithm which is simple, fast and exhibits state-of-the-art boundary adherence. The degree of dissimilarity between each pair of color micro-texture is computed by the FastMap method, a fast version of MDS (Multi-dimensional Scaling), that considers the color micro-textures non-linearity while preserving their distances. These degrees of dissimilarity give us an intermediate saliency map for each RGB, HSL, LUV and CMY color spaces. The final saliency map is their combination to take advantage of the strength of each of them. The MAE (Mean Absolute Error) and F$_β$ measures of our saliency maps, on the complex ECSSD dataset show that our model is both simple and efficient, outperforming several state-of-the-art models.
CaFT: Clustering and Filter on Tokens of Transformer for Weakly Supervised Object Localization
Venue: arXiv cs.CV 2022
Year: 2022

Weakly supervised object localization (WSOL) is a challenging task to localize the object by only category labels. However, there is contradiction between classification and localization because accurate classification network tends to pay attention to discriminative region of objects rather than the entirety. We propose this discrimination is caused by handcraft threshold choosing in CAM-based methods. Therefore, we propose Clustering and Filter of Tokens (CaFT) with Vision Transformer (ViT) backbone to solve this problem in another way. CaFT first sends the patch tokens of the image split to ViT and cluster the output tokens to generate initial mask of the object. Secondly, CaFT considers the initial mask as pseudo labels to train a shallow convolution head (Attention Filter, AtF) following backbone to directly extract the mask from tokens. Then, CaFT splits the image into parts, outputs masks respectively and merges them into one refined mask. Finally, a new AtF is trained on the refined masks and used to predict the box of object. Experiments verify that CaFT outperforms previous work and achieves 97.55\% and 69.86\% localization accuracy with ground-truth class on CUB-200 and ImageNet-1K respectively. CaFT provides a fresh way to think about the WSOL task.
let (rec) insertion without Effects, Lights or Magic
Venue: arXiv cs.PL 2022
Year: 2022

Let insertion in program generation is producing code with definitions (let-statements). Although definitions precede uses in generated code, during code generation `uses' come first: we might not even know a definition is needed until we encounter a reoccurring expression. Definitions are thus generated `in hindsight', which explains why this process is difficult to understand and implement -- even more so for parameterized, recursive and mutually recursive definitions. We have earlier presented an interface for let(rec) insertion -- i.e. for generating (mutually recursive) definitions. We demonstrated its expressiveness and applications, but not its implementation, which relied on effects and compiler magic. We now show how one can understand let insertion, and hence implement it in plain OCaml. We give the first denotational semantics of let(rec)-insertion, which does not rely on any effects at all. The formalization has guided the implementation of let(rec) insertion in the current version of MetaOCaml.
The Area Method in the Wolfram Language
Venue: arXiv cs.LO 2022
Year: 2022

The area method is a decision procedure for geometry developed by Chou et al. in the 1990's. The method aims to reduce the specified hypothesis to an algebraically verifiable form by applying elimination lemmas. The order in which the lemmas are applied is determined by the stated conjecture and the underlying geometric construction. In this paper we present our implementation of the area method for Euclidean geometry as a stand-alone Mathematica package.
Efficient Enumeration Algorithms for Annotated Grammars
Venue: arXiv cs.FL 2022
Year: 2022

We introduce annotated grammars, an extension of context-free grammars which allows annotations on terminals. Our model extends the standard notion of regular spanners, and is more expressive than the extraction grammars recently introduced by Peterfreund. We study the enumeration problem for annotated grammars: fixing a grammar, and given a string as input, enumerate all annotations of the string that form a word derivable from the grammar. Our first result is an algorithm for unambiguous annotated grammars, which preprocesses the input string in cubic time and enumerates all annotations with output-linear delay. This improves over Peterfreund's result, which needs quintic time preprocessing to achieve this delay bound. We then study how we can reduce the preprocessing time while keeping the same delay bound, by making additional assumptions on the grammar. Specifically, we present a class of grammars which only have one derivation shape for all outputs, for which we can enumerate with quadratic time preprocessing. We also give classes that generalize regular spanners for which linear time preprocessing suffices.
Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models
Venue: arXiv cs.CL 2022
Year: 2022

We perform knowledge distillation (KD) benchmark from task-specific BERT-base teacher models to various student models: BiLSTM, CNN, BERT-Tiny, BERT-Mini, and BERT-Small. Our experiment involves 12 datasets grouped in two tasks: text classification and sequence labeling in the Indonesian language. We also compare various aspects of distillations including the usage of word embeddings and unlabeled data augmentation. Our experiments show that, despite the rising popularity of Transformer-based models, using BiLSTM and CNN student models provide the best trade-off between performance and computational resource (CPU, RAM, and storage) compared to pruned BERT models. We further propose some quick wins on performing KD to produce small NLP models via efficient KD training mechanisms involving simple choices of loss functions, word embeddings, and unlabeled data preparation.
Enabling Verification of Deep Neural Networks in Perception Tasks Using Fuzzy Logic and Concept Embeddings
Venue: arXiv cs.CV 2022
Year: 2022

One major drawback of deep convolutional neural networks (CNNs) for use in safety critical applications is their black-box nature. This makes it hard to verify or monitor complex, symbolic requirements on already trained computer vision CNNs. In this work, we present a simple, yet effective, approach to verify that a CNN complies with symbolic predicate logic rules which relate visual concepts. It is the first that (1) does not modify the CNN, (2) may use visual concepts that are no CNN in- or output feature, and (3) can leverage continuous CNN confidence outputs. To achieve this, we newly combine methods from explainable artificial intelligence and logic: First, using supervised concept embedding analysis, the output of a CNN is post-hoc enriched by concept outputs. Second, rules from prior knowledge are modelled as truth functions that accept the CNN outputs, and can be evaluated with little computational overhead. We here investigate the use of fuzzy logic, i.e., continuous truth values, and of proper output calibration, which both theoretically and practically show slight benefits. Applicability is demonstrated on state-of-the-art object detectors for three verification use-cases, where monitoring of rule breaches can reveal detection errors.
Toxicity Detection for Indic Multilingual Social Media Content
Venue: arXiv cs.CL 2022
Year: 2022

Toxic content is one of the most critical issues for social media platforms today. India alone had 518 million social media users in 2020. In order to provide a good experience to content creators and their audience, it is crucial to flag toxic comments and the users who post that. But the big challenge is identifying toxicity in low resource Indic languages because of the presence of multiple representations of the same text. Moreover, the posts/comments on social media do not adhere to a particular format, grammar or sentence structure; this makes the task of abuse detection even more challenging for multilingual social media platforms. This paper describes the system proposed by team 'Moj Masti' using the data provided by ShareChat/Moj in \emph{IIIT-D Multilingual Abusive Comment Identification} challenge. We focus on how we can leverage multilingual transformer based pre-trained and fine-tuned models to approach code-mixed/code-switched classification tasks. Our best performing system was an ensemble of XLM-RoBERTa and MuRIL which achieved a Mean F-1 score of 0.9 on the test data/leaderboard. We also observed an increase in the performance by adding transliterated data. Furthermore, using weak metadata, ensembling and some post-processing techniques boosted the performance of our system, thereby placing us 1st on the leaderboard.
SAE: Sequential Anchored Ensembles
Venue: arXiv cs.LG 2022
Year: 2022

Computing the Bayesian posterior of a neural network is a challenging task due to the high-dimensionality of the parameter space. Anchored ensembles approximate the posterior by training an ensemble of neural networks on anchored losses designed for the optima to follow the Bayesian posterior. Training an ensemble, however, becomes computationally expensive as its number of members grows since the full training procedure is repeated for each member. In this note, we present Sequential Anchored Ensembles (SAE), a lightweight alternative to anchored ensembles. Instead of training each member of the ensemble from scratch, the members are trained sequentially on losses sampled with high auto-correlation, hence enabling fast convergence of the neural networks and efficient approximation of the Bayesian posterior. SAE outperform anchored ensembles, for a given computational budget, on some benchmarks while showing comparable performance on the others and achieved 2nd and 3rd place in the light and extended tracks of the NeurIPS 2021 Approximate Inference in Bayesian Deep Learning competition.
Characterizing the Computational and Memory Requirements of Virtual RANs
Venue: arXiv cs.NI 2022
Year: 2022

The virtualization of radio access networks (RANs) is emerging as a key component of future wireless systems, as it brings agility to the RAN architecture and offers degrees of design freedom. In this paper, we investigate and characterize the computational and memory requirements of virtual RANs. To this end, we build a virtual RAN test-bed leveraging the srsRAN open-source mobile communication platform and general-purpose processor-based servers. Through extensive experiments, we profile the consumption of computing and memory resources, and we assess the system performance. Further, we build regression models to predict the system behavior as the number of connected users increases, under diverse radio transmission settings. In so doing, we develop a methodology and prediction models that can help designing and optimizing virtual RANs.
Modeling Associative Reasoning Processes
Venue: arXiv cs.AI 2022
Year: 2022

The human capability to reason about one domain by using knowledge of other domains has been researched for more than 50 years, but models that are formally sound and predict cognitive process are sparse. We propose a formally sound method that models associative reasoning by adapting logical reasoning mechanisms. In particular it is shown that the combination with large commensense knowledge within a single reasoning system demands for an efficient and powerful association technique. This approach is also used for modelling mind-wandering and the Remote Associates Test (RAT) for testing creativity. In a general discussion we show implications of the model for a broad variety of cognitive phenomena including consciousness.
A Cluster-Based Trip Prediction Graph Neural Network Model for Bike Sharing Systems
Venue: arXiv cs.LG 2022
Year: 2022

Bike Sharing Systems (BSSs) are emerging as an innovative transportation service. Ensuring the proper functioning of a BSS is crucial given that these systems are committed to eradicating many of the current global concerns, by promoting environmental and economic sustainability and contributing to improving the life quality of the population. Good knowledge of users' transition patterns is a decisive contribution to the quality and operability of the service. The analogous and unbalanced users' transition patterns cause these systems to suffer from bicycle imbalance, leading to a drastic customer loss in the long term. Strategies for bicycle rebalancing become important to tackle this problem and for this, bicycle traffic prediction is essential, as it allows to operate more efficiently and to react in advance. In this work, we propose a bicycle trips predictor based on Graph Neural Network embeddings, taking into consideration station groupings, meteorology conditions, geographical distances, and trip patterns. We evaluated our approach in the New York City BSS (CitiBike) data and compared it with four baselines, including the non-clustered approach. To address our problem's specificities, we developed the Adaptive Transition Constraint Clustering Plus (AdaTC+) algorithm, eliminating shortcomings of previous work. Our experiments evidence the clustering pertinence (88% accuracy compared with 83% without clustering) and which clustering technique best suits this problem. Accuracy on the Link Prediction task is always higher for AdaTC+ than benchmark clustering methods when the stations are the same, while not degrading performance when the network is upgraded, in a mismatch with the trained model.
Exception-Driven Fault Localization for Automated Program Repair
Venue: arXiv cs.SE 2022
Year: 2022

Automated Program Repair (APR) techniques typically exploit spectrum-based fault localization (SBFL) to identify the program locations that should be patched, making the effectiveness of APR techniques dependent on the effectiveness of fault localization. Indeed, results show that SBFL often does not localize faults accurately, hindering the effectiveness of APR. In this paper, we propose EXCEPT, a technique that addresses the localization problem by focusing on the semantics of failures rather than on the correlation between the executed statements and the failed tests, as SBFL does. We focus on failures due to exceptions and we exploit their type and source to localize and guess the faults. Experiments with 43 exception-raising faults from the Defects4J benchmark show that EXCEPT can perform better than Ochiai and ssFix.
A Case Study Analysis for Designing a Lunar Navigation Satellite System with Time-Transfer from Earth-GPS
Venue: arXiv cs.RO 2022
Year: 2022

Recently, there has been a growing interest in the use of a SmallSat platform for the future Lunar Navigation Satellite System (LNSS) to allow for cost-effectiveness and rapid deployment. However, many design choices are yet to be finalized for the SmallSat-based LNSS, including the onboard clock and the orbit type. As compared to the legacy Earth-GPS, designing an LNSS poses unique challenges: (a) restricted Size, Weight, and Power (SWaP) of the onboard clock, which limits the timing stability; (b) limited lunar ground monitoring stations, which engenders a greater preference toward stable LNSS satellite orbits. In this current work, we analyze the trade-off between different design considerations related to the onboard clock and the lunar orbit type for designing an LNSS with time-transfer from Earth-GPS. Our proposed time-transfer architecture combines the intermittently available Earth-GPS signals in a timing filter to alleviate the cost and SWaP requirements of the onboard clocks. Specifically, we conduct multiple case studies with different grades of low-SWaP clocks and various previously studied lunar orbit types. We estimate the lunar User Equivalent Range Error (UERE) metric to characterize the ranging accuracy of signals transmitted from an LNSS satellite. Using the Systems Tool Kit (STK)-based simulation setup from Analytical Graphics, Inc. (AGI), we evaluate the lunar UERE across various case studies of the LNSS design to demonstrate comparable performance as that of the legacy Earth-GPS, even while using a low-SWaP onboard clock. We further perform sensitivity analysis to investigate the variation in the lunar UERE metric across different case studies as the Earth-GPS measurement update rates are varied.
DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
Venue: arXiv cs.CR 2022
Year: 2022

Federated Learning (FL) allows multiple clients to collaboratively train a Neural Network (NN) model on their private data without revealing the data. Recently, several targeted poisoning attacks against FL have been introduced. These attacks inject a backdoor into the resulting model that allows adversary-controlled inputs to be misclassified. Existing countermeasures against backdoor attacks are inefficient and often merely aim to exclude deviating models from the aggregation. However, this approach also removes benign models of clients with deviating data distributions, causing the aggregated model to perform poorly for such clients. To address this problem, we propose DeepSight, a novel model filtering approach for mitigating backdoor attacks. It is based on three novel techniques that allow to characterize the distribution of data used to train model updates and seek to measure fine-grained differences in the internal structure and outputs of NNs. Using these techniques, DeepSight can identify suspicious model updates. We also develop a scheme that can accurately cluster model updates. Combining the results of both components, DeepSight is able to identify and eliminate model clusters containing poisoned models with high attack impact. We also show that the backdoor contributions of possibly undetected poisoned models can be effectively mitigated with existing weight clipping-based defenses. We evaluate the performance and effectiveness of DeepSight and show that it can mitigate state-of-the-art backdoor attacks with a negligible impact on the model's performance on benign data.
AI & Racial Equity: Understanding Sentiment Analysis Artificial Intelligence, Data Security, and Systemic Theory in Criminal Justice Systems
Venue: arXiv cs.CY 2022
Year: 2022

Various forms of implications of artificial intelligence that either exacerbate or decrease racial systemic injustice have been explored in this applied research endeavor. Taking each thematic area of identifying, analyzing, and debating an systemic issue have been leveraged in investigating merits and drawbacks of using algorithms to automate human decision making in racially sensitive environments. It has been asserted through the analysis of historical systemic patterns, implicit biases, existing algorithmic risks, and legal implications that natural language processing based AI, such as risk assessment tools, have racially disparate outcomes. It is concluded that more litigative policies are needed to regulate and restrict how internal government institutions and corporations utilize algorithms, privacy and security risks, and auditing requirements in order to diverge from racially injustice outcomes and practices of the past.
A Survey on DNS Encryption: Current Development, Malware Misuse, and Inference Techniques
Venue: arXiv cs.CR 2022
Year: 2022

The domain name system (DNS) that maps alphabetic names to numeric Internet Protocol (IP) addresses plays a foundational role for Internet communications. By default, DNS queries and responses are exchanged in unencrypted plaintext, and hence, can be read and/or hijacked by third parties. To protect user privacy, the networking community has proposed standard encryption technologies such as DNS over TLS (DoT), DNS over HTTPS (DoH), and DNS over QUIC (DoQ) for DNS communications, enabling clients to perform secure and private domain name lookups. We survey the DNS encryption literature published since 2016, focusing on its current landscape and how it is misused by malware, and highlighting the existing techniques developed to make inferences from encrypted DNS traffic. First, we provide an overview of various standards developed in the space of DNS encryption and their adoption status, performance, benefits, and security issues. Second, we highlight ways that various malware families can exploit DNS encryption to their advantage for botnet communications and/or data exfiltration. Third, we discuss existing inference methods for profiling normal patterns and/or detecting malicious encrypted DNS traffic. Several directions are presented to motivate future research in enhancing the performance and security of DNS encryption.
Waveform Design Using Half-duplex Devices for 6G Joint Communications and Sensing
Venue: arXiv cs.IT 2022
Year: 2022

Joint communications and sensing is a promising 6G technology, and the challenge is how to integrate them efficiently. Existing frequency-division and time-division coexistence can hardly bring a gain of integration. Directly using orthogonal frequency-division multiplexing (OFDM) to sense requires complex in-band full-duplex to cancel the selfinterference (SI). To solve these problems, this paper proposes novel coexistence schemes to gain super sensing range (SSR) and simple SI cancellation. SSR enables JCS to gain a sensing range of a sensing-only scheme and shares the resources with communications. Random time-division is proposed to gain a super Doppler range. Flexible sensing implanted OFDM (FSIOFDM) is also proposed. FSI-OFDM uses random sensing occasions to gain super Doppler range, as well as utilizes the fixed tail sensing occasions to achieve supper distance range. The simulation results show that the proposed schemes can gain SSR with limited resources.
Interactive Attention AI to translate low light photos to captions for night scene understanding in women safety
Venue: arXiv cs.CV 2022
Year: 2022

There is amazing progress in Deep Learning based models for Image captioning and Low Light image enhancement. For the first time in literature, this paper develops a Deep Learning model that translates night scenes to sentences, opening new possibilities for AI applications in the safety of visually impaired women. Inspired by Image Captioning and Visual Question Answering, a novel Interactive Image Captioning is developed. A user can make the AI focus on any chosen person of interest by influencing the attention scoring. Attention context vectors are computed from CNN feature vectors and user-provided start word. The Encoder-Attention-Decoder neural network learns to produce captions from low brightness images. This paper demonstrates how women safety can be enabled by researching a novel AI capability in the Interactive Vision-Language model for perception of the environment in the night.