Reviewerly/exHarmony huggingface dataset:
Prob-POS: A Framework for Improving Visual Explanations from Convolutional Neural Networks for Remote Sensing Image Classification

During the past decades, convolutional neural network (CNN)-based models have achieved notable success in remote sensing image classification due to their powerful feature representation ability. However, the lack of explainability during the decision-making process is a common criticism of these high-capacity networks. Local explanation methods that provide visual saliency maps have attracted increasing attention as a means to surmount the barrier of explainability. However, the vast majority of research is conducted on the last convolutional layer, where the salient regions are unintelligible for partial remote sensing images, especially scenes that contain plentiful small targets or are similar to the texture image. To address these issues, we propose a novel framework called Prob-POS, which consists of the class-activation map based on the probe network (Prob-CAM) and the weighted probability of occlusion (wPO) selection strategy. The proposed probe network is a simple but effective architecture to generate elaborate explanation maps and can be applied to any layer of CNNs. The wPO is a quantified metric to evaluate the explanation effectiveness of each layer for different categories to automatically pick out the optimal explanation layer. Variational weights are taken into account to highlight the high-scoring regions in the explanation map. Experimental results on two publicly available datasets and three prevalent networks demonstrate that Prob-POS improves the faithfulness and explainability of CNNs on remote sensing images.
Combined grid and heat conduction optimization for staircase cleaning robot path planning

Staircase climbing, particularly in complex environments of multi-storey buildings, is a challenging task for robotics. By leveraging the similarities between robot path planning and heat conduction, this paper presents a novel path planning to direct with optimal energy consumption the self-reconfigurable staircase cleaning robot called sTetro, able to transform its shape to perform both descent and ascent motions on the staircase. Finding optimal path is modeled as heat traveling between heat source and heat sink through thermal conductive materials. We utilize the temperature gradient on grid-based optimization method to search for optimal paths by minimizing path length and energy consumption. The proposed path planning method successfully applies to six virtual environments with various obstacle placements. Additionally, validation tests on a real sTetro robot in 2 real staircase scenarios show that the energy consumption of the optimized paths from the proposed method is about 21% lower than that from conventional methods.
The feature extraction for classifying words on social media with the Naïve Bayes algorithm

To classify Naïve Bayes classification (NBC), however, it is necessary to have a previous pre-processing and feature extraction. Generally, pre-processing eliminates unnecessary words while feature extraction processes these words. This paper focuses on feature extraction in which calculations and searches are used by applying word2vec while in frequency using term frequency-Inverse document frequency (TF-IDF). The process of classifying words on Twitter with 1734 tweets which are defined as a document to weight the calculation of frequency with TF-IDF with words that often come out in tweet, the value of TF-IDF decreases and vice versa. Following the achievement of the weight value of the word in the tweet, the classification is carried out using Naïve Bayes with 1734 test data, yielding an accuracy of 88.8% in the Slack word category tweet and while in the tweet category of verb 78.79%. It can be concluded that the data in the form of words available on twitter can be classified and those that refer to slack words and verbs with a fairly good level of accuracy. so that it manifests from the habit of twitter social media user.
Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

We consider the problem of formally verifying almost-sure (a.s.) asymptotic stability in discrete-time nonlinear stochastic control systems. While verifying stability in deterministic control systems is extensively studied in the literature, verifying stability in stochastic control systems is an open problem. The few existing works on this topic either consider only specialized forms of stochasticity or make restrictive assumptions on the system, rendering them inapplicable to learning algorithms with neural network policies. In this work, we present an approach for general nonlinear stochastic control problems with two novel aspects: (a) instead of classical stochastic extensions of Lyapunov functions, we use ranking supermartingales (RSMs) to certify a.s. asymptotic stability, and (b) we present a method for learning neural network RSMs. We prove that our approach guarantees a.s. asymptotic stability of the system and provides the first method to obtain bounds on the stabilization time, which stochastic Lyapunov functions do not. Finally, we validate our approach experimentally on a set of nonlinear stochastic reinforcement learning environments with neural network policies.
PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks

Large capacity deep learning models are often prone to a high generalization gap when trained with a limited amount of labeled training data. A recent class of methods to address this problem uses various ways to construct a new training sample by mixing a pair (or more) of training samples. We propose PatchUp, a hidden state block-level regularization technique for Convolutional Neural Networks (CNNs), that is applied on selected contiguous blocks of feature maps from a random pair of samples. Our approach improves the robustness of CNN models against the manifold intrusion problem that may occur in other state-of-the-art mixing approaches. Moreover, since we are mixing the contiguous block of features in the hidden space, which has more dimensions than the input space, we obtain more diverse samples for training towards different dimensions. Our experiments on CIFAR10/100, SVHN, Tiny-ImageNet, and ImageNet using ResNet architectures including PreActResnet18/34, WRN-28-10, ResNet101/152 models show that PatchUp improves upon, or equals, the performance of current state-of-the-art regularizers for CNNs. We also show that PatchUp can provide a better generalization to deformed samples and is more robust against adversarial attacks.
Gradient and Mangitude Based Pruning for Sparse Deep Neural Networks

Deep Neural Networks have memory and computational demands that often render them difficult to use in low-resource environments. Also, highly dense networks are over-parameterized and thus prone to overfitting. To address these problems, we introduce a novel algorithm that prunes (sparsifies) weights from the network by taking into account their magnitudes and gradients taken against a validation dataset. Unlike existing pruning methods, our method does not require the network model to be retrained once initial training is completed. On the CIFAR-10 dataset, our method reduced the number of paramters of MobileNet by a factor of 9X, from 14 million to 1.5 million, with just a 3.8% drop in accuracy.
Light-Weight Seated Posture Guidance System with Machine Learning and Computer Vision

In today's world, the increased time people spend in front of their computers has been one of the main causes for neck and back pains. Especially, since the pandemic, it has been quite evident that slouching at home for long hours on hand-held devices and computers has led many people towards spinal pains and injuries. Backed with scientific research, it has been proven that these pains can be prevented with proper monitoring of the seated posture and taking breaks in between. This paper focuses on building a light-weight end-to-end system that monitors the user's posture and provides feedback whenever it is necessary for them to fix their posture or take a break. Our system utilizes the most common devices: a webcam or a smartphone camera to capture input frames and a machine learning model to differentiate between good and bad postures with 98% accuracy while the user is seated. The newly developed pipeline helps the users in improving their posture without any additional cost or hardware.
A Microring as a Reservoir Computing Node: Memory/Nonlinear Tasks and Effect of Input Non-Ideality

The nonlinear response of an optical microresonator is used in a time multiplexed reservoir computing neural network. Within a virtual node approach combined with an offline training through ridge regression, we solved linear and nonlinear logic operations. We analyzed the nonlinearity of the microresonator as a memory between bits and/or as a neural activation function. This is made possible by controlling both the distance between bits subject to the logical operation and the number of bits supplied to the ridge regression. We show that the optical microresonator exhibits up to two bits of memory in linear tasks and that it allows solving nonlinear tasks providing both memory and nonlinearity. Finally, we demonstrate that the virtual node approach always requires a comparison of the reservoir's performance with the results obtained by applying the same training process on the input signal.
ACLNet: an attention and clustering-based cloud segmentation network

We propose a novel deep learning model named ACLNet, for cloud segmentation from ground images. ACLNet uses both deep neural network and machine learning (ML) algorithm to extract complementary features. Specifically, it uses EfficientNet-B0 as the backbone, "`a trous spatial pyramid pooling" (ASPP) to learn at multiple receptive fields, and "global attention module" (GAM) to extract finegrained details from the image. ACLNet also uses k-means clustering to extract cloud boundaries more precisely. ACLNet is effective for both daytime and nighttime images. It provides lower error rate, higher recall and higher F1-score than state-of-art cloud segmentation models. The source-code of ACLNet is available here: https://github.com/ckmvigil/ACLNet.
Face Mask-Wearing Detection Model Based on Loss Function and Attention Mechanism

Face mask-wearing detection is of great significance for safety protection during the epidemic. Aiming at the problem of low detection accuracy due to the problems of occlusion, complex illumination, and density in mask-wearing detection, this paper proposes a neural network model based on the loss function and attention mechanism for mask-wearing detection in complex environments. Based on YOLOv5s, we first introduce an attention mechanism in the feature fusion process to improve feature utilization, study the effect of different attention mechanisms (CBAM, SE, and CA) on improving deep network models, and then explore the influence of different bounding box loss functions (GIoU, CIoU, and DIoU) on mask-wearing recognition. CIoU is used as the frame regression loss function to improve the positioning accuracy. By collecting 7,958 mask-wearing images and a large number of images of people without masks as a dataset and using YOLOv5s as the benchmark model, the mAP of the model proposed in the paper reached 90.96% on the validation set, which is significantly better than the traditional deep learning method. Mask-wearing detection is carried out in a real environment, and the experimental results of the proposed method can meet the daily detection requirements.
Keyword Localisation in Untranscribed Speech Using Visually Grounded Speech Models

Keyword localisation is the task of finding where in a speech utterance a given query keyword occurs. We investigate to what extent keyword localisation is possible using a visually grounded speech (VGS) model. VGS models are trained on unlabelled images paired with spoken captions. These models are therefore self-supervised—trained without any explicit textual label or location information. To obtain training targets, we first tag training images with soft text labels using a pretrained visual classifier with a fixed vocabulary. This enables a VGS model to predict the presence of a written keyword in an utterance, but not its location. We consider four ways to equip VGS models with localisations capabilities. Two of these—a saliency approach and input masking—can be applied to an arbitrary prediction model after training, while the other two—attention and a score aggregation approach—are incorporated directly into the structure of the model. Masked-based localisation gives some of the best reported localisation scores from a VGS model, with an accuracy of 57% when the system knows that a keyword occurs in an utterance and need to predict its location. In a setting where localisation is performed after detection, an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$F_{1}$</tex-math></inline-formula> of 25% is achieved, and in a setting where a keyword spotting ranking pass is first performed, a localisation <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$P{@}10$</tex-math></inline-formula> of 32% is obtained. While these scores are modest compared to the idealised setting with unordered bag-of-word-supervision (from transcriptions), these VGS models do not receive any textual or location supervision. Further analyses show that these models are limited by the first detection or ranking pass. Moreover, individual keyword localisation performance is correlated with the tagging performance from the visual classifier. We also show qualitatively how and where semantic mistakes occur, e.g. that the model locates <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">surfer</i> when queried with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ocean</i> .
Zero-shot Learning for Grapheme to Phoneme Conversion with Language Ensemble

Grapheme-to-Phoneme (G2P) has many applications in NLP and speech fields.Most existing work focuses heavily on languages with abundant training datasets, which limits the scope of target languages to less than 100 languages.This work attempts to apply zero-shot learning to approximate G2P models for all lowresource and endangered languages in Glottolog (about 8k languages).For any unseen target language, we first build the phylogenetic tree (i.e.language family tree) to identify top-k nearest languages for which we have training sets.Then we run models of those languages to obtain a hypothesis set, which we combine into a confusion network to propose a most likely hypothesis as an approximation to the target language.We test our approach on over 600 unseen languages and demonstrate it significantly outperforms baselines. 1
Irony Detection for Dutch: a Venture into the Implicit

This paper presents the results of a replication experiment for automatic irony detection in Dutch social media text, investigating both a feature-based SVM classifier, as was done by Van Hee et al. ( 2017) and a transformerbased approach.In addition to building a baseline model, an important goal of this research is to explore the implementation of commonsense knowledge in the form of implicit sentiment, as we strongly believe that commonsense and connotative knowledge are essential to the identification of irony and implicit meaning in tweets.We show promising results and how the presented approach can provide a solid baseline and serve as a staging ground to build on in future experiments for irony detection in Dutch.
How You Say It Matters: Measuring the Impact of Verbal Disfluency Tags on Automated Dementia Detection

Automatic speech recognition (ASR) systems usually incorporate postprocessing mechanisms to remove disfluencies, facilitating the generation of clear, fluent transcripts that are conducive to many downstream NLP tasks.However, verbal disfluencies have proved to be predictive of dementia status, although little is known about how various types of verbal disfluencies, nor automatically detected disfluencies, affect predictive performance.We experiment with an off-the-shelf disfluency annotator to tag disfluencies in speech transcripts for a well-known cognitive health assessment task.We evaluate the performance of this model on detecting repetitions and corrections or retracing, and measure the influence of goldannotated versus automatically detected verbal disfluencies on dementia detection through a series of experiments.We find that removing both gold and automatically-detected disfluencies negatively impacts dementia detection performance, degrading classification accuracy by 5.6% and 3% respectively.
A Variational Deep Synthesis Approach for Perception Validation

Abstract This chapter introduces a novel data synthesis framework for validation of perception functions based on machine learning to ensure the safety and functionality of these systems, specifically in the context of automated driving. The main contributions are the introduction of a generative, parametric description of three-dimensional scenarios in a validation parameter space, and layered scene generation process to reduce the computational effort. Specifically, we combine a module for probabilistic scene generation, a variation engine for scene parameters, and a more realistic sensor artifacts simulation. The work demonstrates the effectiveness of the framework for the perception of pedestrians in urban environments based on various deep neural networks (DNNs) for semantic segmentation and object detection. Our approach allows a systematic evaluation of a high number of different objects and combined with our variational approach we can effectively simulate and test a wide range of additional conditions as, e.g., various illuminations. We can demonstrate that our generative approach produces a better approximation of the spatial object distribution to real datasets, compared to hand-crafted 3D scenes.
UKP-SQUARE: An Online Platform for Question Answering Research

Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats (e.g., extractive, abstractive), require different model architectures (e.g., generative, discriminative), and setups (e.g., with or without retrieval).Despite having a large number of powerful, specialized QA pipelines (which we refer to as Skills) that consider a single domain, model or setup, there exists no framework where users can easily explore and compare such pipelines and can extend them according to their needs.To address this issue, we present UKP-SQUARE, an extensible online QA platform for researchers which allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests.In addition, QA researchers can develop, manage, and share their custom Skills using our microservices that support a wide range of models (Transformers, Adapters, ONNX), datastores and retrieval techniques (e.g., sparse and dense).
Leveraging Similar Users for Personalized Language Modeling with Limited Data

Personalized language models are designed and trained to capture language patterns specific to individual users.This makes them more accurate at predicting what a user will write.However, when a new user joins a platform and not enough text is available, it is harder to build effective personalized language models.We propose a solution for this problem, using a model trained on users that are similar to a new user.In this paper, we explore strategies for finding the similarity between new users and existing ones and methods for using the data from existing users who are a good match.We further explore the trade-off between available data for new users and how well their language can be modeled.
Characterization and Implication of Edge WebAssembly Runtimes

WebAssembly, an emerging bytecode format, which is initially developed for partially replacing JavaScript and speeding up browser applications, has been extended to the server-side due to its speed and security promise. It has been considered as a promising alternative to the widely deployed container technique for isolating lightweight applications. To run WebAssmebly from the server-side, aside from the NodeJS runtime, several We-bAssembly native runtimes have been proposed. We characterize major WebAssembly runtimes through extensive applications and metrics. Our results show that different runtimes fit different application scenarios. Based on that, a framework for reducing the startup latency of WebAssembly service while keeping maximum performance is provided. To identify the root causes of the performance gap, the analysis of emerging Cranelift compiler against LLVM in detail is reported. In addition, this paper gives revealing suggestions and architectural proposals for designing an efficient WebAssembly runtime. Our work provides insights on both WebAssembly runtime enhancement and WebAssembly-based cloud service exploitation.
Prime Representing Polynomial

Summary The main purpose of formalization is to prove that the set of prime numbers is diophantine, i.e., is representable by a polynomial formula. We formalize this problem, using the Mizar system [1], [2], in two independent ways, proving the existence of a polynomial without formulating it explicitly as well as with its indication. First, we reuse nearly all the techniques invented to prove the MRDP-theorem [11]. Applying a trick with Mizar schemes that go beyond first-order logic we give a short sophisticated proof for the existence of such a polynomial but without formulating it explicitly. Then we formulate the polynomial proposed in [6] that has 26 variables in the Mizar language as follows ( w · z + h + j − q ) 2 +(( g · k + g + k )·( h + j )+ h − z ) 2 +(2 · k 3 ·(2· k +2)·( n + 1) 2 +1− f 2 ) 2 + ( p + q + z + 2 · n − e ) 2 + ( e 3 · ( e + 2) · ( a + 1) 2 + 1 − o 2 ) 2 + ( x 2 − ( a 2 −′ 1) · y 2 − 1) 2 + (16 · ( a 2 − 1) · r 2 · y 2 · y 2 + 1 − u 2 ) 2 + ((( a + u 2 · ( u 2 − a )) 2 − 1) · ( n + 4 · d · y ) 2 + 1 − ( x + c · u ) 2 ) 2 + ( m 2 − ( a 2 −′ 1) · l 2 − 1) 2 + ( k + i · ( a − 1) − l ) 2 + ( n + l + v − y ) 2 + ( p + l · ( a − n − 1) + b · (2 · a · ( n + 1) − ( n + 1) 2 − 1) − m ) 2 + ( q + y · ( a − p − 1) + s · (2 · a · ( p + 1) − ( p + 1) 2 − 1) − x ) 2 + ( z + p · l · ( a − p ) + t · (2 · a · p − p 2 − 1) − p · m ) 2 and we prove that that for any positive integer k so that k + 1 is prime it is necessary and sufficient that there exist other natural variables a - z for which the polynomial equals zero. 26 variables is not the best known result in relation to the set of prime numbers, since any diophantine equation over ℕ can be reduced to one in 13 unknowns [8] or even less [5], [13]. The best currently known result for all prime numbers, where the polynomial is explicitly constructed is 10 [7] or even 7 in the case of Fermat as well as Mersenne prime number [4]. We are currently focusing our formalization efforts in this direction.