Reviewerly/exHarmony huggingface dataset:
Convolutional Extreme Learning Machines: A Systematic Review

Much work has recently identified the need to combine deep learning with extreme learning in order to strike a performance balance with accuracy, especially in the domain of multimedia applications. When considering this new paradigm—namely, the convolutional extreme learning machine (CELM)—we present a systematic review that investigates alternative deep learning architectures that use the extreme learning machine (ELM) for faster training to solve problems that are based on image analysis. We detail each of the architectures that are found in the literature along with their application scenarios, benchmark datasets, main results, and advantages, and then present the open challenges for CELM. We followed a well-structured methodology and established relevant research questions that guided our findings. Based on 81 primary studies, we found that object recognition is the most common problem that is solved by CELM, and CCN with predefined kernels is the most common CELM architecture proposed in the literature. The results from experiments show that CELM models present good precision, convergence, and computational performance, and they are able to decrease the total processing time that is required by the learning process. The results presented in this systematic review are expected to contribute to the research area of CELM, providing a good starting point for dealing with some of the current problems in the analysis of computer vision based on images.
A brain-inspired computational model for spatio-temporal information processing

Spatio-temporal information processing is fundamental in both brain functions and AI applications. Current strategies for spatio-temporal pattern recognition usually involve explicit feature extraction followed by feature aggregation, which requires a large amount of labeled data. In the present study, motivated by the subcortical visual pathway and early stages of the auditory pathway for motion and sound processing, we propose a novel brain-inspired computational model for generic spatio-temporal pattern recognition. The model consists of two modules, a reservoir module and a decision-making module. The former projects complex spatio-temporal patterns into spatially separated neural representations via its recurrent dynamics, the latter reads out neural representations via integrating information over time, and the two modules are linked together using known examples. Using synthetic data, we demonstrate that the model can extract the frequency and order information of temporal inputs. We apply the model to reproduce the looming pattern discrimination behavior as observed in experiments successfully. Furthermore, we apply the model to the gait recognition task, and demonstrate that our model accomplishes the recognition in an event-based manner and outperforms deep learning counterparts when training data is limited.
LITERATURE REVIEW ON THEORIES MOTIVATION

People possess something that machines don’t-human spirit and inspiration. It is the spirit and inspiration that leads associates to apply their human capital to meet the goals of the organization, the human spirit is highly inspired through motivation. Motivation is essential to almost any aspect of life. Understanding job satisfaction and work motivation can be key elements to improving productivity. This paper seeks to introduce the four major need theories areas: Maslow’s Hierarchy of Needs, Herzberg’s Motivation Theory, Alderfer’s ERG Theory, and McClelland’s Need for Achievement Theory. It also goes further to analyze these theories bringing to the fore various similarities and differences that exist amongst them.
Model of Evaluation and Selection of Expert Group Members for Smart Cities, Green Transportation and Mobility: From Safe Times to Pandemic Times

This paper presents the development of technologies to support the decision-making of local government executives and smart city concept managers in selecting and evaluating the competencies of new members for advisory groups for solving problems that are implemented in safe times in individual areas or in crises, such as pandemics. The reason for developing effective urban transformation strategies and for the transparent selection of independent experts (non-politicians) for policymaking, decision-making, and implementation teams is not only the heterogeneity of smart city dimensions together with the necessary complexity and systems approach, but also the nature of the capacities and tools needed for smart city concepts. The innovative hybrid competency assessment model is based on fuzzy logic and a network for neuro-fuzzy assessment. It is a technological model for evaluating the competencies of specialists, taking into account the influence of human factors on the processes of personnel selection and system management. An innovative web platform named “Smart City Concept Personnel Selection” has been designed, which can be adapted to various users of municipalities or regional institutions for the transparent selection of qualified personnel for effective decision-making and the use of public funds during safe times or emergencies, such as the COVID-19 pandemic.
Discourse Probing of Pretrained Language Models

Existing work on probing of pretrained language models (LMs) has predominantly focused on sentence-level syntactic tasks.In this paper, we introduce document-level discourse probing to evaluate the ability of pretrained LMs to capture document-level relations.We experiment with 7 pretrained LMs, 4 languages, and 7 discourse probing tasks, and find BART to be overall the best model at capturing discourse -but only in its encoder, with BERT performing surprisingly well as the baseline model.Across the different models, there are substantial differences in which layers best capture discourse information, and large disparities between models.
A Novel Explainable Deep Belief Network Framework and Its Application for Feature Importance Analysis

Feature analysis and selection are highly considered topics in deep learning (DL) real-world applications. However, most existing methods are manual and lack of deep insights of training mechanisms. This is because DL is often viewed as a “black box” and the mechanisms providing the output are often hidden from the user and difficult to understand. Some scientists have utilized visualization, sensitivity analysis, and adversary attack machine learning to increase transparency and have demonstrated successful methods in understanding DL, especially related to convolutional neural nets. This paper builds on these methods and focuses on deep belief networks (DBNs), with two training stages: unsupervised learning and supervised learning. First, a novel algorithm named Visual Input-neuron Importance (Vi-II), based on visualization and feature importance criterion are proposed, to calculate changes in the importance of the input features. Second, a criterion named Visual Hidden-layer Importance (Vi-HI) is proposed to dynamically display the contributions of each hidden layer. Third, a novel framework is put forward by combing the two techniques together, to determine the final structure (input and hidden layers) of DBN, for both unsupervised training and supervised training stages. Then, an application based on the analysis of a road safety performance function is demonstrated. The proposed method provides an accurate description of the model’s inner workings, identifies significant features, and eliminates irrelevant features. At last, the revised dataset and optimized model structure are used for car collision prediction; the result demonstrates that the revised model achieves much better performance than comparable methods.
Non-Autoregressive Translation by Learning Target Categorical Codes

Non-autoregressive Transformer is a promising text generation model.However, current non-autoregressive models still fall behind their autoregressive counterparts in translation quality.We attribute this accuracy gap to the lack of dependency modeling among decoder inputs.In this paper, we propose CNAT, which learns implicitly categorical codes as latent variables into the non-autoregressive decoding.The interaction among these categorical codes remedies the missing dependencies and improves the model capacity.Experiment results show that our model achieves comparable or better performance in machine translation tasks than several strong baselines.
CoMeT: Towards Code-Mixed Translation Using Parallel Monolingual Sentences

Code-mixed languages are very popular in multilingual societies around the world, yet the resources lag behind to enable robust systems on such languages.A major contributing factor is the informal nature of these languages which makes it difficult to collect codemixed data.In this paper, we propose our system for Task 1 of CACLS 2021 1 to generate a machine translation system for English to Hinglish in a supervised setting.Translating in the given direction can help expand the set of resources for several tasks by translating valuable datasets from high resource languages.We propose to use mBART, a pre-trained multilingual sequence-to-sequence model, and fully utilize the pre-training of the model by transliterating the roman Hindi words in the code-mixed sentences to Devanagri script.We evaluate how expanding the input by concatenating Hindi translations of the English sentences improves mBART's performance.Our system gives a BLEU score of 12.22 on test set.Further, we perform a detailed error analysis of our proposed systems and explore the limitations of the provided dataset and metrics.
Security Threat Modelling With Bayesian Networks and Sensitivity Analysis for IAAS Virtualization Stack

Designing security mechanisms for cloud computing infrastructures has assumed importance with the widespread adoption of public clouds. Virtualization security is a crucial component of the overall cloud infrastructure security. In this article, the authors employ the concept of Bayesian networks and attack graphs to carry out sensitivity analysis on the different components involved in virtualization security for infrastructure as a service (IaaS) cloud infrastructures. They evaluate the Bayesian attack graph (BAG) for the IaaS model to reveal the sensitive regions and thus help the administrators to secure the high risk components in the stack. They present a formal definition of the sensitivity analysis and then evaluate using the BAG model for IaaS stack. The model and analysis presented here can also be used by security analysts and designers to make a selection of the security solutions based on the risk profile of vulnerable nodes and the corresponding cost involved in adding a defense against the identified vulnerabilities.
Robust Transfer Learning with Pretrained Language Models through Adapters

Transfer learning with large pretrained transformer-based language models like BERT has become a dominating approach for most NLP tasks.Simply fine-tuning those large language models on downstream tasks or combining it with task-specific pretraining is often not robust.In particular, the performance considerably varies as the random seed changes or the number of pretraining and/or fine-tuning iterations varies, and the fine-tuned model is vulnerable to adversarial attack.We propose a simple yet effective adapter-based approach to mitigate these issues.Specifically, we insert small bottleneck layers (i.e., adapter) within each layer of a pretrained model, then fix the pretrained layers and train the adapter layers on the downstream task data, with (1) task-specific unsupervised pretraining and then (2) task-specific supervised training (e.g., classification, sequence labeling).Our experiments demonstrate that such a training scheme leads to improved stability and adversarial robustness in transfer learning to various downstream tasks.
Improved Salp Swarm Algorithm with Simulated Annealing for Solving Engineering Optimization Problems

Swarm-based algorithm can successfully avoid the local optimal constraints, thus achieving a smooth balance between exploration and exploitation. Salp swarm algorithm (SSA), as a swarm-based algorithm on account of the predation behavior of the salp, can solve complex daily life optimization problems in nature. SSA also has the problems of local stagnation and slow convergence rate. This paper introduces an improved salp swarm algorithm, which improve the SSA by using the chaotic sequence initialization strategy and symmetric adaptive population division. Moreover, a simulated annealing mechanism based on symmetric perturbation is introduced to enhance the local jumping ability of the algorithm. The improved algorithm is referred to SASSA. The CEC standard benchmark functions are used to evaluate the efficiency of the SASSA and the results demonstrate that the SASSA has better global search capability. SASSA is also applied to solve engineering optimization problems. The experimental results demonstrate that the exploratory and exploitative proclivities of the proposed algorithm and its convergence patterns are vividly improved.
Strudel: Learning Structured-Decomposable Probabilistic Circuits.

Probabilistic circuits (PCs) represent a probability distribution as a computational graph. Enforcing structural properties on these graphs guarantees that several inference scenarios become tractable. Among these properties, structured decomposability is a particularly appealing one: it enables the efficient and exact computations of the probability of complex logical formulas, and can be used to reason about the expected output of certain predictive models under missing data. This paper proposes Strudel, a simple, fast and accurate learning algorithm for structured-decomposable PCs. Compared to prior work for learning structured-decomposable PCs, Strudel delivers more accurate single PC models in fewer iterations, and dramatically scales learning when building ensembles of PCs. It achieves this scalability by exploiting another structural property of PCs, called determinism, and by sharing the same computational graph across mixture components. We show these advantages on standard density estimation benchmarks and challenging inference scenarios.
Balancing Straight-line Programs

We show that a context-free grammar of size <?TeX $m$?> that produces a single string <?TeX $w$?> of length <?TeX $n$?> (such a grammar is also called a string straight-line program) can be transformed in linear time into a context-free grammar for <?TeX $w$?> of size <?TeX $\mathcal {O}(m)$?> , whose unique derivation tree has depth <?TeX $\mathcal {O}(\log n)$?> . This solves an open problem in the area of grammar-based compression, improves many results in this area, and greatly simplifies many existing constructions. Similar results are shown for two formalisms for grammar-based tree compression: top dags and forest straight-line programs. These balancing results can be all deduced from a single meta-theorem stating that the depth of an algebraic circuit over an algebra with a certain finite base property can be reduced to <?TeX $\mathcal {O}(\log n)$?> with the cost of a constant multiplicative size increase. Here, <?TeX $n$?> refers to the size of the unfolding (or unravelling) of the circuit. In particular, this results applies to standard arithmetic circuits over (noncommutative) semirings.
A Framework for Continuous Authentication Based on Touch Dynamics Biometrics for Mobile Banking Applications

As smart devices have become commonly used to access internet banking applications, these devices constitute appealing targets for fraudsters. Impersonation attacks are an essential concern for internet banking providers. Therefore, user authentication countermeasures based on biometrics, whether physiological or behavioral, have been developed, including those based on touch dynamics biometrics. These measures take into account the unique behavior of a person when interacting with touchscreen devices, thus hindering identitification fraud because it is hard to impersonate natural user behaviors. Behavioral biometric measures also balance security and usability because they are important for human interfaces, thus requiring a measurement process that may be transparent to the user. This paper proposes an improvement to Biotouch, a supervised Machine Learning-based framework for continuous user authentication. The contributions of the proposal comprise the utilization of multiple scopes to create more resilient reasoning models and their respective datasets for the improved Biotouch framework. Another contribution highlighted is the testing of these models to evaluate the imposter False Acceptance Error (FAR). This proposal also improves the flow of data and computation within the improved framework. An evaluation of the multiple scope model proposed provides results between 90.68% and 97.05% for the harmonic mean between recall and precision (F1 Score). The percentages of unduly authenticated imposters and errors of legitimate user rejection (Equal Error Rate (EER)) are between 9.85% and 1.88% for static verification, login, user dynamics, and post-login. These results indicate the feasibility of the continuous multiple-scope authentication framework proposed as an effective layer of security for banking applications, eventually operating jointly with conventional measures such as password-based authentication.
An Investigation of the (In)effectiveness of Counterfactually Augmented Data

While pretrained language models achieve excellent performance on natural language understanding benchmarks, they tend to rely on spurious correlations and generalize poorly to out-of-distribution (OOD) data.Recent work has explored using counterfactuallyaugmented data (CAD)-data generated by minimally perturbing examples to flip the ground-truth label-to identify robust features that are invariant under distribution shift.However, empirical results using CAD during training for OOD generalization have been mixed.To explain this discrepancy, through a toy theoretical example and empirical analysis on two crowdsourced CAD datasets, we show that: (a) while features perturbed in CAD are indeed robust features, it may prevent the model from learning unperturbed robust features; and (b) CAD may exacerbate existing spurious correlations in the data.Our results thus show that the lack of perturbation diversity limits CAD's effectiveness on OOD generalization, calling for innovative crowdsourcing procedures to elicit diverse perturbation of examples.Premise: The lady is standing next to her two children who are eating a pizza.Original Hypothesis: The two children near the lady are eating something.(Entailment) Revised Hypothesis: The two children near the lady are drinking something.(Contradiction) Premise: The lady is standing next to her two children who are eating a pizza.Original Hypothesis: The two children near the lady are eating something.
UAV Path Planning for Reconnaissance and Look-Ahead Coverage Support for Mobile Ground Vehicles

Path planning of unmanned aerial vehicles (UAVs) for reconnaissance and look-ahead coverage support for mobile ground vehicles (MGVs) is a challenging task due to many unknowns being imposed by the MGVs’ variable velocity profiles, change in heading, and structural differences between the ground and air environments. Few path planning techniques have been reported in the literature for multirotor UAVs that autonomously follow and support MGVs in reconnaissance missions. These techniques formulate the path planning problem as a tracking problem utilizing gimbal sensors to overcome the coverage and reconnaissance complexities. Despite their lack of considering additional objectives such as reconnaissance coverage and dynamic environments, they retain several drawbacks, including high computational requirements, hardware dependency, and low performance when the MGV has varying velocities. In this study, a novel 3D path planning technique for multirotor UAVs is presented, the enhanced dynamic artificial potential field (ED-APF), where path planning is formulated as both a follow and cover problem with nongimbal sensors. The proposed technique adopts a vertical sinusoidal path for the UAV that adapts relative to the MGV’s position and velocity, guided by the MGV’s heading for reconnaissance and exploration of areas and routes ahead beyond the MGV sensors’ range, thus extending the MGV’s reconnaissance capabilities. The amplitude and frequency of the sinusoidal path are determined to maximize the required look-ahead visual coverage quality in terms of pixel density and quantity pertaining to the area covered. The ED-APF was tested and validated against the general artificial potential field techniques for various simulation scenarios using Robot Operating System (ROS) and Gazebo-supported PX4-SITL. It demonstrated superior performance and showed its suitability for reconnaissance and look-ahead support to MGVs in dynamic and obstacle-populated environments.
Improving Neural Machine Translation with AMR Semantic Graphs

The Seq2Seq model and its variants (ConvSeq2Seq and Transformer) emerge as a promising novel solution to the machine translation problem. However, these models only focus on exploiting knowledge from bilingual sentences without paying much attention to utilizing external linguistic knowledge sources such as semantic representations. Not only do semantic representations can help preserve meaning but they also minimize the data sparsity problem. However, to date, semantic information remains rarely integrated into machine translation models. In this study, we examine the effect of abstract meaning representation (AMR) semantic graphs in different machine translation models. Experimental results on the IWSLT15 English-Vietnamese dataset have proven the efficiency of the proposed model, expanding the use of external language knowledge sources to significantly improve the performance of machine translation models, especially in the application of low-resource language pairs.
Toxic Comment Classification

A large proportion of online comments available on public domains are usually constructive. However, a significant proportion is toxic and destructive. Several platforms and social media sites are finding it difficult to maintain fair conversation and are often forced to either limit the user comments or get dissolved by shutting down user comments completely. So, to prevent these types of identity hate through comments on social media, we come up with a solution to detect different types of toxicity in the comments using Deep Learning and Natural Language Processing. Dataset is obtained online which is processed to remove noise. Transformation of raw comments is done before feeding it to the classification model using Natural Language Processing. A Convolutional Neural Network model is used which will differentiate toxic comments from non-toxic comments.