Reviewerly/exHarmony huggingface dataset:
Calibrated Convolution with Gaussian of Difference

Attention mechanisms are widely used for Convolutional Neural Networks (CNNs) when performing various visual tasks. Many methods introduce multi-scale information into attention mechanisms to improve their feature transformation performance; however, these methods do not take into account the potential importance of scale invariance. This paper proposes a novel type of convolution, called Calibrated Convolution with Gaussian of Difference (CCGD), that takes into account both the attention mechanisms and scale invariance. A simple yet effective scale-invariant attention module that operates within a single convolution is able to adaptively build powerful scale-invariant features to recalibrate the feature representation. Along with this, a CNN with a heterogeneously grouped structure is used, which enhances the multi-scale representation capability. CCGD can be flexibly deployed in modern CNN architectures without introducing extra parameters. During experimental tests on various datasets, the method increased the ResNet50-based classification accuracy from 76.40% to 77.87% on the ImageNet dataset, and the tests generally confirmed that CCGD can outperform other state-of-the-art attention methods.
Low-Precision Quantization Techniques for Hardware-Implementation-Friendly BERT Models

Transformer-based deep learning models have been widely recognized as highly effective for NLP(natural language processing) tasks, among which BERT(Bidirectional Encoder Representations from Transformers) [1] is achieving outstanding performance on popular benchmarks. However, it is also noticed that the large amount of parameters, along with computational burden, are constraining its implementation on top of hardware platforms with limited computing and memory resources. In this work, having hardware implementation in mind, we introduce and evaluate 2 quantization techniques - clipping, and two piece-wise quantization, which have been implemented in convolutional neural network (CNN) models - to quantize this originally heavy model into the one requiring smaller number of bits. Our experimental results have revealed that, with the implementation of clipping and piece-wise quantization in an independent or joint manner, it is possible to maintain the accuracy of the BERT model after operating quantization with lower bit-width in activation and model quantized for smaller hardware implementation. Evaluations on four typical NLP tasks prove that, with 8-bit integer activations, even if the weights are quantized to only 4-bit integer, the loss of performance is relatively small. We show various quantization results for weights and activations, which indicate that from 4-bit to 8-bit quantization for both of weights and activations can be used with good accuracy with all weights in the model quantized.
Developing a Testing Framework for Cyber-Physical Systems using Gazebo

Developing a reliable Cyber-Physical System (CPS) requires tools and frameworks that can support various scenarios to be verified and tested under different environmental constraints. However, it is challenging to produce and synthesize a testing and verification environment for real-life scenarios to address every possible failure case. In the context of healthcare applications, testing an instrument is often expensive and risk-critical as it involves human interventions. To address these challenges, we present a simulation framework prepared using the combination of the Robot Operating System (ROS) and a tool, Gazebo. This experience paper demonstrates the applicability and usage of the Gazebo and ROS for wheelchair safety scenarios. Our wheelchair control algorithm is built on ROS that monitors the changes in the environment constraints (i.e., identifying the inclined or rough surfaces) and the wheelchair system.
Enhancing Lifelong Language Learning by Improving Pseudo-Sample Generation

Abstract To achieve lifelong language learning, pseudo-rehearsal methods leverage samples generated from a language model to refresh the knowledge of previously learned tasks. Without proper controls, however, these methods could fail to retain the knowledge of complex tasks with longer texts since most of the generated samples are low in quality. To overcome the problem, we propose three specific contributions. First, we utilize double language models, each of which specializes in a specific part of the input, to produce high-quality pseudo samples. Second, we reduce the number of parameters used by applying adapter modules to enhance training efficiency. Third, we further improve the overall quality of pseudo samples using temporal ensembling and sample regeneration. The results show that our framework achieves significant improvement over baselines on multiple task sequences. Also, our pseudo sample analysis reveals helpful insights for designing even better pseudo-rehearsal methods in the future.
Neural Networks in Material Humidity Sensors

The article discusses creating sensors technique for moisture assessment suitable for use in information and control systems regarding drying both bulk and pasty materials. Roller-belt and drum-type dryers developed sensors examples are given. Neural networks training for such sensors is organized according to technological parameters exemplary measures in dryers with a fixed accuracy. Trained neural network inputs are technological parameters normalized measured values that affect drying process directly. From neural network output, material’s normalized moisture content estimate is obtained. Based on the data received from the sensor regarding current humidity inside the drying plant, information system makes a control decision, minimizing the specified drying parameter criteria. The use of such systems in industry makes it possible to increase high-quality material output without drying process productivity losses.
GPGPU based Dual Population Genetic Algorithm for solving Constrained Optimization Problem

Dual Population Genetic Algorithm is a variant of Genetic Algorithm that provides additional diversity to the main population. It covers the premature convergence problem as well as the diversity problem associated with Genetic Algorithm. But also its additional population introduces large search space that increases time required to find an optimal solution. This large scale search space problem can be easily solved using consumer-level graphics cards. The solution obtained using accelerated DPGA for solving a constrained optimization problem from CEC 2006 is compared with the obtained solution using sequential algorithm. The results show speed up maintaining solution quality.
Transfer Learning of Deep Neural Network Human Pose Estimator by Domain-Specific Data for Video Motion Capturing

In this study, we construct sport-specific datasets with keypoint and bounding box annotations, and present a transfer learning approach for a 2D pose estimation using the datasets. Recent progress in deep neural networks has advanced the technology for 2D human pose estimation in an RGB camera image, including approach such as OpenPose and HRNet. Video-based motion capture systems using such a 2D pose estimator reconstruct the 3D joint positions from estimations of multiple cameras. This type of the video motion capture does not require markers and is suitable for measuring sport motions. In a widely used dataset in 2D pose estimators, however, the samples of sport-specific images are insufficient for representing players’ skills in dynamic sport motions. A 2D pose estimator trained with such a dataset sometimes fails in the detection of a sport-specific human pose. The transfer learning in this study provides robust 2D pose estimation and 3D reconstruction of sport-specific motions such as a taekwondo kick motion.
A cyber-risk framework for coordination of the prevention and preservation of behaviours1

Cybersecurity controls are deployed to manage risks posed by malicious behaviours or systems. What is not often considered or articulated is how cybersecurity controls may impact legitimate users (often those whose use of a managed system needs to be protected, and preserved). This oversight characterises the ‘blunt’ nature of many cybersecurity controls. Here we present a framework produced from consideration of concerns across methods from cybercrime opportunity reduction and behaviour change, and existing risk management guidelines. We illustrate the framework and its principles with a range of examples and potential applications, including management of suspicious emails in organizations, and social media controls. The framework describes a capacity to improve the precision of cybersecurity controls by examining shared determinants of negative and positive behaviours in a system. This identifies opportunities for risk owners to better protect legitimate users while simultaneously acting to prevent malicious activity in a managed system. We describe capabilities for a novel approach to managing sociotechnical cyber risk which can be integrated alongside elements of typical risk management processes. This includes consideration of user activities as a system asset to protect, and a consideration of how to engage with other stakeholders in the identification of behaviours to preserve in a system.
TANN: Text-based Attention Neural Network Recommendation Model

The recommendation system can help users automatically select items of interest from a large amount of information. The use of artificial intelligence technology to find out the willingness to express emotions in the text has received widespread attention, but data sparseness and cold start problems have seriously affected the quality of recommendations. Based on this, we propose a Text-based Attention Neural Network (TANN) recommendation model, which is based on text perception. First, use word2vec to obtain the word vector representation of the user comment text. Then, the combination of LSTM and attention mechanism is used to capture the contextual information in the text, and finally the emotional factors are classified and predicted through the CNN layer. Compared with the advanced baseline method, the TANN recommendation model is better than other baselines and can effectively solve the user's data sparseness and cold start problem.
Research on Filtering Feature Selection Methods for E-Mail Spam Detection by Applying K-NN Classifier

In the present paper, the application of filtering methods to select features when detecting email spam using the K-NN classifier is examined. The experiments include computation of the accuracy and F-measure of the e-mail texts classification with different methods for feature selection, different number of selected features and two ways to find the distance between dataset examples when executing K-NN classifier - Euclidean distance and cosine similarity. The obtained results are summarized and analyzed.
Optical-Flow based Analysis for Range Hoods captured Flow Measurement

Abstract The performance assessment of suction systems is a fundamental aspect in industrial field, and the quantitative estimation of their uptake is a still open challenge. This research proposes a measurement methodology for the quantitative evaluation of the steam uptake of a kitchen hood suction system through the definition of an uptake index, obtained by processing optical measurement using the Farnebäck dense optical-flow algorithm. The results and the uncertainty analysis show high reliability and consistency of the proposed approach.
RFDPC: Density Peaks Clustering Algorithm Based on Resultant Force

Density peaks clustering (DPC) is an efficient and effective algorithm due to its outstanding performance in discovering clusters with varying densities. However, the quality of this method is highly dependent on the cutoff distance. To improve the performance of DPC, the gravitation-based clustering (GDPC) algorithm is proposed. However, it cannot identify the clusters of varying densities. We developed a novel density peaks clustering algorithm based on the magnitude and direction of the resultant force acting on a data point (RFDPC). RFDPC is based on the idea that the resultant forces acting on the data points in the same cluster are more likely to point towards the cluster center. The cluster centers are selected based on the force directional factor and distance in the decision graph. Experimental results indicate superior performance of the proposed algorithm in detecting clusters of different densities, irregular shapes, and numbers of clusters.
Fault Prediction of Process Industry Based on Fuzzy Clustering

In the process of process industry production, once the failure will often bring heavy losses to the enterprise and the country in terms of human resources and financial resources, so it is very important to give early warning of the failure and take corresponding solutions. In this paper, a real-time fault prediction algorithm based on incremental model update is proposed for the continuous growth of industrial data flow, real-time update of objects, complex and variable attributes, and value attenuation with time. The algorithm consists of several steps, including data preprocessing, data flow clustering based on sliding window, abnormal data judgment, fault prediction and model prediction update. In each stage of the algorithm, Spark framework is used for parallel acceleration, so as to improve the efficiency of fault prediction.
JPD-SE: High-Level Semantics for Joint Perception-Distortion Enhancement in Image Compression

While humans can effortlessly transform complex visual scenes into simple words and the other way around by leveraging their high-level understanding of the content, conventional or the more recent learned image compression codecs do not seem to utilize the semantic meanings of visual content to their full potential. Moreover, they focus mostly on rate-distortion and tend to underperform in perception quality especially in low bitrate regime, and often disregard the performance of downstream computer vision algorithms, which is a fast-growing consumer group of compressed images in addition to human viewers. In this paper, we (1) present a generic framework that can enable any image codec to leverage high-level semantics and (2) study the joint optimization of perception quality and distortion. Our idea is that given any codec, we utilize high-level semantics to augment the low-level visual features extracted by it and produce essentially a new, semantic-aware codec. We propose a three-phase training scheme that teaches semantic-aware codecs to leverage the power of semantic to jointly optimize rate-perception-distortion (R-PD) performance. As an additional benefit, semantic-aware codecs also boost the performance of downstream computer vision algorithms. To validate our claim, we perform extensive empirical evaluations and provide both quantitative and qualitative results.
Verifying Payment Channels with TLA<sup>+</sup>

A payment channel protocol does not only have to provide the payment functionality, it also has to fulfill security guarantees such as ensuring that an honest party receives their correct balance. For complexity reasons, it is typically difficult to assess the security of such a protocol or to find counterexamples in insecure protocols. In this poster, we present an approach to specify functional as well as security properties for a payment channel protocol in TLA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> and show that a Lightning Network-style protocol fulfills the required properties. In case a counterexample is found, we provide protocol developers with a graphical and intuitive output. We present the challenges we faced and our approach to meeting these challenges.
SentMod: Hidden Backdoor Attack on Unstructured Textual Data

For a very long time, machine learning techniques such as deep learning have been considered to be very reliable and accurate for a wide variety of tasks including security sensitive applications. But recently, it was found that such models were vulnerable to adversarial attacks. It caused these models to generate wrong results when perturbed data was passed through it. One such attack is the backdoor attack where a backdoor is formed in the model when it is trained on a dataset containing malicious data. The malicious data is formed by adding small predefined perturbations to clean instances and changing its label to the target class. The backdoor is usually hidden and is only activated when the model receives a test data containing the same perturbation that was added to the training dataset. In natural language processing, the perturbations are usually a sequence of trigger words that are added to a predefined position in the sentence. But, these perturbations are often very obvious and can be detected by manual inspection since the context of the sentence does not match with its label. So, we have come up with a novel backdoor attack strategy that can generate perturbed data in a way such that its context matches with its label and the perturbation is hidden better. Our algorithm called SentMod, perturbs only 2% of the training data to obtain an attack success ratio of 97%. We have tested with multiple deep learning models to show that our approach is highly effective and is also very difficult to be detected using most common defense methods.
Privacy-Preserving Negotiation of Common Trust Anchors Across Blockchain Networks

Interoperation between permissioned consortium blockchain networks relies on their abilities to discover and validate the identities of each others’ participant organizations. These organizations may possess self-sovereign decentralized identities and verifiable credentials issued by well-known certification authorities. Two mutually untrusting networks of organizations can establish a basis for interoperation if they have one or more certification authorities in common. Yet, for privacy reasons, neither of them may want to expose a priori their entire lists of authorities, necessitating a negotiation process through which common authorities can be identified. In this paper, we analyze this negotiation problem, and propose and analyze two solution approaches, one involving active participation of the trust anchors and the other without involving them.
Machine Learning Based Methods for Arabic Duplicate Question Detection

Incorporating a duplicate question detection system can be beneficial for various systems such as community forums or question answering systems. Detecting question that have already an answer improves the user experience by reducing the search time and returning the correct answer. In this paper, we construct several methods for Arabic duplicate question detection based on machine learning. First, the pre-processing step is applied to clean and normalize questions. Next, we use Term Frequency Inverse Document Frequency (TF-IDF), Word2Vec, and FastText methods to map questions from their textual format into a vector space. Then, we trained various shallow learning methods (SVM, XGBoost, Random Forest, Logistic Regression) and deep learning methods (CNN, RNN, LSTM, GRU) with the objective of detecting if a pair of questions is duplicate or not. Various experiments were conducted to evaluate the performances of our models. The results obtained show that the deep learning model based on GRU with FastText representation performed better compared to the other models.
Assessing the Effect of NO2 Pollution on Solar Irradiance

Not only air pollution can affect men’s health, but it can have a detrimental impact on solar irradiance reaching the earth, resulting, for instance, in a decrease of photovoltaic plants energy yield. In this paper a theoretical model is proposed to evaluate the effect of air pollutants on solar irradiance using data measured at ground level. The model has been applied to the case of NO2 pollution in Acerra, a small town near Naples (Italy) where data have been collected for two years. It is found that NO2 has a negligible effect on solar budget, at least with respect to other pollutants such as PM2.5.
Function Mechanism of Intellectual Property Capability on Relay Innovation Based on CWLBGSO-DAG-Bootstrap SEM: Mediating Effect of Knowledge Matching and Moderating Effect of Relationship Learning

Taking unicorn enterprises as the research objects, this study constructs the conceptual model of function mechanism of intellectual property capability on relay innovation performance and introduces knowledge matching as the mediating variable and relationship learning as the moderating variable. Hybrid methods composed of fusion of rough set and binary firefly algorithm based on community weak connection mechanism (CWLBGSO), directed acyclic graph (DAG), and Bootstrap structural equation modeling (Bootstrap SEM) are used to carry out empirical analysis of the conceptual model of function mechanism. The empirical analysis results show that strong intellectual property capability has a positive role in promoting the relay innovation performance of unicorn enterprises, knowledge matching plays the positive mediating role between intellectual property capability and relay innovation, relationship learning plays the moderating role of the relationships between intellectual property capability and relay innovation, relationship learning plays the moderating role of the relationships between intellectual property capability and knowledge matching, and relationship learning plays the moderating role of the relationships between knowledge matching and relay innovation.
Possibilities of user interface design with the involvement of machine learning elements using Matlab

Machine learning, deep learning, convolutional neural networks are becoming common terms in the field of data processing. The principle is to create models based on base set testing, which are mainly used for prediction-related tasks. Rapid software application development and, last but not least, hardware development allow implement these algorithms into user application software. The use of these tools thus becomes more massive, as no expertise in the principles of artificial intelligence algorithms is required. One of the application programs that has the opportunity to use artificial intelligence techniques is the Matlab application. It offers tools that are grouped in a separate Toolbox called Machine & Deep learning. The presented paper points out the possibilities of creating a user interface that would allow the use of these tools for the needs of visualization and prediction.