Reviewerly/exHarmony huggingface dataset:
TextAT: Adversarial Training for Natural Language Understanding with Token-Level Perturbation

Adversarial training is effective in improving the robustness of neural networks. In NLP, languages are discrete in nature, separate tokens possess discrete semantics. Therefore, to incorporate adversarial training in sequence-level tasks, we introduce a novel training strategy: Text Adversarial Training with token-level perturbation. We fist craft perturbations that are initialized using a fine-grained token-level accumulated perturbations. Then we constrain these perturbations considering that inputs are separate tokens, rather than constraining them under a naive normalization ball. We validate the effectiveness of such normalization method using large-scale Transformer-based language models. Experiments on GLUE benchmark and NER task show that our adversarial training strategy improves the performances on various tasks including text classification and sequence labeling.
Evaluation of diffuse fraction and diffusion coefficient using statistical analysis

Abstract In this study, eighty models are proposed in order to estimate the diffuse fraction and diffuse coefficient. For the proposed models, sunshine ratio and clearness index are considered as predictors. Monthly average global and diffuse solar radiation together with sunshine duration data of Tamanrasset station form 1995 to 2017 were analyzed. The different proposed models are compared and statistically analyzed to assess the performance of the best fitted model. Nine statistical indicators and Global Performance Indicator are computed to evaluate different proposed models. It is concluded that the cubic model with sunshine ratio and clearness index is selected as the best accurate model to estimate diffuse solar radiation on a horizontal surface in the study area.
Term-based pooling in convolutional neural networks for text classification

To achieve good results in convolutional neural networks (CNN) for text classification task, term-based pooling operation in CNNs is proposed. Firstly, the convolution results of several convolution kernels are combined by this method, and then the results after combination are made pooling operation, three sorts of CNN models (we named TB-CNN, MCT-CNN and MMCT-CNN respectively) are constructed and then corresponding algorithmic thought are detailed on this basis. Secondly, relevant experiments and analyses are respectively designed to show the effects of three key parameters (convolution kernel, combination kernel number and word embedding) on three kinds of CNN models and to further demonstrate the effect of the models proposed. The experimental results show that compared with the traditional method of text classification in CNNs, term-based pooling method is addressed that not only the availability of the way is proved, but also the performance shows good superiority.
Reliability Analysis of Common Cause Failure Multistate System Based on CUGF

This paper addresses the problem of mixed uncertainty in the reliability analysis of multistate systems under common cause failure conditions. Combining the cloud model theory, universal generation function (UGF) method, and common cause failure theory, the universal generation function method is extended based on a probabilistic cloud model, i.e., the cloud universal generation function (CUGF) analysis method. The cloud model represents the random and cognitive uncertainty of the state probability, i.e., mixed uncertainty. Next, through CUGF, according to the calculation rules of cloud operators, we provide steps to obtain the reliability of a multistate system under independent failure and common cause failure conditions and obtain cloud digital features for reliability. The accuracy and feasibility of the method are verified by a numerical example. This paper solves the problem of reliability analysis of multistate systems with mixed uncertainty in unit state probability information under common cause failure conditions. We integrate system multistate, information uncertainty, and common cause failure for reliability analysis to avoid large errors, more in line with a project’s actual situation. We propose new ideas and methods to process randomness and fuzzy information or data in multistate system reliability analysis.
Bi-objective optimization of biclustering with binary data

Clustering consists of partitioning data objects into subsets called clusters according to some similarity criteria. This paper addresses a structure for generating overlapping clusters, where data objects can belong to more than one subset, which we join with bi-objective optimization and link to biclustering for problems with binary data. Biclustering simultaneously groups the objects and features so that a specific group of objects has a special group of features. In recent years, biclustering has received a lot of attention in several practical applications. First we present an integer programing formulations for the bi-objective optimization of biclustering. Next we propose a constructive heuristic based on the set intersection operation and its efficient implementation for solving a series of mono-objective problems used inside the ε-constraint method (obtained by keeping only one objective function and the other objective function is integrated into constraints). Finally, our experimental results show that our proposed heuristic provides very good results and significantly reduces the computational expense compared to using the CPLEX solver as an exact algorithm for finding an optimal solution, which drastically increases the computational cost for large instances.
Averaging Attacks on Bounded Noise-based Disclosure Control Algorithms

Abstract We describe and evaluate an attack that reconstructs the histogram of any target attribute of a sensitive dataset which can only be queried through a specific class of real-world privacy-preserving algorithms which we call bounded perturbation algorithms. A defining property of such an algorithm is that it perturbs answers to the queries by adding zero-mean noise distributed within a bounded (possibly undisclosed) range. Other key properties of the algorithm include only allowing restricted queries (enforced via an online interface), suppressing answers to queries which are only satisfied by a small group of individuals (e.g., by returning a zero as an answer), and adding the same perturbation to two queries which are satisfied by the same set of individuals (to thwart differencing or averaging attacks). A real-world example of such an algorithm is the one deployed by the Australian Bureau of Statistics’ (ABS) online tool called TableBuilder, which allows users to create tables, graphs and maps of Australian census data [30]. We assume an attacker (say, a curious analyst) who is given oracle access to the algorithm via an interface. We describe two attacks on the algorithm. Both attacks are based on carefully constructing (different) queries that evaluate to the same answer. The first attack finds the hidden perturbation parameter r (if it is assumed not to be public knowledge). The second attack removes the noise to obtain the original answer of some (counting) query of choice. We also show how to use this attack to find the number of individuals in the dataset with a target attribute value a of any attribute A , and then for all attribute values a i ∈ A . None of the attacks presented here depend on any background information. Our attacks are a practical illustration of the (informal) fundamental law of information recovery which states that “overly accurate estimates of too many statistics completely destroys privacy” [9, 15].
Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020.

We present our team ‘3Idiots’ (referred as ‘sdhanshu’ in the official rankings) approach for the Trolling, Aggression and Cyberbullying (TRAC) 2020 shared tasks. Our approach relies on fine-tuning various Transformer models on the different datasets. We also investigated the utility of task label marginalization, joint label classification, and joint training on multilingual datasets as possible improvements to our models. Our team came second in English sub-task A, a close fourth in the English sub-task B and third in the remaining 4 sub-tasks. We find the multilingual joint training approach to be the best trade-off between computational efficiency of model deployment and model’s evaluation performance. We open source our approach at https://github.com/socialmediaie/TRAC2020.
Open-Set Recognition with Gaussian Mixture Variational Autoencoders

In inference, open-set classification is to either classify a sample into a known class from training or reject it as an unknown class. Existing deep open-set classifiers train explicit closed-set classifiers, in some cases disjointly utilizing reconstruction, which we find dilutes the latent representation's ability to distinguish unknown classes. In contrast, we train our model to cooperatively learn reconstruction and perform class-based clustering in the latent space. With this, our Gaussian mixture variational autoencoder (GMVAE) achieves more accurate and robust open-set classification results, with an average F1 increase of 0.26, through extensive experiments aided by analytical results.
Learning to Identify Follow-Up Questions in Conversational Question Answering

Despite recent progress in conversational question answering, most prior work does not focus on follow-up questions.Practical conversational question answering systems often receive follow-up questions in an ongoing conversation, and it is crucial for a system to be able to determine whether a question is a follow-up question of the current conversation, for more effective answer finding subsequently.In this paper, we introduce a new follow-up question identification task.We propose a three-way attentive pooling network that determines the suitability of a follow-up question by capturing pair-wise interactions between the associated passage, the conversation history, and a candidate follow-up question.It enables the model to capture topic continuity and topic shift while scoring a particular candidate follow-up question.Experiments show that our proposed three-way attentive pooling network outperforms all baseline systems by significant margins.
Algorithmic efficiency indicator for the optimization of route size

In software development, sometimes experienced programmers have difficulty determining before performing their tests, which algorithm will work best to solve a particular problem, and the answer to the question will always depend on the type of problem and nature of the data to be used, in this situation, it is necessary to identify which indicator is the most useful for a specific type of problem and especially in route optimization.Currently, there are techniques and algorithms used in Artificial Intelligence, which, however, cannot display their potential without a well-defined data set.The paper seeks to explain and propose an algorithm selection indicator to build a consistent data set, given the lack of availability of data that allows the best route size decision to be made.
Advanced Multimedia Platform based on Big Data and Artificial Intelligence Improving Cybersecurity

The proposed work describes the design of a multimedia platform managing users and implementing cybersecurity.The paper describes in details the use cases of the whole platform embedding Big Data and artificial intelligence (AI) engine predicting network attacks.The platform has been tested by Tree Ensemble algorithm classifying and predicting anomalous server logs of possible attacks.The data logs are collected in Cassandra Big Data System enabling the AI training model.The work has been developed within the framework of a research industry project.
Pathological Image Classification Based on Hard Example Guided CNN

The diagnosis of biopsy tissue with hematoxylin and eosin (H&E) stained images has been widely used by pathologists to detect the lesions and assess the malignancy.Nevertheless, the diagnostic result relies on the visual observation of pathologists which may vary from person to person under different circumstances.With the advantage of automatically and adaptively learning features at multiple levels of abstraction, Convolutional Neural Networks (CNNs) have rapidly become promising alternatives for pathological image analysis.Therefore, in this paper, we propose an effective method for tumor classification called Hard Example Guided CNN.Our contribution is twofold: firstly, to optimize image representation, we design the CNN architecture as dual-branch, used for extracting global features and local features simultaneously.Secondly, we propose a re-weight training algorithm, which improves learning accuracy and accelerates the convergence by increasing the weight of hard examples.Extensive experiments on multiple datasets demonstrate the superiority of our proposed classification method.
Integrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environments

This paper investigates how to efficiently transition and update policies, trained initially with demonstrations, using off-policy actor-critic reinforcement learning. It is well-known that techniques based on Learning from Demonstrations, for example behavior cloning, can lead to proficient policies given limited data. However, it is currently unclear how to efficiently update that policy using reinforcement learning as these approaches are inherently optimizing different objective functions. Previous works have used loss functions, which combine behavior cloning losses with reinforcement learning losses to enable this update. However, the components of these loss functions are often set anecdotally, and their individual contributions are not well understood. In this work, we propose the Cycle-of-Learning (CoL) framework that uses an actor-critic architecture with a loss function that combines behavior cloning and 1-step Q-learning losses with an off-policy pre-training step from human demonstrations. This enables transition from behavior cloning to reinforcement learning without performance degradation and improves reinforcement learning in terms of overall performance and training time. Additionally, we carefully study the composition of these combined losses and their impact on overall policy learning. We show that our approach outperforms state-of-the-art techniques for combining behavior cloning and reinforcement learning for both dense and sparse reward scenarios. Our results also suggest that directly including the behavior cloning loss on demonstration data helps to ensure stable learning and ground future policy updates.
Analisis Sentimen Masyarakat Terhadap Pembatasan Sosial Berksala Besar Menggunakan Algoritma Support Vector Machine

The spread of COVID-19 has become a pandemic throughout the world, including Indonesia. As the main door in Maluku, Ambon has become a new epicenter of the spread of coronavirus. This condition requires local governments to implement Large Scale Social Restrictions (LSSR) after being permitted by the Ministry of Health. This has become a polemic for some Ambonese people especially those who don't have a steady income. Many posts from social media discussing the LSSR that was implemented in Ambon. This study aims to find out how people's sentiment towards the LSSR implementation plan in Ambon through tweets and comments on social media platforms using sentiment analysis. The data obtained amounted to 1075 tweets and comments and separated between 350 training data and testing data of 725. The data classified using the Support Vector Machine (SVM) algorithm. The results of the study showed positive sentiment of 28%, a negative sentiment of 27%, and a neutral sentiment of 45%. It can be concluded that public sentiment towards the implementation of LSSR in Ambon based on posts on social media platforms is quite balanced between positive and negative sentiments and also dominated by neutral sentiments posts.
Dolores: Deep Contextualized Knowledge Graph Embeddings

We introduce Dolores, a new knowledge graph embeddings, that effectively capture contextual cues and dependencies among entities and relations. First, we note that short paths on knowledge graphs comprising of chains of entities and relations can encode valuable information regarding their contextual usage. We operationalize this notion by representing knowledge graphs not as a collection of triples but as a collection of entity-relation chains, and learn embeddings using deep neural models that capture such contextual usage. Based on BiLSTMs, our model learns deep representations from constructed entity-relation chains. We show that these representations can be easily incorporated into existing models to significantly advance the performance on several knowledge graph tasks like link prediction, triple classification, and multi-hop knowledge base completion.
Fast focus-shifter based on a unimorph deformable mirror

On-the-fly remote laser processing plays an increasingly important role in modern fabrication techniques. These processes require guiding of the focus of a laser beam along the contours of the workpiece in three dimensions. State-of-the-art galvanometer scanners already provide highly dynamic and precise transverse <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mi>x</mml:mi> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mo>−<!-- − --></mml:mo> </mml:mrow> <mml:mi>y</mml:mi> </mml:math> beam steering. However, longitudinal focus shifting (“z-shifting”) relying on conventional optics is restricted to a bandwidth of a few hundred Hz. We have developed and manufactured a fast piezo-based z-shifting mirror with diffraction-limited surface fidelity providing a focus shift of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mi mathvariant="normal">Δ<!-- Δ --></mml:mi> <mml:mi>z</mml:mi> <mml:mo>&gt;<!-- > --></mml:mo> <mml:mn>60</mml:mn> <mml:mspace width="thickmathspace" /> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">m</mml:mi> <mml:mi mathvariant="normal">m</mml:mi> </mml:mrow> </mml:mrow> </mml:math> with an actuation rate of 2 kHz.
A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Non-negative matrix factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy data sets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored data set. Unfortunately, this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as automatic relevance determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial data set, and determines which K produces stable groups of latent features that reconstruct the initial data set well. We then train a multi-layer perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solutions, obtained from NMFk. In order to train the MLP classifier, a training set of 58 660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark data sets, the swimmer and MIT face data, NMFk/MLP correctly recovered the established number of hidden features. Finally, we compared the accuracy of our method to ARD, AIC and stability-based methods.
Authentication Based on the Image Encryption using Delaunay Triangulation and Catalan Objects

This paper presents the authentication method using the Delaunay triangulation incremental algorithm and the Catalan objects.The proposed method is a combination of computational geometry and cryptography.This method presents a new step towards encoding the triangle coordinates using the Catalan-key.We provided specific suggestions for the application of this method in the authentication for bank clients by the image encryption.Client authentication verification is performed by asking the client to enter the (x,y) coordinate values of randomly selected indices of an array.If the entered coordinates match the index values in the banking system array, then the transaction or other operation is approved.If the matching fails, it means that we have an unidentified person who has followed the whole process and wants to break into the banking system.There are many advantages arising from a scenario for the user authentication by the assigned Catalan object and the stack permutation method.Also, we provided concrete examples for the Delaunay encryption of image with an authentication scenario and experimental results for the proposed method.
A novel technique for speech encryption based on k-means clustering and quantum chaotic map

In information transmission such as speech information, higher security and confidentiality are specially required. Therefore, data encryption is a pre-requisite for a secure communication system to protect such information from unauthorized access. A new algorithm for speech encryption is introduced in this paper. It depends on the quantum chaotic map and k-means clustering, which are employed in keys generation. Also, two stages of scrambling were used: the first relied on bits using the proposed algorithm (binary representation scrambling BiRS) and the second relied on k-means using the proposed algorithm (block representation scrambling BlRS). The objective test used statistical analysis measures (signal-to-noise-ratio, segmental signal-to-noise-ratio, frequency-weighted signal-to-noise ratio, correlation coefficient, log-likelihood ratio) applied to evaluate the proposed system. Via MATLAB simulations, it is shown that the proposed technique is secure, reliable and efficient to be implemented in secure speech communication, as well as also being characterized by high clarity of the recovered speech signal.
A Novel Face Recognition Algorithm for Imbalanced Small Samples

Deep learning (DL) has become a hotspot in the research of image recognition.However, the DL strategy must be trained with lots of samples that are distributed evenly across classes, i.e. subjected to balanced distribution.Therefore, this paper attempts to design a method to satisfactorily recognize faces in imbalanced small samples.Firstly, the deep convolutional generative adversarial network (DCGAN) was improved to generate data samples with similar distribution as the original training data, creating a balanced training set of sufficient labelled samples.Then, transfer learning was performed to transform the AlexNet, which is pretrained on big dataset, to the balanced target dataset of small samples.Next, the previous convolutional layer was frozen as a feature extractor, and the truncated normal distribution was reinitialized for the next fully-connected layer.Simulations on face recognition show that our method achieved higher recognition rate and less serious overfitting than ordinary CNNs.