Reviewerly/exHarmony huggingface dataset:
Automatic Fairness Testing of Machine Learning Models

In recent years, there has been an increased application of machine learning (ML) to decision making systems. This has prompted an urgent need for validating requirements on ML models. Fairness is one such requirement to be ensured in numerous application domains. It specifies a software as “learned” by an ML algorithm to not be biased in the sense of discriminating against some attributes (like gender or age), giving different decisions upon flipping the values of these attributes. In this work, we apply verification-based testing (VBT) to the fairness checking of ML models. Verification-based testing employs verification technology to generate test cases potentially violating the property under interest. For fairness testing, we additionally provide a specification language for the formalization of different fairness requirements. From the ML model under test and fairness specification VBT automatically generates test inputs specific to the specified fairness requirement. The empirical evaluation on several benchmark ML models shows verification-based testing to perform better than existing fairness testing techniques with respect to effectiveness.
Modeling Temporal Visual Salience for Human Action Recognition Enabled Visual Anonymity Preservation

This paper proposes a novel approach for visually anonymizing video clips while retaining the ability to machine-based analysis of the video clip, such as, human action recognition.The visual anonymization is achieved by proposing a novel method for generating the anonymization silhouette by modelling the frame-wise temporal visual salience.This is followed by analysing these temporal saliencebased silhouettes by extracting the proposed histograms of gradients in salience (HOG-S ) for learning the action representation in the visually anonymized domain.Since the anonymization maps are based on the temporal salience maps represented in gray scale, only the moving body parts related to the motion of the action are represented in larger gray values forming highly anonymized silhouettes, resulting in the highest mean anonymity score (MAS), the least identifiable visual appearance attributes and a high utility of human-perceived utility in action recognition.In terms of machine-based human action recognition, using the proposed HOG-S features has resulted in the highest accuracy rate in the anonymized domain compared to those achieved from the existing anonymization methods.Overall, the proposed holistic human action recognition method, i.e., the temporal salience modelling followed by the HOG-S feature extraction, has resulted in the best human action recognition accuracy rates for datasets DHA, KTH, UIUC1, UCF Sports and HMDB51 with improvements of 3%, 1.6%, 0.8%, 1.3% and 16.7%, respectively.The proposed method outperforms both feature-based and deep learning based existing approaches.
A Grid Feature-Point Selection Method for Large-Scale Street View Image Retrieval Based on Deep Local Features

Street view image retrieval aims to estimate the image locations by querying the nearest neighbor images with the same scene from a large-scale reference dataset. Query images usually have no location information and are represented by features to search for similar results. The deep local features (DELF) method shows great performance in the landmark retrieval task, but the method extracts many features so that the feature file is too large to load into memory when training the features index. The memory size is limited, and removing the part of features simply causes a great retrieval precision loss. Therefore, this paper proposes a grid feature-point selection method (GFS) to reduce the number of feature points in each image and minimize the precision loss. Convolutional Neural Networks (CNNs) are constructed to extract dense features, and an attention module is embedded into the network to score features. GFS divides the image into a grid and selects features with local region high scores. Product quantization and an inverted index are used to index the image features to improve retrieval efficiency. The retrieval performance of the method is tested on a large-scale Hong Kong street view dataset, and the results show that the GFS reduces feature points by 32.27–77.09% compared with the raw feature. In addition, GFS has a 5.27–23.59% higher precision than other methods.
Argus: Interactive a priori Power Analysis

A key challenge HCl researchers face when designing a controlled experiment is choosing the appropriate number of participants, or sample size. A priori power analysis examines the relationships among multiple parameters, including the complexity associated with human participants, e.g., order and fatigue effects, to calculate the statistical power of a given experiment design. We created Argus, a tool that supports interactive exploration of statistical power: Researchers specify experiment design scenarios with varying confounds and effect sizes. Argus then simulates data and visualizes statistical power across these scenarios, which lets researchers interactively weigh various trade-offs and make informed decisions about sample size. We describe the design and implementation of Argus, a usage scenario designing a visualization experiment, and a think-aloud study.
Findings of the LoResMT 2020 Shared Task on Zero-Shot for Low-Resource languages

This publication has emanated from research in part supported by the EU H2020 programme under grant agreements 731015 (ELEXIS-European Lexical Infrastructure). We are also grateful to Panlingua Language Processing LLP to provide Hindi, Bhojpuri, Magahi monolingual and parallel corpora.
Exploring Explainable Selection to Control Abstractive Summarization

Like humans, document summarization models can interpret a document’s contents in a number of ways. Unfortunately, the neural models of today are largely black boxes that provide little explanation of how or why they generated a summary in the way they did. Therefore, to begin prying open the black box and to inject a level of control into the substance of the final summary, we developed a novel select-and-generate framework that focuses on explainability. By revealing the latent centrality and interactions between sentences, along with scores for novelty and relevance, users are given a window into the choices a model is making and an opportunity to guide those choices in a more desirable direction. A novel pair-wise matrix captures the sentence interactions, centrality and attribute scores, and a mask with tunable attribute thresholds allows the user to control which sentences are likely to be included in the extraction. A sentence-deployed attention mechanism in the abstractor ensures the final summary emphasizes the desired content. Additionally, the encoder is adaptable, supporting both Transformer- and BERT-based configurations. In a series of experiments assessed with ROUGE metrics and two human evaluations, ESCA outperformed eight state-of-the-art models on the CNN/DailyMail and NYT50 benchmark datasets.
USING PADLET TO INCREASE THE STUDENTS’ PROCEDURE TEXT WRITING ACHIEVEMENT

The purpose of this research was to improve the Students’ procedure text writing achievement at English Conversation Club (ECC) MAN Kota Blitar by applying Padlet. The research design used was classroom action research (CAR) which covered 4 steps; 1) Planning, 2) Acting, 3) Observing and evaluating and 4) Reflecting. The Subjects of this research were 50 students from X Grade who join ECC in MAN Kota Blitar.The data collection methods of this research were taken from administering tests and distributing the questionnaire. The research result proved that the use of Padlet for teaching writing of procedure text could increase the students’ writing achievement of procedure text from preliminary action 57.55 increased to 72.85 in cycle 1. Based on the result of cycle 1, it did not fulfill the minimum criterion mastery 75.00; therefore, the researchers continued the research to cycle 2 and got satisfying result 82.65. Furthermore, the students’ of ECC MAN Kota Blitar also presented positive attitude toward the use of Padlet in teaching and learning writing of procedure text.
Artificial Dummies for Urban Dataset Augmentation

Existing datasets for training pedestrian detectors in images suffer from limited appearance and pose variation. The most challenging scenarios are rarely included because they are too difficult to capture due to safety reasons, or they are very unlikely to happen. The strict safety requirements in assisted and autonomous driving applications call for an extra high detection accuracy also in these rare situations. Having the ability to generate people images in arbitrary poses, with arbitrary appearances and embedded in different background scenes with varying illumination and weather conditions, is a crucial component for the development and testing of such applications. The contributions of this paper are three-fold. First, we describe an augmentation method for the controlled synthesis of urban scenes containing people, thus producing rare or never-seen situations. This is achieved with a data generator (called DummyNet) with disentangled control of the pose, the appearance, and the target background scene. Second, the proposed generator relies on novel network architecture and associated loss that takes into account the segmentation of the foreground person and its composition into the background scene. Finally, we demonstrate that the data generated by our DummyNet improve the performance of several existing person detectors across various datasets as well as in challenging situations, such as night-time conditions, where only a limited amount of training data is available. In the setup with only day-time data available, we improve the night-time detector by 17% log-average miss rate over the detector trained with the day-time data only.
Pruning Filters Base on Extending Filter Group Lasso

Deep Convolution Neural Networks (CNNs) have been widely used in image recognition, while models of CNNs are desired to be more compact as the growing demands arise from various kinds of AI applications. As sparsity is generally accepted as inherent characteristics of the pruned models, L1 based approaches normally add penalty on convolution filters or channels to objectives structurally but straightforwardly. That kind of induced sparse models only represent the balanced results of training loss and penalty mechanically. In this paper, we construct the Extending Filter Group (EFG) by thorough investigations on underlying constraints between every two successive layers. The penalty in terms of EFG addresses train process on filters of the current layer and channels in the following layer, which called as synchronous reinforcement. Thus it provides an alternative way to induce a model with ideal sparsity, especially in case of complex datasets. Moreover, we present Noise Filter Recognition Mechanism (NFRM) to improve model accuracy, as a shrinking model is to be obtained from the original based our contributions. Our method achieves the accuracy of 72.67% in cifar-100 dataset, while the performance is 72.04% in current baseline. It is worth noticing that the pruning rate achieved through our method marks 43.8%, which is higher than the one generated by other popular pruning methods.
Students' perception on the adoption of an e-textbook (digital) as an alternative to the printed textbook

The debate between using printed textbooks (traditional) or e-textbooks (digital textbooks) is ongoing in the educational sector. Students are often required to buy textbooks in order to complete a course in the university, and this textbook can either be printed or e-textbook. Students who are usually reluctant to read a printed text or electronic texts are not familiar with the advantages and disadvantages of each text, respectively. Thus, the absence of information prompts the students to employ only one kind of text either digital or printed. The purpose of this research was to investigate whether university students prefer printed textbooks or digital textbooks (e-textbooks). The importance of the research is to understand why students are making the decision to use either traditional textbooks or e-textbooks and what makes them develop these preferences. The benefits that will be derived from this research will help illustrate what the students feel about traditional (printed) and e-textbooks (digital) and the choices that go into making that decision. This article is based on a study that was conducted at a university based in Midrand, South Africa. The study applied the mixed methods to analyse the relationship between perceived ease of use (PEOU), perceived usefulness (PU), attitude (ATT), complexity (CMX) and compatibility (CMP) on Behavioural intention to use. Questionnaire was the main data collection tool from as sample of 80 third year students from the Information Technology department of the university in question. The overall result led to the conclusion that students prefer e-textbook to the printed textbook.
Fast Real-Time Trajectory Planning Method with 3rd-Order Curve Optimization for Automated Vehicles

Automated driving (AD) is one of the fastest-growing tendencies in the Intelligent Transportation Systems (ITS) field with some interesting demonstrations and prototypes. Currently, the main research topics are aligned with vehicle communications, environment recognition, control, and decision-making. A real-time trajectory planning method for Automated vehicles (AVs) is presented in this paper; the contribution is part of AD's decision-making module. This novel approach uses the properties of the 3er order Bézier curves to generate fast and reliable vehicle trajectories. Online execution and vehicle tracking capacities are considered on the approach. A feasible trajectory is selected based on the criteria: (i) the vehicle must be contained by a collision-free corridor given by an upper decision layer, (ii) the vehicle must be capable to track the generated trajectory, and (iii) the continuity of the path and curvature must be preserved in the joints. Our approach was tested considering a vehicle length (automated bus) of 12 meters. The scenario has the dimension of a real test location with multiple roundabouts.
Dual Supervision Framework for Relation Extraction with Distant Supervision and Human Annotation

Relation extraction (RE) has been extensively studied due to its importance in real-world applications such as knowledge base construction and question answering.Most of the existing works train the models on either distantly supervised data or human-annotated data.To take advantage of the high accuracy of human annotation and the cheap cost of distant supervision, we propose the dual supervision framework which effectively utilizes both types of data.However, simply combining the two types of data to train a RE model may decrease the prediction accuracy since distant supervision has labeling bias.We employ two separate prediction networks HA-Net and DS-Net to predict the labels by human annotation and distant supervision, respectively, to prevent the degradation of accuracy by the incorrect labeling of distant supervision.Furthermore, we propose an additional loss term called disagreement penalty to enable HA-Net to learn from distantly supervised labels.In addition, we exploit additional networks to adaptively assess the labeling bias by considering contextual information.Our performance study on sentence-level and document-level REs confirms the effectiveness of the dual supervision framework.
Incorporating Noisy Length Constraints into Transformer with Length-aware Positional Encodings

Neural Machine Translation often suffers from an under-translation problem due to its limited modeling of output sequence lengths.In this work, we propose a novel approach to training a Transformer model using length constraints based on length-aware positional encoding (PE).Since length constraints with exact target sentence lengths degrade translation performance, we add random noise within a certain window size to the length constraints in the PE during the training.In the inference step, we predict the output lengths using input sequences and a BERTbased length prediction model.Experimental results in an ASPEC English-to-Japanese translation showed the proposed method produced translations with lengths close to the reference ones and outperformed a vanilla Transformer by 3.22 points in BLEU on short sentences within ten subwords.The average translation results using our length prediction model were also better than another baseline method using input lengths for the length constraints.The proposed noise injection improved robustness for length prediction errors, especially within the window size.
Modelling Long-distance Node Relations for KBQA with Global Dynamic Graph

The structural information of Knowledge Bases (KBs) has proven effective to Question Answering (QA).Previous studies rely on deep graph neural networks (GNNs) to capture rich structural information, which may not model node relations in particularly long distance due to oversmoothing issue.To address this challenge, we propose a novel framework GlobalGraph, which models long-distance node relations from two views: 1) Node type similarity: GlobalGraph assigns each node a global type label and models long-distance node relations through the global type label similarity; 2) Correlation between nodes and questions: we learn similarity scores between nodes and the question, and model long-distance node relations through the sum score of two nodes.We conduct extensive experiments on two widely used multi-hop KBQA datasets to prove the effectiveness of our method.
Requirements Prioritization Techniques in the last decade: A Systematic Literature Review

Requirements Prioritization is an activity, part of Requirements Engineering, whose purpose is to determine the relevance that each requirement has within a software system to be developed. In order to conduct this activity, several requirements prioritization techniques have been proposed. However, companies and software developers do not know which techniques can give the best results possible or when to apply them. Choosing the wrong technique might lead the prioritization process to an inappropriate approach. This systematic literature review aims to know the state of the art of the prioritization techniques which have been used in software projects developed during the last decade, as well as knowing the benefits provided by their implementation. Our results show that AHP, Cumulative Voting, Cost-Value Approach, and Numerical Assignment were the most documented techniques. According to the studies reviewed, these techniques have specific advantages, for example, accuracy and effort reduction benefits. They also have disadvantages such as scalability issues and time consumption problems. However, the use of Requirements Prioritization Techniques, in general terms, generate cost and effort benefits as well as a considerable reduction in the time needed to conduct the Prioritization Process.
A comparative analysis of Latent Semantic analysis and Latent Dirichlet allocation topic modeling methods using Bible data

Objective: To compare the topic modeling techniques, as no free lunch theorem states that under a uniform distribution over search problems, all machine learning algorithms perform equally. Hence, here, we compare Latent Semantic Analysis (LSA) or Latent Dirichlet Allocation (LDA) to identify better performer for English bible data set which has not been studied yet. Methods: This comparative study divided into three levels: In the first level, bible data was extracted from the sources and preprocessed to remove the words and characters which were not useful to obtain the semantic structures or necessary patterns to make the meaningful corpus. In the second level, the preprocessed data were converted into a bag of words and numerical statistic TF-IDF (Term Frequency – Inverse Document Frequency) is used to assess how relevant a word is to a document in a corpus. In the third level, Latent Semantic analysis and Latent Dirichlet Allocations methods were applied over the resultant corpus to study the feasibility of the techniques. Findings: Based on our evaluation, we observed that the LDA achieves 60 to 75% superior performance when compared to LSA using document similarity within-corpus, document similarity with the unseen document. Additionally, LDA showed better coherence score (0.58018) than LSA (0.50395). Moreover, when compared to any word within-corpus, the word association showed better results with LDA. Some words have homonyms based on the context; for example, in the bible; bear has a meaning of punishment and birth. In our study, LDA word association results are almost near to human word associations when compared to LSA. Novelty: LDA was found to be the computationally efficient and interpretable method in adopting the English Bible dataset of New International Version that was not yet created. Keywords: Topic modeling; LSA; LDA; word association; document similarity;Bible data set
FBK-DH at SemEval-2020 Task 12: Using Multi-channel BERT for Multilingual Offensive Language Detection

In this paper we present our submission to subtask A at SemEval 2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval2).For Danish, Turkish, Arabic and Greek, we develop an architecture based on transfer learning and relying on a two-channel BERT model, in which the English BERT and the multilingual one are combined after creating a machine-translated parallel corpus for each language in the task.For English, instead, we adopt a more standard, single-channel approach.We find that, in a multilingual scenario, with some languages having small training data, using parallel BERT models with machine translated data can give systems more stability, especially when dealing with noisy data.The fact that machine translation on social media data may not be perfect does not hurt the overall classification performance.
Language-Driven Region Pointer Advancement for Controllable Image Captioning

Controllable Image Captioning is a recent sub-field in the multi-modal task of Image Captioning wherein constraints are placed on which regions in an image should be described in the generated natural language caption.This puts a stronger focus on producing more detailed descriptions, and opens the door for more end-user control over results.A vital component of the Controllable Image Captioning architecture is the mechanism that decides the timing of attending to each region through the advancement of a region pointer.In this paper, we propose a novel method for predicting the timing of region pointer advancement by treating the advancement step as a natural part of the language structure via a NEXT-token, motivated by a strong correlation to the sentence structure in the training data.We find that our timing agrees with the ground-truth timing in the Flickr30k Entities test data with a precision of 86.55% and a recall of 97.92%.Our model implementing this technique improves the state-of-the-art on standard captioning metrics while additionally demonstrating a considerably larger effective vocabulary size.