Reviewerly/exHarmony huggingface dataset:
Storytelling in human--centric software engineering research

BACKGROUND: Software engineering is a human activity. People naturally make sense of their activities and experience through storytelling. But storytelling does not appear to have been properly studied by software engineering research. AIM: We explore the question: what contribution can storytelling make to human--centric software engineering research? METHOD: We define concepts, identify types of story and their purposes, outcomes and effects, briefly review prior literature, identify several contributions and propose next steps. RESULTS: Storytelling can, amongst other contributions, contribute to data collection, data analyses, ways of knowing, research outputs, interventions in practice, and advocacy, and can integrate with evidence and arguments. Like all methods, storytelling brings risks. These risks can be managed. CONCLUSION: Storytelling provides a potential counter--balance to abstraction, and an approach to retain and honour human meaning in software engineering.
Moonshine: An Online Randomness Distiller for Zero-Involvement Authentication

Context-based authentication is a method for transparently validating another device's legitimacy to join a network based on location. Devices can pair with one another by continuously harvesting environmental noise to generate a random key with no user involvement. However, there are gaps in our understanding of the theoretical limitations of environmental noise harvesting, making it difficult for researchers to build efficient algorithms for sampling environmental noise and distilling keys from that noise. This work explores the information-theoretic capacity of context-based authentication mechanisms to generate random bit strings from environmental noise sources with known properties. Using only mild assumptions about the source process's characteristics, we demonstrate that commonly-used bit extraction algorithms extract only about 10% of the available randomness from a source noise process. We present an efficient algorithm to improve the quality of keys generated by context-based methods and evaluate it on real key extraction hardware. Moonshine is a randomness distiller which is more efficient at extracting bits from an environmental entropy source than existing methods. Our techniques nearly double the quality of keys as measured by the NIST test suite, producing keys that can be used in real-world authentication scenarios.
Current Status and Performance Analysis of Table Recognition in Document Images with Deep Neural Networks

The first phase of table recognition is to detect the tabular area in a document. Subsequently, the tabular structures are recognized in the second phase in order to extract information from the respective cells. Table detection and structural recognition are pivotal problems in the domain of table understanding. However, table analysis is a perplexing task due to the colossal amount of diversity and asymmetry in tables. Therefore, it is an active area of research in document image analysis. Recent advances in the computing capabilities of graphical processing units have enabled deep neural networks to outperform traditional state-of-the-art machine learning methods. Table understanding has substantially benefited from the recent breakthroughs in deep neural networks. However, there has not been a consolidated description of the deep learning methods for table detection and table structure recognition. This review paper provides a thorough analysis of the modern methodologies that utilize deep neural networks. This work provided a thorough understanding of the current state-of-the-art and related challenges of table understanding in document images. Furthermore, the leading datasets and their intricacies have been elaborated along with the quantitative results. Moreover, a brief overview is given regarding the promising directions that can serve as a guide to further improve table analysis in document images.
Quantum Computing Platforms: Assessing the Impact on Quality Attributes and SDLC Activities

Practical quantum computing is rapidly becoming a reality. To harness quantum computers' real potential in software applications, one needs to have an in-depth understanding of all such characteristics of quantum computing platforms (QCPs), relevant from the Software Engineering (SE) perspective. Restrictions on copying, deletion, the transmission of qubit states, a hard dependency on quantum algorithms are few, out of many, examples of QCP characteristics that have significant implications for building quantum software. Thus, developing quantum software requires a paradigm shift in thinking by software engineers. This paper presents the key findings from the SE perspective, resulting from an in-depth examination of state-of-the-art QCPs available today. The main contributions that we present include i) Proposing a general architecture of the QCPs, ii) Proposing a programming model for developing quantum software, iii) Determining architecturally significant characteristics of QCPs, and \textbf{iv)} Determining the impact of these characteristics on various Quality Attributes (QAs) and Software Development Life Cycle (SDLC) activities. We show that the nature of QCPs makes them useful mainly in specialized application areas such as scientific computing. Except for performance and scalability, most of the other QAs (e.g., maintainability, testability, and reliability) are adversely affected by different characteristics of a QCP.
Simulating the DFT Algorithm for Audio Processing

Since the evolution of digital computers, the storage of data has always been in terms of discrete bits that can store values of either 1 or 0. Hence, all computer programs (such as MATLAB), convert any input continuous signal into a discrete dataset. Applying this to oscillating signals, such as audio, opens a domain for processing as well as editing. The Fourier transform, which is an integral over infinite limits, for the use of signal processing is discrete. The essential feature of the Fourier transform is to decompose any signal into a combination of multiple sinusoidal waves that are easy to deal with. The discrete Fourier transform (DFT) can be represented as a matrix, with each data point acting as an orthogonal point, allowing one to perform complicated transformations on individual frequencies. Due to this formulation, all the concepts of linear algebra and linear transforms prove to be extremely useful here. In this paper, we first explain the theoretical basis of audio processing using linear algebra, and then focus on a simulation coded in MATLAB, to process and edit various audio samples. The code is open ended and easily expandable by just defining newer matrices which can transform over the original audio signal. Finally, this paper attempts to highlight and briefly explain the results that emerge from the simulation
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

This paper proposes a novel method of learning by predicting view assignments with support samples (PAWS). The method trains a model to minimize a consistency loss, which ensures that different views of the same unlabeled instance are assigned similar pseudo-labels. The pseudo-labels are generated non-parametrically, by comparing the representations of the image views to those of a set of randomly sampled labeled images. The distance between the view representations and labeled representations is used to provide a weighting over class labels, which we interpret as a soft pseudo-label. By non-parametrically incorporating labeled samples in this way, PAWS extends the distance-metric loss used in self-supervised methods such as BYOL and SwAV to the semi-supervised setting. Despite the simplicity of the approach, PAWS outperforms other semi-supervised methods across architectures, setting a new state-of-the-art for a ResNet-50 on ImageNet trained with either 10% or 1% of the labels, reaching 75.5% and 66.5% top-1 respectively. PAWS requires 4x to 12x less training than the previous best methods.
Safety-Augmented Operation of Mobile Robots Using Variable Structure Control

The design process and complexity of existing safety controls are heavily determined by the geometrical properties of the environment, which affects the proof of convergence, design scalability, performance robustness, and numerical efficiency of the control. Hence, this paper proposes a variable structure control to isolate the environment's geometrical complexity from the control structure. A super-twisting algorithm is used to achieve accurate trajectory tracking and robust safety control. The safety control is designed solely based on distance measurement. First, a nominal safety model for obstacle avoidance is derived, where realistic system constraints are considered. The nominal model is well-suited for safety control design for obstacle avoidance, geofencing, and border patrol with analytically proven stability results. The safety control utilizes distance measurement to maintain a safe distance by compensating the robot's angular velocity. A supervisory logic is constructed to guarantee the overall stability and safety of the system. Operational safety and precision tracking are proven under parametric uncertainty and environmental uncertainty. The proposed design is modular with minimal tuning parameters, which reduces the computational burden and improves the control scalability. The effectiveness of the proposed method is verified against various case studies.
Improving Fairness in Speaker Recognition

The human voice conveys unique characteristics of an individual, making voice biometrics a key technology for verifying identities in various industries. Despite the impressive progress of speaker recognition systems in terms of accuracy, a number of ethical and legal concerns has been raised, specifically relating to the fairness of such systems. In this paper, we aim to explore the disparity in performance achieved by state-of-the-art deep speaker recognition systems, when different groups of individuals characterized by a common sensitive attribute (e.g., gender) are considered. In order to mitigate the unfairness we uncovered by means of an exploratory study, we investigate whether balancing the representation of the different groups of individuals in the training set can lead to a more equal treatment of these demographic groups. Experiments on two state-of-the-art neural architectures and a large-scale public dataset show that models trained with demographically-balanced training sets exhibit a fairer behavior on different groups, while still being accurate. Our study is expected to provide a solid basis for instilling beyond-accuracy objectives (e.g., fairness) in speaker recognition.
Multi-Task Learning of Query Intent and Named Entities using Transfer Learning

Named entity recognition (NER) has been studied extensively and the earlier algorithms were based on sequence labeling like Hidden Markov Models (HMM) and conditional random fields (CRF). These were followed by neural network based deep learning models. Recently, BERT has shown new state of the art accuracy in sequence labeling tasks like NER. In this short article, we study various approaches to task specific NER. Task specific NER has two components - identifying the intent of a piece of text (like search queries), and then labeling the query with task specific named entities. For example, we consider the task of labeling Target store locations in a search query (which could be entered in a search box or spoken in a device like Alexa or Google Home). Store locations are highly ambiguous and sometimes it is difficult to differentiate between say a location and a non-location. For example, "pickup my order at orange store" has "orange" as the store location, while "buy orange at target" has "orange" as a fruit. We explore this difficulty by doing multi-task learning which we call global to local transfer of information. We jointly learn the query intent (i.e. store lookup) and the named entities by using multiple loss functions in our BERT based model and find interesting results.
Algorithmic Factors Influencing Bias in Machine Learning

It is fair to say that many of the prominent examples of bias in Machine Learning (ML) arise from bias that is there in the training data. In fact, some would argue that supervised ML algorithms cannot be biased, they reflect the data on which they are trained. In this paper we demonstrate how ML algorithms can misrepresent the training data through underestimation. We show how irreducible error, regularization and feature and class imbalance can contribute to this underestimation. The paper concludes with a demonstration of how the careful management of synthetic counterfactuals can ameliorate the impact of this underestimation bias.
A review on physical and data-driven based nowcasting methods using sky images

Amongst all the renewable energy resources (RES), solar is the most popular form of energy source and is of particular interest for its widely integration into the power grid. However, due to the intermittent nature of solar source, it is of the greatest significance to forecast solar irradiance to ensure uninterrupted and reliable power supply to serve the energy demand. There are several approaches to perform solar irradiance forecasting, for instance satellite-based methods, sky image-based methods, machine learning-based methods, and numerical weather prediction-based methods. In this paper, we present a review on short-term intra-hour solar prediction techniques known as nowcasting methods using sky images. Along with this, we also report and discuss which sky image features are significant for the nowcasting methods.
Filter Distribution Templates in Convolutional Networks for Image Classification Tasks

Neural network designers have reached progressive accuracy by increasing models depth, introducing new layer types and discovering new combinations of layers. A common element in many architectures is the distribution of the number of filters in each layer. Neural network models keep a pattern design of increasing filters in deeper layers such as those in LeNet, VGG, ResNet, MobileNet and even in automatic discovered architectures such as NASNet. It remains unknown if this pyramidal distribution of filters is the best for different tasks and constrains. In this work we present a series of modifications in the distribution of filters in four popular neural network models and their effects in accuracy and resource consumption. Results show that by applying this approach, some models improve up to 8.9% in accuracy showing reductions in parameters up to 54%.
Sign Segmentation with Changepoint-Modulated Pseudo-Labelling

The objective of this work is to find temporal boundaries between signs in continuous sign language. Motivated by the paucity of annotation available for this task, we propose a simple yet effective algorithm to improve segmentation performance on unlabelled signing footage from a domain of interest. We make the following contributions: (1) We motivate and introduce the task of source-free domain adaptation for sign language segmentation, in which labelled source data is available for an initial training phase, but is not available during adaptation. (2) We propose the Changepoint-Modulated Pseudo-Labelling (CMPL) algorithm to leverage cues from abrupt changes in motion-sensitive feature space to improve pseudo-labelling quality for adaptation. (3) We showcase the effectiveness of our approach for category-agnostic sign segmentation, transferring from the BSLCORPUS to the BSL-1K and RWTH-PHOENIX-Weather 2014 datasets, where we outperform the prior state of the art.
A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning

Many reinforcement learning algorithms rely on value estimation, however, the most widely used algorithms -- namely temporal difference algorithms -- can diverge under both off-policy sampling and nonlinear function approximation. Many algorithms have been developed for off-policy value estimation based on the linear mean squared projected Bellman error (MSPBE) and are sound under linear function approximation. Extending these methods to the nonlinear case has been largely unsuccessful. Recently, several methods have been introduced that approximate a different objective -- the mean-squared Bellman error (MSBE) -- which naturally facilitate nonlinear approximation. In this work, we build on these insights and introduce a new generalized MSPBE that extends the linear MSPBE to the nonlinear setting. We show how this generalized objective unifies previous work and obtain new bounds for the value error of the solutions of the generalized objective. We derive an easy-to-use, but sound, algorithm to minimize the generalized objective, and show that it is more stable across runs, is less sensitive to hyperparameters, and performs favorably across four control domains with neural network function approximation.
Batebit Controller: Popularizing Digital Musical Instruments Development Process

In this paper, we present an ongoing research project related to popularizing the mindset of building new digital musical instruments. We developed a physical kit and software intended to provide beginner users with the first grasp on the development process of a digital musical instrument. We expect that, by using the kit and the software, the users could experiment in a short period the various steps in developing a DMI such as physical structure, electronics, programming, mapping, and sound design. Our approach to popularizing the DMI development process is twofold: reducing the cognitive load for beginners by encapsulating technical details and lowering the costs of the kit by using simple components and open-source software. In the end, we expect that by increasing the interest of beginners in the building process of digital musical instruments, we could make the community of new interfaces for musical expression stronger.
A Chromium-based Memento-aware Web Browser

Web browsers provide a user-friendly means of navigating the web. Users rely on their web browser to provide information about the websites they are visiting, such as the security state. Browsers also provide a user interface (UI) with visual cues about each tab that is open, including icons for if the tab is playing audio or requires authentication to view. However, current browsers do not differentiate between the live web and the past web. If a user loads an archived webpage, known as a memento, they have to rely on UI elements present within the page itself to inform them that the page they are viewing is not the live web. Additionally, memento-awareness extends beyond recognizing a page that has already been archived. The browser should give users the ability to archive live webpages, essentially creating mementos of webpages they found important as they surf the web. In this report, the process to create a proof-of-concept browser that is aware of mementos is presented. The browser is created by adding on to the implementation of the open source web browser by Google, Chromium. Creating this prototype for a Memento-aware Browser shows that the features implemented fit well into the current Chromium implementation. The user experience is enhanced by adding the memento-awareness, and the changes to the Chromium code base are minimal.
Rethinking BiSeNet For Real-time Semantic Segmentation

BiSeNet has been proved to be a popular two-stream network for real-time segmentation. However, its principle of adding an extra path to encode spatial information is time-consuming, and the backbones borrowed from pretrained tasks, e.g., image classification, may be inefficient for image segmentation due to the deficiency of task-specific design. To handle these problems, we propose a novel and efficient structure named Short-Term Dense Concatenate network (STDC network) by removing structure redundancy. Specifically, we gradually reduce the dimension of feature maps and use the aggregation of them for image representation, which forms the basic module of STDC network. In the decoder, we propose a Detail Aggregation module by integrating the learning of spatial information into low-level layers in single-stream manner. Finally, the low-level features and deep features are fused to predict the final segmentation results. Extensive experiments on Cityscapes and CamVid dataset demonstrate the effectiveness of our method by achieving promising trade-off between segmentation accuracy and inference speed. On Cityscapes, we achieve 71.9% mIoU on the test set with a speed of 250.4 FPS on NVIDIA GTX 1080Ti, which is 45.2% faster than the latest methods, and achieve 76.8% mIoU with 97.0 FPS while inferring on higher resolution images.
From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence

We take inspiration from the study of human explanation to inform the design and evaluation of interpretability methods in machine learning. First, we survey the literature on human explanation in philosophy, cognitive science, and the social sciences, and propose a list of design principles for machine-generated explanations that are meaningful to humans. Using the concept of weight of evidence from information theory, we develop a method for generating explanations that adhere to these principles. We show that this method can be adapted to handle high-dimensional, multi-class settings, yielding a flexible framework for generating explanations. We demonstrate that these explanations can be estimated accurately from finite samples and are robust to small perturbations of the inputs. We also evaluate our method through a qualitative user study with machine learning practitioners, where we observe that the resulting explanations are usable despite some participants struggling with background concepts like prior class probabilities. Finally, we conclude by surfacing~design~implications for interpretability tools in general.
Controlling earthquake-like instabilities using artificial intelligence

Earthquakes are lethal and costly. This study aims at avoiding these catastrophic events by the application of injection policies retrieved through reinforcement learning. With the rapid growth of artificial intelligence, prediction-control problems are all the more tackled by function approximation models that learn how to control a specific task, even for systems with unmodeled/unknown dynamics and important uncertainties. Here, we show for the first time the possibility of controlling earthquake-like instabilities using state-of-the-art deep reinforcement learning techniques. The controller is trained using a reduced model of the physical system, i.e, the spring-slider model, which embodies the main dynamics of the physical problem for a given earthquake magnitude. Its robustness to unmodeled dynamics is explored through a parametric study. Our study is a first step towards minimizing seismicity in industrial projects (geothermal energy, hydrocarbons production, CO2 sequestration) while, in a second step for inspiring techniques for natural earthquakes control and prevention.