Reviewerly/exHarmony huggingface dataset:
Leveraging SPARQL Queries for UML Consistency Checking

Context and motivation: Multiple-viewed requirements modeling method describes the system to-be from different perspectives. Some requirements models are then specified in various UML diagrams. Question/problem: Managing those models can be tedious and error-prone, since a lot of CASE tools provide poor support for reasoning and consistency checking. Principal ideas/results: Ontology is a formal notation for describing concepts and their relations in a domain. Since software requirements are a kind of knowledge, we propose to adopt a knowledge engineering approach for managing the consistency of requirements models. In this paper, an ontology for three most commonly used UML diagrams is developed in Web Ontology Language (OWL). The transformation of UML class, sequence and state diagrams to OWL knowledge base is presented. Owing to the underlying logical reasoning capability of OWL, a semantic query language, SPARQL (SPARQL Protocol and RDF Query Language), is used to query the knowledge base for consistency checking. Contribution: This paper introduces a semantic web-based knowledge engineering approach to represent and manage software requirements knowledge in OWL. By experimenting with a concrete software system, we demonstrate the feasibility and applicability of this knowledge approach.
Short-Time Spectral Aggregation for Speaker Embedding

State-of-the-art speaker verification systems take frame-level acoustics features as input and produce fixed-dimensional embeddings as utterance-level representations. Thus, how to aggregate information from frame-level features is vital for achieving high performance. This paper introduces short-time spectral pooling (STSP) for better aggregation of frame-level information. STSP transforms the temporal feature maps of a speaker embedding network into the spectral domain and extracts the lowest spectral components of the averaged spectrograms for aggregation. Benefiting from the low-pass characteristic of the averaged spectrograms, STSP is able to preserve most of the speaker information in the feature maps using a few spectral components only. We show that statistics pooling is a special case of STSP where only the DC spectral components are used. Experiments on VoxCeleb1 and VOiCES 2019 show that STSP outperforms statistics pooling and multi-head attentive pooling, which suggests that leveraging more spectral information in the CNN feature maps can produce highly discriminative speaker embeddings.
Leveraging Deep CNN and Transfer Learning for Side-Channel Attack

The adoption of deep neural networks for profiled side-channel attacks provides powerful options for leakage detection and key retrieval of secure products. Although deep learning is being widely adopted for computer vision, less research has been prominent in template-based profiling power SCA attacks. In addition, most of the existing works fall into a one-dimensional (1-D) CNN technique rather than two-dimensional (2-D) CNN methods. Training a deep 2-D CNN from scratch is computationally expensive and requires a large amount of training data. To overcome these challenges, we adopt deep 2-D CNNs, GoogLeNet, InceptionV3, VGG16, and MobileNetV2 pre-trained to identify all possible AES key bytes. In order to use 2-D CNN and transfer learning, we also propose a novel multi-scale continuous wavelet transform of the power traces and generate scalograms from the wavelet coefficients. Moreover, we propose ResNet 1-D CNN architecture using power traces signal to break AES-128 implementation. To evaluate our proposed work, a key rank metric with the ASCAD dataset is utilized. Our proposed deep CNN framework achieves ≥ 99 accuracy when key rank is less than 10.
Revisiting Sequence-to-Sequence Video Object Segmentation with Multi-Task Loss and Skip-Memory

Video Object Segmentation (VOS) is an active research area of the visual domain. One of its fundamental subtasks is semi-supervised / one-shot learning: given only the segmentation mask for the first frame, the task is to provide pixel-accurate masks for the object over the rest of the sequence. Despite much progress in the last years, we noticed that many of the existing approaches lose objects in longer sequences, especially when the object is small or briefly occluded. In this work, we build upon a sequence-to-sequence approach that employs an encoder-decoder architecture together with a memory module for exploiting the sequential data. We further improve this approach by proposing a model that manipulates multiscale spatio-temporal information using memory-equipped skip connections. Furthermore, we incorporate an auxiliary task based on distance classification which greatly enhances the quality of edges in segmentation masks. We compare our approach to the state of the art and show considerable improvement in the contour accuracy metric and the overall segmentation accuracy. Our source code and the pre-trained weights are publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sup> https://github.com/fatemehazimi990/RS2S.
Tessera: Discretizing Data Analysis Workflows on a Task Level

Researchers have investigated a number of strategies for capturing and analyzing data analyst event logs in order to design better tools, identify failure points, and guide users. However, this remains challenging because individual- and session-level behavioral differences lead to an explosion of complexity and there are few guarantees that log observations map to user cognition. In this paper we introduce a technique for segmenting sequential analyst event logs which combines data, interaction, and user features in order to create discrete blocks of goal-directed activity. Using measures of inter-dependency and comparisons between analysis states, these blocks identify patterns in interaction logs coupled with the current view that users are examining. Through an analysis of publicly available data and data from a lab study across a variety of analysis tasks, we validate that our segmentation approach aligns with users’ changing goals and tasks. Finally, we identify several downstream applications for our approach.
Handling Class Imbalance in Low-Resource Dialogue Systems by Combining Few-Shot Classification and Interpolation

Utterance classification performance in low-resource dialogue systems is constrained by an inevitably high degree of data imbalance in class labels. We present a new end-to-end pairwise learning framework that is designed specifically to tackle this phenomenon by inducing a few-shot classification capability in the utterance representations and augmenting data through an interpolation of utterance representations. Our approach is a general purpose training methodology, agnostic to the neural architecture used for encoding utterances. We show significant improvements in macro-F1 score over standard cross-entropy training for three different neural architectures, demonstrating improvements on a Virtual Patient dialogue dataset as well as a low-resourced emulation of the Switchboard dialogue act classification dataset.
Explainable Artificial Intelligence Requirements for Safe, Intelligent Robots

While requirements for robot performance to perform a task are generally well understood, the requirements around the explanatory capabilities of these systems are often at best an afterthought. This results in a dangerous situation where neither users nor experts can predict situations where the robot will or will not work, nor understand what causes failures and unexpected behaviour. In this paper, we discuss and survey the field of Explainable Artificial Intelligence, as it relates to the generation of requirements for the development of safe, intelligent robots. We then present a categorisation of explanatory capabilities and requirements that aims to help users and developers alike to ensure an appropriate match between the types of explanations that a given application requires, and the capabilities of various underlying AI techniques.
A Survey on Occluded Face recognition

The human face is a research hotspot in the existing biometric technology. The collection of the faces is affected by unpredictable factors in reality, such as expression change, posture change, illumination, occlusion, and so on, which will reduce the performance of the face recognition system. Occluded face recognition is a long-standing and challenging problem. There are scanty studies on occlusion issues compared with other problems. Occlusion face recognition has profound application potential at present, so it is imperative to deal with the occlusion problem effectively. The first primary problem of face recognition is faced with face detection, so we introduce general face detection and occluded face detection. For general face detection, we briefly describe the methods based on template matching and deep learning. Then we analyze the existing methods of occluded face detection from two aspects, which are 1) based on manual design features and 2) based on deep learning. Secondly, we focus on occluded face recognition from the perspectives of 2D and 3D. With regard to 2D face images, we make an analysis from two categories in detail, which are 1) robust feature extraction and 2) subspace regression. 3D scanners and other 3D acquisition technologies are developing rapidly, and 3D face images contain rich information inherent in the face itself and are robust to occlusion. 3D face recognition is inevitable to become the mainstream in the field of face recognition. Therefore, we analyze and summarize the three popular aspects, which are 1) based on surface information, 2) based on local surface information, and 3) based on deep learning. Finally, we analyze the current problems and future research directions.
Collective Intelligence of Autonomous Animals in VR Hunting

In the scenario of a VR hunting game, autonomous behaviour of in-game animals is essential. In this study, new adaptive steering algorithms are designed for autonomous animals to navigate around the environment, with a research focus on collective intelligence in decision making and tactical actions. Advanced strategies for a group of autonomous animals are developed in order to simulate a more realistic forest environment. Computational experiments and comparisons with animation results are presented, accompanied by a demo video, which show significant advantages over previous work. The new models and algorithms can also be used for autonomous motion controls for other XR-based training.
Decomposition-Based Watermarking of Quantum Circuits

Quantum computing is an emerging computing paradigm that is expected to offer decisive advantages for critical applications. The design flow of quantum circuits poses problematic security challenges in the presence of variable errors and limited resources (qubits) to support error correction. Yet, quantum computers can be very powerful for several applications if proper hardware/software approaches are deployed.In the current embodiment, quantum circuits are expected to be used for several applications including financial analysis and machine learning in which a designer wants to protect the Intellectual Property (IP) of the quantum circuits. In this paper, we propose a watermarking approach for quantum circuits, which embeds a secret signature during the decomposition phase to verify the ownership of the IP. We evaluate the effectiveness of the proposed approach using Quantum Approximate optimization Algorithm (QAOA) used for solving a Max-Cut problem.
Learning to recognize while learning to speak: Self-supervision and developing a speaking motor

Traditionally, learning speech synthesis and speech recognition were investigated as two separate tasks. This separation hinders incremental development for concurrent synthesis and recognition, where partially-learned synthesis and partially-learned recognition must help each other throughout lifelong learning. This work is a paradigm shift—we treat synthesis and recognition as two intertwined aspects of a lifelong learning agent. Furthermore, in contrast to existing recognition or synthesis systems, babies do not need their mothers to directly supervise their vocal tracts at every moment during the learning. We argue that self-generated non-symbolic states/actions at fine-grained time level help such a learner as necessary temporal contexts. Here, we approach a new and challenging problem—how to enable an autonomous learning system to develop an artificial speaking motor for generating temporally-dense (e.g., frame-wise) actions on the fly without human handcrafting a set of symbolic states. The self-generated states/actions are Muscles-like, High-dimensional, Temporally-dense and Globally-smooth (MHTG), so that these states/actions are directly attended for concurrent synthesis and recognition for each time frame. Human teachers are relieved from supervising learner's motor ends. The Candid Covariance-free Incremental (CCI) Principal Component Analysis (PCA) is applied to develop such an artificial speaking motor where PCA features drive the motor. Since each life must develop normally, each Developmental Network-2 (DN-2) reaches the same network (maximum likelihood, ML) regardless of randomly initialized weights, where ML is not just for a function approximator but rather an emergent Turing Machine. The machine-synthesized sounds are evaluated by both the neural network and humans with recognition experiments. Our experimental results showed learning-to-synthesize and learning-to-recognize-through-synthesis for phonemes. This work corresponds to a key step toward our goal to close a great gap toward fully autonomous machine learning directly from the physical world.
Streaming End-to-End Speech Recognition with Jointly Trained Neural Feature Enhancement

In this paper, we present a streaming end-to-end speech recognition model based on Monotonic Chunkwise Attention (MoCha) jointly trained with enhancement layers. Even though the MoCha attention enables streaming speech recognition with recognition accuracy comparable to a full attention-based approach, training this model is sensitive to various factors such as the difficulty of training examples, hyper-parameters, and so on. Because of these issues, speech recognition accuracy of a MoCha-based model for clean speech drops significantly when a multi-style training approach is applied. Inspired by Curriculum Learning [1], we introduce two training strategies: Gradual Application of Enhanced Features (GAEF) and Gradual Reduction of Enhanced Loss (GREL). With GAEF, the model is initially trained using clean features. Subsequently, the portion of outputs from the enhancement layers gradually increases. With GREL, the portion of the Mean Squared Error (MSE) loss for the enhanced output gradually reduces as training proceeds. In experimental results on the LibriSpeech corpus and noisy far-field test sets, the proposed model with GAEF-GREL training strategies shows significantly better results than the conventional multi-style training approach.
Inconsistency-tolerant query answering for existential rules

Querying inconsistent knowledge bases is an intriguing problem that gave rise to a flourishing research activity in the knowledge representation and reasoning community during the last years. It has been extensively studied in the context of description logics (DLs), and its computational complexity is rather well-understood. Although DLs are popular formalisms for modeling ontologies, it is generally agreed that rule-based ontologies are well-suited for data-intensive applications, since they allow us to conveniently deal with higher-arity relations, which naturally occur in standard relational databases. The goal of this work is to perform an in-depth complexity analysis of querying inconsistent knowledge bases in the case of the main decidable classes of existential rules, based on the notions of guardedness, linearity, acyclicity, and stickiness, enriched with negative (a.k.a. denial) constraints. Our investigation concentrates on three central inconsistency-tolerant semantics: the ABox repair (AR) semantics, considered as the standard one, and its main sound approximations, the intersection of repairs (IAR) semantics and the intersection of closed repairs (ICR) semantics.
Online Domain Adaptation for Person Re-Identification with a Human in the Loop

Supervised deep learning methods have recently achieved remarkable performance in person re-identification. Unsupervised domain adaptation (UDA) approaches have also been proposed for application scenarios where only unlabelled data are available from target camera views. We consider a more challenging scenario when even collecting a suitable amount of representative, unlabelled target data for offline training or fine-tuning is infeasible. In this context we revisit the human-in-the-loop (HITL) approach, which exploits online the operator's feedback on a small amount of target data. We argue that HITL is a kind of online domain adaptation specifically suited to person re-identification. We then reconsider relevance feedback methods for content-based image retrieval that are computationally much cheaper than state-of-the-art HITL methods for person reidentification, and devise a specific feedback protocol for them. Experimental results show that HITL can achieve comparable or better performance than UDA, and is therefore a valid alternative when the lack of unlabelled target data makes UDA infeasible.
Improved Data Selection for Domain Adaptation in ASR

Automatic speech recognition (ASR) systems are highly sensitive to train-test domain mismatch. However, because transcription is often prohibitively expensive, it is important to be able to make use of available transcribed out-of-domain data. We address the problem of domain adaptation with semi-supervised training (SST). Contrary to work in in-domain SST, we find significant performance improvement even with just one hour of target-domain data—though, the selection of the data is critical. We show that minimum phone error rate is a good oracle measure for selection, and we approximate this measure by using the average phone confidence of an utterance. With larger domain shifts, we also find that deletions and low lexical diversity are a serious issue, which we address by incorporating phone rate into our selection metric. With our proposed selection criterion, we see up to 57% relative improvements over the out-of-domain baseline model. Furthermore, this selection method generalizes well, and matches or outperforms word-level confidence selection across six separate domain shift conditions.
Game Theoretic Approaches to Mitigate Cloud Security Risks: An Initial Insight

Cloud computing is one of the most promising innovations impacting data storage and processing. And with it, clients rely on the IT solutions offered by an external provider instead of on-premise applications. Despite its enormous impacts, customers are still reluctant to outsource their business processes because of security concerns. As data is typically stored and governed by cloud vendors, users need to deal with security issues linked to the loss of control over their sensitive data. Cloud providers need to implement the appropriate security measures that might attract more clients while making the minimum investment. While used in various disciplines, game theory has recently expanded to investigate the effect of the defenders’ and attackers’ behaviors on strategic decision-making. This study aims to develop insights into how game theory can develop better security policies in cloud computing. First, we perform threat modeling to identify the potential threats facing cloud. Second, we identify the limitations of existing game based solutions and then suggest an improved model define an adequate strategy that would figure out the right balance between the required security level and the profit margins. Besides, we present future directions that can be explored to build highly reliable and optimal strategies for cloud services.
Automated defect prioritization based on defects resolved at various project periods

Defect prioritization is mainly a manual and error-prone task in the current state-of-the-practice. We evaluated the effectiveness of an automated approach that employs supervised machine learning. We used two alternative techniques, namely a Naive Bayes classifier and a Long Short-Term Memory model. We performed an industrial case study with a real project from the consumer electronics domain. We compiled more than 15,000 issues collected over 3 years. We could reach an accuracy level up to 79.36% and we had 3 observations. First, Long Short-Term Memory model has a better accuracy when compared with a Naive Bayes classifier. Second, structured features lead to better accuracy compared to textual descriptions. Third, accuracy is not improved by considering increasingly earlier defects as part of the training data. Increasing the size of the training data even decreases the accuracy compared to the results, when we use data only regarding the recently resolved defects.
Comparative Study of Different Image Captioning Models

This paper has compared various deep learning models for generating caption of images gathered from Flickr 8k Dataset. Also, this research work attempts to combine a CNN type encoder for extracting features from images and a Recurrent Neural Network for generating caption for the extracted features. The CNN encoders used are VGG16 and InceptionV3. The extracted features are then passed to a unidirectional or a bidirectional LSTM for generating captions. The proposed model has used beam search as well as greedy algorithms to generate captions from vocabulary. The generated captions are then compared with actual captions with the help of BLEU scores. The Bilingual Evaluation Understudy score (BLEU) is used to compare how close a given sentence is to another sentence. The BLEU score of captions generated using beam search as well as greedy algorithms are analyzed and compared to see which is better.
Information Extraction on Requirement Prioritization Approaches in Agile Software Development Processes

Requirement Engineering is the process to gather the software requirements from end-users. Prime objectives of requirement engineering include analysis, documentation, and development of system requirement specification. Prioritization of gathered requirements of a project is an uphill task for justifiable requirement engineering especially at the early phases of the development. Through relevant requirement prioritization, we can identify the exact requirements that must be included in a specific release. The main objective behind requirement prioritization is to decrease the associated risks in the development process. In Agile Software Development processes, it is not a cakewalk to prioritize the relevant requirements because requirements are welcomed even late in development phases in agile methodologies. This paper extracts information on discrete requirement prioritization approaches along with their strength and weaknesses. Through this paper, product owners will be in a condition to finalize a relevant prioritization approach according to the project's behavior and other constraints.