OpenRaiser/Intern-Atlas huggingface dataset:
AccoMontage2: A Complete Harmonization and Accompaniment Arrangement System
Venue: arXiv cs.SD 2022
Year: 2022

We propose AccoMontage2, a system capable of doing full-length song harmonization and accompaniment arrangement based on a lead melody. Following AccoMontage, this study focuses on generating piano arrangements for popular/folk songs and it carries on the generalized template-based retrieval method. The novelties of this study are twofold. First, we invent a harmonization module (which AccoMontage does not have). This module generates structured and coherent full-length chord progression by optimizing and balancing three loss terms: a micro-level loss for note-wise dissonance, a meso-level loss for phrase-template matching, and a macro-level loss for full piece coherency. Second, we develop a graphical user interface which allows users to select different styles of chord progression and piano texture. Currently, chord progression styles include Pop, R&B, and Dark, while piano texture styles include several levels of voicing density and rhythmic complexity. Experimental results show that both our harmonization and arrangement results significantly outperform the baselines. Lastly, we release AccoMontage2 as an online application as well as the organized chord progression templates as a public dataset.
Gait Recognition in the Wild with Multi-hop Temporal Switch
Venue: arXiv cs.CV 2022
Year: 2022

Existing studies for gait recognition are dominated by in-the-lab scenarios. Since people live in real-world senses, gait recognition in the wild is a more practical problem that has recently attracted the attention of the community of multimedia and computer vision. Current methods that obtain state-of-the-art performance on in-the-lab benchmarks achieve much worse accuracy on the recently proposed in-the-wild datasets because these methods can hardly model the varied temporal dynamics of gait sequences in unconstrained scenes. Therefore, this paper presents a novel multi-hop temporal switch method to achieve effective temporal modeling of gait patterns in real-world scenes. Concretely, we design a novel gait recognition network, named Multi-hop Temporal Switch Network (MTSGait), to learn spatial features and multi-scale temporal features simultaneously. Different from existing methods that use 3D convolutions for temporal modeling, our MTSGait models the temporal dynamics of gait sequences by 2D convolutions. By this means, it achieves high efficiency with fewer model parameters and reduces the difficulty in optimization compared with 3D convolution-based models. Based on the specific design of the 2D convolution kernels, our method can eliminate the misalignment of features among adjacent frames. In addition, a new sampling strategy, i.e., non-cyclic continuous sampling, is proposed to make the model learn more robust temporal features. Finally, the proposed method achieves superior performance on two public gait in-the-wild datasets, i.e., GREW and Gait3D, compared with state-of-the-art methods.
How to Evaluate Explainability? -- A Case for Three Criteria
Venue: arXiv cs.SE 2022
Year: 2022

The increasing complexity of software systems and the influence of software-supported decisions in our society have sparked the need for software that is safe, reliable, and fair. Explainability has been identified as a means to achieve these qualities. It is recognized as an emerging non-functional requirement (NFR) that has a significant impact on system quality. However, in order to develop explainable systems, we need to understand when a system satisfies this NFR. To this end, appropriate evaluation methods are required. However, the field is crowded with evaluation methods, and there is no consensus on which are the "right" ones. Much less, there is not even agreement on which criteria should be evaluated. In this vision paper, we will provide a multidisciplinary motivation for three such quality criteria concerning the information that systems should provide: comprehensibility, fidelity, and assessability. Our aim is to to fuel the discussion regarding these criteria, such that adequate evaluation methods for them will be conceived.
The Nudging Effect on Tracking Activity
Venue: arXiv cs.HC 2022
Year: 2022

Wearables activity trackers are becoming widely adopted to understand individual behavior. Understanding behavior may help in self-regulation such as self-monitoring, goal-setting, self-corrective, etc.; Nevertheless, challenges exist in attaining consistent use and adoption of wearables, which hinders behavior understanding. Research has suggested that nudging strategies may change and sustain human engagement. However, it is still unknown how nudging may affect human wearing behavior on an individual level. We conducted a six-month study in which we tested several nudging techniques on the same participants. The preliminary results of our research show that participants perform better when a nudging strategy is applied. In addition, participants responded differently to different nudging techniques. Future research can focus on developing an individual-based nudging mechanism to encourage users to wear their devices consistently.
On the detection of morphing attacks generated by GANs
Venue: arXiv cs.CV 2022
Year: 2022

Recent works have demonstrated the feasibility of GAN-based morphing attacks that reach similar success rates as more traditional landmark-based methods. This new type of "deep" morphs might require the development of new adequate detectors to protect face recognition systems. We explore simple deep morph detection baselines based on spectral features and LBP histograms features, as well as on CNN models, both in the intra-dataset and cross-dataset case. We observe that simple LBP-based systems are already quite accurate in the intra-dataset setting, but struggle with generalization, a phenomenon that is partially mitigated by fusing together several of those systems at score-level. We conclude that a pretrained ResNet effective for GAN image detection is the most effective overall, reaching close to perfect accuracy. We note however that LBP-based systems maintain a level of interest : additionally to their lower computational requirements and increased interpretability with respect to CNNs, LBP+ResNet fusions sometimes also showcase increased performance versus ResNet-only, hinting that LBP-based systems can focus on meaningful signal that is not necessarily picked up by the CNN detector.
Agile Effort Estimation: Have We Solved the Problem Yet? Insights From A Second Replication Study (GPT2SP Replication Report)
Venue: arXiv cs.SE 2022
Year: 2022

Fu and Tantithamthavorn have recently proposed GPT2SP, a Transformer-based deep learning model for SP estimation of user stories. They empirically evaluated the performance of GPT2SP on a dataset shared by Choetkiertikul et al including 16 projects with a total of 23,313 issues. They benchmarked GPT2SP against two baselines (namely the naive Mean and Median estimators) and the method previously proposed by Choetkiertikul et al. (which we will refer to as DL2SP from now on) for both within- and cross-project estimation scenarios, and evaluated the extent to which each components of GPT2SP contribute towards the accuracy of the SP estimates. Their results show that GPT2SP outperforms DL2SP with a 6%-47% improvement over MAE for the within-project scenario and a 3%-46% improvement for the cross-project scenarios. However, when we attempted to use the GPT2SP source code made available by Fu and Tantithamthavorn to reproduce their experiments, we found a bug in the computation of the Mean Absolute Error (MAE), which may have inflated the GPT2SP's accuracy reported in their work. Therefore, we had issued a pull request to fix such a bug, which has been accepted and merged into their repository at https://github.com/awsm-research/gpt2sp/pull/2. In this report, we describe the results we achieved by using the fixed version of GPT2SP to replicate the experiments conducted in the original paper for RQ1 and RQ2. Following the original study, we analyse the results considering the Medan Absolute Error (MAE) of the estimation methods over all issues in each project, but we also report the Median Absolute Error (MdAE) and the Standard accuracy (SA) for completeness.
On the Complexity of the Storyplan Problem
Venue: arXiv cs.CC 2022
Year: 2022

Motivated by dynamic graph visualization, we study the problem of representing a graph $G$ in the form of a \emph{storyplan}, that is, a sequence of frames with the following properties. Each frame is a planar drawing of the subgraph of $G$ induced by a suitably defined subset of its vertices. Between two consecutive frames, a new vertex appears while some other vertices may disappear, namely those whose incident edges have already been drawn in at least one frame. In a storyplan, each vertex appears and disappears exactly once. For a vertex (edge) visible in a sequence of consecutive frames, the point (curve) representing it does not change throughout the sequence. Note that the order in which the vertices of $G$ appear in the sequence of frames is a total order. In the \textsc{StoryPlan} problem, we are given a graph and we want to decide whether there exists a total order of its vertices for which a storyplan exists. We prove that the problem is NP-complete, and complement this hardness with two parameterized algorithms, one in the vertex cover number and one in the feedback edge set number of $G$. Also, we prove that partial $3$-trees always admit a storyplan, which can be computed in linear time. Finally, we show that the problem remains NP-complete in the case in which the total order of the vertices is given as part of the input and we have to choose how to draw the frames.
Negation detection in Dutch clinical texts: an evaluation of rule-based and machine learning methods
Venue: arXiv cs.CL 2022
Year: 2022

As structured data are often insufficient, labels need to be extracted from free text in electronic health records when developing models for clinical information retrieval and decision support systems. One of the most important contextual properties in clinical text is negation, which indicates the absence of findings. We aimed to improve large scale extraction of labels by comparing three methods for negation detection in Dutch clinical notes. We used the Erasmus Medical Center Dutch Clinical Corpus to compare a rule-based method based on ContextD, a biLSTM model using MedCAT and (finetuned) RoBERTa-based models. We found that both the biLSTM and RoBERTa models consistently outperform the rule-based model in terms of F1 score, precision and recall. In addition, we systematically categorized the classification errors for each model, which can be used to further improve model performance in particular applications. Combining the three models naively was not beneficial in terms of performance. We conclude that the biLSTM and RoBERTa-based models in particular are highly accurate accurate in detecting clinical negations, but that ultimately all three approaches can be viable depending on the use case at hand.
Searching for Structure in Unfalsifiable Claims
Venue: arXiv cs.CL 2022
Year: 2022

Social media platforms give rise to an abundance of posts and comments on every topic imaginable. Many of these posts express opinions on various aspects of society, but their unfalsifiable nature makes them ill-suited to fact-checking pipelines. In this work, we aim to distill such posts into a small set of narratives that capture the essential claims related to a given topic. Understanding and visualizing these narratives can facilitate more informed debates on social media. As a first step towards systematically identifying the underlying narratives on social media, we introduce PAPYER, a fine-grained dataset of online comments related to hygiene in public restrooms, which contains a multitude of unfalsifiable claims. We present a human-in-the-loop pipeline that uses a combination of machine and human kernels to discover the prevailing narratives and show that this pipeline outperforms recent large transformer models and state-of-the-art unsupervised topic models.
Actor Prioritized Experience Replay
Venue: arXiv cs.LG 2022
Year: 2022

A widely-studied deep reinforcement learning (RL) technique known as Prioritized Experience Replay (PER) allows agents to learn from transitions sampled with non-uniform probability proportional to their temporal-difference (TD) error. Although it has been shown that PER is one of the most crucial components for the overall performance of deep RL methods in discrete action domains, many empirical studies indicate that it considerably underperforms actor-critic algorithms in continuous control. We theoretically show that actor networks cannot be effectively trained with transitions that have large TD errors. As a result, the approximate policy gradient computed under the Q-network diverges from the actual gradient computed under the optimal Q-function. Motivated by this, we introduce a novel experience replay sampling framework for actor-critic methods, which also regards issues with stability and recent findings behind the poor empirical performance of PER. The introduced algorithm suggests a new branch of improvements to PER and schedules effective and efficient training for both actor and critic networks. An extensive set of experiments verifies our theoretical claims and demonstrates that the introduced method significantly outperforms the competing approaches and obtains state-of-the-art results over the standard off-policy actor-critic algorithms.
Go-Explore Complex 3D Game Environments for Automated Reachability Testing
Venue: arXiv cs.AI 2022
Year: 2022

Modern AAA video games feature huge game levels and maps which are increasingly hard for level testers to cover exhaustively. As a result, games often ship with catastrophic bugs such as the player falling through the floor or being stuck in walls. We propose an approach specifically targeted at reachability bugs in simulated 3D environments based on the powerful exploration algorithm, Go-Explore, which saves unique checkpoints across the map and then identifies promising ones to explore from. We show that when coupled with simple heuristics derived from the game's navigation mesh, Go-Explore finds challenging bugs and comprehensively explores complex environments without the need for human demonstration or knowledge of the game dynamics. Go-Explore vastly outperforms more complicated baselines including reinforcement learning with intrinsic curiosity in both covering the navigation mesh and number of unique positions across the map discovered. Finally, due to our use of parallel agents, our algorithm can fully cover a vast 1.5km x 1.5km game world within 10 hours on a single machine making it extremely promising for continuous testing suites.
Addressing Hidden Imperfections in Online Experimentation
Venue: arXiv cs.SE 2022
Year: 2022

Technology companies are increasingly using randomized controlled trials (RCTs) as part of their development process. Despite having fine control over engineering systems and data instrumentation, these RCTs can still be imperfectly executed. In fact, online experimentation suffers from many of the same biases seen in biomedical RCTs including opt-in and user activity bias, selection bias, non-compliance with the treatment, and more generally, challenges in the ability to test the question of interest. The result of these imperfections can lead to a bias in the estimated causal effect, a loss in statistical power, an attenuation of the effect, or even a need to reframe the question that can be answered. This paper aims to make practitioners of experimentation more aware of imperfections in technology-industry RCTs, which can be hidden throughout the engineering stack or in the design process.
Analyse fonctionnelle de l'outil de gestion de planning LibStaffer
Venue: arXiv cs.HC 2022
Year: 2022

LibStaffer is a staff scheduling tool for libraries provided by Springshare. In this article, we present the analysis that we implemented to determine the potential of LibStaffer in order to simplify staff scheduling in library branches with various configurations, as well as in transversal units with a focus on public services. As a starting point, we focused on the questions available in Philippe Lenepveu and Marc Maisonneuve's work on scheduling tools for libraries. We then enriched the initial list with questions that emerged among several colleagues. After a two-month LibStaffer trial period, we were able to answer those questions and were convinced to take out a subscription to the tool. -- LibStaffer est un outil de gestion de planning de service pour les bibliothèques proposé par Springshare. Dans cet article, nous exposons l'analyse que nous avons mise en œuvre pour déterminer le potentiel de LibStaffer en matière de simplification de la gestion de planning d'accueil, sur des implantations de configurations diverses, ainsi que d'activités transversales de service au public. Nous nous sommes fondés sur les questions établies par Philippe Lenepveu et Marc Maisonneuve, dans leur ouvrage consacré aux logiciels de gestion de planning pour les bibliothèques, et les avons enrichies par les interrogations de plusieurs collègues. Bénéficier d'un test de LibStaffer, de près de deux mois, nous a permis de répondre à ces questions et nous a convaincus de prendre une souscription à l'outil.
A Technique to Create Weaker Abstract Board Game Agents via Reinforcement Learning
Venue: arXiv cs.AI 2022
Year: 2022

Board games, with the exception of solo games, need at least one other player to play. Because of this, we created Artificial Intelligent (AI) agents to play against us when an opponent is missing. These AI agents are created in a number of ways, but one challenge with these agents is that an agent can have superior ability compared to us. In this work, we describe how to create weaker AI agents that play board games. We use Tic-Tac-Toe, Nine-Men's Morris, and Mancala, and our technique uses a Reinforcement Learning model where an agent uses the Q-learning algorithm to learn these games. We show how these agents can learn to play the board game perfectly, and we then describe our approach to making weaker versions of these agents. Finally, we provide a methodology to compare AI agents.
In conversation with Artificial Intelligence: aligning language models with human values
Venue: arXiv cs.CY 2022
Year: 2022

Large-scale language technologies are increasingly used in various forms of communication with humans across different contexts. One particular use case for these technologies is conversational agents, which output natural language text in response to prompts and queries. This mode of engagement raises a number of social and ethical questions. For example, what does it mean to align conversational agents with human norms or values? Which norms or values should they be aligned with? And how can this be accomplished? In this paper, we propose a number of steps that help answer these questions. We start by developing a philosophical analysis of the building blocks of linguistic communication between conversational agents and human interlocutors. We then use this analysis to identify and formulate ideal norms of conversation that can govern successful linguistic communication between humans and conversational agents. Furthermore, we explore how these norms can be used to align conversational agents with human values across a range of different discursive domains. We conclude by discussing the practical implications of our proposal for the design of conversational agents that are aligned with these norms and values.
A Low-Cost Multi-Agent System for Physical Security in Smart Buildings
Venue: arXiv cs.CR 2022
Year: 2022

Modern organizations face numerous physical security threats, from fire hazards to more intricate concerns regarding surveillance and unauthorized personnel. Conventional standalone fire and intrusion detection solutions must be installed and maintained independently, which leads to high capital and operational costs. Nonetheless, due to recent developments in smart sensors, computer vision techniques, and wireless communication technologies, these solutions can be integrated in a modular and low-cost manner. This work introduces Integrated Physical Security System (IP2S), a multi-agent system capable of coordinating diverse Internet of Things (IoT) sensors and actuators for an efficient mitigation of multiple physical security events. The proposed system was tested in a live case study that combined fire and intrusion detection in an industrial shop floor environment with four different sectors, two surveillance cameras, and a firefighting robot. The experimental results demonstrate that the integration of several events in a single automated system can be advantageous for the security of smart buildings, reducing false alarms and delays.
Detection of False Data Injection Attacks in Smart Grid: A Secure Federated Deep Learning Approach
Venue: arXiv cs.CR 2022
Year: 2022

As an important cyber-physical system (CPS), smart grid is highly vulnerable to cyber attacks. Amongst various types of attacks, false data injection attack (FDIA) proves to be one of the top-priority cyber-related issues and has received increasing attention in recent years. However, so far little attention has been paid to privacy preservation issues in the detection of FDIAs in smart grid. Inspired by federated learning, a FDIA detection method based on secure federated deep learning is proposed in this paper by combining Transformer, federated learning and Paillier cryptosystem. The Transformer, as a detector deployed in edge nodes, delves deep into the connection between individual electrical quantities by using its multi-head self-attention mechanism. By using federated learning framework, our approach utilizes the data from all nodes to collaboratively train a detection model while preserving data privacy by keeping the data locally during training. To improve the security of federated learning, a secure federated learning scheme is designed by combing Paillier cryptosystem with federated learning. Through extensive experiments on the IEEE 14-bus and 118-bus test systems, the effectiveness and superiority of the proposed method are verifed.
Constrained Riemannian Noncoherent Constellations for the MIMO Multiple Access Channel
Venue: arXiv cs.IT 2022
Year: 2022

We consider the design of multiuser constellations for a multiple access channel (MAC) with K users, with M antennas each, that transmit simultaneously to a receiver equipped with N antennas through a Rayleigh block-fading channel, when no channel state information (CSI) is available to either the transmitter or the receiver. In full-diversity scenarios where the coherence time is at least T>= (K+1)M, the proposed constellation design criterion is based on the asymptotic expression of the multiuser pairwise error probability (PEP) derived by Brehler and Varanasi. In non-full diversity scenarios, for which the previous PEP expression is no longer valid, the proposed design criteria are based on proxies of the PEP recently proposed by Ngo and Yang. Although both the PEP expression and its bounds or proxies were previously considered intractable for optimization, in this work we derive their respective unconstrained gradients. These gradients are in turn used in the optimization of the proposed cost functions in different Riemannian manifolds representing different power constraints. In particular, in addition to the standard unitary space-time modulation (USTM) leading to optimization on the Grassmann manifold, we consider a more relaxed per-codeword power constraint leading to optimization on the so-called oblique manifold, and an average power constraint leading to optimization on the so-called trace manifold. Equipped with these theoretical tools, we design multiuser constellations for the MIMO MAC in full-diversity and non-full-diversity scenarios with state-of-the-art performance in terms of symbol error rate (SER).