Reviewerly/exHarmony huggingface dataset:
Long Short-Term Memory for Spatial Encoding in Multi-Agent Path Planning

Reinforcement learning-based path planning for multi-agent systems of varying size constitutes a research topic with increasing significance as progress in domains such as urban air mobility and autonomous aerial vehicles continues.Reinforcement learning with continuous state and action spaces is used to train a policy network that accommodates desirable path planning behaviors and can be used for time-critical applications.A Long Short-Term Memory module is proposed to encode an unspecified number of states for a varying, indefinite number of agents.The described training strategies and policy architecture lead to a guidance that scales to an infinite number of agents and unlimited physical dimensions, although training takes place at a smaller scale.The guidance is implemented on a low-cost, off-theshelf onboard computer.The feasibility of the proposed approach is validated by presenting flight test results of up to four drones, autonomously navigating collision-free in a real-world environment.
Action Keypoint Network for Efficient Video Recognition

Reducing redundancy is crucial for improving the efficiency of video recognition models. An effective approach is to select informative content from the holistic video, yielding a popular family of dynamic video recognition methods. However, existing dynamic methods focus on either temporal or spatial selection independently while neglecting a reality that the redundancies are usually spatial and temporal, simultaneously. Moreover, their selected content is usually cropped with fixed shapes (e.g., temporally-cropped frames, spatially-cropped patches), while the realistic distribution of informative content can be much more diverse. With these two insights, this paper proposes to integrate temporal and spatial selection into an Action Keypoint Network (AK-Net). From different frames and positions, AK-Net selects some informative points scattered in arbitrary-shaped regions as a set of "action keypoints" and then transforms the video recognition into point cloud classification. More concretely, AK-Net has two steps, i.e., the keypoint selection and the point cloud classification. First, it inputs the video into a baseline network and outputs a feature map from an intermediate layer. We view each pixel on this feature map as a spatial-temporal point and select some informative keypoints using self-attention. Second, AK-Net devises a ranking criterion to arrange the keypoints into an ordered 1D sequence. Since the video is represented with a 1D sequence after the specified layer, AK-Net transforms the subsequent layers into a point cloud classification sub-net by compacting the original 2D convolutional kernels into 1D kernels. Consequentially, AK-Net brings two-fold benefits for efficiency: The keypoint selection step collects informative content within arbitrary shapes and increases the efficiency for modeling spatial-temporal dependencies, while the point cloud classification step further reduces the computational cost by compacting the convolutional kernels. Experimental results show that AK-Net can consistently improve the efficiency and performance of baseline methods on several video recognition benchmarks.
A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slots

A slot value might be provided segment by segment over multiple-turn interactions in a dialog, especially for some important information such as phone numbers and names.It is a common phenomenon in daily life, but little attention has been paid to it in previous work.To fill the gap, this paper defines a new task named Sub-Slot based Task-Oriented Dialog (SSTOD) and builds a Chinese dialog dataset SSD for boosting research on SSTOD.The dataset includes a total of 40K dialogs and 500K utterances from four different domains: Chinese names, phone numbers, ID numbers and license plate numbers.The data is well annotated with sub-slot values, slot values, dialog states and actions.We find some new linguistic phenomena and interactive manners in SSTOD which raise critical challenges of building dialog agents for the task.We test three state-of-the-art dialog models on SSTOD and find they cannot handle the task well on any of the four domains.We also investigate an improved model by involving slot knowledge in a plug-in manner.More work should be done to meet the new challenges raised from SSTOD which widely exists in real-life applications.
Attentive Temporal Pooling for Conformer-Based Streaming Language Identification in Long-Form Speech

In this paper, we introduce a novel language identification system based on conformer layers. We propose an attentive temporal pooling mechanism to allow the model to carry information in long-form audio via a recurrent form, such that the inference can be performed in a streaming fashion. Additionally, a simple domain adaptation mechanism is introduced to allow adapting an existing language identification model to a new domain where the prior language distribution is different. We perform a comparative study of different model topologies under different constraints of model size, and find that conformer-base models outperform LSTM and transformer based models. Our experiments also show that attentive temporal pooling and domain adaptation significantly improve the model accuracy.
Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback

For summarization, human preferences is critical to tame outputs of the summarizer in favor of human interests, as ground-truth summaries are scarce and ambiguous.Practical settings require dynamic exchanges between humans and AI agents wherein feedback is provided in an online manner, a few at a time.In this paper, we introduce a new framework to train summarization models with preference feedback interactively.By properly leveraging offline data and a novel reward model, we improve the performance regarding ROUGE scores and sample-efficiency.Our experiments on three various datasets confirm the benefit of the proposed framework in active, few-shot and online settings of preference learning.
Estimation of playable piano fingering by pitch-difference fingering match model

Abstract Most existing statistical models used to predict piano fingering apply explicit constraints among fingers and between fingers and notes; however, they disregard the relationship among notes. Furthermore, the state transfer matrix of HMM often makes the fingering of notes in compact scales unplayable without moving the hands. The direct adoption of notes interferes with mapping between specific pitches and the corresponding fingering. Inspired by human annotation and the note span constraints used in rule-based methods (in which fingering knowledge is acquired from span), we developed a model by which to match pitch difference and finger sequences (PdF). Playable fingering is achieved by combining learned finger-transfer knowledge with priori finger-transfer knowledge. The playability of the model was evaluated using a novel index, referred to as the irrational fingering rate (IFR). Experiment results demonstrate that the proposed model outperforms the third-order hidden Markov finger annotation model in terms of average match rate (by 4.06%) and highest match rate (by 2.87%). The proposed scheme also resolves the unplayable-without-hand-movement problem in compact scales.
Tailoring 2D fast iterative filtering algorithm for low-contrast optical fringe pattern preprocessing

Retrieving object phase from the optical fringe pattern is a critical task in quantitative phase imaging and often requires appropriate image preprocessing (background and noise minimization), especially when retrieving phase from the single-shot fringe pattern image. In this article, for the first time, we propose to adapt the 2D Fast Iterative Filtering (FIF) method for fringe pattern decomposition and develop a novel version of FIF called the 2D fringe pattern Fast Iterative Filtering (fpFIF2), that is tailored for fringe pattern preprocessing. We show the positive influence of fpFIF2 onto fringe pattern filtering comparing to the previous 2D FIF implementation regarding processing speed, quality, and usage comfortability. We also compare the fpFIF2 with other state-of-the-art fringe pattern filtering methods in terms of aiding the Hilbert spiral transform method in phase retrieval. Employing numerical simulations and experimental fringe analysis, we prove that fpFIF2 outperforms reference methods, especially in terms of low-fringe-contrast phase reconstruction quality and decomposition time.
Multi-dimensional parameter-space partitioning of spatio-temporal simulation ensembles

Numerical simulations are commonly used to understand the parameter dependence of given spatio-temporal phenomena. Sampling a multi-dimensional parameter space and running the respective simulations leads to an ensemble of a large number of spatio-temporal simulation runs. A main objective for analyzing the ensemble is to partition (or segment) the multi-dimensional parameter space into connected regions of simulation runs with similar behavior. To facilitate such an analysis, we propose a novel visualization method for multi-dimensional parameter-space partitions. Our visualization is based on the concept of a hyper-slicer, which allows for undistorted views of the parameter-space segments' extent and transitions. For navigation within the parameter space, interactions with a 2D embedding of the parameter-space samples, including their segment memberships, are supported. Parameter-space partitions are generated in a semi-automatic fashion by analyzing the similarity space of the ensemble's simulation runs. Clusters of similar simulation runs induce the segments of the parameter-space partition. We link the parameter-space partitioning visualizations to similarity-space visualizations of the ensemble's simulation runs and embed them into an interactive visual analysis tool that supports the analysis of all facets of the spatio-temporal simulation ensemble targeted at the overarching goal of analyzing the parameter-space partitioning. The partitioning can then be visually analyzed and interactively refined. We evaluated our approach with experts within case studies from three different domains.
Interpretable AI for policy-making in pandemics

Since the first wave of the COVID-19 pandemic, governments have applied restrictions in order to slow down its spreading. However, creating such policies is hard, especially because the government needs to trade-off the spreading of the pandemic with the economic losses. For this reason, several works have applied machine learning techniques, often with the help of special-purpose simulators, to generate policies that were more effective than the ones obtained by governments. While the performance of such approaches are promising, they suffer from a fundamental issue: since such approaches are based on black-box machine learning, their real-world applicability is limited, because these policies cannot be analyzed, nor tested, and thus they are not trustable. In this work, we employ a recently developed hybrid approach, which combines reinforcement learning with evolutionary computation, for the generation of interpretable policies for containing the pandemic. These policies, trained on an existing simulator, aim to reduce the spreading of the pandemic while minimizing the economic losses. Our results show that our approach is able to find solutions that are extremely simple, yet very powerful. In fact, our approach has significantly better performance (in simulated scenarios) than both previous work and government policies.
Metappearance

There currently exist two main approaches to reproducing visual appearance using Machine Learning (ML): The first is training models that generalize over different instances of a problem, e.g., different images of a dataset. As one-shot approaches, these offer fast inference, but often fall short in quality. The second approach does not train models that generalize across tasks, but rather over-fit a single instance of a problem, e.g., a flash image of a material. These methods offer high quality, but take long to train. We suggest to combine both techniques end-to-end using meta-learning: We over-fit onto a single problem instance in an inner loop, while also learning how to do so efficiently in an outer-loop across many exemplars. To this end, we derive the required formalism that allows applying meta-learning to a wide range of visual appearance reproduction problems: textures, BRDFs, svBRDFs, illumination or the entire light transport of a scene. The effects of meta-learning parameters on several different aspects of visual appearance are analyzed in our framework, and specific guidance for different tasks is provided. Metappearance enables visual quality that is similar to over-fit approaches in only a fraction of their runtime while keeping the adaptivity of general models.
Where is your app frustrating users?

User reviews of mobile apps provide a communication channel for developers to perceive user satisfaction. Many app features that users have problems with are usually expressed by key phrases such as "upload pictures", which could be buried in the review texts. The lack of fine-grained view about problematic features could obscure the developers' understanding of where the app is frustrating users, and postpone the improvement of the apps. Existing pattern-based approaches to extract target phrases suffer from low accuracy due to insufficient semantic understanding of the reviews, thus can only summarize the high-level topics/aspects of the reviews. This paper proposes a semantic-aware, fine-grained app review analysis approach (SIRA) to extract, cluster, and visualize the problematic features of apps. The main component of SIRA is a novel BERT+Attr-CRF model for fine-grained problematic feature extraction, which combines textual descriptions and review attributes to better model the semantics of reviews and boost the performance of the traditional BERT-CRF model. SIRA also clusters the extracted phrases based on their semantic relations and presents a visualization of the summaries. Our evaluation on 3,426 reviews from six apps confirms the effectiveness of SIRA in problematic feature extraction and clustering. We further conduct an empirical study with SIRA on 318,534 reviews of 18 popular apps to explore its potential application and examine its usefulness in real-world practice.
Decentralized EHR Storage Using Blockchain

Traditionally the health records of the patient are stored in a hospital database and the hospital administration or the healthcare provider generally retain access to this medical data. The health records of patients are scattered among different hospitals. Hence, it becomes difficult to access the historical data of the patient when needed. These results are delayed and possibly incomplete data provided by the patient to their doctor. These flaws can be overcome by the implementation of an Electronic Health Record (EHR) system. Blockchain provides a decentralized platform for storing and accessing EHR, which is almost impossible to tamper with. Every user in the network acts as a node and has a ledger that stores all the data, which means the possibility of a loss of data is extremely low. Our work aims to simplify the accessing and managing of patients’ historical medical records using a decentralized application and blockchain technology.
A calibration method for line-structured light system by using sinusoidal fringes and homography matrix

The existing calibration methods of line-structured light system have complicated calibration procedures. To address this issue, this paper presents a practicable calibration method based on a homography matrix. In this calibration method, it does not need the camera calibration as well as the light plane calibration. First, a sequence of the light-stripe images and sinusoidal fringe images are obtained at different positions moved by a translation stage. Second, a series of feature points located at the light plane will be extracted from these images. Then, a homography matrix, which is to map the relationship between the light plane and the image plane, can be calculated. The three-dimensional (3D) data of a tested object can be restored by this matrix after extracting the image coordinates of the light-stripe images. The experimental results have shown its feasibility and validity. Moreover, the entire calibration process is practicable to simplify the experimental procedures and easy to be applied in industrial inspection.
Photonic Neuromorphic Computing: Architectures, Technologies, and Training Models

We summarize recent developments in neuromorphic photonics, including our work and the advances it brings beyond the state-of-the-art demonstrators in terms of architectures, technologies, and training models for a synergistic hardware/software codesign approach.
Integration Blockchain for Data Sharing and Collaboration in Mobile Healthcare Applications

Enabled by way of cellular and wearable era, private fitness record provides huge and growing importance of healthcare, which is beneficial for both care vendors and medical studies. The secure method and handy sharing of personal health statistics is vital to the improvement of the interplay and collaborating with the healthcare enterprise. Confronted with the capability privacy troubles and vulnerabilities present in current private health statistics garage and system of sharing, which also include the concept of self-sovereign facts ownership, we advocate an modern person-centric fitness data sharing solution via making use of a decentralized and permissioned blockchain to guard privateness the usage of channel formation scheme and beautify the identification management the use of the club service supported with the aid of the blockchain. A cellular utility is deployed to accumulate fitness data from non-public wearable devices, guide input, and scientific gadgets, and synchronize information to the cloud for records sharing with healthcare insurance agencies and healthcare companies. To maintain the integrity of fitness statistics, inside each document, an evidence of validation and integrity is permanently retrieved from database stored in cloud location and is very important to the blockchain network. Furthermore, for performance and scalable and considerations, we undertake a tree-based totally processing statically and batching method to handle big facts units of personal fitness statistics captured and uploaded by means of a mobile platform.
Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

Spoken language understanding (SLU) is an essential task for machines to understand human speech for better interactions. However, errors from the automatic speech recognizer (ASR) usually hurt the understanding performance. In reality, ASR systems may not be easy to adjust for the target scenarios. Therefore, this paper focuses on learning utterance representations that are robust to ASR errors using a contrastive objective, and further strengthens the generalization ability by combining supervised contrastive learning and self-distillation in model fine-tuning. Experiments on three benchmark datasets demonstrate the effectiveness of our proposed approach.
Distance learning, digital inequality, and COVID-19: Visualizing learning channels among California public school students

COVID-19 has put a spotlight not only on digital inequalities but on the formation of digital skills. With the pandemic, the abrupt transition of schools to distance learning in continues to underscore the key importance of digital skills for students who will increasingly need digital skills to navigate new educational terrains and adapt to its demands. This study examines fundamental issues in the formation of digital skills and argues that access to the Internet and digital devices are insufficient if students lack digital skills to use them and benefit from them. To explain why we analyze the relationships and learning channels that are central to students acquisition of digital skills. We examine the interrelationships between four essential “channels” through which students acquire digitals skills: through self direction; through peers; through family; and through formal instruction and interaction with teachers. Drawing from original survey and interview data among disadvantaged secondary school students, we employ configurational and narrative methods to explore how the four learning channels map onto the formation of four key Internet skills (Internet search, Internet-mediated school research, multimedia, and social media). The study’s findings shed light on the ways that self-instruction, the primary learning channel, articulates with the other learning channels. Findings show that school-related research skills are best developed through educator-led instruction, while multimedia and social media skills can be developed through the combination of instruction on the part of peers and family members as long as the learning is simultaneously self-directed. Finally, these findings help explain why remote learning has been inadequate for some low-SES students during the pandemic. As we show, these students could not fully benefit from self-instruction due to the lack of family members or peers who could teach them digital skills. When coupled with disenfranchisement from educator instruction, these students could not acquire the digital skills necessary for them to engage in informed self-instruction and skill development.
Speciesist language and nonhuman animal bias in English Masked Language Models

Various existing studies have analyzed what social biases are inherited by NLP models. These biases may directly or indirectly harm people, therefore previous studies have focused only on human attributes. However, until recently no research on social biases in NLP regarding nonhumans existed. In this paper, we analyze biases to nonhuman animals, i.e. speciesist bias, inherent in English Masked Language Models such as BERT. We analyzed speciesist bias against 46 animal names using template-based and corpus-extracted sentences containing speciesist (or non-speciesist) language. We found that pre-trained masked language models tend to associate harmful words with nonhuman animals and have a bias toward using speciesist language for some nonhuman animal names. Our code for reproducing the experiments will be made available on GitHub.
AI for Biometric Authentication Systems

Biometric authentication is currently one of the most attractive authentication methodologies, as it is very user friendly and is the most secure option for consumer grade mobile device authentication. Furthermore, biometric authentication can provide unique security properties, as it is the only authentication factor that is hard to share or steal. Machine learning has further advanced biometric authentication performance and helps detect well known attack vectors such as spoofing attacks. Sadly, the use of machine learning enlarges the already elaborate attack surface of biometric authentication systems. We further explore the how, pros and cons of machine learning for – the interesting security problem – biometric authentication.