Reviewerly/exHarmony huggingface dataset:
Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systems

Despite the surprising power of many modern AI systems that often learn their own representations, there is significant discontent about their inscrutability and the attendant problems in their ability to interact with humans. While alternatives such as neuro-symbolic approaches have been proposed, there is a lack of consensus on what they are about. There are often two independent motivations (i) symbols as a lingua franca for human-AI interaction and (ii) symbols as (system-produced) abstractions use in its internal reasoning. The jury is still out on whether AI systems will need to use symbols in their internal reasoning to achieve general intelligence capabilities. Whatever the answer there is, the need for (human-understandable) symbols in human-AI interaction seems quite compelling. Symbols, like emotions, may well not be sine qua non for intelligence per se, but they will be crucial for AI systems to interact with us humans--as we can neither turn off our emotions not get by without our symbols. In particular, in many human-designed domains, humans would be interested in providing explicit (symbolic) knowledge and advice--and expect machine explanations in kind. This alone requires AI systems to at least do their I/O in symbolic terms. In this blue sky paper, we argue this point of view, and discuss research directions that need to be pursued to allow for this type of human-AI interaction.
MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks

An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collaboration and communication. To enable theory of mind modeling in situated interactions, we introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in the 3D virtual blocks world of Minecraft. It provides information that captures partners' beliefs of the world and of each other as an interaction unfolds, bringing abundant opportunities to study human collaborative behaviors in situated language communication. As a first step towards our goal of developing embodied AI agents able to infer belief states of collaborative partners in situ, we build and present results on computational models for several theory of mind tasks.
Adversarial Attack and Defense on Deep Neural Network-Based Voice Processing Systems: An Overview

Voice Processing Systems (VPSes), now widely deployed, have become deeply involved in people’s daily lives, helping drive the car, unlock the smartphone, make online purchases, etc. Unfortunately, recent research has shown that those systems based on deep neural networks are vulnerable to adversarial examples, which attract significant attention to VPS security. This review presents a detailed introduction to the background knowledge of adversarial attacks, including the generation of adversarial examples, psychoacoustic models, and evaluation indicators. Then we provide a concise introduction to defense methods against adversarial attacks. Finally, we propose a systematic classification of adversarial attacks and defense methods, with which we hope to provide a better understanding of the classification and structure for beginners in this field.
Efficient and High Path Quality Autonomous Exploration and Trajectory Planning of UAV in an Unknown Environment

The ability of an autonomous Unmanned Aerial Vehicle (UAV) in an unknown environment is a prerequisite for its execution of complex tasks and is the main research direction in related fields. The autonomous navigation of UAVs in unknown environments requires solving the problem of autonomous exploration of the surrounding environment and path planning, which determines whether the drones can complete mission-based flights safely and efficiently. Existing UAV autonomous flight systems hardly perform well in terms of efficient exploration and flight trajectory quality. This paper establishes an integrated solution for autonomous exploration and path planning. In terms of autonomous exploration, frontier-based and sampling-based exploration strategies are integrated to achieve fast and effective exploration performance. In the study of path planning in complex environments, an advanced Rapidly Exploring Random Tree (RRT) algorithm combining the adaptive weights and dynamic step size is proposed, which effectively solves the problem of balancing flight time and trajectory quality. Then, this paper uses the Hermite difference polynomial to optimization the trajectory generated by the RRT algorithm. We named proposed UAV autonomous flight system as Frontier and Sampling-based Exploration and Advanced RRT Planner system (FSEPlanner). Simulation performs in both apartment and maze environment, and results show that the proposed FSEPlanner algorithm achieves greatly improved time consumption and path distances, and the smoothed path is more in line with the actual flight needs of a UAV.
Diagrammatic Differentiation for Quantum Machine Learning

We introduce diagrammatic differentiation for tensor calculus by generalising the dual number construction from rigs to monoidal categories.Applying this to ZX diagrams, we show how to calculate diagrammatically the gradient of a linear map with respect to a phase parameter.For diagrams of parametrised quantum circuits, we get the well-known parameter-shift rule at the basis of many variational quantum algorithms.We then extend our method to the automatic differentation of hybrid classical-quantum circuits, using diagrams with bubbles to encode arbitrary non-linear operators.Moreover, diagrammatic differentiation comes with an open-source implementation in DisCoPy, the Python library for monoidal categories.Diagrammatic gradients of classical-quantum circuits can then be simplified using the PyZX library and executed on quantum hardware via the tket compiler.This opens the door to many practical applications harnessing both the structure of string diagrams and the computational power of quantum machine learning.
Neural Machine Translation Quality and Post-Editing Performance

We test the natural expectation that using MT in professional translation saves human processing time.The last such study was carried out by Sanchez-Torron and Koehn (2016) with phrase-based MT, artificially reducing the translation quality.In contrast, we focus on neural MT (NMT) of high quality, which has become the state-of-the-art approach since then and also got adopted by most translation companies.Through an experimental study involving over 30 professional translators for English→Czech translation, we examine the relationship between NMT performance and post-editing time and quality.Across all models, we found that better MT systems indeed lead to fewer changes in the sentences in this industry setting.The relation between system quality and post-editing time is however not straightforward and, contrary to the results on phrase-based MT, BLEU is definitely not a stable predictor of the time or final output quality.
Spatiotemporal Correlation-Based Accurate 3D Face Imaging Using Speckle Projection and Real-Time Improvement

The reconstruction of 3D face data is widely used in the fields of biometric recognition and virtual reality. However, the rapid acquisition of 3D data is plagued by reconstruction accuracy, slow speed, excessive scenes and contemporary reconstruction-technology. To solve this problem, an accurate 3D face-imaging implementation framework based on coarse-to-fine spatiotemporal correlation is designed, improving the spatiotemporal correlation stereo matching process and accelerating the processing using a spatiotemporal box filter. The reliability of the reconstruction parameters is further verified in order to resolve the contention between the measurement accuracy and time cost. A binocular 3D data acquisition device with a rotary speckle projector is used to continuously and synchronously acquire an infrared speckle stereo image sequence for reconstructing an accurate 3D face model. Based on the face mask data obtained by the high-precision industrial 3D scanner, the relationship between the number of projected speckle patterns, the matching window size, the reconstruction accuracy and the time cost is quantitatively analysed. An optimal combination of parameters is used to achieve a balance between reconstruction speed and accuracy. Thus, to overcome the problem of a long acquisition time caused by the switching of the rotary speckle pattern, a compact 3D face acquisition device using a fixed three-speckle projector is designed. Using the optimal combination parameters of the three speckles, the parallel pipeline strategy is adopted in each core processing unit to maximise system resource utilisation and data throughput. The most time-consuming spatiotemporal correlation stereo matching activity was accelerated by the graphical processing unit. The results show that the system achieves real-time image acquisition, as well as 3D face reconstruction, while maintaining acceptable systematic precision.
Estimation of Chronic Illness Severity Based on Machine Learning Methods

Chronic diseases are diseases that last one year or more and require a continuous medical care and monitoring. Based on this point, a dataset from an APP called Flaredown helps patients of chronic disease improve their symptoms and conditions. In this study, an illness severity-level model was proposed to give the patient an alert to his or her health condition into three different levels according to their severity. Personal information, treatment conditions, and dietary conditions were analyzed by a statistical measure, TD-IDF. Seven different machine learning models were used and compared to generate the illness severity-level model. The results revealed that the XGBoost model with a <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <msub> <mrow> <mi>F</mi> </mrow> <mrow> <mn>1</mn> </mrow> </msub> </math> score of 0.85 and LightGBM model with a <math xmlns="http://www.w3.org/1998/Math/MathML" id="M2"> <msub> <mrow> <mi>F</mi> </mrow> <mrow> <mn>1</mn> </mrow> </msub> </math> score of 0.84 have the best performance. We also applied feature selection and parameter tuning for these two models to attain better performance, and the final best <math xmlns="http://www.w3.org/1998/Math/MathML" id="M3"> <msub> <mrow> <mi>F</mi> </mrow> <mrow> <mn>1</mn> </mrow> </msub> </math> scores achieved by the XGBoost model and LightGBM model were both 0.85. Sensitivity analysis has shown that the treatment feature and symptom feature have important effects on the classification of the illness severity-level. Based on this, a fusion model was designed to study the data and the final accuracy of the fusion model was 93.3%. Thus, this study provides an effective illness severity-level model for a reference and guidance for the management of high-risk groups of chronic diseases. Patients may use this illness severity-level model to self-monitor their illness conditions and take proactive steps to avoid deterioration of their illness and take further medical care.
UFaceNet: Research on Multi-Task Face Recognition Algorithm Based on CNN

With the application of deep convolutional neural networks, the performance of computer vision tasks has been improved to a new level. The construction of a deeper and more complex network allows the face recognition algorithm to obtain a higher accuracy, However, the disadvantages of large computation and storage costs of neural networks limit the further popularization of the algorithm. To solve this problem, we have studied the unified and efficient neural network face recognition algorithm under the condition of a single camera; we propose that the complete face recognition process consists of four tasks: face detection, in vivo detection, keypoint detection, and face verification; combining the key algorithms of these four tasks, we propose a unified network model based on a deep separable convolutional structure—UFaceNet. The model uses multisource data to carry out multitask joint training and uses the keypoint detection results to aid the learning of other tasks. It further introduces the attention mechanism through feature level clipping and alignment to ensure the accuracy of the model, using the shared convolutional layer network among tasks to reduce model calculations amount and realize network acceleration. The learning goal of multi-tasking implicitly increases the amount of training data and different data distribution, making it easier to learn the characteristics with generalization. The experimental results show that the UFaceNet model is better than other models in terms of calculation amount and number of parameters with higher efficiency, and some potential areas to be used.
Classification System for Leukemia Cell Images based on Hu Moment Invariants and Support Vector Machines

Leukemia is cancer that attacks the tissues of white blood cells. It occurs when the body produces abnormal blood cells exceeding normal limits; thus, causing them not to function properly. It has a huge effect on the immune system of humans. Medical personnel currently need a long time to recognize leukemia and distinguish acute leukemia cells from normal cells. This study aims to build a classification system of white blood cell images using a feature extraction technique with Hu moment invariants and Support Vector Machine (SVM) classification methods. In this study, the data of 800 blood image samples were divided into two classes, acute and normal, with each class having 400 sample images. The calculation of the average accuracy and average time value on the system obtained the accuracy value of 88% and the required time of 3.73 seconds. The highest accuracy values for the testing data is 95% with duration time 0.89 seconds. The system could classify the leukemia images using Hu moment invariants and SVM.
Deep Learning-Based Analysis of Face Images as a Screening Tool for Genetic Syndromes

Approximately 4% of the world's population suffers from rare diseases. A vast majority of these disorders have a genetic background. The number of genes that have been linked to human diseases is constantly growing, but there are still genetic syndromes that remain to be discovered. The diagnostic yield of genetic testing is continuously developing, and the need for testing is becoming more significant. Due to limited resources, including trained clinical geneticists, patients referred to clinical genetics units must be accurately selected. Around 30-40% of genetic disorders are associated with specific facial characteristics called dysmorphic features. As part of our research, we analyzed the performance of classifiers based on deep learning face recognition models in detecting dysmorphic features. We tested two classification problems: a multiclass problem (15 genetic disorders vs. controls) and a two-class problem (disease vs. controls). In the multiclass task, the best result reached an accuracy level of 84%. The best accuracy result in the two-class problem reached 96%. More importantly, the binary classifier detected disease features in patients with diseases that were not previously present in the training dataset. The classifier was able to generalize differences between patients and controls, and to detect abnormalities without information about the specific disorder. This indicates that a screening tool based on deep learning and facial recognition could not only detect known diseases, but also detect patients with diseases that were not previously known. In the future, this tool could help in screening patients before they are referred to the genetic unit.
Warp-Refine Propagation: Semi-Supervised Auto-labeling via Cycle-consistency

Deep learning models for semantic segmentation rely on expensive, large-scale, manually annotated datasets. Labelling is a tedious process that can take hours per image. Automatically annotating video sequences by propagating sparsely labeled frames through time is a more scalable alternative. In this work, we propose a novel label propagation method, termed Warp-Refine Propagation, that combines semantic cues with geometric cues to efficiently auto-label videos. Our method learns to refine geometrically-warped labels and infuse them with learned semantic priors in a semi-supervised setting by leveraging cycle-consistency across time. We quantitatively show that our method improves label-propagation by a noteworthy margin of 13.1 mIoU on the ApolloScape dataset. Furthermore, by training with the auto-labelled frames, we achieve competitive results on three semantic-segmentation benchmarks, improving the state-of-the-art by a large margin of 1.8 and 3.61 mIoU on NYU-V2 and KITTI, while matching the current best results on Cityscapes.
Online Engineering Pedagogy: A Proposal for Specialization of the Teacher Training in Engineering

The context of the COVID-19 pandemic produced new immediate needs in the field of university teaching related to distance learning and forces the universities to transform their “traditional” face-to-face teaching methods, particularly with the implementation of online education. This situation represented a challenge not only for the universities but also for the teachers because they need to transform their teaching work in the classroom to online strategies for online learning environments. To meet these needs for effective online education an online pilot training course in Engineering Education based on the IGIP Curriculum of the TU Dresden was designed and implemented. The course “Introduction to online teaching and learning in engineering” (in Spanish: “Introducción a la Enseñanza-Aprendizaje Online en Ingeniería”) consisted of 4 modules implemented on a mix of online communication strategy of synchronous activities carried out on the Zoom platform, together with asynchronous work on a Moodle-based LMS platform. The course was offered between May and June 2020 for a group of academics of the Faculty of Engineering of a public Chilean University. This paper describes the designed online pilot training course in Engineering Pedagogy and presents the results of the evaluation of its implementation. For this a survey was applied and filled by the participants to evaluate the course and to know their per-ceptions about their competencies development to improve online learning in engineering.
A Highly Scalable Method for Extractive Text Summarization Using Convex Optimization

The paper describes a convex optimization formulation of the extractive text summarization problem and a simple and scalable algorithm to solve it. The optimization program is constructed as a convex relaxation of an intuitive but computationally hard integer programming problem. The objective function is highly symmetric, being invariant under unitary transformations of the text representations. Another key idea is to replace the constraint on the number of sentences in the summary with a convex surrogate. For solving the program we have designed a specific projected gradient descent algorithm and analyzed its performance in terms of execution time and quality of the approximation. Using the datasets DUC 2005 and Cornell Newsroom Summarization Dataset, we have shown empirically that the algorithm can provide competitive results for single document summarization and multi-document query-based summarization. On the Cornell Newsroom Summarization Dataset, it ranked second among the unsupervised methods tested. For the more challenging task of multi-document query-based summarization, the method was tested on the DUC 2005 Dataset. Our algorithm surpassed the other reported methods with respect to the ROUGE-SU4 metric, and it was at less than 0.01 from the top performing algorithms with respect to ROUGE-1 and ROUGE-2 metrics.
Quantum causal unravelling

Abstract Complex processes often arise from sequences of simpler interactions involving a few particles at a time. These interactions, however, may not be directly accessible to experiments. Here we develop the first efficient method for unravelling the causal structure of the interactions in a multipartite quantum process, under the assumption that the process has bounded information loss and induces causal dependencies whose strength is above a fixed (but otherwise arbitrary) threshold. Our method is based on a quantum algorithm whose complexity scales polynomially in the total number of input/output systems, in the dimension of the systems involved in each interaction, and in the inverse of the chosen threshold for the strength of the causal dependencies. Under additional assumptions, we also provide a second algorithm that has lower complexity and requires only local state preparation and local measurements. Our algorithms can be used to identify processes that can be characterized efficiently with the technique of quantum process tomography. Similarly, they can be used to identify useful communication channels in quantum networks, and to test the internal structure of uncharacterized quantum circuits.
Design and Implementation of Intelligent Classroom Framework Through Light-Weight Neural Networks Based on Multimodal Sensor Data Fusion Approach

Intelligent classrooms are becoming famous.Multimodal sensor data fusion technique, where data generated from different sensors are fused to derive at some valuable insights from the classroom settings.In this paper, the proposed framework model tries to enable intelligence in a traditional Classroom Environment by experimenting on modules such as Deep Learning based Face recognition systems, Interactive Smart Mirror Assistant, Indoor Classroom Air quality monitors.Sensor hub (Jedi One) helps to visualize and analyse streaming data in real-time.Based on the proposed framework design, experimentations are carried out.The accuracy achieved in the Face Recognition System of 71% has to be increased with 80-90% by finetuning the parameters.In future, Interactive dashboards can be activated via PowerBI or Excel worksheets.Based on the questionnaire study & responses from the participants on AI & IoT systems inside the classrooms, more than 50% responded positively to support the usage of these technologies in a Classroom Environment.The future classrooms will be (DLeIC) Deep Learning enabled IoT Classrooms to lift the educational space into a new dimension.Incorporating Deep Learning on IoT systems can be a savvy and fruitful path to collaborate with generations to come.
Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking

Recent progress in task-oriented neural dialogue systems is largely focused on a handful of languages, as annotation of training data is tedious and expensive.Machine translation has been used to make systems multilingual, but this can introduce a pipeline of errors.Another promising solution is using cross-lingual transfer learning through pretrained multilingual models.Existing methods train multilingual models with additional codemixed task data or refine the cross-lingual representations through parallel ontologies.In this work, we enhance the transfer learning process by intermediate fine-tuning of pretrained multilingual models, where the multilingual models are fine-tuned with different but related data and/or tasks.Specifically, we use parallel and conversational movie subtitles datasets to design cross-lingual intermediate tasks suitable for downstream dialogue tasks.We use only 200K lines of parallel data for intermediate fine-tuning which is already available for 1782 language pairs.We test our approach on the cross-lingual dialogue state tracking task for the parallel Mul-tiWoZ (English→Chinese, Chinese→English) and Multilingual WoZ (English→German, English→Italian) datasets.We achieve impressive improvements (> 20% on joint goal accuracy) on the parallel MultiWoZ dataset and the Multilingual WoZ dataset over the vanilla baseline with only 10% of the target language task data and zero-shot setup respectively.