OpenRaiser/Intern-Atlas huggingface dataset:
Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Sampling
Venue: arXiv cs.LG 2022
Year: 2022

Principled Bayesian deep learning (BDL) does not live up to its potential when we only focus on marginal predictive distributions (marginal predictives). Recent works have highlighted the importance of joint predictives for (Bayesian) sequential decision making from a theoretical and synthetic perspective. We provide additional practical arguments grounded in real-world applications for focusing on joint predictives: we discuss online Bayesian inference, which would allow us to make predictions while taking into account additional data without retraining, and we propose new challenging evaluation settings using active learning and active sampling. These settings are motivated by an examination of marginal and joint predictives, their respective cross-entropies, and their place in offline and online learning. They are more realistic than previously suggested ones, building on work by Wen et al. (2021) and Osband et al. (2022), and focus on evaluating the performance of approximate BNNs in an online supervised setting. Initial experiments, however, raise questions on the feasibility of these ideas in high-dimensional parameter spaces with current BDL inference techniques, and we suggest experiments that might help shed further light on the practicality of current research for these problems. Importantly, our work highlights previously unidentified gaps in current research and the need for better approximate joint predictives.
Accuracy Enhancement for Ear Acoustic Authentication Using Between-class Features
Venue: arXiv cs.CR 2022
Year: 2022

In existing biometric authentication methods, the user must perform an authentication operation such as placing a finger in a scanner or facing a camera. With ear acoustic authentication, acoustic characteristics of the ear canal are used as biometric information. Therefore, a person wearing earphones does not need to perform any authentication operation. In biometric authentication, it is necessary to minimize the false acceptance rate (FAR) so that no unauthorized user is misidentified as an authorized user. However, if the FAR is set low, it increases the false rejection rate (FRR), the rate at which authorized users are falsely recognized as unauthorized users. It has been reported that when FAR is 0.1%, the FRR in ear acoustic authentication reaches as much as 22%. In this study, we propose a method that reduces FRR and enhances authentication accuracy; it generates new ear canal acoustic characteristics called between-class (BC) features, which combine the ear canal acoustic characteristics of authorized and unauthorized users features. The proposed method uses a support vector machine to learn the BC features as the data of authorized users, then generates a hyperplane in an area close to that data. We hypothesize that this would decrease the possibility of misidentifying an unauthorized user as an authorized user and decrease the FRR when the FAR is 0.1%. To evaluate the performance of the proposed method, BC features were applied to ear acoustic authentication, and FAR and FRR were calculated. The FRR with FAR = 0.1% was 7.95% lower than with the previous method, and the equal error rate -- the error rate when FAR and FRR are equivalent -- decreased by 0.15%. These results confirmed that the proposed method can make ear acoustic authentication more convenient while maintaining a high level of security.
Evaluation of Transfer Learning for Polish with a Text-to-Text Model
Venue: arXiv cs.CL 2022
Year: 2022

We introduce a new benchmark for assessing the quality of text-to-text models for Polish. The benchmark consists of diverse tasks and datasets: KLEJ benchmark adapted for text-to-text, en-pl translation, summarization, and question answering. In particular, since summarization and question answering lack benchmark datasets for the Polish language, we describe their construction and make them publicly available. Additionally, we present plT5 - a general-purpose text-to-text model for Polish that can be fine-tuned on various Natural Language Processing (NLP) tasks with a single training objective. Unsupervised denoising pre-training is performed efficiently by initializing the model weights with a multi-lingual T5 (mT5) counterpart. We evaluate the performance of plT5, mT5, Polish BART (plBART), and Polish GPT-2 (papuGaPT2). The plT5 scores top on all of these tasks except summarization, where plBART is best. In general (except for summarization), the larger the model, the better the results. The encoder-decoder architectures prove to be better than the decoder-only equivalent.
Automating In-Network Machine Learning
Venue: arXiv cs.NI 2022
Year: 2022

Using programmable network devices to aid in-network machine learning has been the focus of significant research. However, most of the research was of a limited scope, providing a proof of concept or describing a closed-source algorithm. To date, no general solution has been provided for mapping machine learning algorithms to programmable network devices. In this paper, we present Planter, an open-source, modular framework for mapping trained machine learning models to programmable devices. Planter supports a wide range of machine learning models, multiple targets and can be easily extended. The evaluation of Planter compares different mapping approaches, and demonstrates the feasibility, performance, and resource efficiency for applications such as anomaly detection, financial transactions, and quality of experience. The results show that Planter-based in-network machine learning algorithms can run at line rate, have a negligible effect on latency, coexist with standard switching functionality, and have no or minor accuracy trade-offs.
A Scalable Workflow to Build Machine Learning Classifiers with Clinician-in-the-Loop to Identify Patients in Specific Diseases
Venue: arXiv cs.CL 2022
Year: 2022

Clinicians may rely on medical coding systems such as International Classification of Diseases (ICD) to identify patients with diseases from Electronic Health Records (EHRs). However, due to the lack of detail and specificity as well as a probability of miscoding, recent studies suggest the ICD codes often cannot characterise patients accurately for specific diseases in real clinical practice, and as a result, using them to find patients for studies or trials can result in high failure rates and missing out on uncoded patients. Manual inspection of all patients at scale is not feasible as it is highly costly and slow. This paper proposes a scalable workflow which leverages both structured data and unstructured textual notes from EHRs with techniques including NLP, AutoML and Clinician-in-the-Loop mechanism to build machine learning classifiers to identify patients at scale with given diseases, especially those who might currently be miscoded or missed by ICD codes. Case studies in the MIMIC-III dataset were conducted where the proposed workflow demonstrates a higher classification performance in terms of F1 scores compared to simply using ICD codes on gold testing subset to identify patients with Ovarian Cancer (0.901 vs 0.814), Lung Cancer (0.859 vs 0.828), Cancer Cachexia (0.862 vs 0.650), and Lupus Nephritis (0.959 vs 0.855). Also, the proposed workflow that leverages unstructured notes consistently outperforms the baseline that uses structured data only with an increase of F1 (Ovarian Cancer 0.901 vs 0.719, Lung Cancer 0.859 vs 0.787, Cancer Cachexia 0.862 vs 0.838 and Lupus Nephritis 0.959 vs 0.785). Experiments on the large testing set also demonstrate the proposed workflow can find more patients who are miscoded or missed by ICD codes. Moreover, interpretability studies are also conducted to clinically validate the top impact features of the classifiers.
Internet Performance in the 2022 Conflict in Ukraine: An Asymmetric Analysis
Venue: arXiv cs.NI 2022
Year: 2022

On 24 February 2022 Russia invaded Ukraine, starting one of the largest military conflicts in Europe in recent years. In this paper we present preliminary findings about the impact of the conflict on the Internet performance in Ukraine and in Russia, introducing an ironically asymmetric picture: the Internet performance in Ukraine has significantly degraded, while the performance in Russia has improved.
A Tutorial on Structural Optimization
Venue: arXiv cs.CE 2022
Year: 2022

Structural optimization is a useful and interesting tool. Unfortunately, it can be hard for new researchers to get started on the topic because existing tutorials assume the reader has substantial domain knowledge. They obscure the fact that structural optimization is really quite simple, elegant, and easy to implement. With that in mind, let's write our own structural optimization code, from scratch, in 180 lines.
One Explanation to Rule them All -- Ensemble Consistent Explanations
Venue: arXiv cs.AI 2022
Year: 2022

Transparency is a major requirement of modern AI based decision making systems deployed in real world. A popular approach for achieving transparency is by means of explanations. A wide variety of different explanations have been proposed for single decision making systems. In practice it is often the case to have a set (i.e. ensemble) of decisions that are used instead of a single decision only, in particular in complex systems. Unfortunately, explanation methods for single decision making systems are not easily applicable to ensembles -- i.e. they would yield an ensemble of individual explanations which are not necessarily consistent, hence less useful and more difficult to understand than a single consistent explanation of all observed phenomena. We propose a novel concept for consistently explaining an ensemble of decisions locally with a single explanation -- we introduce a formal concept, as well as a specific implementation using counterfactual explanations.
Internet of Intelligence: A Survey on the Enabling Technologies, Applications, and Challenges
Venue: arXiv cs.NI 2022
Year: 2022

The Internet of intelligence is conceived as an emerging networking paradigm, which will make intelligence as easy to obtain as information. This paper provides an overview of the Internet of intelligence, focusing on motivations, architecture, enabling technologies, applications, and existing challenges. This can provide a good foundation for those who are interested to gain insights into the concept of the Internet of intelligence and the key enablers of this emerging networking paradigm. Specifically, this paper starts by investigating the evolution of networking paradigms and artificial intelligence (AI), based on which we present the motivations of the Internet of intelligence by demonstrating that networking needs intelligence and intelligence needs networking. We then present the layered architecture to characterize the Internet of intelligence systems and discuss the enabling technologies of each layer. Moreover, we discuss the critical applications and their integration with the Internet of intelligence paradigm. Finally, some technical challenges and open issues are summarized to fully exploit the benefits of the Internet of intelligence.
Symbolic-Numeric Factorization of Differential Operators
Venue: arXiv cs.SC 2022
Year: 2022

We present a symbolic-numeric Las Vegas algorithm for factoring Fuchsian ordinary differential operators with rational function coefficients. The new algorithm combines ideas of van Hoeij's "local-to-global" method and of the ''analytic'' approach proposed by van der Hoeven. It essentially reduces to the former in ''easy'' cases where the local-to-global method succeeds, and to an optimized variant of the latter in the "hardest" cases, while handling intermediate cases more efficiently than both.
Understanding Factors that Shape Children's Long Term Engagement with an In-Home Learning Companion Robot
Venue: arXiv cs.RO 2022
Year: 2022

Social robots are emerging as learning companions for children, and research shows that they facilitate the development of interest and learning even through brief interactions. However, little is known about how such technologies might support these goals in authentic environments over long-term periods of use and interaction. We designed a learning companion robot capable of supporting children reading popular-science books by expressing social and informational commentaries. We deployed the robot in homes of 14 families with children aged 10-12 for four weeks during the summer. Our analysis revealed critical factors that affected children's long-term engagement and adoption of the robot, including external factors such as vacations, family visits, and extracurricular activities; family/parental involvement; and children's individual interests. We present four in-depth cases that illustrate these factors and demonstrate their impact on children's reading experiences and discuss the implications of our findings for robot design.
On the complexity of recognizing Stick, BipHook and Max Point-Tolerance graphs
Venue: arXiv cs.CC 2022
Year: 2022

Stick graphs are defined as follows. Let A (respectively B) be a set of vertical (respectively horizontal) segments in the plane such that the bottom endpoints of the segments in A and the left endpoints of the segments in B lie on the same ground straight line with slope -1. The Stick graph defined by A and B, which is necessarily bipartite, is the intersection graph of the segments in A with the segments in B. We answer an open problem by showing that recognizing Stick graphs is NP-complete. This result allows us to easily solve two other open problems, namely the recognition of BipHook graphs and of max point-tolerance graphs. We show that both of them are NP-complete problems.
Industry 5.0 is Coming: A Survey on Intelligent NextG Wireless Networks as Technological Enablers
Venue: arXiv cs.NI 2022
Year: 2022

Industry 5.0 vision, a step toward the next industrial revolution and enhancement to Industry 4.0, envisioned the new goals of resilient, sustainable, and human-centric approaches in diverse emerging applications, e.g., factories-of-the-future, digital society. The vision seeks to leverage human intelligence and creativity in nexus with intelligent, efficient, and reliable cognitive collaborating robots (cobots) to achieve zero waste, zerodefect, and mass customization-based manufacturing solutions. However, the vision requires the merging of cyber-physical worlds through utilizing Industry 5.0 technological enablers, e.g., cognitive cobots, person-centric artificial intelligence (AI), cyberphysical systems, digital twins, hyperconverged data storage and computing, communication infrastructure, and others. In this regard, the convergence of the emerging computational intelligence (CI) paradigm and next-generation wireless networks (NGWNs) can fulfill the stringent communication and computation requirements of the technological enablers in the Industry 5.0 vision, which is the aim of this survey-based tutorial. In this article, we address this issue by reviewing and analyzing current emerging concepts and technologies, e.g., CI tools and frameworks, network-in-box architecture, open radio access networks, softwarized service architectures, potential enabling services, and others, essential for designing the objectives of CINGWNs to fulfill the Industry 5.0 vision requirements. Finally, we provide a list of lessons learned from our detailed review, research challenges, and open issues that should be addressed in CI-NGWNs to realize Industry 5.0.
The security strength of Blockchain technology : A Survey Report
Venue: arXiv cs.CR 2022
Year: 2022

The advent of blockchain technology by the Nakamoto group in 2008 has created a new trend on how to deal with various security issues and vulnerabilities. Blockchain systems have gained momentum in various spheres of technology deployment in business organizations. This paper presents a critical literature survey on the security strength of blockchains and security issues associated with blockchain technology deployment. Numerous studies have experimented with the various technical features of blockchain systems across various transaction domains. Findings obtained from the literature survey and thematic content analysis of the existing research studies indicate that blockchain systems provide unique capabilities that support processes and transactions across various sectors with a high level of integrity, transparency, confidentiality, and privacy. However, some loopholes and limitations associated with the deployment and use of blockchains have been highlighted in various studies. The present study cross-examined the security issues of the underlying scientific research evidence.
ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
Venue: arXiv cs.CL 2022
Year: 2022

Neural retrievers based on pre-trained language models (PLMs), such as dual-encoders, have achieved promising performance on the task of open-domain question answering (QA). Their effectiveness can further reach new state-of-the-arts by incorporating cross-architecture knowledge distillation. However, most of the existing studies just directly apply conventional distillation methods. They fail to consider the particular situation where the teacher and student have different structures. In this paper, we propose a novel distillation method that significantly advances cross-architecture distillation for dual-encoders. Our method 1) introduces a self on-the-fly distillation method that can effectively distill late interaction (i.e., ColBERT) to vanilla dual-encoder, and 2) incorporates a cascade distillation process to further improve the performance with a cross-encoder teacher. Extensive experiments are conducted to validate that our proposed solution outperforms strong baselines and establish a new state-of-the-art on open-domain QA benchmarks.
Computing the ensemble spread from deterministic weather predictions using conditional generative adversarial networks
Venue: arXiv cs.LG 2022
Year: 2022

Ensemble prediction systems are an invaluable tool for weather forecasting. Practically, ensemble predictions are obtained by running several perturbations of the deterministic control forecast. However, ensemble prediction is associated with a high computational cost and often involves statistical post-processing steps to improve its quality. Here we propose to use deep-learning-based algorithms to learn the statistical properties of an ensemble prediction system, the ensemble spread, given only the deterministic control forecast. Thus, once trained, the costly ensemble prediction system will not be needed anymore to obtain future ensemble forecasts, and the statistical properties of the ensemble can be derived from a single deterministic forecast. We adapt the classical pix2pix architecture to a three-dimensional model and also experiment with a shared latent space encoder-decoder model, and train them against several years of operational (ensemble) weather forecasts for the 500 hPa geopotential height. The results demonstrate that the trained models indeed allow obtaining a highly accurate ensemble spread from the control forecast only.
On eventual non-negativity and positivity for the weighted sum of powers of matrices
Venue: arXiv cs.FL 2022
Year: 2022

The long run behaviour of linear dynamical systems is often studied by looking at eventual properties of matrices and recurrences that underlie the system. A basic problem that lies at the core of many questions in this setting is the following: given a set of pairs of rational weights and matrices {(w_1 , A_1 ), . . . , (w_m , A_m )}, we ask if the weighted sum of powers of these matrices is eventually non-negative P (resp. n positive), i.e., does there exist an integer N s.t for all n greater than N , (w_1 A_1^n + ... + w_m A_m^n) is atmost 0 (resp. greater than 0). The restricted setting when m = w_1 = 1, results in so-called eventually non-negative (or eventually positive) matrices, which enjoy nice spectral properties and have been well-studied in control theory. More applications arise in varied contexts, ranging from program verification to partially observable and multi-modal systems. Our goal is to investigate this problem and its link to linear recurrence sequences. Our first result is that for m at least 2, the problem is as hard as the ultimate positivity of linear recurrences, a long standing open question (known to be coNP-hard). Our second result is a reduction in the other direction showing that for any m at least 1, the problem reduces to ultimate positivity of linear recurrences. This shows precise upper bounds for several subclasses of matrices by exploiting known results on linear recurrence sequences. Our third main result is a novel reduction technique for a large class of problems (including those mentioned above) over rational diagonalizable matrices to the corresponding problem over simple real-algebraic matrices. This yields effective decision procedures for diagonalizable matrices.
High-Order Multilinear Discriminant Analysis via Order-$\textit{n}$ Tensor Eigendecomposition
Venue: arXiv cs.LG 2022
Year: 2022

Higher-order data with high dimensionality is of immense importance in many areas of machine learning, computer vision, and video analytics. Multidimensional arrays (commonly referred to as tensors) are used for arranging higher-order data structures while keeping the natural representation of the data samples. In the past decade, great efforts have been made to extend the classic linear discriminant analysis for higher-order data classification generally referred to as multilinear discriminant analysis (MDA). Most of the existing approaches are based on the Tucker decomposition and $\textit{n}$-mode tensor-matrix products. The current paper presents a new approach to tensor-based multilinear discriminant analysis referred to as High-Order Multilinear Discriminant Analysis (HOMLDA). This approach is based upon the tensor decomposition where an order-$\textit{n}$ tensor can be written as a product of order-$\textit{n}$ tensors and has a natural extension to traditional linear discriminant analysis (LDA). Furthermore, the resulting framework, HOMLDA, might produce a within-class scatter tensor that is close to singular. Thus, computing the inverse inaccurately may distort the discriminant analysis. To address this problem, an improved method referred to as Robust High-Order Multilinear Discriminant Analysis (RHOMLDA) is introduced. Experimental results on multiple data sets illustrate that our proposed approach provides improved classification performance with respect to the current Tucker decomposition-based supervised learning methods.