OpenRaiser/Intern-Atlas huggingface dataset:
Message Transmission and Common Randomness Generation over MIMO Slow Fading Channels with Arbitrary Channel State Distribution
Venue: arXiv cs.IT 2022
Year: 2022

We investigate the problem of message transmission and the problem of common randomness (CR) generation over single-user multiple-input multiple-output (MIMO) slow fading channels with average input power constraint, additive white Gaussian noise (AWGN), arbitrary state distribution and with complete channel state information available at the receiver side (CSIR). First, we derive a lower and an upper bound on the outage transmission capacity of MIMO slow fading channels for arbitrary state distribution and show that the bounds coincide except possibly at the points of discontinuity of the outage transmission capacity, of which there are, at most, countably many. To prove the lower bound on the outage transmission capacity, we also establish the capacity of a specific compound MIMO Gaussian channel. Second, we define the outage CR capacity for a two-source model with unidirectional communication over a MIMO slow fading channel with arbitrary state distribution and establish a lower and an upper bound on it using our bounds on the outage transmission capacity of the MIMO slow fading channel.
Real-time large-scale supplier order assignments across two-tiers of a supply chain with penalty and dual-sourcing
Venue: arXiv cs.CE 2022
Year: 2022

Supplier selection and order allocation (SSOA) are key strategic decisions in supply chain management which greatly impact the performance of the supply chain. Although, the SSOA problem has been studied extensively but less attention paid to scalability presents a significant gap preventing adoption of SSOA algorithms by industrial practitioners. This paper presents a novel multi-item, multi-supplier double order allocations with dual-sourcing and penalty constraints across two-tiers of a supply chain, resulting in cooperation and in facilitating supplier preferences to work with other suppliers through bidding. We propose Mixed-Integer Programming models for allocations at individual-tiers as well as an integrated allocations. An application to a real-time large-scale case study of a manufacturing company is presented, which is the largest scale studied in terms of supply chain size and number of variables so far in literature. The use case allows us to highlight how problem formulation and implementation can help reduce computational complexity using Mathematical Programming (MP) and Genetic Algorithm (GA) approaches. The results show an interesting observation that MP outperforms GA to solve SSOA. Sensitivity analysis is presented for sourcing strategy, penalty threshold and penalty factor. The developed model was successfully deployed in a large international sourcing conference with multiple bidding rounds, which helped in more than 10% procurement cost reductions to the manufacturing company.
A GA-like Dynamic Probability Method With Mutual Information for Feature Selection
Venue: arXiv cs.IT 2022
Year: 2022

Feature selection plays a vital role in promoting the classifier's performance. However, current methods ineffectively distinguish the complex interaction in the selected features. To further remove these hidden negative interactions, we propose a GA-like dynamic probability (GADP) method with mutual information which has a two-layer structure. The first layer applies the mutual information method to obtain a primary feature subset. The GA-like dynamic probability algorithm, as the second layer, mines more supportive features based on the former candidate features. Essentially, the GA-like method is one of the population-based algorithms so its work mechanism is similar to the GA. Different from the popular works which frequently focus on improving GA's operators for enhancing the search ability and lowering the converge time, we boldly abandon GA's operators and employ the dynamic probability that relies on the performance of each chromosome to determine feature selection in the new generation. The dynamic probability mechanism significantly reduces the parameter number in GA that making it easy to use. As each gene's probability is independent, the chromosome variety in GADP is more notable than in traditional GA, which ensures GADP has a wider search space and selects relevant features more effectively and accurately. To verify our method's superiority, we evaluate our method under multiple conditions on 15 datasets. The results demonstrate the outperformance of the proposed method. Generally, it has the best accuracy. Further, we also compare the proposed model to the popular heuristic methods like POS, FPA, and WOA. Our model still owns advantages over them.
Optimizing text representations to capture (dis)similarity between political parties
Venue: arXiv cs.CL 2022
Year: 2022

Even though fine-tuned neural language models have been pivotal in enabling "deep" automatic text analysis, optimizing text representations for specific applications remains a crucial bottleneck. In this study, we look at this problem in the context of a task from computational social science, namely modeling pairwise similarities between political parties. Our research question is what level of structural information is necessary to create robust text representation, contrasting a strongly informed approach (which uses both claim span and claim category annotations) with approaches that forgo one or both types of annotation with document structure-based heuristics. Evaluating our models on the manifestos of German parties for the 2021 federal election. We find that heuristics that maximize within-party over between-party similarity along with a normalization step lead to reliable party similarity prediction, without the need for manual annotation.
Extending F1 metric, probabilistic approach
Venue: arXiv cs.LG 2022
Year: 2022

This article explores the extension of well-known F1 score used for assessing the performance of binary classifiers. We propose the new metric using probabilistic interpretation of precision, recall, specificity, and negative predictive value. We describe its properties and compare it to common metrics. Then we demonstrate its behavior in edge cases of the confusion matrix. Finally, the properties of the metric are tested on binary classifier trained on the real dataset.
Integrated Brier Score based Survival Cobra -- A regression based approach
Venue: arXiv cs.LG 2022
Year: 2022

Recently Goswami et al. \cite{goswami2022concordance} introduced two novel implementations of combined regression strategy to find the conditional survival function. The paper uses regression-based weak learners and provides an alternative version of the combined regression strategy (COBRA) ensemble using the Integrated Brier Score to predict conditional survival function. We create a novel predictor based on a weighted version of all machine predictions taking weights as a specific function of normalized Integrated Brier Score. We use two different norms (Frobenius and Sup norm) to extract the proximity points in the algorithm. Our implementations consider right-censored data too. We illustrate the proposed algorithms through some real-life data analysis.
Ethics for Digital Medicine: A Path for Ethical Emerging Medical IoT Design
Venue: arXiv cs.CY 2022
Year: 2022

The dawn of the digital medicine era, ushered in by increasingly powerful embedded systems and Internet of Things (IoT) computing devices, is creating new therapies and biomedical solutions that promise to positively transform our quality of life. However, the digital medicine revolution also creates unforeseen and complex ethical, regulatory, and societal issues. In this article, we reflect on the ethical challenges facing digital medicine. We discuss the perils of ethical oversights in medical devices, and the role of professional codes and regulatory oversight towards the ethical design, deployment, and operation of digital medicine devices that safely and effectively meet the needs of patients. We advocate for an ensemble approach of intensive education, programmable ethical behaviors, and ethical analysis frameworks, to prevent mishaps and sustain ethical innovation, design, and lifecycle management of emerging digital medicine devices.
Bootstrapping NLP tools across low-resourced African languages: an overview and prospects
Venue: arXiv cs.CL 2022
Year: 2022

Computing and Internet access are substantially growing markets in Southern Africa, which brings with it increasing demands for local content and tools in indigenous African languages. Since most of those languages are low-resourced, efforts have gone into the notion of bootstrapping tools for one African language from another. This paper provides an overview of these efforts for Niger-Congo B (`Bantu') languages. Bootstrapping grammars for geographically distant languages has been shown to still have positive outcomes for morphology and rules or grammar-based natural language generation. Bootstrapping with data-driven approaches to NLP tasks is difficult to use meaningfully regardless geographic proximity, which is largely due to lexical diversity due to both orthography and vocabulary. Cladistic approaches in comparative linguistics may inform bootstrapping strategies and similarity measures might serve as proxy for bootstrapping potential as well, with both fertile ground for further research.
The privacy issue of counterfactual explanations: explanation linkage attacks
Venue: arXiv cs.LG 2022
Year: 2022

Black-box machine learning models are being used in more and more high-stakes domains, which creates a growing need for Explainable AI (XAI). Unfortunately, the use of XAI in machine learning introduces new privacy risks, which currently remain largely unnoticed. We introduce the explanation linkage attack, which can occur when deploying instance-based strategies to find counterfactual explanations. To counter such an attack, we propose k-anonymous counterfactual explanations and introduce pureness as a new metric to evaluate the validity of these k-anonymous counterfactual explanations. Our results show that making the explanations, rather than the whole dataset, k- anonymous, is beneficial for the quality of the explanations.
Language-Integrated Query for Temporal Data (Extended version)
Venue: arXiv cs.PL 2022
Year: 2022

Modern applications often manage time-varying data. Despite decades of research on temporal databases, which culminated in the addition of temporal data operations into the SQL:2011 standard, temporal data query and manipulation operations are unavailable in most mainstream database management systems, leaving developers with the unenviable task of implementing such functionality from scratch. In this paper, we extend \emph{language-integrated query} to support writing temporal queries and updates in a uniform host language, with the language performing the required rewriting to emulate temporal capabilities automatically on any standard relational database. We introduce two core languages, $λ_{\mathsf{TLINQ}}$ and $λ_{\mathsf{VLINQ}}$, for manipulating transaction time and valid time data respectively, and formalise existing implementation strategies by giving provably correct semantics-preserving translations into a non-temporal core language, $λ_{\mathsf{LINQ}}$. We show how existing work on query normalisation supports a surprisingly simple implementation strategy for \emph{sequenced joins}. We implement our approach in the Links programming language, and describe a non-trivial case study based on curating COVID-19 statistics.
Do Content Management Systems Impact the Security of Free Content Websites? A Correlation Analysis
Venue: arXiv cs.CR 2022
Year: 2022

This paper investigates the potential causes of the vulnerabilities of free content websites to address risks and maliciousness. Assembling more than 1,500 websites with free and premium content, we identify their content management system (CMS) and malicious attributes. We use frequency analysis at both the aggregate and per category of content (books, games, movies, music, and software), utilizing the unpatched vulnerabilities, total vulnerabilities, malicious count, and percentiles to uncover trends and affinities of usage and maliciousness of CMS{'s} and their contribution to those websites. Moreover, we find that, despite the significant number of custom code websites, the use of CMS{'s} is pervasive, with varying trends across types and categories. Finally, we find that even a small number of unpatched vulnerabilities in popular CMS{'s} could be a potential cause for significant maliciousness.
Optimal Policies for Age and Distortion in a Discrete-Time Model
Venue: arXiv cs.IT 2022
Year: 2022

We study a discrete-time model where each packet has a cost of not being sent -- this cost might depend on the packet content. We study the tradeoff between the age and the cost where the sender is confined to packet-based strategies. The optimal tradeoff is found by an appropriate formulation of the problem as a Markov Decision Process (MDP). We show that the optimal tradeoff can be attained with finite-memory policies and we devise an efficient policy iteration algorithm to find these optimal policies. We further study a related problem where the transmitted packets are subject to erasures. We show that the optimal policies for our problem are also optimal for this new setup. Allowing coding across packets significantly extends the packet-based strategies. We show that when the packet payloads are small, the performance can be improved by coding.
AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning
Venue: arXiv cs.LG 2022
Year: 2022

Diagnostic and prognostic models are increasingly important in medicine and inform many clinical decisions. Recently, machine learning approaches have shown improvement over conventional modeling techniques by better capturing complex interactions between patient covariates in a data-driven manner. However, the use of machine learning introduces a number of technical and practical challenges that have thus far restricted widespread adoption of such techniques in clinical settings. To address these challenges and empower healthcare professionals, we present a machine learning framework, AutoPrognosis 2.0, to develop diagnostic and prognostic models. AutoPrognosis leverages state-of-the-art advances in automated machine learning to develop optimized machine learning pipelines, incorporates model explainability tools, and enables deployment of clinical demonstrators, without requiring significant technical expertise. Our framework eliminates the major technical obstacles to predictive modeling with machine learning that currently impede clinical adoption. To demonstrate AutoPrognosis 2.0, we provide an illustrative application where we construct a prognostic risk score for diabetes using the UK Biobank, a prospective study of 502,467 individuals. The models produced by our automated framework achieve greater discrimination for diabetes than expert clinical risk scores. Our risk score has been implemented as a web-based decision support tool and can be publicly accessed by patients and clinicians worldwide. In addition, AutoPrognosis 2.0 is provided as an open-source python package. By open-sourcing our framework as a tool for the community, clinicians and other medical practitioners will be able to readily develop new risk scores, personalized diagnostics, and prognostics using modern machine learning techniques.
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play
Venue: arXiv cs.CL 2022
Year: 2022

The task of context-dependent text-to-SQL aims to convert multi-turn user utterances to formal SQL queries. This is a challenging task due to both the scarcity of training data from which to learn complex contextual dependencies and to generalize to unseen databases. In this paper we explore augmenting the training datasets using self-play, which leverages contextual information to synthesize new interactions to adapt the model to new databases. We first design a SQL-to-text model conditioned on a sampled goal query, which represents a user's intent, that then converses with a text-to-SQL semantic parser to generate new interactions. We then filter the synthesized interactions and retrain the models with the augmented data. We find that self-play improves the accuracy of a strong baseline on SParC and CoSQL, two widely used cross-domain text-to-SQL datasets. Our analysis shows that self-play simulates various conversational thematic relations, enhances cross-domain generalization and improves beam-search.
Joint Coreference Resolution for Zeros and non-Zeros in Arabic
Venue: arXiv cs.CL 2022
Year: 2022

Most existing proposals about anaphoric zero pronoun (AZP) resolution regard full mention coreference and AZP resolution as two independent tasks, even though the two tasks are clearly related. The main issues that need tackling to develop a joint model for zero and non-zero mentions are the difference between the two types of arguments (zero pronouns, being null, provide no nominal information) and the lack of annotated datasets of a suitable size in which both types of arguments are annotated for languages other than Chinese and Japanese. In this paper, we introduce two architectures for jointly resolving AZPs and non-AZPs, and evaluate them on Arabic, a language for which, as far as we know, there has been no prior work on joint resolution. Doing this also required creating a new version of the Arabic subset of the standard coreference resolution dataset used for the CoNLL-2012 shared task (Pradhan et al.,2012) in which both zeros and non-zeros are included in a single dataset.
Urban Socio-Technical Systems: An Autonomy and Mobility Perspective
Venue: arXiv cs.CY 2022
Year: 2022

The future of the human race is urban. The world's population is projected to grow an additional 2.5 billion by 2050, with all expected to live in urban areas. This will increase the percentage of urban population from 55% today to 70% within three decades and further strengthen the role of cities as the hub for information, transportation, and overall socio-economic development. Unlike any other time in human history, the increasing levels of autonomy and machine intelligence are transforming cities to be no longer just human agglomerations but a fusion of humans, machines, and algorithms making collective decisions, thus complex socio-technical systems. This manuscript summarizes and discusses my efforts from the urban autonomy and mobility perspective to develop the urban socio-technical system.
A New Perspective for Understanding Generalization Gap of Deep Neural Networks Trained with Large Batch Sizes
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks (DNNs) are typically optimized using various forms of mini-batch gradient descent algorithm. A major motivation for mini-batch gradient descent is that with a suitably chosen batch size, available computing resources can be optimally utilized (including parallelization) for fast model training. However, many works report the progressive loss of model generalization when the training batch size is increased beyond some limits. This is a scenario commonly referred to as generalization gap. Although several works have proposed different methods for alleviating the generalization gap problem, a unanimous account for understanding generalization gap is still lacking in the literature. This is especially important given that recent works have observed that several proposed solutions for generalization gap problem such learning rate scaling and increased training budget do not indeed resolve it. As such, our main exposition in this paper is to investigate and provide new perspectives for the source of generalization loss for DNNs trained with a large batch size. Our analysis suggests that large training batch size results in increased near-rank loss of units' activation (i.e. output) tensors, which consequently impacts model optimization and generalization. Extensive experiments are performed for validation on popular DNN models such as VGG-16, residual network (ResNet-56) and LeNet-5 using CIFAR-10, CIFAR-100, Fashion-MNIST and MNIST datasets.
Augmentation by Counterfactual Explanation -- Fixing an Overconfident Classifier
Venue: arXiv cs.LG 2022
Year: 2022

A highly accurate but overconfident model is ill-suited for deployment in critical applications such as healthcare and autonomous driving. The classification outcome should reflect a high uncertainty on ambiguous in-distribution samples that lie close to the decision boundary. The model should also refrain from making overconfident decisions on samples that lie far outside its training distribution, far-out-of-distribution (far-OOD), or on unseen samples from novel classes that lie near its training distribution (near-OOD). This paper proposes an application of counterfactual explanations in fixing an over-confident classifier. Specifically, we propose to fine-tune a given pre-trained classifier using augmentations from a counterfactual explainer (ACE) to fix its uncertainty characteristics while retaining its predictive performance. We perform extensive experiments with detecting far-OOD, near-OOD, and ambiguous samples. Our empirical results show that the revised model have improved uncertainty measures, and its performance is competitive to the state-of-the-art methods.
Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation
Venue: arXiv cs.SD 2022
Year: 2022

Code-switching describes the practice of using more than one language in the same sentence. In this study, we investigate how to optimize a neural transducer based bilingual automatic speech recognition (ASR) model for code-switching speech. Focusing on the scenario where the ASR model is trained without supervised code-switching data, we found that semi-supervised training and synthetic code-switched data can improve the bilingual ASR system on code-switching speech. We analyze how each of the neural transducer's encoders contributes towards code-switching performance by measuring encoder-specific recall values, and evaluate our English/Mandarin system on the ASCEND data set. Our final system achieves 25% mixed error rate (MER) on the ASCEND English/Mandarin code-switching test set -- reducing the MER by 2.1% absolute compared to the previous literature -- while maintaining good accuracy on the monolingual test sets.
Partitioning and Placement of Deep Neural Networks on Distributed Edge Devices to Maximize Inference Throughput
Venue: arXiv cs.NI 2022
Year: 2022

Edge inference has become more widespread, as its diverse applications range from retail to wearable technology. Clusters of networked resource-constrained edge devices are becoming common, yet no system exists to split a DNN across these clusters while maximizing the inference throughput of the system. We present an algorithm which partitions DNNs and distributes them across a set of edge devices with the goal of minimizing the bottleneck latency and therefore maximizing inference throughput. The system scales well to systems of different node memory capacities and numbers of nodes. We find that we can reduce the bottleneck latency by 10x over a random algorithm and 35% over a greedy joint partitioning-placement algorithm. Furthermore we find empirically that for the set of representative models we tested, the algorithm produces results within 9.2% of the optimal bottleneck latency.