OpenRaiser/Intern-Atlas huggingface dataset:
Iterative Adaptively Regularized LASSO-ADMM Algorithm for CFAR Estimation of Sparse Signals: IAR-LASSO-ADMM-CFAR Algorithm
Venue: arXiv cs.IT 2022
Year: 2022

The least-absolute shrinkage and selection operator (LASSO) is a regularization technique for estimating sparse signals of interest emerging in various applications and can be efficiently solved via the alternating direction method of multipliers (ADMM), which will be termed as LASSO-ADMM algorithm. The choice of the regularization parameter has significant impact on the performance of LASSO-ADMM algorithm. However, the optimization for the regularization parameter in the existing LASSO-ADMM algorithms has not been solved yet. In order to optimize this regularization parameter, we propose an efficient iterative adaptively regularized LASSO-ADMM (IAR-LASSO-ADMM) algorithm by iteratively updating the regularization parameter in the LASSO-ADMM algorithm. Moreover, a method is designed to iteratively update the regularization parameter by adding an outer iteration to the LASSO-ADMM algorithm. Specifically, at each outer iteration the zero support of the estimate obtained by the inner LASSO-ADMM algorithm is utilized to estimate the noise variance, and the noise variance is utilized to update the threshold according to a pre-defined const false alarm rate (CFAR). Then, the resulting threshold is utilized to update both the non-zero support of the estimate and the regularization parameter, and proceed to the next inner iteration. In addition, a suitable stopping criterion is designed to terminate the outer iteration process to obtain the final non-zero support of the estimate of the sparse measurement signals. The resulting algorithm is termed as IAR-LASSO-ADMM-CFAR algorithm. Finally, simulation results have been presented to show that the proposed IAR-LASSO-ADMM-CFAR algorithm outperforms the conventional LASSO-ADMM algorithm and other existing algorithms in terms of reconstruction accuracy, and its sparsity order estimate is more accurate than the existing algorithms.
Unsupervised Structure-Consistent Image-to-Image Translation
Venue: arXiv cs.CV 2022
Year: 2022

The Swapping Autoencoder achieved state-of-the-art performance in deep image manipulation and image-to-image translation. We improve this work by introducing a simple yet effective auxiliary module based on gradient reversal layers. The auxiliary module's loss forces the generator to learn to reconstruct an image with an all-zero texture code, encouraging better disentanglement between the structure and texture information. The proposed attribute-based transfer method enables refined control in style transfer while preserving structural information without using a semantic mask. To manipulate an image, we encode both the geometry of the objects and the general style of the input images into two latent codes with an additional constraint that enforces structure consistency. Moreover, due to the auxiliary loss, training time is significantly reduced. The superiority of the proposed model is demonstrated in complex domains such as satellite images where state-of-the-art are known to fail. Lastly, we show that our model improves the quality metrics for a wide range of datasets while achieving comparable results with multi-modal image generation techniques.
Graphical Models of False Information and Fact Checking Ecosystems
Venue: arXiv cs.SI 2022
Year: 2022

The wide spread of false information online including misinformation and disinformation has become a major problem for our highly digitised and globalised society. A lot of research has been done to better understand different aspects of false information online such as behaviours of different actors and patterns of spreading, and also on better detection and prevention of such information using technical and socio-technical means. One major approach to detect and debunk false information online is to use human fact-checkers, who can be helped by automated tools. Despite a lot of research done, we noticed a significant gap on the lack of conceptual models describing the complicated ecosystems of false information and fact checking. In this paper, we report the first graphical models of such ecosystems, focusing on false information online in multiple contexts, including traditional media outlets and user-generated content. The proposed models cover a wide range of entity types and relationships, and can be a new useful tool for researchers and practitioners to study false information online and the effects of fact checking.
ImitAL: Learned Active Learning Strategy on Synthetic Data
Venue: arXiv cs.LG 2022
Year: 2022

Active Learning (AL) is a well-known standard method for efficiently obtaining annotated data by first labeling the samples that contain the most information based on a query strategy. In the past, a large variety of such query strategies has been proposed, with each generation of new strategies increasing the runtime and adding more complexity. However, to the best of our our knowledge, none of these strategies excels consistently over a large number of datasets from different application domains. Basically, most of the the existing AL strategies are a combination of the two simple heuristics informativeness and representativeness, and the big differences lie in the combination of the often conflicting heuristics. Within this paper, we propose ImitAL, a domain-independent novel query strategy, which encodes AL as a learning-to-rank problem and learns an optimal combination between both heuristics. We train ImitAL on large-scale simulated AL runs on purely synthetic datasets. To show that ImitAL was successfully trained, we perform an extensive evaluation comparing our strategy on 13 different datasets, from a wide range of domains, with 7 other query strategies.
DP2-Pub: Differentially Private High-Dimensional Data Publication with Invariant Post Randomization
Venue: arXiv cs.CR 2022
Year: 2022

A large amount of high-dimensional and heterogeneous data appear in practical applications, which are often published to third parties for data analysis, recommendations, targeted advertising, and reliable predictions. However, publishing these data may disclose personal sensitive information, resulting in an increasing concern on privacy violations. Privacy-preserving data publishing has received considerable attention in recent years. Unfortunately, the differentially private publication of high dimensional data remains a challenging problem. In this paper, we propose a differentially private high-dimensional data publication mechanism (DP2-Pub) that runs in two phases: a Markov-blanket-based attribute clustering phase and an invariant post randomization (PRAM) phase. Specifically, splitting attributes into several low-dimensional clusters with high intra-cluster cohesion and low inter-cluster coupling helps obtain a reasonable allocation of privacy budget, while a double-perturbation mechanism satisfying local differential privacy facilitates an invariant PRAM to ensure no loss of statistical information and thus significantly preserves data utility. We also extend our DP2-Pub mechanism to the scenario with a semi-honest server which satisfies local differential privacy. We conduct extensive experiments on four real-world datasets and the experimental results demonstrate that our mechanism can significantly improve the data utility of the published data while satisfying differential privacy.
Ontology-Driven Self-Supervision for Adverse Childhood Experiences Identification Using Social Media Datasets
Venue: arXiv cs.CL 2022
Year: 2022

Adverse Childhood Experiences (ACEs) are defined as a collection of highly stressful, and potentially traumatic, events or circumstances that occur throughout childhood and/or adolescence. They have been shown to be associated with increased risks of mental health diseases or other abnormal behaviours in later lives. However, the identification of ACEs from textual data with Natural Language Processing (NLP) is challenging because (a) there are no NLP ready ACE ontologies; (b) there are few resources available for machine learning, necessitating the data annotation from clinical experts; (c) costly annotations by domain experts and large number of documents for supporting large machine learning models. In this paper, we present an ontology-driven self-supervised approach (derive concept embeddings using an auto-encoder from baseline NLP results) for producing a publicly available resource that would support large-scale machine learning (e.g., training transformer based large language models) on social media corpus. This resource as well as the proposed approach are aimed to facilitate the community in training transferable NLP models for effectively surfacing ACEs in low-resource scenarios like NLP on clinical notes within Electronic Health Records. The resource including a list of ACE ontology terms, ACE concept embeddings and the NLP annotated corpus is available at https://github.com/knowlab/ACE-NLP.
Attractor Stability in Finite Asynchronous Biological System Models
Venue: arXiv cs.DM 2022
Year: 2022

We present mathematical techniques for exhaustive studies of long-term dynamics of asynchronous biological system models. Specifically, we extend the notion of $κ$-equivalence developed for graph dynamical systems to support systematic analysis of all possible attractor configurations that can be generated when varying the asynchronous update order (Macauley and Mortveit (2009)). We extend earlier work by Veliz-Cuba and Stigler (2011), Goles et al. (2014), and others by comparing long-term dynamics up to topological conjugation: rather than comparing the exact states and their transitions on attractors, we only compare the attractor structures. In general, obtaining this information is computationally intractable. Here, we adapt and apply combinatorial theory for dynamical systems to develop computational methods that greatly reduce this computational cost. We give a detailed algorithm and apply it to ($i$) the lac operon model for Escherichia coli proposed by Veliz-Cuba and Stigler (2011), and ($ii$) the regulatory network involved in the control of the cell cycle and cell differentiation in the Caenorhabditis elegans vulva precursor cells proposed by Weinstein et al. (2015). In both cases, we uncover all possible limit cycle structures for these networks under sequential updates. Specifically, for the lac operon model, rather than examining all $10! > 3.6 \cdot 10^6$ sequential update orders, we demonstrate that it is sufficient to consider $344$ representative update orders, and, more notably, that these $344$ representatives give rise to $4$ distinct attractor structures. A similar analysis performed for the C. elegans model demonstrates that it has precisely $125$ distinct attractor structures. We conclude with observations on the variety and distribution of the models' attractor structures and use the results to discuss their robustness.
AI-coupled HPC Workflows
Venue: arXiv cs.DC 2022
Year: 2022

Increasingly, scientific discovery requires sophisticated and scalable workflows. Workflows have become the ``new applications,'' wherein multi-scale computing campaigns comprise multiple and heterogeneous executable tasks. In particular, the introduction of AI/ML models into the traditional HPC workflows has been an enabler of highly accurate modeling, typically reducing computational needs compared to traditional methods. This chapter discusses various modes of integrating AI/ML models to HPC computations, resulting in diverse types of AI-coupled HPC workflows. The increasing need of coupling AI/ML and HPC across scientific domains is motivated, and then exemplified by a number of production-grade use cases for each mode. We additionally discuss the primary challenges of extreme-scale AI-coupled HPC campaigns -- task heterogeneity, adaptivity, performance -- and several framework and middleware solutions which aim to address them. While both HPC workflow and AI/ML computing paradigms are independently effective, we highlight how their integration, and ultimate convergence, is leading to significant improvements in scientific performance across a range of domains, ultimately resulting in scientific explorations otherwise unattainable.
The Esports Frontier: Rendering for Competitive Games
Venue: arXiv cs.GR 2022
Year: 2022

Real-time graphics is commonly thought of as anything exceeding about 30 fps, where the interactivity of the application becomes fluid enough for high rates of interaction. Inspired by esports and competitive gaming, where players regularly exceed the threshold for real-time by 10x (esports displays commonly reach 360 Hz or beyond), this talk begins the exploration of how rendering has the opportunity to evolve beyond the current state of focus on either image quality or frame rate. Esports gamers regularly decline nearly all options for increased image quality in exchange for maximum frame rates. However, there remains a distinct opportunity to move beyond the focus on video as a sequence of images and instead rethink the pipeline for more continuous updates.
Multiresolution Neural Networks for Imaging
Venue: arXiv cs.CV 2022
Year: 2022

We present MR-Net, a general architecture for multiresolution neural networks, and a framework for imaging applications based on this architecture. Our coordinate-based networks are continuous both in space and in scale as they are composed of multiple stages that progressively add finer details. Besides that, they are a compact and efficient representation. We show examples of multiresolution image representation and applications to texturemagnification, minification, and antialiasing. This document is the extended version of the paper [PNS+22]. It includes additional material that would not fit the page limitations of the conference track for publication.
MIMO Multifunction RF Systems: Detection Performance and Waveform Design
Venue: arXiv cs.IT 2022
Year: 2022

This paper studies the detection performance of a multiple-input-multiple-output (MIMO) multifunction radio frequency (MFRF) system, which simultaneously supports radar, communication, and jamming. We show that the detection performance of the MIMO MFRF system improves as the transmit signal-to-interference-plus-noise-ratio (SINR) increases. To analyze the achievable SINR of the system, we formulate an SINR maximization problem under the communication and jamming functionality constraint as well as a transmit energy constraint. We derive a closed-form solution of this optimization problem for energy-constrained waveforms and present a detailed analysis of the achievable SINR. Moreover, we analyze the SINR for systems transmitting constant-modulus waveforms, which are often used in practice. We propose an efficient constant-modulus waveform design algorithm to maximize the SINR. Numerical results demonstrate the capability of a MIMO array to provide multiple functions, and also show the tradeoff between radar detection and the communication/jamming functionality.
Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima
Venue: arXiv cs.LG 2022
Year: 2022

Learning rate is one of the most important hyper-parameters that has a significant influence on neural network training. Learning rate schedules are widely used in real practice to adjust the learning rate according to pre-defined schedules for fast convergence and good generalization. However, existing learning rate schedules are all heuristic algorithms and lack theoretical support. Therefore, people usually choose the learning rate schedules through multiple ad-hoc trials, and the obtained learning rate schedules are sub-optimal. To boost the performance of the obtained sub-optimal learning rate schedule, we propose a generic learning rate schedule plugin, called LEArning Rate Perturbation (LEAP), which can be applied to various learning rate schedules to improve the model training by introducing a certain perturbation to the learning rate. We found that, with such a simple yet effective strategy, training processing exponentially favors flat minima rather than sharp minima with guaranteed convergence, which leads to better generalization ability. In addition, we conduct extensive experiments which show that training with LEAP can improve the performance of various deep learning models on diverse datasets using various learning rate schedules (including constant learning rate).
A simplified convergence theory for Byzantine resilient stochastic gradient descent
Venue: arXiv cs.LG 2022
Year: 2022

In distributed learning, a central server trains a model according to updates provided by nodes holding local data samples. In the presence of one or more malicious servers sending incorrect information (a Byzantine adversary), standard algorithms for model training such as stochastic gradient descent (SGD) fail to converge. In this paper, we present a simplified convergence theory for the generic Byzantine Resilient SGD method originally proposed by Blanchard et al. [NeurIPS 2017]. Compared to the existing analysis, we shown convergence to a stationary point in expectation under standard assumptions on the (possibly nonconvex) objective function and flexible assumptions on the stochastic gradients.
Embedding Privacy Into Design Through Software Developers: Challenges & Solutions
Venue: arXiv cs.HC 2022
Year: 2022

To make privacy a first-class citizen in software, we argue for equipping developers with usable tools, as well as providing support from organizations, educators, and regulators. We discuss the challenges with the successful integration of privacy features and propose solutions for stakeholders to help developers perform privacy-related tasks.
Credit card fraud detection - Classifier selection strategy
Venue: arXiv cs.LG 2022
Year: 2022

Machine learning has opened up new tools for financial fraud detection. Using a sample of annotated transactions, a machine learning classification algorithm learns to detect frauds. With growing credit card transaction volumes and rising fraud percentages there is growing interest in finding appropriate machine learning classifiers for detection. However, fraud data sets are diverse and exhibit inconsistent characteristics. As a result, a model effective on a given data set is not guaranteed to perform on another. Further, the possibility of temporal drift in data patterns and characteristics over time is high. Additionally, fraud data has massive and varying imbalance. In this work, we evaluate sampling methods as a viable pre-processing mechanism to handle imbalance and propose a data-driven classifier selection strategy for characteristic highly imbalanced fraud detection data sets. The model derived based on our selection strategy surpasses peer models, whilst working in more realistic conditions, establishing the effectiveness of the strategy.
Empirical study of Machine Learning Classifier Evaluation Metrics behavior in Massively Imbalanced and Noisy data
Venue: arXiv cs.LG 2022
Year: 2022

With growing credit card transaction volumes, the fraud percentages are also rising, including overhead costs for institutions to combat and compensate victims. The use of machine learning into the financial sector permits more effective protection against fraud and other economic crime. Suitably trained machine learning classifiers help proactive fraud detection, improving stakeholder trust and robustness against illicit transactions. However, the design of machine learning based fraud detection algorithms has been challenging and slow due the massively unbalanced nature of fraud data and the challenges of identifying the frauds accurately and completely to create a gold standard ground truth. Furthermore, there are no benchmarks or standard classifier evaluation metrics to measure and identify better performing classifiers, thus keeping researchers in the dark. In this work, we develop a theoretical foundation to model human annotation errors and extreme imbalance typical in real world fraud detection data sets. By conducting empirical experiments on a hypothetical classifier, with a synthetic data distribution approximated to a popular real world credit card fraud data set, we simulate human annotation errors and extreme imbalance to observe the behavior of popular machine learning classifier evaluation matrices. We demonstrate that a combined F1 score and g-mean, in that specific order, is the best evaluation metric for typical imbalanced fraud detection model classification.
A logical theory for conditional weak ontic necessity based on context update
Venue: arXiv cs.CL 2022
Year: 2022

Weak ontic necessity is the ontic necessity expressed by ``should'' or ``ought to'' in English. An example of it is ``I should be dead by now''. A feature of this necessity is whether it holds does not have anything to do with whether its prejacent holds. In this paper, we present a logical theory for conditional weak ontic necessity based on context update. A context is a set of ordered defaults, determining expected possible states of the present world. Sentences are evaluated with respect to contexts. When evaluating the conditional weak ontic necessity with respect to a context, we first update the context with the antecedent, then check whether the consequent holds with respect to the updated context. The logic is complete. Our theory combines premise semantics and update semantics for conditionals.
Dynamic collaborative filtering Thompson Sampling for cross-domain advertisements recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Recently online advertisers utilize Recommender systems (RSs) for display advertising to improve users' engagement. The contextual bandit model is a widely used RS to exploit and explore users' engagement and maximize the long-term rewards such as clicks or conversions. However, the current models aim to optimize a set of ads only in a specific domain and do not share information with other models in multiple domains. In this paper, we propose dynamic collaborative filtering Thompson Sampling (DCTS), the novel yet simple model to transfer knowledge among multiple bandit models. DCTS exploits similarities between users and between ads to estimate a prior distribution of Thompson sampling. Such similarities are obtained based on contextual features of users and ads. Similarities enable models in a domain that didn't have much data to converge more quickly by transferring knowledge. Moreover, DCTS incorporates temporal dynamics of users to track the user's recent change of preference. We first show transferring knowledge and incorporating temporal dynamics improve the performance of the baseline models on a synthetic dataset. Then we conduct an empirical analysis on a real-world dataset and the result showed that DCTS improves click-through rate by 9.7% than the state-of-the-art models. We also analyze hyper-parameters that adjust temporal dynamics and similarities and show the best parameter which maximizes CTR.
Backwards Reachability for Probabilistic Timed Automata: A Replication Report
Venue: arXiv cs.LO 2022
Year: 2022

Backwards reachability is an efficient zone-based approach for model checking probabilistic timed automata w.r.t. PTCTL properties. Current implementations, however, are restricted to maximum probabilities of reachability properties. In this paper, we report on our new implementation of backwards reachability as part of the Modest Toolset. Its support for minimum and maximum probabilities of until formulas makes it the most general implementation available today. We compare its behaviour to the experimental results reported in the original papers presenting the backwards reachability technique.