OpenRaiser/Intern-Atlas huggingface dataset:
Dataset Distillation: A Comprehensive Review
Venue: arXiv cs.LG 2023
Year: 2023

Recent success of deep learning is largely attributed to the sheer amount of data used for training deep neural networks.Despite the unprecedented success, the massive data, unfortunately, significantly increases the burden on storage and transmission and further gives rise to a cumbersome model training process. Besides, relying on the raw data for training \emph{per se} yields concerns about privacy and copyright. To alleviate these shortcomings, dataset distillation~(DD), also known as dataset condensation (DC), was introduced and has recently attracted much research attention in the community. Given an original dataset, DD aims to derive a much smaller dataset containing synthetic samples, based on which the trained models yield performance comparable with those trained on the original dataset. In this paper, we give a comprehensive review and summary of recent advances in DD and its application. We first introduce the task formally and propose an overall algorithmic framework followed by all existing DD methods. Next, we provide a systematic taxonomy of current methodologies in this area, and discuss their theoretical interconnections. We also present current challenges in DD through extensive experiments and envision possible directions for future works.
Optimal Algorithms for Latent Bandits with Cluster Structure
Venue: arXiv cs.LG 2023
Year: 2023

We consider the problem of latent bandits with cluster structure where there are multiple users, each with an associated multi-armed bandit problem. These users are grouped into \emph{latent} clusters such that the mean reward vectors of users within the same cluster are identical. At each round, a user, selected uniformly at random, pulls an arm and observes a corresponding noisy reward. The goal of the users is to maximize their cumulative rewards. This problem is central to practical recommendation systems and has received wide attention of late \cite{gentile2014online, maillard2014latent}. Now, if each user acts independently, then they would have to explore each arm independently and a regret of $Ω(\sqrt{\mathsf{MNT}})$ is unavoidable, where $\mathsf{M}, \mathsf{N}$ are the number of arms and users, respectively. Instead, we propose LATTICE (Latent bAndiTs via maTrIx ComplEtion) which allows exploitation of the latent cluster structure to provide the minimax optimal regret of $\widetilde{O}(\sqrt{(\mathsf{M}+\mathsf{N})\mathsf{T}})$, when the number of clusters is $\widetilde{O}(1)$. This is the first algorithm to guarantee such strong regret bound. LATTICE is based on a careful exploitation of arm information within a cluster while simultaneously clustering users. Furthermore, it is computationally efficient and requires only $O(\log{\mathsf{T}})$ calls to an offline matrix completion oracle across all $\mathsf{T}$ rounds.
Verifiable Fully Homomorphic Encryption
Venue: arXiv cs.CR 2023
Year: 2023

Fully Homomorphic Encryption (FHE) is seeing increasing real-world deployment to protect data in use by allowing computation over encrypted data. However, the same malleability that enables homomorphic computations also raises integrity issues, which have so far been mostly overlooked. While FHEs lack of integrity has obvious implications for correctness, it also has severe implications for confidentiality: a malicious server can leverage the lack of integrity to carry out interactive key-recovery attacks. As a result, virtually all FHE schemes and applications assume an honest-but-curious server who does not deviate from the protocol. In practice, however, this assumption is insufficient for a wide range of deployment scenarios. While there has been work that aims to address this gap, these have remained isolated efforts considering only aspects of the overall problem and fail to fully address the needs and characteristics of modern FHE schemes and applications. In this paper, we analyze existing FHE integrity approaches, present attacks that exploit gaps in prior work, and propose a new notion for maliciously-secure verifiable FHE. We then instantiate this new notion with a range of techniques, analyzing them and evaluating their performance in a range of different settings. We highlight their potential but also show where future work on tailored integrity solutions for FHE is still required.
An Energy-Efficient Reconfigurable Autoencoder Implementation on FPGA
Venue: arXiv cs.LG 2023
Year: 2023

Autoencoders are unsupervised neural networks that are used to process and compress input data and then reconstruct the data back to the original data size. This allows autoencoders to be used for different processing applications such as data compression, image classification, image noise reduction, and image coloring. Hardware-wise, re-configurable architectures like Field Programmable Gate Arrays (FPGAs) have been used for accelerating computations from several domains because of their unique combination of flexibility, performance, and power efficiency. In this paper, we look at the different autoencoders available and use the convolutional autoencoder in both FPGA and GPU-based implementations to process noisy static MNIST images. We compare the different results achieved with the FPGA and GPU-based implementations and then discuss the pros and cons of each implementation. The evaluation of the proposed design achieved 80%accuracy and our experimental results show that the proposed accelerator achieves a throughput of 21.12 Giga-Operations Per Second (GOP/s) with a 5.93 W on-chip power consumption at 100 MHz. The comparison results with off-the-shelf devices and recent state-of-the-art implementations illustrate that the proposed accelerator has obvious advantages in terms of energy efficiency and design flexibility. We also discuss future work that can be done with the use of our proposed accelerator.
Are Home Security Systems Reliable?
Venue: arXiv cs.CR 2023
Year: 2023

Home security systems have become increasingly popular since they provide an additional layer of protection and peace of mind. These systems typically include battery-powered motion sensors, contact sensors, and smart locks. Z-Wave is a very popular wireless communication technology for these low-power systems. In this paper, we demonstrate two new attacks targeting Z-Wave devices. First, we show how an attacker can remotely attack Z-Wave security devices to increase their power consumption by three orders of magnitude, reducing their battery life from a few years to just a few hours. Second, we show multiple Denial of Service (DoS) attacks which enables an attacker to interrupt the operation of security systems in just a few seconds. Our experiments show that these attacks are effective even when the attacker device is in a car 100 meters away from the targeted house.
Bounded Model Checking for Asynchronous Hyperproperties
Venue: arXiv cs.LO 2023
Year: 2023

Many types of attacks on confidentiality stem from the nondeterministic nature of the environment that computer programs operate in (e.g., schedulers and asynchronous communication channels). In this paper, we focus on verification of confidentiality in nondeterministic environments by reasoning about asynchronous hyperproperties. First, we generalize the temporal logic A-HLTL to allow nested trajectory quantification, where a trajectory determines how different execution traces may advance and stutter. We propose a bounded model checking algorithm for A-HLTL based on QBF-solving for a fragment of the generalized A-HLTL and evaluate it by various case studies on concurrent programs, scheduling attacks, compiler optimization, speculative execution, and cache timing attacks. We also rigorously analyze the complexity of model checking for different fragments of A-HLTL.
A variational autoencoder-based nonnegative matrix factorisation model for deep dictionary learning
Venue: arXiv cs.LG 2023
Year: 2023

Construction of dictionaries using nonnegative matrix factorisation (NMF) has extensive applications in signal processing and machine learning. With the advances in deep learning, training compact and robust dictionaries using deep neural networks, i.e., dictionaries of deep features, has been proposed. In this study, we propose a probabilistic generative model which employs a variational autoencoder (VAE) to perform nonnegative dictionary learning. In contrast to the existing VAE models, we cast the model under a statistical framework with latent variables obeying a Gamma distribution and design a new loss function to guarantee the nonnegative dictionaries. We adopt an acceptance-rejection sampling reparameterization trick to update the latent variables iteratively. We apply the dictionaries learned from VAE-NMF to two signal processing tasks, i.e., enhancement of speech and extraction of muscle synergies. Experimental results demonstrate that VAE-NMF performs better in learning the latent nonnegative dictionaries in comparison with state-of-the-art methods.
The Chance of Winning Election Impacts on Social Media Strategy
Venue: arXiv cs.CY 2023
Year: 2023

Social media has been a paramount arena for election campaigns for political actors. While many studies have been paying attention to the political campaigns related to partisanship, politicians also can conduct different campaigns according to their chances of winning. Leading candidates, for example, do not behave the same as fringe candidates in their elections, and vice versa. We, however, know little about this difference in social media political campaign strategies according to their odds in elections. We tackle this problem by analyzing candidates' tweets in terms of users, topics, and sentiment of replies. Our study finds that, as their chances of winning increase, candidates narrow the targets they communicate with, from people in general to the electrical districts and specific persons (verified accounts or accounts with many followers). Our study brings new insights into the candidates' campaign strategies through the analysis based on the novel perspective of the candidate's electoral situation.
A Novel Sparse Regularizer
Venue: arXiv cs.LG 2023
Year: 2023

$L_p$-norm regularization schemes such as $L_0$, $L_1$, and $L_2$-norm regularization and $L_p$-norm-based regularization techniques such as weight decay, LASSO, and elastic net compute a quantity which depends on model weights considered in isolation from one another. This paper introduces a regularizer based on minimizing a novel measure of entropy applied to the model during optimization. In contrast with $L_p$-norm-based regularization, this regularizer is concerned with the spatial arrangement of weights within a weight matrix. This novel regularizer is an additive term for the loss function and is differentiable, simple and fast to compute, scale-invariant, requires a trivial amount of additional memory, and can easily be parallelized. Empirically this method yields approximately a one order-of-magnitude improvement in the number of nonzero model parameters required to achieve a given level of test accuracy when training LeNet300 on MNIST.
Review, Meta-Taxonomy, and Use Cases of Cyberattack Taxonomies of Manufacturing Cybersecurity Threat Attributes and Countermeasures
Venue: arXiv cs.CR 2023
Year: 2023

A thorough and systematic understanding of different elements of cyberattacks is essential for developing the necessary tools to prevent, detect, diagnose, and mitigate cyberattacks in manufacturing systems. In response, researchers have proposed several attack taxonomies as methods for recognizing and categorizing various cyberattack attributes. However, those taxonomies cover selected attack attributes depending on the research focus, sometimes accompanied by inconsistent naming and definitions. These seemingly different taxonomies often overlap and can complement each other to create a comprehensive knowledge base of cyberattack attributes that is currently missing in the literature. Additionally, there is a missing link from creating structured knowledge by using a taxonomy to applying this structure for cybersecurity tools development and aiding practitioners in using it. To tackle these challenges, this article highlights how cyberattack taxonomies can be used to better understand and characterize manufacturing cybersecurity threats. It also reviews and analyzes current taxonomical classifications of manufacturing cybersecurity threat attributes and countermeasures, as well as the proliferation of the scope and coverage in existing taxonomies. As a result, these taxonomies are compiled into a more comprehensive and consistent meta-taxonomy for the smart manufacturing space. The resulting meta-taxonomy provides a holistic analysis of current taxonomies and integrates them into a unified structure. Based on this structure, this paper identifies gaps in current attack taxonomies and provides directions for future improvements. Finally, the paper introduces potential use cases for attack taxonomies in smart manufacturing systems for assessing security threats and their associated risks, devising risk mitigation strategies, and informing the application of cybersecurity frameworks.
Face Recognition in the age of CLIP & Billion image datasets
Venue: arXiv cs.CV 2023
Year: 2023

CLIP (Contrastive Language-Image Pre-training) models developed by OpenAI have achieved outstanding results on various image recognition and retrieval tasks, displaying strong zero-shot performance. This means that they are able to perform effectively on tasks for which they have not been explicitly trained. Inspired by the success of OpenAI CLIP, a new publicly available dataset called LAION-5B was collected which resulted in the development of open ViT-H/14, ViT-G/14 models that outperform the OpenAI L/14 model. The LAION-5B dataset also released an approximate nearest neighbor index, with a web interface for search & subset creation. In this paper, we evaluate the performance of various CLIP models as zero-shot face recognizers. Our findings show that CLIP models perform well on face recognition tasks, but increasing the size of the CLIP model does not necessarily lead to improved accuracy. Additionally, we investigate the robustness of CLIP models against data poisoning attacks by testing their performance on poisoned data. Through this analysis, we aim to understand the potential consequences and misuse of search engines built using CLIP models, which could potentially function as unintentional face recognition engines.
Deep Dynamic Scene Deblurring from Optical Flow
Venue: arXiv cs.CV 2023
Year: 2023

Deblurring can not only provide visually more pleasant pictures and make photography more convenient, but also can improve the performance of objection detection as well as tracking. However, removing dynamic scene blur from images is a non-trivial task as it is difficult to model the non-uniform blur mathematically. Several methods first use single or multiple images to estimate optical flow (which is treated as an approximation of blur kernels) and then adopt non-blind deblurring algorithms to reconstruct the sharp images. However, these methods cannot be trained in an end-to-end manner and are usually computationally expensive. In this paper, we explore optical flow to remove dynamic scene blur by using the multi-scale spatially variant recurrent neural network (RNN). We utilize FlowNets to estimate optical flow from two consecutive images in different scales. The estimated optical flow provides the RNN weights in different scales so that the weights can better help RNNs to remove blur in the feature spaces. Finally, we develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features. Both quantitative and qualitative evaluations on the benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art algorithms in terms of accuracy, speed, and model size.
Connected OFCity Challenge: an updated perspective on technology for connected cities
Venue: arXiv cs.NI 2023
Year: 2023

This paper gives an update on technologies discussed during three OFC events, called 'The Connected OFCity Challenge', from 2016 to 2018. It focuses on research development and field deployment of Passive Optical Networks and Cloud-Based technologies.
Pseudorandom Generators for Sliding-Window Algorithms
Venue: arXiv cs.CC 2023
Year: 2023

A sliding-window algorithm of window size $t$ is an algorithm whose current operation depends solely on the last $t$ symbols read. We construct pseudorandom generators (PRGs) for low-space randomized sliding-window algorithms that have access to a binary randomness source. More specifically, we lift these algorithms to the non-uniform setting of branching programs and study them as a subclass thereof that we call sliding-window branching programs (SWBPs), accordingly. For general SWBPs, given a base PRG $G_\mathrm{base}$ with seed length $d_\mathrm{base}$ that $\varepsilon_\mathrm{base}$-fools width-$w$, length-$t$ (general) branching programs, we give two PRG constructions for fooling any same-width SWBP of length $n$ and window size $t$ (where we assume $w \ge n$). The first uses an additional $d_\mathrm{base} + O(\log(n/t) \log(1/\varepsilon_\mathrm{base}))$ random bits, whereas the second has a seed length of $O((d_\mathrm{base} + \log\log(n/t) + \log(1/\varepsilon_\mathrm{base})) \log(d_\mathrm{base} + \log(1/\varepsilon_\mathrm{base})))$. Both PRGs incur only a $(n/2t)^{O(1)}$ multiplicative loss in the error parameter. As an application, we show how to decide the language of a sublinear-time probabilistic cellular automaton using small space. More specifically, these results target the model of PACAs, which are probabilistic cellular automata that accept if and only if all cells are simultaneously accepting. For (sublinear) $T(n) = Ω(\log n)^{1.01}$, we prove that every language accepted by a $T$-time one-sided error PACA (the PACA equivalent of $\mathsf{RP}$) can be decided using only $O(T)$ space. Meanwhile, forgoing the previous requirement on $T$, we show the same holds for $T$-time two-sided error PACA (the PACA equivalent of $\mathsf{BPP}$) if we use $\tilde{O}(T) + O(\log n)$ space instead (where the $\tilde{O}$ notation hides only $\mathsf{polylog}(T)$ factors).
Biases in Scholarly Recommender Systems: Impact, Prevalence, and Mitigation
Venue: arXiv cs.IR 2023
Year: 2023

With the remarkable increase in the number of scientific entities such as publications, researchers, and scientific topics, and the associated information overload in science, academic recommender systems have become increasingly important for millions of researchers and science enthusiasts. However, it is often overlooked that these systems are subject to various biases. In this article, we first break down the biases of academic recommender systems and characterize them according to their impact and prevalence. In doing so, we distinguish between biases originally caused by humans and biases induced by the recommender system. Second, we provide an overview of methods that have been used to mitigate these biases in the scholarly domain. Based on this, third, we present a framework that can be used by researchers and developers to mitigate biases in scholarly recommender systems and to evaluate recommender systems fairly. Finally, we discuss open challenges and possible research directions related to scholarly biases.
Machine learning techniques for the Schizophrenia diagnosis: A comprehensive review and future research directions
Venue: arXiv cs.LG 2023
Year: 2023

Schizophrenia (SCZ) is a brain disorder where different people experience different symptoms, such as hallucination, delusion, flat-talk, disorganized thinking, etc. In the long term, this can cause severe effects and diminish life expectancy by more than ten years. Therefore, early and accurate diagnosis of SCZ is prevalent, and modalities like structural magnetic resonance imaging (sMRI), functional MRI (fMRI), diffusion tensor imaging (DTI), and electroencephalogram (EEG) assist in witnessing the brain abnormalities of the patients. Moreover, for accurate diagnosis of SCZ, researchers have used machine learning (ML) algorithms for the past decade to distinguish the brain patterns of healthy and SCZ brains using MRI and fMRI images. This paper seeks to acquaint SCZ researchers with ML and to discuss its recent applications to the field of SCZ study. This paper comprehensively reviews state-of-the-art techniques such as ML classifiers, artificial neural network (ANN), deep learning (DL) models, methodological fundamentals, and applications with previous studies. The motivation of this paper is to benefit from finding the research gaps that may lead to the development of a new model for accurate SCZ diagnosis. The paper concludes with the research finding, followed by the future scope that directly contributes to new research directions.
Multi-objective Software Architecture Refactoring driven by Quality Attributes
Venue: arXiv cs.SE 2023
Year: 2023

Architecture optimization is the process of automatically generating design options, typically to enhance software's quantifiable quality attributes, such as performance and reliability. Multi-objective optimization approaches have been used in this situation to assist the designer in selecting appropriate trade-offs between a number of non-functional features. Through automated refactoring, design alternatives can be produced in this process, and assessed using non-functional models. This type of optimization tasks are hard and time- and resource-intensive, which frequently hampers their use in software engineering procedures. In this paper, we present our optimization framework where we examined the performance of various genetic algorithms. We also exercised our framework with two case studies with various levels of size, complexity, and domain served as our test subjects.
Adversarial AI in Insurance: Pervasiveness and Resilience
Venue: arXiv cs.LG 2023
Year: 2023

The rapid and dynamic pace of Artificial Intelligence (AI) and Machine Learning (ML) is revolutionizing the insurance sector. AI offers significant, very much welcome advantages to insurance companies, and is fundamental to their customer-centricity strategy. It also poses challenges, in the project and implementation phase. Among those, we study Adversarial Attacks, which consist of the creation of modified input data to deceive an AI system and produce false outputs. We provide examples of attacks on insurance AI applications, categorize them, and argue on defence methods and precautionary systems, considering that they can involve few-shot and zero-shot multilabelling. A related topic, with growing interest, is the validation and verification of systems incorporating AI and ML components. These topics are discussed in various sections of this paper.