OpenRaiser/Intern-Atlas huggingface dataset:
Efficient micro data centres deployment for mobile healthcare monitoring systems in IoT urban scenarios
Venue: arXiv cs.AI 2023
Year: 2023

In the last decade, the Internet of Things paradigm has caused an exponential increase in the number of connected devices. This trend brings the Internet closer to everyday activities and enables data collection that can be used to create and improve a great variety of services and applications. Despite its great benefits, this paradigm also comes with several challenges. More powerful storage and processing capabilities are required to service all these devices. Additionally, the need to deploy and manage the infrastructure to efficiently support these resources continues to pose a challenge. Modeling and simulation can help to design and analyze these scenarios, providing flexible and powerful mechanisms to study and compare different strategies and infrastructures. In this scenario, Micro Data Centers (MDCs) can be used as an effective way of reducing overwhelmed Cloud Data Center infrastructures. This paper explores an M\&S methodology to study the overall power consumption of a healthcare IoT scenario. The patients wear non-intrusive monitoring devices that periodically generate tasks to be executed in MDCs. We extract the layout of existing urban infrastructures, simulate the monitored population's behavior, and compare the power consumption of several data center configurations.
Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning
Venue: arXiv cs.AI 2023
Year: 2023

We approach the problem of understanding how people interact with each other in collaborative settings, especially when individuals know little about their teammates, via Multiagent Inverse Reinforcement Learning (MIRL), where the goal is to infer the reward functions guiding the behavior of each individual given trajectories of a team's behavior during some task. Unlike current MIRL approaches, we do not assume that team members know each other's goals a priori; rather, that they collaborate by adapting to the goals of others perceived by observing their behavior, all while jointly performing a task. To address this problem, we propose a novel approach to MIRL via Theory of Mind (MIRL-ToM). For each agent, we first use ToM reasoning to estimate a posterior distribution over baseline reward profiles given their demonstrated behavior. We then perform MIRL via decentralized equilibrium by employing single-agent Maximum Entropy IRL to infer a reward function for each agent, where we simulate the behavior of other teammates according to the time-varying distribution over profiles. We evaluate our approach in a simulated 2-player search-and-rescue operation where the goal of the agents, playing different roles, is to search for and evacuate victims in the environment. Our results show that the choice of baseline profiles is paramount to the recovery of the ground-truth rewards, and that MIRL-ToM is able to recover the rewards used by agents interacting both with known and unknown teammates.
LiT Tuned Models for Efficient Species Detection
Venue: arXiv cs.CV 2023
Year: 2023

Recent advances in training vision-language models have demonstrated unprecedented robustness and transfer learning effectiveness; however, standard computer vision datasets are image-only, and therefore not well adapted to such training methods. Our paper introduces a simple methodology for adapting any fine-grained image classification dataset for distributed vision-language pretraining. We implement this methodology on the challenging iNaturalist-2021 dataset, comprised of approximately 2.7 million images of macro-organisms across 10,000 classes, and achieve a new state-of-the art model in terms of zero-shot classification accuracy. Somewhat surprisingly, our model (trained using a new method called locked-image text tuning) uses a pre-trained, frozen vision representation, proving that language alignment alone can attain strong transfer learning performance, even on fractious, long-tailed datasets. Our approach opens the door for utilizing high quality vision-language pretrained models in agriculturally relevant applications involving species detection.
OppLoD: the Opponency based Looming Detector, Model Extension of Looming Sensitivity from LGMD to LPLC2
Venue: arXiv cs.CV 2023
Year: 2023

Looming detection plays an important role in insect collision prevention systems. As a vital capability evolutionary survival, it has been extensively studied in neuroscience and is attracting increasing research interest in robotics due to its close relationship with collision detection and navigation. Visual cues such as angular size, angular velocity, and expansion have been widely studied for looming detection by means of optic flow or elementary neural computing research. However, a critical visual motion cue has been long neglected because it is so easy to be confused with expansion, that is radial-opponent-motion (ROM). Recent research on the discovery of LPLC2, a ROM-sensitive neuron in Drosophila, has revealed its ultra-selectivity because it only responds to stimuli with focal, outward movement. This characteristic of ROM-sensitivity is consistent with the demand for collision detection because it is strongly associated with danger looming that is moving towards the center of the observer. Thus, we hope to extend the well-studied neural model of the lobula giant movement detector (LGMD) with ROM-sensibility in order to enhance robustness and accuracy at the same time. In this paper, we investigate the potential to extend an image velocity-based looming detector, the lobula giant movement detector (LGMD), with ROM-sensibility. To achieve this, we propose the mathematical definition of ROM and its main property, the radial motion opponency (RMO). Then, a synaptic neuropile that analogizes the synaptic processing of LPLC2 is proposed in the form of lateral inhibition and attention. Thus, our proposed model is the first to perform both image velocity selectivity and ROM sensitivity. Systematic experiments are conducted to exhibit the huge potential of the proposed bio-inspired looming detector.
CertViT: Certified Robustness of Pre-Trained Vision Transformers
Venue: arXiv cs.CV 2023
Year: 2023

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks. Code : https://github.com/sagarverma/transformer-lipschitz
Quantum Machine Learning hyperparameter search
Venue: arXiv cs.LG 2023
Year: 2023

This paper presents a quantum-based Fourier-regression approach for machine learning hyperparameter optimization applied to a benchmark of models trained on a dataset related to a forecast problem in the airline industry. Our approach utilizes the Fourier series method to represent the hyperparameter search space, which is then optimized using quantum algorithms to find the optimal set of hyperparameters for a given machine learning model. Our study evaluates the proposed method on a benchmark of models trained to predict a forecast problem in the airline industry using a standard HyperParameter Optimizer (HPO). The results show that our approach outperforms traditional hyperparameter optimization methods in terms of accuracy and convergence speed for the given search space. Our study provides a new direction for future research in quantum-based machine learning hyperparameter optimization.
Meta-World Conditional Neural Processes
Venue: arXiv cs.LG 2023
Year: 2023

We propose Meta-World Conditional Neural Processes (MW-CNP), a conditional world model generator that leverages sample efficiency and scalability of Conditional Neural Processes to enable an agent to sample from its own "hallucination". We intend to reduce the agent's interaction with the target environment at test time as much as possible. To reduce the number of samples required at test time, we first obtain a latent representation of the transition dynamics from a single rollout from the test environment with hidden parameters. Then, we obtain rollouts for few-shot learning by interacting with the "hallucination" generated by the meta-world model. Using the world model representation from MW-CNP, the meta-RL agent can adapt to an unseen target environment with significantly fewer samples collected from the target environment compared to the baselines. We emphasize that the agent does not have access to the task parameters throughout training and testing, and MW-CNP is trained on offline interaction data logged during meta-training.
Hello Me, Meet the Real Me: Audio Deepfake Attacks on Voice Assistants
Venue: arXiv cs.CR 2023
Year: 2023

The radical advances in telecommunications and computer science have enabled a myriad of applications and novel seamless interaction with computing interfaces. Voice Assistants (VAs) have become a norm for smartphones, and millions of VAs incorporated in smart devices are used to control these devices in the smart home context. Previous research has shown that they are prone to attacks, leading vendors to countermeasures. One of these measures is to allow only a specific individual, the device's owner, to perform possibly dangerous tasks, that is, tasks that may disclose personal information, involve monetary transactions etc. To understand the extent to which VAs provide the necessary protection to their users, we experimented with two of the most widely used VAs, which the participants trained. We then utilised voice synthesis using samples provided by participants to synthesise commands that were used to trigger the corresponding VA and perform a dangerous task. Our extensive results showed that more than 30\% of our deepfake attacks were successful and that there was at least one successful attack for more than half of the participants. Moreover, they illustrate statistically significant variation among vendors and, in one case, even gender bias. The outcomes are rather alarming and require the deployment of further countermeasures to prevent exploitation, as the number of VAs in use is currently comparable to the world population.
Ratio Shift Keying Modulation for Time-Varying Molecular Communication Channels
Venue: arXiv cs.ET 2023
Year: 2023

Molecular Communications (MC) is a bio-inspired communication technique that uses molecules to encode and transfer information. Many efforts have been devoted to developing novel modulation techniques for MC based on various distinguishable characteristics of molecules, such as their concentrations or types. In this paper, we investigate a particular modulation scheme called Ratio Shift Keying (RSK), where the information is encoded in the concentration ratio of two different types of molecules. RSK modulation is hypothesized to enable accurate information transfer in dynamic MC scenarios where the time-varying channel characteristics affect both types of molecules equally. To validate this hypothesis, we first conduct an information-theoretical analysis of RSK modulation and derive the capacity of the end-to-end MC channel where the receiver estimates concentration ratio based on ligand-receptor binding statistics in an optimal or suboptimal manner. We then analyze the error performance of RSK modulation in a practical time-varying MC scenario, that is mobile MC, in which both the transmitter and the receiver undergo diffusion-based propagation. Our numerical and analytical results, obtained for varying levels of similarity between the ligand types used for ratio-encoding, and varying number of receptors, show that RSK can significantly outperform the most commonly considered MC modulation technique, concentration shift keying (CSK), in dynamic MC scenarios.
Heterogeneous Social Event Detection via Hyperbolic Graph Representations
Venue: arXiv cs.SI 2023
Year: 2023

Social events reflect the dynamics of society and, here, natural disasters and emergencies receive significant attention. The timely detection of these events can provide organisations and individuals with valuable information to reduce or avoid losses. However, due to the complex heterogeneities of the content and structure of social media, existing models can only learn limited information; large amounts of semantic and structural information are ignored. In addition, due to high labour costs, it is rare for social media datasets to include high-quality labels, which also makes it challenging for models to learn information from social media. In this study, we propose two hyperbolic graph representation-based methods for detecting social events from heterogeneous social media environments. For cases where a dataset has labels, we designed a Hyperbolic Social Event Detection (HSED) model that converts complex social information into a unified social message graph. This model addresses the heterogeneity of social media, and, with this graph, the information in social media can be used to capture structural information based on the properties of hyperbolic space. For cases where the dataset is unlabelled, we designed an Unsupervised Hyperbolic Social Event Detection (UHSED). This model is based on the HSED model but includes graph contrastive learning to make it work in unlabelled scenarios. Extensive experiments demonstrate the superiority of the proposed approaches.
DrasCLR: A Self-supervised Framework of Learning Disease-related and Anatomy-specific Representation for 3D Medical Images
Venue: arXiv cs.CV 2023
Year: 2023

Large-scale volumetric medical images with annotation are rare, costly, and time prohibitive to acquire. Self-supervised learning (SSL) offers a promising pre-training and feature extraction solution for many downstream tasks, as it only uses unlabeled data. Recently, SSL methods based on instance discrimination have gained popularity in the medical imaging domain. However, SSL pre-trained encoders may use many clues in the image to discriminate an instance that are not necessarily disease-related. Moreover, pathological patterns are often subtle and heterogeneous, requiring the ability of the desired method to represent anatomy-specific features that are sensitive to abnormal changes in different body parts. In this work, we present a novel SSL framework, named DrasCLR, for 3D medical imaging to overcome these challenges. We propose two domain-specific contrastive learning strategies: one aims to capture subtle disease patterns inside a local anatomical region, and the other aims to represent severe disease patterns that span larger regions. We formulate the encoder using conditional hyper-parameterized network, in which the parameters are dependant on the anatomical location, to extract anatomically sensitive features. Extensive experiments on large-scale computer tomography (CT) datasets of lung images show that our method improves the performance of many downstream prediction and segmentation tasks. The patient-level representation improves the performance of the patient survival prediction task. We show how our method can detect emphysema subtypes via dense prediction. We demonstrate that fine-tuning the pre-trained model can significantly reduce annotation efforts without sacrificing emphysema detection accuracy. Our ablation study highlights the importance of incorporating anatomical context into the SSL framework.
Time to Embrace Natural Language Processing (NLP)-based Digital Pathology: Benchmarking NLP- and Convolutional Neural Network-based Deep Learning Pipelines
Venue: arXiv cs.CL 2023
Year: 2023

NLP-based computer vision models, particularly vision transformers, have been shown to outperform CNN models in many imaging tasks. However, most digital pathology artificial-intelligence models are based on CNN architectures, probably owing to a lack of data regarding NLP models for pathology images. In this study, we developed digital pathology pipelines to benchmark the five most recently proposed NLP models (vision transformer (ViT), Swin Transformer, MobileViT, CMT, and Sequencer2D) and four popular CNN models (ResNet18, ResNet50, MobileNetV2, and EfficientNet) to predict biomarkers in colorectal cancer (microsatellite instability, CpG island methylator phenotype, and BRAF mutation). Hematoxylin and eosin-stained whole-slide images from Molecular and Cellular Oncology and The Cancer Genome Atlas were used as training and external validation datasets, respectively. Cross-study external validations revealed that the NLP-based models significantly outperformed the CNN-based models in biomarker prediction tasks, improving the overall prediction and precision up to approximately 10% and 26%, respectively. Notably, compared with existing models in the current literature using large training datasets, our NLP models achieved state-of-the-art predictions for all three biomarkers using a relatively small training dataset, suggesting that large training datasets are not a prerequisite for NLP models or transformers, and NLP may be more suitable for clinical studies in which small training datasets are commonly collected. The superior performance of Sequencer2D suggests that further research and innovation on both transformer and bidirectional long short-term memory architectures are warranted in the field of digital pathology. NLP models can replace classic CNN architectures and become the new workhorse backbone in the field of digital pathology.
CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularization
Venue: arXiv cs.LG 2023
Year: 2023

Federated learning enables edge devices to train a global model collaboratively without exposing their data. Despite achieving outstanding advantages in computing efficiency and privacy protection, federated learning faces a significant challenge when dealing with non-IID data, i.e., data generated by clients that are typically not independent and identically distributed. In this paper, we tackle a new type of Non-IID data, called cluster-skewed non-IID, discovered in actual data sets. The cluster-skewed non-IID is a phenomenon in which clients can be grouped into clusters with similar data distributions. By performing an in-depth analysis of the behavior of a classification model's penultimate layer, we introduce a metric that quantifies the similarity between two clients' data distributions without violating their privacy. We then propose an aggregation scheme that guarantees equality between clusters. In addition, we offer a novel local training regularization based on the knowledge-distillation technique that reduces the overfitting problem at clients and dramatically boosts the training scheme's performance. We theoretically prove the superiority of the proposed aggregation over the benchmark FedAvg. Extensive experimental results on both standard public datasets and our in-house real-world dataset demonstrate that the proposed approach improves accuracy by up to 16% compared to the FedAvg algorithm.
HCGMNET: A Hierarchical Change Guiding Map Network For Change Detection
Venue: arXiv cs.CV 2023
Year: 2023

Very-high-resolution (VHR) remote sensing (RS) image change detection (CD) has been a challenging task for its very rich spatial information and sample imbalance problem. In this paper, we have proposed a hierarchical change guiding map network (HCGMNet) for change detection. The model uses hierarchical convolution operations to extract multiscale features, continuously merges multi-scale features layer by layer to improve the expression of global and local information, and guides the model to gradually refine edge features and comprehensive performance by a change guide module (CGM), which is a self-attention with changing guide map. Extensive experiments on two CD datasets show that the proposed HCGMNet architecture achieves better CD performance than existing state-of-the-art (SOTA) CD methods.
Crowd simulation incorporating a route choice model and similarity evaluation using real large-scale data
Venue: arXiv cs.MA 2023
Year: 2023

Modeling and simulation approaches that express crowd movement with mathematical models are widely and actively studied to understand crowd movement and resolve crowd accidents. Existing literature on crowd modeling focuses on only the decision-making of walking behavior. However, the decision-making of route choice, which is a higher-level decision, should also be modeled for constructing more practical simulations. Furthermore, the reproducibility evaluation of the crowd simulation incorporating the route choice model using real data is insufficient. Therefore, we generalize and propose a crowd simulation framework that includes actual crowd movement measurements, route choice model estimation, and crowd simulator construction. We use the Discrete choice model as the route choice model and the Social force model as the walking model. In experiments, we measure crowd movements during an evacuation drill in a theater and a firework event where tens of thousands of people moved and prove that the crowd simulation incorporating the route choice model can reproduce the real large-scale crowd movement more accurately.
Interval Type-2 Fuzzy Neural Networks for Multi-Label Classification
Venue: arXiv cs.LG 2023
Year: 2023

Prediction of multi-dimensional labels plays an important role in machine learning problems. We found that the classical binary labels could not reflect the contents and their relationships in an instance. Hence, we propose a multi-label classification model based on interval type-2 fuzzy logic. In the proposed model, we use a deep neural network to predict the type-1 fuzzy membership of an instance and another one to predict the fuzzifiers of the membership to generate interval type-2 fuzzy memberships. We also propose a loss function to measure the similarities between binary labels in datasets and interval type-2 fuzzy memberships generated by our model. The experiments validate that our approach outperforms baselines on multi-label classification benchmarks.
A note on the partition bound for one-way classical communication complexity
Venue: arXiv cs.CC 2023
Year: 2023

We present a linear program for the one-way version of the partition bound (denoted $\mathsf{prt}^1_\varepsilon(f)$). We show that it characterizes one-way randomized communication complexity $\mathsf{R}_\varepsilon^1(f)$ with shared randomness of every partial function $f:\mathcal{X}\times\mathcal{Y}\to\mathcal{Z}$, i.e., for $δ,\varepsilon\in(0,1/2)$, $\mathsf{R}_\varepsilon^1(f) \geq \log\mathsf{prt}_\varepsilon^1(f)$ and $\mathsf{R}_{\varepsilon+δ}^1(f) \leq \log\mathsf{prt}_\varepsilon^1(f) + \log\log(1/δ)$. This improves upon the characterization of $\mathsf{R}_\varepsilon^1(f)$ in terms of the rectangle bound (due to Jain and Klauck, 2010) by reducing the additive $O(\log(1/δ))$-term to $\log\log(1/δ)$.