OpenRaiser/Intern-Atlas huggingface dataset:
Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay
Venue: arXiv cs.CV 2023
Year: 2023

Endoscopy is a widely used technique for the early detection of diseases or robotic-assisted minimally invasive surgery (RMIS). Numerous deep learning (DL)-based research works have been developed for automated diagnosis or processing of endoscopic view. However, existing DL models may suffer from catastrophic forgetting. When new target classes are introduced over time or cross institutions, the performance of old classes may suffer severe degradation. More seriously, data privacy and storage issues may lead to the unavailability of old data when updating the model. Therefore, it is necessary to develop a continual learning (CL) methodology to solve the problem of catastrophic forgetting in endoscopic image segmentation. To tackle this, we propose a Endoscopy Continual Semantic Segmentation (EndoCSS) framework that does not involve the storage and privacy issues of exemplar data. The framework includes a mini-batch pseudo-replay (MB-PR) mechanism and a self-adaptive noisy cross-entropy (SAN-CE) loss. The MB-PR strategy circumvents privacy and storage issues by generating pseudo-replay images through a generative model. Meanwhile, the MB-PR strategy can also correct the model deviation to the replay data and current training data, which is aroused by the significant difference in the amount of current and replay images. Therefore, the model can perform effective representation learning on both new and old tasks. SAN-CE loss can help model fitting by adjusting the model's output logits, and also improve the robustness of training. Extensive continual semantic segmentation (CSS) experiments on public datasets demonstrate that our method can robustly and effectively address the catastrophic forgetting brought by class increment in endoscopy scenes. The results show that our framework holds excellent potential for real-world deployment in a streaming learning manner.
Only Encode Once: Making Content-based News Recommender Greener
Venue: arXiv cs.IR 2023
Year: 2023

Large pretrained language models (PLM) have become de facto news encoders in modern news recommender systems, due to their strong ability in comprehending textual content. These huge Transformer-based architectures, when finetuned on recommendation tasks, can greatly improve news recommendation performance. However, the PLM-based pretrain-finetune framework incurs high computational cost and energy consumption, primarily due to the extensive redundant processing of news encoding during each training epoch. In this paper, we propose the ``Only Encode Once'' framework for news recommendation (OLEO), by decoupling news representation learning from downstream recommendation task learning. The decoupled design makes content-based news recommender as green and efficient as id-based ones, leading to great reduction in computational cost and training resources. Extensive experiments show that our OLEO framework can reduce carbon emissions by up to 13 times compared with the state-of-the-art pretrain-finetune framework and maintain a competitive or even superior performance level. The source code is released for reproducibility.
Explaining with Attribute-based and Relational Near Misses: An Interpretable Approach to Distinguishing Facial Expressions of Pain and Disgust
Venue: arXiv cs.AI 2023
Year: 2023

Explaining concepts by contrasting examples is an efficient and convenient way of giving insights into the reasons behind a classification decision. This is of particular interest in decision-critical domains, such as medical diagnostics. One particular challenging use case is to distinguish facial expressions of pain and other states, such as disgust, due to high similarity of manifestation. In this paper, we present an approach for generating contrastive explanations to explain facial expressions of pain and disgust shown in video sequences. We implement and compare two approaches for contrastive explanation generation. The first approach explains a specific pain instance in contrast to the most similar disgust instance(s) based on the occurrence of facial expressions (attributes). The second approach takes into account which temporal relations hold between intervals of facial expressions within a sequence (relations). The input to our explanation generation approach is the output of an interpretable rule-based classifier for pain and disgust.We utilize two different similarity metrics to determine near misses and far misses as contrasting instances. Our results show that near miss explanations are shorter than far miss explanations, independent from the applied similarity metric. The outcome of our evaluation indicates that pain and disgust can be distinguished with the help of temporal relations. We currently plan experiments to evaluate how the explanations help in teaching concepts and how they could be enhanced by further modalities and interaction.
Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrations
Venue: arXiv cs.CL 2023
Year: 2023

The language ability of Large Language Models (LLMs) is often unbalanced towards English because of the imbalance in the distribution of the pre-training data. This disparity is demanded in further fine-tuning and affecting the cross-lingual abilities of LLMs. In this paper, we propose to empower Instructiontuned LLMs (It-LLMs) in languages other than English by building semantic alignment between them. Hence, we propose CrossAlpaca, an It-LLM with cross-lingual instruction-following and Translation-following demonstrations to improve semantic alignment between languages. We validate our approach on the multilingual Question Answering (QA) benchmarks XQUAD and MLQA and adapted versions of MMLU and BBH. Our models, tested over six different languages, outperform the It-LLMs tuned on monolingual data. The final results show that instruction tuning on non-English data is not enough and that semantic alignment can be further improved by Translation-following demonstrations.
Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy
Venue: arXiv cs.CV 2023
Year: 2023

Diabetic Retinopathy (DR), a leading cause of vision impairment, requires early detection and treatment. Developing robust AI models for DR classification holds substantial potential, but a key challenge is ensuring their generalization in unfamiliar domains with varying data distributions. To address this, our paper investigates cross-domain generalization, also known as domain generalization (DG), within the context of DR classification. DG, a challenging problem in the medical domain, is complicated by the difficulty of gathering labeled data across different domains, such as patient demographics and disease stages. Some recent studies have shown the effectiveness of using CLIP to handle the DG problem in natural images. In this study, we investigate CLIP's transfer learning capabilities and its potential for cross-domain generalization in diabetic retinopathy (DR) classification. We carry out comprehensive experiments to assess the efficacy and potential of CLIP in addressing DG for DR classification. Further, we introduce a multi-modal fine-tuning strategy named Context Optimization with Learnable Visual Tokens (CoOpLVT), which enhances context optimization by conditioning on visual features. Our findings demonstrate that the proposed method increases the F1-score by 1.8% over the baseline, thus underlining its promise for effective DG in DR classification. Our code is publicly available at https://github.com/Sanoojan/CLIP-DRDG.
Partially Constrained GRAND of Linear Block Codes
Venue: arXiv cs.IT 2023
Year: 2023

This paper is concerned with a search-number-reduced guessing random additive noise decoding (GRAND) algorithm for linear block codes, called partially constrained GRAND (PC-GRAND). In contrast to the original GRAND, which guesses error patterns without constraints, the PC-GRAND guesses only those error patterns satisfying partial constraints of the codes. In particular, the PC-GRAND takes partial rows of the parity-check matrix as constraints for generating candidate error patterns and the remaining rows as checks for validating the candidates. The number of searches can be reduced when the serial list Viterbi algorithm (SLVA) is implemented for searching over a trellis specified by the partial parity-check matrix. This is confirmed by numerical results. Numerical simulations are also provided for comparison with other decoding algorithms.
Reinforcement Strategies in General Lotto Games
Venue: arXiv cs.GT 2023
Year: 2023

Strategic decisions are often made over multiple periods of time, wherein decisions made earlier impact a competitor's success in later stages. In this paper, we study these dynamics in General Lotto games, a class of models describing the competitive allocation of resources between two opposing players. We propose a two-stage formulation where one of the players has reserved resources that can be strategically pre-allocated across the battlefields in the first stage of the game as reinforcements. The players then simultaneously allocate their remaining real-time resources, which can be randomized, in a decisive final stage. Our main contributions provide complete characterizations of the optimal reinforcement strategies and resulting equilibrium payoffs in these multi-stage General Lotto games. Interestingly, we determine that real-time resources are at least twice as effective as reinforcement resources when considering equilibrium payoffs.
How much Training is Needed in Downlink Cell-Free mMIMO under LoS/NLoS channels?
Venue: arXiv cs.IT 2023
Year: 2023

The assumption that no LoS channels exist between wireless access points~(APs) and user equipments~(UEs) becomes questionable in the context of the recent developments in the direction of cell free massive multiple input multiple output MIMO~(CF-mMIMO) systems. In CF-mMIMO systems, the access point density is assumed to be comparable to, or much larger than the the user density, thereby leading to the possibility of existence of LoS links between the UEs and the APs, depending on the local propagation conditions. In this paper, we compare the rates achievable by CF-mMIMO systems under probabilistic LoS/ NLos channels, with and without acquiring the channel state information~(CSI) of the fast fading components. We show that, under sufficiently large AP densities, statistical beamforming that does not require the knowledge about the fast fading components of the channels, performs almost at par with full beamforming, utilizing the information about the fast fading channel coefficients, thus potentially avoiding the need for training during every frame. We validate our results via detailed Monte Carlo simulations, and also elaborate the conditions under which statistical beamforming can be successfully employed in massive MIMO systems with LoS/ NLoS channels.
CPFES: Physical Fitness Evaluation Based on Canadian Agility and Movement Skill Assessment
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, the assessment of fundamental movement skills integrated with physical education has focused on both teaching practice and the feasibility of assessment. The object of assessment has shifted from multiple ages to subdivided ages, while the content of assessment has changed from complex and time-consuming to concise and efficient. Therefore, we apply deep learning to physical fitness evaluation, we propose a system based on the Canadian Agility and Movement Skill Assessment (CAMSA) Physical Fitness Evaluation System (CPFES), which evaluates children's physical fitness based on CAMSA, and gives recommendations based on the scores obtained by CPFES to help children grow. We have designed a landmark detection module and a pose estimation module, and we have also designed a pose evaluation module for the CAMSA criteria that can effectively evaluate the actions of the child being tested. Our experimental results demonstrate the high accuracy of the proposed system.
Generalized Deterministic-Random Tradeoff in Integrated Sensing and Communications: The Sensing-Optimal Operating Point
Venue: arXiv cs.IT 2023
Year: 2023

Integrated sensing and communications (ISAC) has been recognized as a key component in the envisioned 6G communication systems. Understanding the fundamental performance tradeoff between sensing and communication functionalities is essential for designing practical cost-efficient ISAC systems. In this paper, we aim for augmenting the current understanding of the deterministic-random tradeoff (DRT) between sensing and communication, by analyzing the sensing-optimal operating point of the fundamental capacity-distortion region. We show that the DRT exists for generic sensing performance metrics that are in general not convex/concave in the ISAC waveform. Especially, we elaborate on a representative non-convex performance metric, namely the detection probability for target detection tasks.
Target-independent XLA optimization using Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

An important challenge in Machine Learning compilers like XLA is multi-pass optimization and analysis. There has been recent interest chiefly in XLA target-dependent optimization on the graph-level, subgraph-level, and kernel-level phases. We specifically focus on target-independent optimization XLA HLO pass ordering: our approach aims at finding the optimal sequence of compiler optimization passes, which is decoupled from target-dependent optimization. However, there is little domain specific study in pass ordering for XLA HLO. To this end, we propose introducing deep Reinforcement Learning (RL) based search for optimal XLA HLO pass ordering. We also propose enhancements to the deep RL algorithms to further improve optimal search performance and open the research direction for domain-specific guidance for RL. We create an XLA Gym experimentation framework as a tool to enable RL algorithms to interact with the compiler for passing optimizations and thereby train agents. Overall, in our experimentation we observe an average of $13.3\%$ improvement in operation count reduction on a benchmark of GPT-2 training graphs and $10.4\%$ improvement on a diverse benchmark including GPT-2, BERT, and ResNet graphs using the proposed approach over the compiler's default phase ordering.
Evaluation of Non-Fungible Token (NFT)
Venue: arXiv cs.CR 2023
Year: 2023

The derivative of token standard of Ethereum blockchain, termed as Non Fungible Token is distinguishable token. These tokens are bound with digital properties that provide them unique identification which helps in fulfilling the aim of distinguishable tokens. These tokens are used as an evidence of ownership for the digital asset, with which they are bound to. And it is with these non fungible tokens that the problem of proving ownership of digital asset is being solved and with this technique, it is with hope that developers are looking forward to solve many more problems of the real world with it, may it be providing tradability solutions for arts, real estate and many other sectors. During the time of writing this, the NFT has shown unpredictable growth in the recent years and this has caused the stimulation of prosperity of DApps(Decentralized Application).With an unpredictable growth and garnering attention worldwide with many mainstream key people investing in it , the NFT is still in developing stage and is still premature. This paper is an attempt to squeeze the NFT developments systematically, so the aspiring developers can have the resource to start with and aid the development process further
UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localization
Venue: arXiv cs.MM 2023
Year: 2023

The emergence of artificial intelligence-generated content (AIGC) has raised concerns about the authenticity of multimedia content in various fields. However, existing research for forgery content detection has focused mainly on binary classification tasks of complete videos, which has limited applicability in industrial settings. To address this gap, we propose UMMAFormer, a novel universal transformer framework for temporal forgery localization (TFL) that predicts forgery segments with multimodal adaptation. Our approach introduces a Temporal Feature Abnormal Attention (TFAA) module based on temporal feature reconstruction to enhance the detection of temporal differences. We also design a Parallel Cross-Attention Feature Pyramid Network (PCA-FPN) to optimize the Feature Pyramid Network (FPN) for subtle feature enhancement. To evaluate the proposed method, we contribute a novel Temporal Video Inpainting Localization (TVIL) dataset specifically tailored for video inpainting scenes. Our experiments show that our approach achieves state-of-the-art performance on benchmark datasets, including Lav-DF, TVIL, and Psynd, significantly outperforming previous methods. The code and data are available at https://github.com/ymhzyj/UMMAFormer/.
Complex coalitions: political alliances across relational contexts
Venue: arXiv cs.SI 2023
Year: 2023

Coalitions are central to politics, including government formation, international relations, and public policy. Coalitions emerge when actors engage one another across multiple relational contexts, but existing literature often approaches coalitions in singular contexts. We introduce complex coalitions, a theoretical-methodological framework that emphasises the relevance of multiple contexts and cross-context dependencies in coalition politics. We also implement tools to statistically infer such coalition structures using multilayer networks. To demonstrate the usefulness of our approach, we compare coalitions among Finnish organisations engaging in climate politics across three con-texts: resource coordination, legacy media discourse, and social media communication. We show that considering coalitions as complex and accounting for cross-context dependencies improves the empirical validity of coalition studies. In our case study, the three contexts represent complementary, but not congruent, channels for enacting coalitions. In conclusion, we argue that the complex coalitions approach is useful for advancing understanding of coalitions in different political realms.
Closeness of Some Graph Operations
Venue: arXiv cs.DM 2023
Year: 2023

Closeness is an important measure of network centrality. In this article we will calculate the closeness of graphs, created by using operations on graphs. We will prove a formula for the closeness of shadow graphs. We will calculate the closeness of line graphs of some wellknown graphs (like path, star, cycle, and complete graphs) and the closeness of line graphs of two of these graphs, connected by a bridge (like lollipop, tadpole, broom, and bistar graphs).
Channel Charting in Real-World Coordinates
Venue: arXiv cs.IT 2023
Year: 2023

Channel charting is an emerging self-supervised method that maps channel state information (CSI) to a low-dimensional latent space, which represents pseudo-positions of user equipments (UEs). While this latent space preserves local geometry, i.e., nearby UEs are nearby in latent space, the pseudo-positions are in arbitrary coordinates and global geometry is not preserved. In order to enable channel charting in real-world coordinates, we propose a novel bilateration loss for multipoint wireless systems in which only the access point (AP) locations are known--no geometrical models or ground-truth UE position information is required. The idea behind this bilateration loss is to compare the received power at pairs of APs in order to determine whether a UE should be placed closer to one AP or the other in latent space. We demonstrate the efficacy of our method using channel vectors from a commercial ray-tracer.
Prediction of Tourism Flow with Sparse Geolocation Data
Venue: arXiv cs.LG 2023
Year: 2023

Modern tourism in the 21st century is facing numerous challenges. Among these the rapidly growing number of tourists visiting space-limited regions like historical cities, museums and bottlenecks such as bridges is one of the biggest. In this context, a proper and accurate prediction of tourism volume and tourism flow within a certain area is important and critical for visitor management tasks such as sustainable treatment of the environment and prevention of overcrowding. Static flow control methods like conventional low-level controllers or limiting access to overcrowded venues could not solve the problem yet. In this paper, we empirically evaluate the performance of state-of-the-art deep-learning methods such as RNNs, GNNs, and Transformers as well as the classic statistical ARIMA method. Granular limited data supplied by a tourism region is extended by exogenous data such as geolocation trajectories of individual tourists, weather and holidays. In the field of visitor flow prediction with sparse data, we are thereby capable of increasing the accuracy of our predictions, incorporating modern input feature handling as well as mapping geolocation data on top of discrete POI data.
Face Presentation Attack Detection by Excavating Causal Clues and Adapting Embedding Statistics
Venue: arXiv cs.CV 2023
Year: 2023

Recent face presentation attack detection (PAD) leverages domain adaptation (DA) and domain generalization (DG) techniques to address performance degradation on unknown domains. However, DA-based PAD methods require access to unlabeled target data, while most DG-based PAD solutions rely on a priori, i.e., known domain labels. Moreover, most DA-/DG-based methods are computationally intensive, demanding complex model architectures and/or multi-stage training processes. This paper proposes to model face PAD as a compound DG task from a causal perspective, linking it to model optimization. We excavate the causal factors hidden in the high-level representation via counterfactual intervention. Moreover, we introduce a class-guided MixStyle to enrich feature-level data distribution within classes instead of focusing on domain information. Both class-guided MixStyle and counterfactual intervention components introduce no extra trainable parameters and negligible computational resources. Extensive cross-dataset and analytic experiments demonstrate the effectiveness and efficiency of our method compared to state-of-the-art PADs. The implementation and the trained weights are publicly available.
Quantitative Data Analysis: CRASAR Small Unmanned Aerial Systems at Hurricane Ian
Venue: arXiv cs.RO 2023
Year: 2023

This paper provides a summary of the 281 sorties that were flown by the 10 different models of small unmanned aerial systems (sUAS) at Hurricane Ian, and the failures made in the field. These 281 sorties, supporting 44 missions, represents the largest use of sUAS in a disaster to date (previously Hurricane Florence with 260 sorties). The sUAS operations at Hurricane Ian differ slightly from prior operations as they included the first documented uses of drones performing interior search for victims, and the first use of a VTOL fixed wing aircraft during a large scale disaster. However, there are substantive similarities to prior drone operations. Most notably, rotorcraft continue to perform the vast majority of flights, wireless data transmission capacity continues to be a limitation, and the lack of centralized control for unmanned and manned aerial systems continues to cause operational friction. This work continues by documenting the failures, both human and technological made in the field and concludes with a discussion summarizing potential areas for further work to improve sUAS response to large scale disasters.