Reviewerly/exHarmony huggingface dataset:
Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability

We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable. Inspired by brains, BIMT embeds neurons in a geometric space and augments the loss function with a cost proportional to the length of each neuron connection. This is inspired by the idea of minimum connection cost in evolutionary biology, but we are the first the combine this idea with training neural networks with gradient descent for interpretability. We demonstrate that BIMT discovers useful modular neural networks for many simple tasks, revealing compositional structures in symbolic formulas, interpretable decision boundaries and features for classification, and mathematical structure in algorithmic datasets. Qualitatively, BIMT-trained networks have modules readily identifiable by the naked eye, but regularly trained networks seem much more complicated. Quantitatively, we use Newman's method to compute the modularity of network graphs; BIMT achieves the highest modularity for all our test problems. A promising and ambitious future direction is to apply the proposed method to understand large models for vision, language, and science.
Competitive Reinforcement Learning Agents with Adaptive Networks

The depth of a neural network's architecture is a crucial decision that must balance network performance and the computational resources required during training and inference. In the context of Reinforcement Learning (RL), this architectural choice can profoundly impact the policy (i.e. agent's behavior) learned by the network. Depending on the state of the agent, different policies learned by the network may improve the agent's performance, particularly for time-sensitive applications (e.g. real-time, low-latency scenarios) when considering the additional computational time needed to access the output of deeper networks. Therefore, this paper proposes Greater Use of Time (GUT), a method that involves training multiple networks of different lengths and allowing them to make decisions collab-oratively. If the shorter network is not confident enough, the longer network is relied on. For each network, the policy is learned through deep Q-learning, and the method's performance is evaluated in a competitive multi-agent environment. The results demonstrate that using multiple networks with different lengths not only reduces computational cost at inference time, but also yields significantly better performance than either the short or long network alone (p < 0.05). Importantly, the proposed use of confidence-based decision-making also significantly outperforms random decision-making (p < 0.05).
Deep Reinforcement Learning Based on Optical Neural Networks in Path Planning

We propose the optical deep Q network (ODQN) algorithm based on optical neural networks (ONNs) to accelerate calculation in 2D grid path planning task. The calculated results demonstrate that the innovative algorithm is competitive with the original deep Q network (DQN) algorithm based on artificial neural networks (ANNs). Further, we analyze the noise errors of ONNs and prove the relatively good robustness of the ODQN algorithm in the 2D grid world. Compared to previous works, we introduce an efficient learning strategy for ONNs and demonstrate its broad application prospects in reinforcement learning and so on.
A Survey on Bias Mitigation in Federated Learning

Federated learning (FL) enables collaborative model training while keeping data decentralized. However, system heterogeneity and statistical differences in decentralized data can introduce biases and unfairness. This paper surveys existing bias mitigation techniques in FL across various phases of the training process. We identify sources of bias and present a critical analysis of current fairness-aware FL algorithms, categorizing them as preventive (Pre-processing) or reactive (in-processing and Post-processing) based on when bias mitigation is applied. In addition, this paper reveals open challenges in balancing fairness and efficiency in FL, handling non-independent and identically distributed (non-IID) data, and ensuring privacy. This survey lays out the foundation for developing unbiased and privacy-preserving FL systems without discrimination in the future.
Machine Learning Based Techniques for Workload Prediction in Serverless Environments

This paper proposes a novel approach for predicting serverless workloads using machine learning techniques. The research highlights the importance of accurate workload prediction in serverless computing and provides valuable insights into the potential benefits of machine learning in this context. The proposed approach utilises machine learning models to analyse historical data on serverless workloads and various features related to system behaviour to identify patterns and trends and predict the expected resource requirements for future serverless workloads. The study includes multiple machine-learning techniques such as LSTM, ARIMA and VAR and time-series analysis to accurately predict workload patterns and resource demands to determine which is best suited for predicting serverless workloads. The performance evaluation uses real-world serverless workload data, demonstrating its effectiveness in accurately predicting resource needs and optimising resource allocation. This research's findings could improve the efficiency and cost-effectiveness of serverless computing by enabling proactive resource allocation and optimisation.
ML-based Digital Twin for anomaly detection: a case-study on Turtle soccer robots

In recent years, machine learning (ML) based digital twins (DTs) have seen widespread application in the anomaly detection domain. A search-based literature survey revealed that the majority of the case studies focus on large-scale systems (i.e., nuclear power plant, aerospace, and power grid) producing extensive data. Our work aims to investigate the performance of this technology in smaller-scale systems that generate less data. In this case study, we developed a ML-based DT of the mobility system, the omni wheels, of the Turtle soccer robots. The DT is capable of analyzing historical data collected from the physical robots and differentiating between damaged and undamaged wheels. Our experiments suggest that ML-based DT of small-scale systems is indeed capable of achieving relatively accurate results for anomaly detection use-cases.
Efficient preservation of detail features for image arbitrary style transfer

Some mainstream image arbitrary style transfer models still have limitations in maintaining the saliency information and details of content images, and the generated images often have issues such as content blur and distorted distortion. To solve the problems, the style transfer model proposed in the article can preserve the details of the content image effectively. Our model includes introducing a positional self-attention module which can enhance the representation of extracted image features and effectively capture remote relationships and coordinate global and local features. The feature fusion module guided by global information is proposed, which can high-quality fuse content features and style features. A detail preservation module is proposed to preserve details. In addition, we proposed a new loss function, which can preserve the global structure of images and eliminate artifacts. The experimental results demonstrate the effectiveness of the proposed image arbitrary style transfer model in achieving a balance between style and content. The model effectively preserves the complete semantic information and detailed features of the content images, resulting in stylized images that are visually more appealing.
Area-Efficient VLSI Architecture of Key Switching for BGV Fully Homomorphic Encryption

Fully Homomorphic Encryption (FHE) has gained much attention in recent years due to its ability to perform desired operations on encrypted data, thereby ensuring data security and privacy in cloud computing. However, the widespread adoption of FHE faces challenges arising from its extremely high computational complexity and large memory requirements. This work focuses on reducing the computational complexity of key switching, which is a crucial function in FHE. By employing a fully parallel architecture with pipelined processing elements, the usage of buffer memory can be significantly reduced for large point sizes. Additionally, an optimized key switching architecture with superior area efficiency is obtained by minimizing the number of required Number Theoretic Transfer (NTT/INTT) modules based on a given input throughput. Compared to related work, the proposed architecture can achieve up to 36.31% overall performance improvement. Moreover, the developed key switching architecture is designed to facilitate flexible parameter selection based on BGV-FHE parameters.
Analyzing the Experience of Participating in a Mentoring Program for Academic Alert College Students Through Text Network Analysis

The purpose of this study is to identify the effect of personal participation and interaction through activity sheets created in a course called &lt;i&gt;Mentoring Program for Graders&lt;/i&gt;. In doing so, this study provides educational implications by examining the experiences of all students who participated in the programs operated in the university through the visualization of text network analysis methods and network analysis. The target audience is a total of 86 teams and 176 participants in the mentoring program for merit credits at a four-year university in G City. The program consisted of a total of 5 sessions, and participants wrote an activity report after each session. After obtaining consent for data collection and use, the contents of the activity sheets and impressions were collected, and data analysis was carried out using MAXQDA. As a result of the study, 456 codes were coded in the mentee experience, and the main keywords according to the frequency of the codes were ‘improvement of learning attitude’ and ‘enhancement of self-efficacy’. In the mentor experience, there were 411 codes, and as a result of the coding, ‘growing by helping the mentee’ and ‘pace maker’ were derived as the main keywords. Finally, a total of 1,483 codes were coded in the interaction experience, and the main keywords were ‘companionship’ and ‘steady communication’. Finally, based on the results of the code system analysis of each experience, network analysis was carried out to visually confirm Code map, and the structural analysis according to the cluster was presented. Based on these results, we proposed a plan to increase the effectiveness of the mentoring program for graders.
A Study on the Value Manifestations of Red Archives

Red archives constitute a significant part of archival resources, bearing unique significance for information retention and emotional transmission. In various social practices, red archives provide crucial resource support and exhibit diversified forms of value. In diverse social utilization activities, red archives demonstrate evidential value in testifying history, educational value in reinforcing social education, and emotional value in carrying forward the spirit of predecessors. The realization of the value of archives depends on the needs of the subjects in the process of social practice activities. We need to carry out as many diversified social practice activities as possible, and on this basis, we can maximize the value of the red archive, so that its value can be maximized.
Generating Instance-level Prompts for Rehearsal-free Continual Learning

We introduce Domain-Adaptive Prompt (DAP), a novel method for continual learning using Vision Transformers (ViT). Prompt-based continual learning has recently gained attention due to its rehearsal-free nature. Currently, the prompt pool, which is suggested by prompt-based continual learning, is key to effectively exploiting the frozen pretrained ViT backbone in a sequence of tasks. However, we observe that the use of a prompt pool creates a domain scalability problem between pre-training and continual learning. This problem arises due to the inherent encoding of group-level instructions within the prompt pool. To address this problem, we propose DAP, a pool-free approach that generates a suitable prompt in an instance-level manner at inference time. We optimize an adaptive prompt generator that creates instance-specific fine-grained instructions required for each input, enabling enhanced model plasticity and reduced forgetting. Our experiments on seven datasets with varying degrees of domain similarity to ImageNet demonstrate the superiority of DAP over state-of-the-art prompt-based methods. Code is publicly available at https://github.com/naver-ai/dap-cl.
PGFed: Personalize Each Client’s Global Objective for Federated Learning

Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client's empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N2)) communication overhead and potential privacy leakage while achieving this, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients' empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods. The code is publicly available at https://github.com/ljaiverson/pgfed.
Towards Realistic Evaluation of Industrial Continual Learning Scenarios with an Emphasis on Energy Consumption and Computational Footprint

Incremental Learning (IL) aims to develop Machine Learning (ML) models that can learn from continuous streams of data and mitigate catastrophic forgetting. We analyse the current state-of-the-art Class-IL implementations and demonstrate why the current body of research tends to be one-dimensional, with an excessive focus on accuracy metrics. A realistic evaluation of Continual Learning methods should also emphasise energy consumption and overall computational load for a comprehensive understanding. This paper addresses research gaps between current IL research and industrial project environments, including varying incremental tasks and the introduction of Joint Training in tandem with IL. We introduce InVar100 (Industrial Objects in Varied Contexts), a novel dataset meant to simulate the visual environments in industrial setups and perform various experiments for IL. Additionally, we incorporate explainability (using class activations) to interpret the model predictions. Our approach, RECIL (Real-World Scenarios and Energy Efficiency Considerations for Class Incremental Learning) provides meaningful insights about the applicability of IL approaches in practical use cases. The overarching aim is to bring the Incremental Learning and Green AI fields together and encourage the application of CIL methods in real-world scenarios. Code and dataset are available.
Gramian Attention Heads are Strong yet Efficient Vision Learners

We introduce a novel architecture design that enhances expressiveness by incorporating multiple head classifiers (i.e., classification heads) instead of relying on channel expansion or additional building blocks. Our approach employs attention-based aggregation, utilizing pairwise feature similarity to enhance multiple lightweight heads with minimal resource overhead. We compute the Gramian matrices to reinforce class tokens in an attention layer for each head. This enables the heads to learn more discriminative representations, enhancing their aggregation capabilities. Furthermore, we propose a learning algorithm that encourages heads to complement each other by reducing correlation for aggregation. Our models eventually surpass state-of-the-art CNNs and ViTs regarding the accuracy-throughput trade-off on ImageNet-1K and deliver remarkable performance across various downstream tasks, such as COCO object instance segmentation, ADE20k semantic segmentation, and fine-grained visual classification datasets. The effectiveness of our framework is substantiated by practical experimental results and further underpinned by generalization error bound. We release the code publicly at: https://github.com/Lab-LVM/imagenet-models.
LVOS: A Benchmark for Long-term Video Object Segmentation

Existing video object segmentation (VOS) benchmarks focus on short-term videos which just last about 3-5 seconds and where objects are visible most of the time. These videos are poorly representative of practical applications, and the absence of long-term datasets restricts further investigation of VOS on the application in realistic scenarios. So, in this paper, we present a new benchmark dataset named LVOS, which consists of 220 videos with a total duration of 421 minutes. To the best of our knowledge, LVOS is the first densely annotated long-term VOS dataset. The videos in our LVOS last 1.59 minutes on average, which is 20 times longer than videos in existing VOS datasets. Each video includes various attributes, especially challenges deriving from the wild, such as long-term reappearing and cross-temporal similar objeccts. Based on LVOS, we assess existing video object segmentation algorithms and propose a Diverse Dynamic Memory network (DDMemory) that consists of three complementary memory banks to exploit temporal information adequately. The experimental results demonstrate the strength and weaknesses of prior methods, pointing promising directions for further study. Data and code are available at https://lingyihongfd.github.io/lvos.github.io/.
Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis

Conditional generative models typically demand large annotated training sets to achieve high-quality synthesis. As a result, there has been significant interest in designing models that perform plug-and-play generation, i.e., to use a predefined or pretrained model, which is not explicitly trained on the generative task, to guide the generative process (e.g., using language). However, such guidance is typically useful only towards synthesizing high-level semantics rather than editing fine-grained details as in image-to-image translation tasks. To this end, and capitalizing on the powerful fine-grained generative control offered by the recent diffusion-based generative models, we introduce Steered Diffusion, a generalized framework for photorealistic zero-shot conditional image generation using a diffusion model trained for unconditional generation. The key idea is to steer the image generation of the diffusion model at inference time via designing a loss using a pre-trained inverse model that characterizes the conditional task. This loss modulates the sampling trajectory of the diffusion process. Our framework allows for easy incorporation of multiple conditions during inference. We present experiments using steered diffusion on several tasks including inpainting, colorization, text-guided semantic editing, and image super-resolution. Our results demonstrate clear qualitative and quantitative improvements over state-of-the-art diffusion-based plug-and-play models while adding negligible additional computational cost.
Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning

Compositional zero-shot learning (CZSL) aims to recognize unseen compositions with prior knowledge of known primitives (attribute and object). Previous works for CZSL often suffer from grasping the contextuality between attribute and object, as well as the discriminability of visual features, and the long-tailed distribution of real-world compositional data. We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. CoT employs object and attribute experts in distinctive manners to generate representative embeddings, using the visual network hierarchically. The object expert extracts representative object embeddings from the final layer in a bottom-up manner, while the attribute expert makes attribute embeddings in a top-down manner with a proposed object-guided attention module that models contextuality explicitly. To remedy biased prediction caused by imbalanced data distribution, we develop a simple minority attribute augmentation (MAA) that synthesizes virtual samples by mixing two images and oversampling minority attribute classes. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL. We also demonstrate the effectiveness of CoT in improving visual discrimination and addressing the model bias from the imbalanced data distribution. The code is available at https://github.com/HanjaeKim98/CoT.
EGC: Image Generation and Classification via a Diffusion Energy-Based Model

Learning image classification and image generation using the same set of network parameters presents a formidable challenge. Recent advanced approaches perform well in one task often exhibit poor performance in the other. This work introduces an energy-based classifier and generator, namely EGC, which can achieve superior performance in both tasks using a single neural network. Unlike conventional classifiers that produce a label given an image (i.e., a conditional distribution p(y|x)), the forward pass in EGC is a classification model that yields a joint distribution p(x,y), enabling a diffusion model in its backward pass by marginalizing out the label y to estimate the score function. Furthermore, EGC can be adapted for unsupervised learning by considering the label as latent variables. EGC achieves competitive generation results compared with state-of-the-art approaches on ImageNet-1k, CelebA-HQ and LSUN Church, while achieving superior classification accuracy and robustness against adversarial attacks on CIFAR-10. This work marks the inaugural success in mastering both domains using a unified network parameter set. We believe that EGC bridges the gap between discriminative and generative learning. Code will be released at https://github.com/GuoQiushan/EGC.