Reviewerly/exHarmony huggingface dataset:
Leveraging Old Knowledge to Continually Learn New Classes in Medical Images

Class-incremental continual learning is a core step towards developing artificial intelligence systems that can continuously adapt to changes in the environment by learning new concepts without forgetting those previously learned. This is especially needed in the medical domain where continually learning from new incoming data is required to classify an expanded set of diseases. In this work, we focus on how old knowledge can be leveraged to learn new classes without catastrophic forgetting. We propose a framework that comprises of two main components: (1) a dynamic architecture with expanding representations to preserve previously learned features and accommodate new features; and (2) a training procedure alternating between two objectives to balance the learning of new features while maintaining the model’s performance on old classes. Experiment results on multiple medical datasets show that our solution is able to achieve superior performance over state-of-the-art baselines in terms of class accuracy and forgetting.
Similarity Distribution Based Membership Inference Attack on Person Re-identification

While person Re-identification (Re-ID) has progressed rapidly due to its wide real-world applications, it also causes severe risks of leaking personal information from training data. Thus, this paper focuses on quantifying this risk by membership inference (MI) attack. Most of the existing MI attack algorithms focus on classification models, while Re-ID follows a totally different training and inference paradigm. Re-ID is a fine-grained recognition task with complex feature embedding, and model outputs commonly used by existing MI like logits and losses are not accessible during inference. Since Re-ID focuses on modelling the relative relationship between image pairs instead of individual semantics, we conduct a formal and empirical analysis which validates that the distribution shift of the inter-sample similarity between training and test set is a critical criterion for Re-ID membership inference. As a result, we propose a novel membership inference attack method based on the inter-sample similarity distribution. Specifically, a set of anchor images are sampled to represent the similarity distribution conditioned on a target image, and a neural network with a novel anchor selection module is proposed to predict the membership of the target image. Our experiments validate the effectiveness of the proposed approach on both the Re-ID task and conventional classification task.
DisGUIDE: Disagreement-Guided Data-Free Model Extraction

Recent model-extraction attacks on Machine Learning as a Service (MLaaS) systems have moved towards data-free approaches, showing the feasibility of stealing models trained with difficult-to-access data. However, these attacks are ineffective or limited due to the low accuracy of extracted models and the high number of queries to the models under attack. The high query cost makes such techniques infeasible for online MLaaS systems that charge per query. We create a novel approach to get higher accuracy and query efficiency than prior data-free model extraction techniques. Specifically, we introduce a novel generator training scheme that maximizes the disagreement loss between two clone models that attempt to copy the model under attack. This loss, combined with diversity loss and experience replay, enables the generator to produce better instances to train the clone models. Our evaluation on popular datasets CIFAR-10 and CIFAR-100 shows that our approach improves the final model accuracy by up to 3.42% and 18.48% respectively. The average number of queries required to achieve the accuracy of the prior state of the art is reduced by up to 64.95%. We hope this will promote future work on feasible data-free model extraction and defenses against such attacks.
Entropy-based Client Selection Mechanism for Vehicular Federated Environments

Autonomous driving requires machine learning models to be trained at the edge for improved efficiency and reduced communication latency. Federated learning (FL) allows knowledge sharing among all devices, but Not Independent and Identically Distributed (non-IID) scenarios with biased device data distributions can lead to statistical heterogeneity and lower classification accuracy. This paper proposes an entropy-based client selection approach for vehicular federated learning environments that aims to address the challenges posed by non-IID data in vehicular networks. The proposed method is compared to a random selection mechanism in both IID and non-IID scenarios, as well as in a scenario with random client drops. The results show that the entropy-based selection method outperforms the random selection method in all compared metrics, particularly in non-IID scenarios.
Denoising Multi-Similarity Formulation: A Self-Paced Curriculum-Driven Approach for Robust Metric Learning

Deep Metric Learning (DML) is a group of techniques that aim to measure the similarity between objects through the neural network. Although the number of DML methods has rapidly increased in recent years, most previous studies cannot effectively handle noisy data, which commonly exists in practical applications and often leads to serious performance deterioration. To overcome this limitation, in this paper, we build a connection between noisy samples and hard samples in the framework of self-paced learning, and propose a Balanced Self-Paced Metric Learning (BSPML) algorithm with a denoising multi-similarity formulation, where noisy samples are treated as extremely hard samples and adaptively excluded from the model training by sample weighting. Especially, due to the pairwise relationship and a new balance regularization term, the sub-problem w.r.t. sample weights is a nonconvex quadratic function. To efficiently solve this nonconvex quadratic problem, we propose a doubly stochastic projection coordinate gradient algorithm. Importantly, we theoretically prove the convergence not only for the doubly stochastic projection coordinate gradient algorithm, but also for our BSPML algorithm. Experimental results on several standard data sets demonstrate that our BSPML algorithm has better generalization ability and robustness than the state-of-the-art robust DML approaches.
Concurrent Multi-Label Prediction in Event Streams

Streams of irregularly occurring events are commonly modeled as a marked temporal point process. Many real-world datasets such as e-commerce transactions and electronic health records often involve events where multiple event types co-occur, e.g. multiple items purchased or multiple diseases diagnosed simultaneously. In this paper, we tackle multi-label prediction in such a problem setting, and propose a novel Transformer-based Conditional Mixture of Bernoulli Network (TCMBN) that leverages neural density estimation to capture complex temporal dependence as well as probabilistic dependence between concurrent event types. We also propose potentially incorporating domain knowledge in the objective by regularizing the predicted probability. To represent probabilistic dependence of concurrent event types graphically, we design a two-step approach that first learns the mixture of Bernoulli network and then solves a least-squares semi-definite constrained program to numerically approximate the sparse precision matrix from a learned covariance matrix. This approach proves to be effective for event prediction while also providing an interpretable and possibly non-stationary structure for insights into event co-occurrence. We demonstrate the superior performance of our approach compared to existing baselines on multiple synthetic and real benchmarks.
“Nothing Abnormal”: Disambiguating Medical Reports via Contrastive Knowledge Infusion

Sharing medical reports is essential for patient-centered care. A recent line of work has focused on automatically generating reports with NLP methods. However, different audiences have different purposes when writing/reading medical reports – for example, healthcare professionals care more about pathology, whereas patients are more concerned with the diagnosis ("Is there any abnormality?"). The expectation gap results in a common situation where patients find their medical reports to be ambiguous and therefore unsure about the next steps. In this work, we explore the audience expectation gap in healthcare and summarize common ambiguities that lead patients to be confused about their diagnosis into three categories: medical jargon, contradictory findings, and misleading grammatical errors. Based on our analysis, we define a disambiguation rewriting task to regenerate an input to be unambiguous while preserving information about the original content. We further propose a rewriting algorithm based on contrastive pretraining and perturbation-based rewriting. In addition, we create two datasets, OpenI-Annotated based on chest reports and VA-Annotated based on general medical reports, with available binary labels for ambiguity and abnormality presence annotated by radiology specialists. Experimental results on these datasets show that our proposed algorithm effectively rewrites input sentences in a less ambiguous way with high content fidelity. Our code and annotated data will be released to facilitate future research.
Domain Adaptation with Adversarial Training on Penultimate Activations

Enhancing model prediction confidence on target data is an important objective in Unsupervised Domain Adaptation (UDA). In this paper, we explore adversarial training on penultimate activations, i.e., input features of the final linear classification layer. We show that this strategy is more efficient and better correlated with the objective of boosting prediction confidence than adversarial training on input images or intermediate features, as used in previous works. Furthermore, with activation normalization commonly used in domain adaptation to reduce domain gap, we derive two variants and systematically analyze the effects of normalization on our adversarial training. This is illustrated both in theory and through empirical analysis on real adaptation tasks. Extensive experiments are conducted on popular UDA benchmarks under both standard setting and source-data free setting. The results validate that our method achieves the best scores against previous arts. Code is available at https://github.com/tsun/APA.
Sharing Data With Shared Benefits: Artificial Intelligence Perspective

Artificial intelligence (AI) and data sharing go hand in hand. In order to develop powerful AI models for medical and health applications, data need to be collected and brought together over multiple centers. However, due to various reasons, including data privacy, not all data can be made publicly available or shared with other parties. Federated and swarm learning can help in these scenarios. However, in the private sector, such as between companies, the incentive is limited, as the resulting AI models would be available for all partners irrespective of their individual contribution, including the amount of data provided by each party. Here, we explore a potential solution to this challenge as a viewpoint, aiming to establish a fairer approach that encourages companies to engage in collaborative data analysis and AI modeling. Within the proposed approach, each individual participant could gain a model commensurate with their respective data contribution, ultimately leading to better diagnostic tools for all participants in a fair manner.
TgrApp: Anomaly Detection and Visualization of Large-Scale Call Graphs

Given a million-scale dataset of who-calls-whom data containing imperfect labels, how can we detect existing and new fraud patterns? We propose TgrApp, which extracts carefully designed features and provides visualizations to assist analysts in spotting fraudsters and suspicious behavior. Our TgrApp method has the following properties: (a) Scalable, as it is linear on the input size; and (b) Effective, as it allows natural interaction with human analysts, and is applicable in both supervised and unsupervised settings.
Towards Deployment-Efficient and Collision-Free Multi-Agent Path Finding (Student Abstract)

Multi-agent pathfinding (MAPF) is essential to large-scale robotic coordination tasks. Planning-based algorithms show their advantages in collision avoidance while avoiding exponential growth in the number of agents. Reinforcement-learning (RL)-based algorithms can be deployed efficiently but cannot prevent collisions entirely due to the lack of hard constraints. This paper combines the merits of planning-based and RL-based MAPF methods to propose a deployment-efficient and collision-free MAPF algorithm. The experiments show the effectiveness of our approach.
H-AES: Towards Automated Essay Scoring for Hindi

The use of Natural Language Processing (NLP) for Automated Essay Scoring (AES) has been well explored in the English language, with benchmark models exhibiting performance comparable to human scorers. However, AES in Hindi and other low-resource languages remains unexplored. In this study, we reproduce and compare state-of-the-art methods for AES in the Hindi domain. We employ classical feature-based Machine Learning (ML) and advanced end-to-end models, including LSTM Networks and Fine-Tuned Transformer Architecture, in our approach and derive results comparable to those in the English language domain. Hindi being a low-resource language, lacks a dedicated essay-scoring corpus. We train and evaluate our models using translated English essays and empirically measure their performance on our own small-scale, real-world Hindi corpus. We follow this up with an in-depth analysis discussing prompt-specific behavior of different language models implemented.
Resilient Binary Neural Network

Binary neural networks (BNNs) have received ever-increasing popularity for their great capability of reducing storage burden as well as quickening inference time. However, there is a severe performance drop compared with {real-valued} networks, due to its intrinsic frequent weight oscillation during training. In this paper, we introduce a Resilient Binary Neural Network (ReBNN) to mitigate the frequent oscillation for better BNNs' training. We identify that the weight oscillation mainly stems from the non-parametric scaling factor. To address this issue, we propose to parameterize the scaling factor and introduce a weighted reconstruction loss to build an adaptive training objective. For the first time, we show that the weight oscillation is controlled by the balanced parameter attached to the reconstruction loss, which provides a theoretical foundation to parameterize it in back propagation. Based on this, we learn our ReBNN by calculating the balanced parameter based on its maximum magnitude, which can effectively mitigate the weight oscillation with a resilient training process. Extensive experiments are conducted upon various network models, such as ResNet and Faster-RCNN for computer vision, as well as BERT for natural language processing. The results demonstrate the overwhelming performance of our ReBNN over prior arts. For example, our ReBNN achieves 66.9% Top-1 accuracy with ResNet-18 backbone on the ImageNet dataset, surpassing existing state-of-the-arts by a significant margin. Our code is open-sourced at https://github.com/SteveTsui/ReBNN.
Face recognition algorithm based on sample expansion and label embedding for multi-resolution dictionary learning

Face recognition, being a widely utilized and profoundly challenging task within the field of computer vision, has witnessed the emergence of numerous practical methodologies in recent years. Among these, dictionary learning-based methods have garnered extensive attention and have found widespread application. Nevertheless, it is worth noting that the majority of dictionary learning methods suffer from certain limitations. On the one hand, their primary focus lies solely on the resolution of the original images, thereby rendering them susceptible to the impact of resolution changes encountered in real-world scenarios. On the other hand, their ability to acquire a robust dictionary is hindered by the inadequacy of available training samples. We propose a multi-resolution dictionary learning method based on sample extension and label embedding. We generate virtual samples for the original training samples and convert all samples into different resolutions. In addition, noise constraints are applied to the virtual samples to enhance robustness. Then, we construct the label embedding term using the label information of the atoms and embed it on a multi-resolution dictionary learning model for training. The recognition rates of the method on the ORL, extended Yale B, CMU PIE, and GT datasets are 93.72%, 92.57%, 97.79%, and 73.91%, respectively.
Towards a Proper Evaluation of Automated Conversational Systems

Efficient evaluation of dialogue agents is a major problem in conversational AI, with current research still relying largely on human studies for method validation. Recently, there has been a trend toward the use of automatic self-play and bot-bot evaluation as an approximation for human ratings of conversational systems. Such methods promise to alleviate the time and financial costs associated with human evaluation, and current proposed methods show moderate to strong correlation with human judgements. In this study, we further investigate the fitness of end-to-end self-play and bot-bot interaction for dialogue system evaluation. Specifically, we perform a human study to confirm self-play evaluations of a recently proposed agent that implements a GPT-2 based response generator on the Persuasion For Good charity solicitation task. This agent leverages Progression Function (PF) models to predict the evolving acceptability of an ongoing dialogue and uses dialogue rollouts to proactively simulate how candidate responses may impact the future success of the conversation. The agent was evaluated in an automatic self-play setting, using automatic metrics to estimate sentiment and intent to donate in each simulated dialogue. This evaluation indicated that sentiment and intent to donate were higher (p &lt; 0.05) across dialogues involving the progression-aware agents with rollouts, compared to a baseline agent with no rollout-based planning mechanism. To validate the use of self-play in this setting, we follow up by conducting a human evaluation of this same agent on a range of factors including convincingness, aggression, competence, confidence, friendliness, and task utility on the same Persuasion For Good solicitation task. Results show that human users agree with previously reported automatic self-play results with respect to agent sentiment, specifically showing improvement in friendliness and confidence in the experimental condition; however, we also discover that for the same agent, humans reported a lower desire to use it in the future compared to the baseline. We perform a qualitative sentiment analysis of participant feedback to explore possible reasons for this, and discuss implications for self-play and bot-bot interaction as a general framework for evaluating conversational systems.
SDSS: Secure Data Sharing Scheme for Edge Enabled IoV Networks

With the large-scale deployment of the Internet of Vehicles (IoV) and 5G technologies, it is inevitable to share data frequently for superior in-vehicle services. However, due to the dynamically changing and widely distributed Vehicular Ad-hoc Networks (VANETs), data sharing still faces challenges in security, efficiency, and reliability. In this paper, we propose a secure and reliable data-sharing scheme (SDSS) for edge-enabled IoV networks. It assigns multiple attribute authorities to alleviate the management burden and support a large attribute universe catering to the various services in IoV. To enhance efficiency and flexibility, edge computing is introduced for quickly responding to vehicles’ requests and assisting resource-constrained vehicle computation. And an online/offline mechanism is designed to further alleviate the computational pressure of sharing data online. In addition, we put forward a cooperative key generation approach to guarantee the security of users’ private keys. The security analysis proves that SDSS ensures resistance to collusion attacks and indistinguishability under chosen-ciphertext attacks (IND-CCA). Moreover, it can avoid the single point of failure and resist denial of service (DoS) attacks with the help of multiple distributed edge nodes. The experiment demonstrates SDSS is practicable for IoV data sharing.
Augmenting VR/XR experiences using directional vibrotactile feedback and temperature variation using wearable devices

As virtual and mixed reality hardware systems become more mainstream, users are spending substantial amounts of time in simulated environments. Unlike the transition from desktop to mobile devices, VR/XR utilizes 360 wrap-around space which can be challenging to master even for experienced users. Tasks and tools commonly utilized in 2D environments within mobile and personal computing devices may not always be intuitive for VR space. For that reason, it is important to study and evaluate which common graphical user interface (GUI) techniques can be extended to VR/XR and how the efficiency of common 2D tools need to be improved within a 360-degree space. In this study authors explore six commonly used GUI tools and evaluate them in a VR environment. The research looks at how participants deconstruct 360-degree GUI tasks by identifying the location of the controls, navigating through the VR space to the relevant area and finally adjusting the GUI controls as instructed. The study looks at augmenting the interaction by providing vibrotactile navigation cues along with kinaesthetic and temperature-based feedback to complete the GUI tasks. Comparing to conventional visual only techniques that are currently being used in VR environments, vibrotactile, kinaesthetic and temperature feedback provided faster task completion times and more pleasant user experience. Participants also rated the additional feedback channels as more informative and less distracting within the virtual environment. Overall results show that participants preferred the novel use of haptic feedback for most of the GUI controls assessed within the study. Moreover, results also show that some more complex GUI controls (i.e., dial, menus, and lists) may not be best suited for VR 360-degree interaction, using visual only information channels, especially with non-robust inside-out hand tracking techniques. Additional research is needed to validate these results across different VR/XR hardware and simulated environments, however, current results point towards utilizing multi-modal and multi-technology interaction tools to create more immersive and intuitive 360 virtual spaces across a wide range of VR/XR devices.
X Thinking in the Experience Economy Era: 23 Personas that Identify Generation Z Interaction Qualities

In today's business landscape, enterprises are no longer relying solely on their understanding of products but are instead prioritizing user experiences. To comprehend and model the challenges presented by this shift, we conducted a sequence of literature reviews and six qualitative user studies. Through this research, we identified 23 archetypical personas and how they manifest in the experience economy era. Building upon these insights, we identified six crucial interaction qualities and intend to create design guidelines and demonstrators that facilitate Generation Z interactions in everyday life.