OpenRaiser/Intern-Atlas huggingface dataset:
Self-supervision for medical image classification: state-of-the-art performance with ~100 labeled training samples per class
Venue: arXiv cs.CV 2023
Year: 2023

Is self-supervised deep learning (DL) for medical image analysis already a serious alternative to the de facto standard of end-to-end trained supervised DL? We tackle this question for medical image classification, with a particular focus on one of the currently most limiting factors of the field: the (non-)availability of labeled data. Based on three common medical imaging modalities (bone marrow microscopy, gastrointestinal endoscopy, dermoscopy) and publicly available data sets, we analyze the performance of self-supervised DL within the self-distillation with no labels (DINO) framework. After learning an image representation without use of image labels, conventional machine learning classifiers are applied. The classifiers are fit using a systematically varied number of labeled data (1-1000 samples per class). Exploiting the learned image representation, we achieve state-of-the-art classification performance for all three imaging modalities and data sets with only a fraction of between 1% and 10% of the available labeled data and about 100 labeled samples per class.
Wearable multi-color RAPD screening device
Venue: arXiv cs.CV 2023
Year: 2023

In this work, we developed a wearable, head-mounted device that automatically calculates the precise Relative Afferent Pupillary Defect (RAPD) value of a patient. The device consists of two RGB LEDs, two infrared cameras, and one microcontroller. In the RAPD test, the parameters like LED on-off durations, brightness level, and color of the light can be controlled by the user. Upon data acquisition, a computational unit processes the data, calculates the RAPD score and visualizes the test results with a user-friendly interface.Multiprocessing methods used on GUI to optimize the processing pipeline. We have shown that our head-worn instrument is easy to use, fast, and suitable for early-diagnostics and screening purposes for various neurological conditions such as RAPD, glaucoma, asymmetric glaucoma, and anisocoria.
Open Set Classification of GAN-based Image Manipulations via a ViT-based Hybrid Architecture
Venue: arXiv cs.CV 2023
Year: 2023

Classification of AI-manipulated content is receiving great attention, for distinguishing different types of manipulations. Most of the methods developed so far fail in the open-set scenario, that is when the algorithm used for the manipulation is not represented by the training set. In this paper, we focus on the classification of synthetic face generation and manipulation in open-set scenarios, and propose a method for classification with a rejection option. The proposed method combines the use of Vision Transformers (ViT) with a hybrid approach for simultaneous classification and localization. Feature map correlation is exploited by the ViT module, while a localization branch is employed as an attention mechanism to force the model to learn per-class discriminative features associated with the forgery when the manipulation is performed locally in the image. Rejection is performed by considering several strategies and analyzing the model output layers. The effectiveness of the proposed method is assessed for the task of classification of facial attribute editing and GAN attribution.
Lady and the Tramp Nextdoor: Online Manifestations of Economic Inequalities in the Nextdoor Social Network
Venue: arXiv cs.SI 2023
Year: 2023

From health to education, income impacts a huge range of life choices. Earlier research has leveraged data from online social networks to study precisely this impact. In this paper, we ask the opposite question: do different levels of income result in different online behaviors? We demonstrate it does. We present the first large-scale study of Nextdoor, a popular location-based social network. We collect 2.6 Million posts from 64,283 neighborhoods in the United States and 3,325 neighborhoods in the United Kingdom, to examine whether online discourse reflects the income and income inequality of a neighborhood. We show that posts from neighborhoods with different incomes indeed differ, e.g. richer neighborhoods have a more positive sentiment and discuss crimes more, even though their actual crime rates are much lower. We then show that user-generated content can predict both income and inequality. We train multiple machine learning models and predict both income (R-squared=0.841) and inequality (R-squared=0.77).
Automaton-Guided Curriculum Generation for Reinforcement Learning Agents
Venue: arXiv cs.AI 2023
Year: 2023

Despite advances in Reinforcement Learning, many sequential decision making tasks remain prohibitively expensive and impractical to learn. Recently, approaches that automatically generate reward functions from logical task specifications have been proposed to mitigate this issue; however, they scale poorly on long-horizon tasks (i.e., tasks where the agent needs to perform a series of correct actions to reach the goal state, considering future transitions while choosing an action). Employing a curriculum (a sequence of increasingly complex tasks) further improves the learning speed of the agent by sequencing intermediate tasks suited to the learning capacity of the agent. However, generating curricula from the logical specification still remains an unsolved problem. To this end, we propose AGCL, Automaton-guided Curriculum Learning, a novel method for automatically generating curricula for the target task in the form of Directed Acyclic Graphs (DAGs). AGCL encodes the specification in the form of a deterministic finite automaton (DFA), and then uses the DFA along with the Object-Oriented MDP (OOMDP) representation to generate a curriculum as a DAG, where the vertices correspond to tasks, and edges correspond to the direction of knowledge transfer. Experiments in gridworld and physics-based simulated robotics domains show that the curricula produced by AGCL achieve improved time-to-threshold performance on a complex sequential decision-making problem relative to state-of-the-art curriculum learning (e.g, teacher-student, self-play) and automaton-guided reinforcement learning baselines (e.g, Q-Learning for Reward Machines). Further, we demonstrate that AGCL performs well even in the presence of noise in the task's OOMDP description, and also when distractor objects are present that are not modeled in the logical specification of the tasks' objectives.
RRHF: Rank Responses to Align Language Models with Human Feedback without tears
Venue: arXiv cs.CL 2023
Year: 2023

Reinforcement Learning from Human Feedback (RLHF) facilitates the alignment of large language models with human preferences, significantly enhancing the quality of interactions between humans and models. InstructGPT implements RLHF through several stages, including Supervised Fine-Tuning (SFT), reward model training, and Proximal Policy Optimization (PPO). However, PPO is sensitive to hyperparameters and requires multiple models in its standard implementation, making it hard to train and scale up to larger parameter counts. In contrast, we propose a novel learning paradigm called RRHF, which scores sampled responses from different sources via a logarithm of conditional probabilities and learns to align these probabilities with human preferences through ranking loss. RRHF can leverage sampled responses from various sources including the model responses from itself, other large language model responses, and human expert responses to learn to rank them. RRHF only needs 1 to 2 models during tuning and can efficiently align language models with human preferences robustly without complex hyperparameter tuning. Additionally, RRHF can be considered an extension of SFT and reward model training while being simpler than PPO in terms of coding, model counts, and hyperparameters. We evaluate RRHF on the Helpful and Harmless dataset, demonstrating comparable alignment performance with PPO by reward model score and human labeling. Extensive experiments show that the performance of RRHF is highly related to sampling quality which suggests RRHF is a best-of-n learner. Codes available at https://github.com/GanjinZero/RRHF.
FIR-based Future Trajectory Prediction in Nighttime Autonomous Driving
Venue: arXiv cs.CV 2023
Year: 2023

The performance of the current collision avoidance systems in Autonomous Vehicles (AV) and Advanced Driver Assistance Systems (ADAS) can be drastically affected by low light and adverse weather conditions. Collisions with large animals such as deer in low light cause significant cost and damage every year. In this paper, we propose the first AI-based method for future trajectory prediction of large animals and mitigating the risk of collision with them in low light. In order to minimize false collision warnings, in our multi-step framework, first, the large animal is accurately detected and a preliminary risk level is predicted for it and low-risk animals are discarded. In the next stage, a multi-stream CONV-LSTM-based encoder-decoder framework is designed to predict the future trajectory of the potentially high-risk animals. The proposed model uses camera motion prediction as well as the local and global context of the scene to generate accurate predictions. Furthermore, this paper introduces a new dataset of FIR videos for large animal detection and risk estimation in real nighttime driving scenarios. Our experiments show promising results of the proposed framework in adverse conditions. Our code is available online.
iDML: Incentivized Decentralized Machine Learning
Venue: arXiv cs.LG 2023
Year: 2023

With the rising emergence of decentralized and opportunistic approaches to machine learning, end devices are increasingly tasked with training deep learning models on-devices using crowd-sourced data that they collect themselves. These approaches are desirable from a resource consumption perspective and also from a privacy preservation perspective. When the devices benefit directly from the trained models, the incentives are implicit - contributing devices' resources are incentivized by the availability of the higher-accuracy model that results from collaboration. However, explicit incentive mechanisms must be provided when end-user devices are asked to contribute their resources (e.g., computation, communication, and data) to a task performed primarily for the benefit of others, e.g., training a model for a task that a neighbor device needs but the device owner is uninterested in. In this project, we propose a novel blockchain-based incentive mechanism for completely decentralized and opportunistic learning architectures. We leverage a smart contract not only for providing explicit incentives to end devices to participate in decentralized learning but also to create a fully decentralized mechanism to inspect and reflect on the behavior of the learning architecture.
Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understanding
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) have made significant progress in various domains, including healthcare. However, the specialized nature of clinical language understanding tasks presents unique challenges and limitations that warrant further investigation. In this study, we conduct a comprehensive evaluation of state-of-the-art LLMs, namely GPT-3.5, GPT-4, and Bard, within the realm of clinical language understanding tasks. These tasks span a diverse range, including named entity recognition, relation extraction, natural language inference, semantic textual similarity, document classification, and question-answering. We also introduce a novel prompting strategy, self-questioning prompting (SQP), tailored to enhance LLMs' performance by eliciting informative questions and answers pertinent to the clinical scenarios at hand. Our evaluation underscores the significance of task-specific learning strategies and prompting techniques for improving LLMs' effectiveness in healthcare-related tasks. Additionally, our in-depth error analysis on the challenging relation extraction task offers valuable insights into error distribution and potential avenues for improvement using SQP. Our study sheds light on the practical implications of employing LLMs in the specialized domain of healthcare, serving as a foundation for future research and the development of potential applications in healthcare settings.
Can ChatGPT and Bard Generate Aligned Assessment Items? A Reliability Analysis against Human Performance
Venue: arXiv cs.CL 2023
Year: 2023

ChatGPT and Bard are AI chatbots based on Large Language Models (LLM) that are slated to promise different applications in diverse areas. In education, these AI technologies have been tested for applications in assessment and teaching. In assessment, AI has long been used in automated essay scoring and automated item generation. One psychometric property that these tools must have to assist or replace humans in assessment is high reliability in terms of agreement between AI scores and human raters. In this paper, we measure the reliability of OpenAI ChatGP and Google Bard LLMs tools against experienced and trained humans in perceiving and rating the complexity of writing prompts. Intraclass correlation (ICC) as a performance metric showed that the inter-reliability of both the OpenAI ChatGPT and the Google Bard were low against the gold standard of human ratings.
Chatbots and ChatGPT: A Bibliometric Analysis and Systematic Review of Publications in Web of Science and Scopus Databases
Venue: arXiv cs.DL 2023
Year: 2023

This paper presents a bibliometric analysis of the scientific literature related to chatbots, focusing specifically on ChatGPT. Chatbots have gained increasing attention recently, with an annual growth rate of 19.16% and 27.19% on the Web of Sciences (WoS) and Scopus, respectively. In this study, we have explored the structure, conceptual evolution, and trends in this field by analyzing data from both Scopus and WoS databases. The research consists of two study phases: (i) an analysis of chatbot literature and (ii) a comprehensive review of scientific documents on ChatGPT. In the first phase, a bibliometric analysis is conducted on all published literature, including articles, book chapters, conference papers, and reviews on chatbots from both Scopus (5839) and WoS (2531) databases covering the period from 1998 to 2023. An in-depth analysis focusing on sources, countries, authors' impact, and keywords has revealed that ChatGPT is the latest trend in the chatbot field. Consequently, in the second phase, bibliometric analysis has been carried out on ChatGPT publications, and 45 published studies have been analyzed thoroughly based on their methods, novelty, and conclusions. The key areas of interest identified from the study can be classified into three groups: artificial intelligence and related technologies, design and evaluation of conversational agents, and digital technologies and mental health. Overall, the study aims to provide guidelines for researchers to conduct their research more effectively in the field of chatbots and specifically highlight significant areas for future investigation into ChatGPT.
Performance Study of Partitioned Caches in Asymmetric Multi-Core Processors
Venue: arXiv cs.AR 2023
Year: 2023

The current workloads and applications are highly diversified, facing critical challenges such as the Power Wall and the Memory Wall Problem. Different strategies over the multiple levels of Caches have evolved to mitigate these problems. Also, to work with such diversified applications, the Asymmetric Multi-Core Processor (AMP) presents itself as a viable solution. In this paper, we study the performance of L2 and Last Level Cache for different cache partitions against various AMP configurations. In addition, this study investigates the optimal cache partitioning for a collection of Multi-threaded benchmarks from PARSEC and SPLASH2 benchmark suites under medium-sized inputs. We have studied the effect of block replacement strategies and their impact on the key metrics such as total on-chip power consumption and L2 \& LLC Miss rates. Our study presents an intermediate cache design for AMPs between the two extremities of fully shared and fully private L2 \& LLC level Cache, which helps achieve the desired power values and optimal cache miss penalties.
Measuring Latency Reduction and the Digital Divide of Cloud Edge Datacenters
Venue: arXiv cs.NI 2023
Year: 2023

Cloud providers are highly incentivized to reduce latency. One way they do this is by locating datacenters as close to users as possible. These "cloud edge" datacenters are placed in metropolitan areas and enable edge computing for residents of these cities. Therefore, which cities are selected to host edge datacenters determines who has the fastest access to applications requiring edge compute - creating a digital divide between those closest and furthest from a datacenter. In this study we measure latency to the current and predicted cloud edge datacenters of three major cloud providers around the world. Our measurements use the RIPE Atlas platform targeting cloud regions, AWS Local Zones, and network optimization services that minimize the path to the cloud edge. An analysis of the digital divide shows rising inequality as the relative difference between users closest and farthest from cloud compute increases. We also find this inequality unfairly affects lower income census tracts in the US. This result is extended globally using remotely sensed night time lights as a proxy for wealth. Finally, we demonstrate that low earth orbit satellite internet can help to close this digital divide and provide more fair access to the cloud edge.
Revisiting Single-gated Mixtures of Experts
Venue: arXiv cs.CV 2023
Year: 2023

Mixture of Experts (MoE) are rising in popularity as a means to train extremely large-scale models, yet allowing for a reasonable computational cost at inference time. Recent state-of-the-art approaches usually assume a large number of experts, and require training all experts jointly, which often lead to training instabilities such as the router collapsing In contrast, in this work, we propose to revisit the simple single-gate MoE, which allows for more practical training. Key to our work are (i) a base model branch acting both as an early-exit and an ensembling regularization scheme, (ii) a simple and efficient asynchronous training pipeline without router collapse issues, and finally (iii) a per-sample clustering-based initialization. We show experimentally that the proposed model obtains efficiency-to-accuracy trade-offs comparable with other more complex MoE, and outperforms non-mixture baselines. This showcases the merits of even a simple single-gate MoE, and motivates further exploration in this area.
End-to-End O-RAN Security Architecture, Threat Surface, Coverage, and the Case of the Open Fronthaul
Venue: arXiv cs.CR 2023
Year: 2023

O-RAN establishes an advanced radio access network (RAN) architecture that supports inter-operable, multi-vendor, and artificial intelligence (AI) controlled wireless access networks. The unique components, interfaces, and technologies of O-RAN differentiate it from the 3GPP RAN. Because O-RAN supports 3GPP protocols, currently 4G and 5G, while offering additional network interfaces and controllers, it has a larger attack surface. The O-RAN security requirements, vulnerabilities, threats, and countermeasures must be carefully assessed for it to become a platform for 5G Advanced and future 6G wireless. This article presents the ongoing standardization activities of the O-RAN Alliance for modeling the potential threats to the network and to the open fronthaul interface, in particular. We identify end-to-end security threats and discuss those on the open fronthaul in more detail. We then provide recommendations for countermeasures to tackle the identified security risks and encourage industry to establish standards and best practices for safe and secure implementations of the open fronthaul interface.
Understanding Causality with Large Language Models: Feasibility and Opportunities
Venue: arXiv cs.LG 2023
Year: 2023

We assess the ability of large language models (LLMs) to answer causal questions by analyzing their strengths and weaknesses against three types of causal question. We believe that current LLMs can answer causal questions with existing causal knowledge as combined domain experts. However, they are not yet able to provide satisfactory answers for discovering new knowledge or for high-stakes decision-making tasks with high precision. We discuss possible future directions and opportunities, such as enabling explicit and implicit causal modules as well as deep causal-aware LLMs. These will not only enable LLMs to answer many different types of causal questions for greater impact but also enable LLMs to be more trustworthy and efficient in general.
Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis
Venue: arXiv cs.CL 2023
Year: 2023

In the first half of 2023, text-generative artificial intelligence (AI), including ChatGPT, equipped with GPT-3.5 and GPT-4, from OpenAI, has attracted considerable attention worldwide. In this study, first, we compared Japanese stylometric features of texts generated by GPT (-3.5 and -4) and those written by humans. In this work, we performed multi-dimensional scaling (MDS) to confirm the distributions of 216 texts of three classes (72 academic papers written by 36 single authors, 72 texts generated by GPT-3.5, and 72 texts generated by GPT-4 on the basis of the titles of the aforementioned papers) focusing on the following stylometric features: (1) bigrams of parts-of-speech, (2) bigram of postpositional particle words, (3) positioning of commas, and (4) rate of function words. MDS revealed distinct distributions at each stylometric feature of GPT (-3.5 and -4) and human. Although GPT-4 is more powerful than GPT-3.5 because it has more parameters, both GPT (-3.5 and -4) distributions are likely to overlap. These results indicate that although the number of parameters may increase in the future, GPT-generated texts may not be close to that written by humans in terms of stylometric features. Second, we verified the classification performance of random forest (RF) for two classes (GPT and human) focusing on Japanese stylometric features. This study revealed the high performance of RF in each stylometric feature: The RF classifier focusing on the rate of function words achieved 98.1% accuracy. Furthermore the RF classifier focusing on all stylometric features reached 100% in terms of all performance indexes (accuracy, recall, precision, and F1 score). This study concluded that at this stage we human discriminate ChatGPT from human limited to Japanese language.
Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimation
Venue: arXiv cs.CV 2023
Year: 2023

Human pose estimation has seen widespread use of transformer models in recent years. Pose transformers benefit from the self-attention map, which captures the correlation between human joint tokens and the image. However, training such models is computationally expensive. The recent token-Pruned Pose Transformer (PPT) solves this problem by pruning the background tokens of the image, which are usually less informative. However, although it improves efficiency, PPT inevitably leads to worse performance than TokenPose due to the pruning of tokens. To overcome this problem, we present a novel method called Distilling Pruned-Token Transformer for human pose estimation (DPPT). Our method leverages the output of a pre-trained TokenPose to supervise the learning process of PPT. We also establish connections between the internal structure of pose transformers and PPT, such as attention maps and joint features. Our experimental results on the MPII datasets show that our DPPT can significantly improve PCK compared to previous PPT models while still reducing computational complexity.