pwc-archive/papers-with-abstracts huggingface dataset:
Are These the Same Apple? Comparing Images Based on Object Intrinsics
Date: 2023-09-26 00:00:00
Proceeding: NeurIPS 2023 11

The human visual system can effortlessly recognize an object under different extrinsic factors such as lighting, object poses, and background, yet current computer vision systems often struggle with these variations. An important step to understanding and improving artificial vision systems is to measure image similarity purely based on intrinsic object properties that define object identity. This problem has been studied in the computer vision literature as re-identification, though mostly restricted to specific object categories such as people and cars. We propose to extend it to general object categories, exploring an image similarity metric based on object intrinsics. To benchmark such measurements, we collect the Common paired objects Under differenT Extrinsics (CUTE) dataset of 18, 000 images of 180 objects under different extrinsic factors such as lighting, poses, and imaging conditions. While existing methods such as LPIPS and CLIP scores do not measure object intrinsics well, we find that combining deep features learned from contrastive self-supervised learning with foreground filtering is a simple yet effective approach to approximating the similarity. We conduct an extensive survey of pre-trained features and foreground extraction methods to arrive at a strong baseline that best measures intrinsic object-centric image similarity among current methods. Finally, we demonstrate that our approach can aid in downstream applications such as acting as an analog for human subjects and improving generalizable re-identification. Please see our project website at https://s-tian.github.io/projects/cute/ for visualizations of the data and demos of our metric.
Voice Anonymization for All -- Bias Evaluation of the Voice Privacy Challenge Baseline System
Authors: Anna Leschanowsky, Ünal Ege Gaznepoglu, Nils Peters
Date: 2023-11-27 00:00:00
Tasks: All

In an age of voice-enabled technology, voice anonymization offers a solution to protect people's privacy, provided these systems work equally well across subgroups. This study investigates bias in voice anonymization systems within the context of the Voice Privacy Challenge. We curate a novel benchmark dataset to assess performance disparities among speaker subgroups based on sex and dialect. We analyze the impact of three anonymization systems and attack models on speaker subgroup bias and reveal significant performance variations. Notably, subgroup bias intensifies with advanced attacker capabilities, emphasizing the challenge of achieving equal performance across all subgroups. Our study highlights the need for inclusive benchmark datasets and comprehensive evaluation strategies that address subgroup bias in voice anonymization.
Thinking Outside the Box: Orthogonal Approach to Equalizing Protected Attributes
Authors: Jiahui Liu, Xiaohao Cai, Mahesan Niranjan
Date: 2023-11-21 00:00:00
Tasks: Attribute, Decision Making, Dimensionality Reduction

There is growing concern that the potential of black box AI may exacerbate health-related disparities and biases such as gender and ethnicity in clinical decision-making. Biased decisions can arise from data availability and collection processes, as well as from the underlying confounding effects of the protected attributes themselves. This work proposes a machine learning-based orthogonal approach aiming to analyze and suppress the effect of the confounder through discriminant dimensionality reduction and orthogonalization of the protected attributes against the primary attribute information. By doing so, the impact of the protected attributes on disease diagnosis can be realized, undesirable feature correlations can be mitigated, and the model prediction performance can be enhanced.
Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals
Authors: Nir Chemaya, Daniel Martin
Date: 2023-11-19 00:00:00

The emergent abilities of Large Language Models (LLMs), which power tools like ChatGPT and Bard, have produced both excitement and worry about how AI will impact academic writing. In response to rising concerns about AI use, authors of academic publications may decide to voluntarily disclose any AI tools they use to revise their manuscripts, and journals and conferences could begin mandating disclosure and/or turn to using detection services, as many teachers have done with student writing in class settings. Given these looming possibilities, we investigate whether academics view it as necessary to report AI use in manuscript preparation and how detectors react to the use of AI in academic writing.
Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
Authors: Markus Anderljung, Everett Thornton Smith, Joe O'Brien, Lisa Soder, Benjamin Bucknall, Emma Bluemke, Jonas Schuett, Robert Trager, Lacey Strahm, Rumman Chowdhury
Date: 2023-11-15 00:00:00
Tasks: Red Teaming

With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching implications. These decisions should not be left solely in the hands of frontier LLM developers. LLM users, civil society and policymakers need trustworthy sources of information to steer such decisions for the better. Involving outside actors in the evaluation of these systems - what we term 'external scrutiny' - via red-teaming, auditing, and external researcher access, offers a solution. Though there are encouraging signs of increasing external scrutiny of frontier LLMs, its success is not assured. In this paper, we survey six requirements for effective external scrutiny of frontier AI systems and organize them under the ASPIRE framework: Access, Searching attitude, Proportionality to the risks, Independence, Resources, and Expertise. We then illustrate how external scrutiny might function throughout the AI lifecycle and offer recommendations to policymakers.
Large Language Model-Driven Classroom Flipping: Empowering Student-Centric Peer Questioning with Flipped Interaction
Authors: Chee Wei Tan
Date: 2023-11-14 00:00:00
Tasks: Chatbot, Language Modeling, Language Modelling, Large Language Model, Prompt Engineering

Reciprocal questioning is essential for effective teaching and learning, fostering active engagement and deeper understanding through collaborative interactions, especially in large classrooms. Can large language model (LLM), such as OpenAI's GPT (Generative Pre-trained Transformer) series, assist in this? This paper investigates a pedagogical approach of classroom flipping based on flipped interaction in LLMs. Flipped interaction involves using language models to prioritize generating questions instead of answers to prompts. We demonstrate how traditional classroom flipping techniques, including Peer Instruction and Just-in-Time Teaching (JiTT), can be enhanced through flipped interaction techniques, creating student-centric questions for hybrid teaching. In particular, we propose a workflow to integrate prompt engineering with clicker and JiTT quizzes by a poll-prompt-quiz routine and a quiz-prompt-discuss routine to empower students to self-regulate their learning capacity and enable teachers to swiftly personalize training pathways. We develop an LLM-driven chatbot software that digitizes various elements of classroom flipping and facilitates the assessment of students using these routines to deliver peer-generated questions. We have applied our LLM-driven chatbot software for teaching both undergraduate and graduate students from 2020 to 2022, effectively useful for bridging the gap between teachers and students in remote teaching during the COVID-19 pandemic years. In particular, LLM-driven classroom flipping can be particularly beneficial in large class settings to optimize teaching pace and enable engaging classroom experiences.
Knowledge Tracing Challenge: Optimal Activity Sequencing for Students
Authors: Yann Hicke
Date: 2023-11-13 00:00:00
Tasks: Knowledge Tracing

Knowledge tracing is a method used in education to assess and track the acquisition of knowledge by individual learners. It involves using a variety of techniques, such as quizzes, tests, and other forms of assessment, to determine what a learner knows and does not know about a particular subject. The goal of knowledge tracing is to identify gaps in understanding and provide targeted instruction to help learners improve their understanding and retention of material. This can be particularly useful in situations where learners are working at their own pace, such as in online learning environments. By providing regular feedback and adjusting instruction based on individual needs, knowledge tracing can help learners make more efficient progress and achieve better outcomes. Effectively solving the KT problem would unlock the potential of computer-aided education applications such as intelligent tutoring systems, curriculum learning, and learning materials recommendations. In this paper, we will present the results of the implementation of two Knowledge Tracing algorithms on a newly released dataset as part of the AAAI2023 Global Knowledge Tracing Challenge.
ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management
Authors: Angela Zhang, Mert Yuksekgonul, Joshua Guild, James Zou, Joseph C. Wu
Date: 2023-11-10 00:00:00
Tasks: Decision Making, Management

Recent breakthroughs in large language models (LLMs) have led to their rapid dissemination and widespread use. One early application has been to medicine, where LLMs have been investigated to streamline clinical workflows and facilitate clinical analysis and decision-making. However, a leading barrier to the deployment of Artificial Intelligence (AI) and in particular LLMs has been concern for embedded gender and racial biases. Here, we evaluate whether a leading LLM, ChatGPT 3.5, exhibits gender and racial bias in clinical management of acute coronary syndrome (ACS). We find that specifying patients as female, African American, or Hispanic resulted in a decrease in guideline recommended medical management, diagnosis, and symptom management of ACS. Most notably, the largest disparities were seen in the recommendation of coronary angiography or stress testing for the diagnosis and further intervention of ACS and recommendation of high intensity statins. These disparities correlate with biases that have been observed clinically and have been implicated in the differential gender and racial morbidity and mortality outcomes of ACS and coronary artery disease. Furthermore, we find that the largest disparities are seen during unstable angina, where fewer explicit clinical guidelines exist. Finally, we find that through asking ChatGPT 3.5 to explain its reasoning prior to providing an answer, we are able to improve clinical accuracy and mitigate instances of gender and racial biases. This is among the first studies to demonstrate that the gender and racial biases that LLMs exhibit do in fact affect clinical management. Additionally, we demonstrate that existing strategies that improve LLM performance not only improve LLM performance in clinical management, but can also be used to mitigate gender and racial biases.
Transdisciplinary AI Education: The Confluence of Curricular and Community Needs in the Instruction of Artificial Intelligence
Authors: Roozbeh Aliabadi, Aditi Singh, Eryka Wilson
Date: 2023-11-10 00:00:00

The integration of artificial intelligence (AI) into education has the potential to transform the way we learn and teach. In this paper, we examine the current state of AI in education and explore the potential benefits and challenges of incorporating this technology into the classroom. The approaches currently available for AI education often present students with experiences only focusing on discrete computer science concepts agnostic to a larger curriculum. However, teaching AI must not be siloed or interdisciplinary. Rather, AI instruction ought to be transdisciplinary, including connections to the broad curriculum and community in which students are learning. This paper delves into the AI program currently in development for Neom Community School and the larger Education, Research, and Innovation Sector in Neom, Saudi Arabia s new megacity under development. In this program, AI is both taught as a subject and to learn other subjects within the curriculum through the school systems International Baccalaureate (IB) approach, which deploys learning through Units of Inquiry. This approach to education connects subjects across a curriculum under one major guiding question at a time. The proposed method offers a meaningful approach to introducing AI to students throughout these Units of Inquiry, as it shifts AI from a subject that students like or not like to a subject that is taught throughout the curriculum.
ChatGPT as Co-Advisor in Scientific Initiation: Action Research with Project-Based Learning in Elementary Education
Authors: Fabiano Villan, Renato P. dos Santos
Date: 2023-11-10 00:00:00

Background: In the contemporary educational landscape, technology has the power to drive innovative pedagogical practices. Overcoming the resistance of teachers and students to adopting new methods and technologies is a challenge that needs to be addressed. Objectives: To evaluate the effectiveness of ChatGPT as a co-advisor in research projects and its influence on the implementation of Project-Based Learning (PBL), as well as overcoming resistance to the use of new pedagogical methodologies. Design: An action-research methodology was employed, including unstructured interviews and the application of questionnaires via Google Forms. Setting and Participants: The research was conducted in an elementary school, involving 353 students and 16 teachers. Data Collection and Analysis: Data were gathered through observations and notes in meetings and interviews, complemented by electronic questionnaires, with quantitative and qualitative analyses performed via Microsoft Excel and Google Forms. Results: The introduction of ChatGPT as a pedagogical tool led to increased student engagement and decreased teacher resistance, reflected in recognition at local science fairs. Conclusion: The study confirmed the utility of ChatGPT in school research co-orientation, highlighting its role in facilitating PBL and promoting cultural changes in educational practice, with proactive school management identified as a catalysing element in adapting to educational innovations.
Towards a Feminist Metaethics of AI
Authors: Anastasia Siapka
Date: 2023-11-10 00:00:00
Tasks: Ethics

The proliferation of Artificial Intelligence (AI) has sparked an overwhelming number of AI ethics guidelines, boards and codes of conduct. These outputs primarily analyse competing theories, principles and values for AI development and deployment. However, as a series of recent problematic incidents about AI ethics/ethicists demonstrate, this orientation is insufficient. Before proceeding to evaluate other professions, AI ethicists should critically evaluate their own; yet, such an evaluation should be more explicitly and systematically undertaken in the literature. I argue that these insufficiencies could be mitigated by developing a research agenda for a feminist metaethics of AI. Contrary to traditional metaethics, which reflects on the nature of morality and moral judgements in a non-normative way, feminist metaethics expands its scope to ask not only what ethics is but also what our engagement with it should be like. Applying this perspective to the context of AI, I suggest that a feminist metaethics of AI would examine: (i) the continuity between theory and action in AI ethics; (ii) the real-life effects of AI ethics; (iii) the role and profile of those involved in AI ethics; and (iv) the effects of AI on power relations through methods that pay attention to context, emotions and narrative.
AI for All: Operationalising Diversity and Inclusion Requirements for AI Systems
Authors: Muneera Bano, Didar Zowghi, Vincenzo Gervasi, Rifat Shams
Date: 2023-11-07 00:00:00
Tasks: All, Decision Making, Diversity, Language Modeling, Language Modelling, Large Language Model

As Artificial Intelligence (AI) permeates many aspects of society, it brings numerous advantages while at the same time raising ethical concerns and potential risks, such as perpetuating inequalities through biased or discriminatory decision-making. To develop AI systems that cater for the needs of diverse users and uphold ethical values, it is essential to consider and integrate diversity and inclusion (D&I) principles throughout AI development and deployment. Requirements engineering (RE) is a fundamental process in developing software systems by eliciting and specifying relevant needs from diverse stakeholders. This research aims to address the lack of research and practice on how to elicit and capture D&I requirements for AI systems. We have conducted comprehensive data collection and synthesis from the literature review to extract requirements themes related to D&I in AI. We have proposed a tailored user story template to capture D&I requirements and conducted focus group exercises to use the themes and user story template in writing D&I requirements for two example AI systems. Additionally, we have investigated the capability of our solution by generating synthetic D&I requirements captured in user stories with the help of a Large Language Model.
Standardized Analysis Ready (STAR) data cube for high-resolution Flood mapping using Sentinel-1 data
Authors: Surajit Ghosh, Arpan Dawn, Sneha Kour, Susmita Ghosh
Date: 2023-11-07 00:00:00
Tasks: Earth Observation

Floods are one of the most common disasters globally. Flood affects humans in many ways. Therefore, rapid assessment is needed to assess the effect of floods and to take early action to support the vulnerable community in time. Sentinel-1 is one such Earth Observation (EO) mission widely used for mapping the flooding conditions at a 10m scale. However, various preprocessing steps are involved before analyses of the Sentinel-1 data. Researchers sometimes avoid a few necessary corrections since it is time-consuming and complex. Standardization of the Sentinel-1 data is the need of the hour, specifically for supporting researchers to use the Standardized Analysis-Ready (STAR) data cube without experiencing the complexity of the Sentinel-1 data processing. In the present study, we proposed a workflow to use STAR in Google Earth Engine (GEE) environment. The Nigeria Flood of 2022 has been used as a case study for assessing the model performance.
Benefits and Harms of Large Language Models in Digital Mental Health
Authors: Munmun De Choudhury, Sachin R. Pendse, Neha Kumar
Date: 2023-11-07 00:00:00

The past decade has been transformative for mental health research and practice. The ability to harness large repositories of data, whether from electronic health records (EHR), mobile devices, or social media, has revealed a potential for valuable insights into patient experiences, promising early, proactive interventions, as well as personalized treatment plans. Recent developments in generative artificial intelligence, particularly large language models (LLMs), show promise in leading digital mental health to uncharted territory. Patients are arriving at doctors' appointments with information sourced from chatbots, state-of-the-art LLMs are being incorporated in medical software and EHR systems, and chatbots from an ever-increasing number of startups promise to serve as AI companions, friends, and partners. This article presents contemporary perspectives on the opportunities and risks posed by LLMs in the design, development, and implementation of digital mental health tools. We adopt an ecological framework and draw on the affordances offered by LLMs to discuss four application areas -- care-seeking behaviors from individuals in need of care, community care provision, institutional and medical care provision, and larger care ecologies at the societal level. We engage in a thoughtful consideration of whether and how LLM-based technologies could or should be employed for enhancing mental health. The benefits and harms our article surfaces could serve to help shape future research, advocacy, and regulatory efforts focused on creating more responsible, user-friendly, equitable, and secure LLM-based tools for mental health treatment and intervention.
COVID-19 Imposes Rethinking of Conferencing -- Environmental Impact Assessment of Artificial Intelligence Conferences
Authors: Pavlina Mitsou, Nikoleta-Victoria Tsakalidou, Eleni Vrochidou, George A. Papakostas
Date: 2023-11-06 00:00:00

It has been noticed that through COVID-19 greenhouse gas emissions had a sudden reduction. Based on this significant observation, we decided to conduct a research to quantify the impact of scientific conferences' air-travelling, explore and suggest alternative ways for greener conferences to re-duce the global carbon footprint. Specifically, we focused on the most popular conferences for the Artificial Intelligence community based on their scientific impact factor, their scale, and the well-organized proceedings towards measuring the impact of air travelling participation. This is the first time that systematic quantification of a state-of-the-art subject like Artificial Intelligence takes place to define its conferencing footprint in the broader frames of environmental awareness. Our findings highlight that the virtual way is the first on the list of green conferences' conduction although there are serious concerns about it. Alternatives to optimal conferences' location selection have demonstrated savings on air-travelling CO2 emissions of up to 63.9%.
Evolutionary City: Towards a Flexible, Agile and Symbiotic System
Authors: Xi Chen, Wei Hu, Jingru Yu, Ding Wang, Shengyue Yao, Yilun Lin, Fei-Yue Wang
Date: 2023-11-06 00:00:00
Tasks: Decision Making, Management

Urban growth sometimes leads to rigid infrastructure that struggles to adapt to changing demand. This paper introduces a novel approach, aiming to enable cities to evolve and respond more effectively to such dynamic demand. It identifies the limitations arising from the complexity and inflexibility of existing urban systems. A framework is presented for enhancing the city's adaptability perception through advanced sensing technologies, conducting parallel simulation via graph-based techniques, and facilitating autonomous decision-making across domains through decentralized and autonomous organization and operation. Notably, a symbiotic mechanism is employed to implement these technologies practically, thereby making urban management more agile and responsive. In the case study, we explore how this approach can optimize traffic flow by adjusting lane allocations. This case not only enhances traffic efficiency but also reduces emissions. The proposed evolutionary city offers a new perspective on sustainable urban development, highliting the importance of integrated intelligence within urban systems.