The ChatGPT Artificial Intelligence Chatbot: How Well Does It Answer Accounting Assessment Questions?
Authors: Wood David A. , Achhpilia Muskan P. , Adams Mollie T. , Aghazadeh Sanaz , Akinyele Kazeem , Akpan Mfon , Allee Kristian D. , Allen Abigail M. , Almer Elizabeth D. , Ames Daniel , Arity Viktor , Barr-Pulliam Dereck , Basoglu K. Asli , Belnap Andrew , Bentley Jeremiah W. , Berg Terje , Berglund Nathan R. , Berry Erica , Bhandari Avishek , Bhuyan Md Nazmul Hasan , Black Paul W. , Blondeel Eva , Bond David , Bonrath Annika , Borthick A. Faye , Boyle Erik S. , Bradford Marianne , Brandon Duane M. , Brazel Joseph F. , Brockbank Bryan G. , Burger Marcus , Byzalov Dmitri , Cannon James N. , Caro Cecil , Carr Abraham H. , Cathey Jack , Cating Ryan , Charron Kimberly , Chavez Stacy , Chen Jason , Chen Jennifer C. , Chen Jennifer W. , Cheng Christine , Cheng Xu , Christensen Brant E. , Church Kimberly Swanson , Cicone N. J. , Constance Patience , Cooper Lauren A. , Correia Candice L. , Coyne Joshua , Cram W. Alec , Curtis Asher , Daigle Ronald J. , Dannemiller Steve , Davenport Stephan A. , Dawson Gregory S. , De Meyst Karen J. L. , Dell Scott , Demirkan Sebahattin , Denison Christine A. , Desai Hrishikesh , DeSimone Steven , Diehl Leah M. , Dimes Ruth , Dong Bei , Donnelly Amy , du Pon Adam , Duan Huijue Kelly , Duffey Ada , Dunn Ryan T. , Durkin Mary P. , Dzuranin Ann C. , Eberle Rachel M. , Ege Matthew S. , El Mahdy Dina , Esplin Adam , Eulerich Marc , Everaert Patricia , Farah Nusrat , Farish Lauren , Favere-Marchesi Michael , Fayard Dutch , Filosa Jessica R. , Ford Melinda , Franz Diana R. , Fulmer Bachman P. , Fulmer Sarah , Furner Zhan Z. , Gantman Sonia , Garner Steve , Garrett Jace , Geng Xin , Golden Joanna , Goldman William , Gomez Josue , Gooley Mark , Granitto Shawn P. , Green Karen Y. , Greenman Cindy L. , Gupta Gaurav , Guymon Ronald N. , Hale Kevin , Harper Christopher J. , Hartt S. Allen , Hawk Holly , Hawkins Steven R. , Hawkins Erin M. , Hay David C. , Heinzelmann Rafael , Henderson Cassy D. , Hendricks Bradley E. , Heninger William G. , Hill Mary S. , Holden Nicole , Holderness D. Kip , Holt Travis P. , Hoopes Jeffrey L. , Hsieh Sheng-Feng , Huang Feiqi , Huang Hua-Wei , Huang Ting-Chiao , Huels Brian W. , Hunter Kara , Hurley Patrick J. , Inger Kerry , Islam Sharif , Ison Isaac , Issa Hussein , Jackson Andrew B. , Jackson Scott C. , Janvrin Diane J. , Jimenez Peggy D. , Johanson Daniel , Judd J. Scott , Kawada Brett S. , Kelton Andrea Seaton , Kern Sara , Kerr Jon N. , Keune Marsha B. , Kim Mindy , Knox Brian D. , Kogan Gregory , Kotb Amr , Krane Ronja , Kremin Joleen , Krieg Kimberly S. , Kugel Jonathan , Kulset Ellen M. , Kuruppu Chamara , LaDuca Garrison , Lamberton Barbara A. , Lamboy-Ruiz Melvin A. , Lang Bradley , Larocque Stephannie A. , Larson Melissa P. , Lawson Bradley P. , Lawson James G. , Lee Lorraine , Lenk Margarita M. , Li-Kuehne Michelle , Liljegren Jonathan , Lin Yi-Hung , Liu Wu-Po , Liu Zishang , Lock Brandon , Long James H. , Loraas Tina , Lowensohn Suzanne , Loy Thomas R. , Lyngstadaas Hakim , Maas Wim , MacGregor Jason E. , Madsen Dag Øivind , Malone Carissa L. , Margolin Maximilian , Marshall Mary E. , Martin Rachel M. , Mpofu Colleen McClain , McCoy Chris , McGuigan Nicholas C. , McSwain Dwayne N. , Meckfessel Michele D. , Mellon Mark J. , Melton Olivia S. , Mercado Julie M. , Mitsuda Steven , Modugu Kennedy , Moehrle Stephen , Chaghervand Amirali Moeini , Moffitt Kevin , Moon Joon Seok , Muehlmann Brigitte , Murray Johnna , Mwaungulu Emmanuel S. , Myers Noah , Naegle J. Conrad , Ndicu Martin J. , Nelson Aaron S. , Nguyen Anh L. , Niederkofler Thomas , Nikbakht Ehsan , O'Brien Ann D. , Ogunade Kehinde M. , O'Leary Daniel , Oler Mitchell J. , Oler Derek K. , Olsen Kari Joseph , Otalor John I. , Outlaw Kyle W. , Ozlanski Michael. E. , Parlier Jenny , Paterson Jeffrey S. , Pearson Christopher A. , Petersen Michael J. , Petra Steven T. , Pickard Matthew D. , Pickerd Jeffrey , Pinsker Robert , Plante Catherine , Plečnik James M. , Price Richard A. , Quick Linda A. , Raedy Jana , Raschke Robyn , Ravenscraft Julie , Richardson Vernon , Rixom Brett A. , Robertson John F. , Rock Iyad , Romney Miles A. , Rozario Andrea , Ruff Michael F. , Rupley Kathleen , Saeedi Ali , Saiewitz Aaron , Salzsieder Leigh W. , Sarkar Sayan , Saulls Michael , Scanlan Tialei A. , Schaefer Tammie J. , Schaupp Daniel , Schneider Gary P. , Seebeck Andreas , Sellers R. Drew , Seto Samantha C. , Sevel Romi-Lee , Shan Yuxin , Sherwood Matthew G. , Singorahardjo Maggie , Skaftadottir Hanna Kristin , Skomra Justyna , Smith Jason L. , Smith Dallin O. , Smith James , Snow Mason C. , Sommerfeldt Ryan D. , Sorensen Kate B. , Sorensen Trevor L. , Spieler Andrew C. , Stallings Matthew A. , Stallings Lesya , Stancill Alan , Stanley Jonathan D. , Stefaniak Chad M. , Stephens Nathaniel M. , Stewart Bryan W. , Stratopoulos Theophanis C. , Street Daniel A. , Subedi Meena , Summers Scott L. , Sundkvist Charlotte H. , Synn Christina , Tadesse Amanuel , Tapis Gregory P. , Tassin Kerri , Taylor Samantha , Teal Mary , Teeter Ryan , Tharapos Meredith , Theis Jochen C. , Thomas Jack , Thompson Kristen S. , Thornock Todd A. , Tietz Wendy , Travalent Anthony M. , Trinkle Brad S. , Truelson J. Mike , Turner Michael C. , Vagner Brandon , Vakilzadeh Hamid , van der Geest Jesse , van Pelt Victor , Vandervelde Scott D. , Vega Jose , Vera-Muñoz Sandra , Villanueva Brigham , Vincent Nishani Edirisinghe , Wagener Martin , Walton Stephanie , Warne Rick C. , Watanabe Olena V. , Watson David , Watson Marcia Weidenmier , Weber Jill , Weirich Thomas , West Ashley N. , Wilford Amanda L. , Wilson Aaron B. , Winrow Brian , Winrow Timothy , Winrow Tasia S. , Wiseman Denise , Witte Annie L. , Wood Bryan D. , Wood Jessica , Woolley Darryl , Wright Nicole S. , Wu Juan , Xiong Xiaomei , Yatsenko Dimitri , Yazzie Courtney E. , Young Glen M. , Zhang Chanyuan , Zimmerman Aleksandra B. , Zoet Emily 
Published: April 2023

ABSTRACT
ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5 percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.