Allison Sullivan

allison.sullivan@uta.edu

Assistant Professor https://allisonius.github.io/

Department of Computer Science
The University of Texas at Arlington https://scopelabuta.github.io/

RESEARCH INTEREST
Automated Software Engineering: Test/Oracle Generation, Automated Bug Localization and Repair, Regression Testing, andMutation Testing
Formal Methods and Programming Languages: Model Based Testing, Formal Specification Languages, Program Synthesis,
Model Checking and Symbolic Execution
ACADEMIC EMPLOYMENT HISTORY
2020 – PRESENT Assistant Professor, The University of Texas at Arlington

Arlington, TX

2018 – 2020 Assistant Professor, North Carolina A&T State University

Greensboro, NC

EDUCATION
2014-2017 Ph.D. in Software Engineering, The University of Texas at Austin

Austin, TX

2012-2014 M.S. in Software Engineering, The University of Texas at Austin

Austin, TX

2008-2012 B.S. in Software Engineering, The University of Texas at Dallas

Richardson, TX

INDUSTRY EMPLOYMENT HISTORY
June – July 2019 Google, Inc, Faculty in Residence (FIR)

Mountain View, CA

FIR is an immersive program that trained me to: design and implement classroom experiments related toproject-based learning, conduct resume workshops, and give mock interviews.
2015 – 2016 Naval Research Laboratory, Intern in the Center for High Assurance Computer Systems

Washington, D.CI extended the SCR toolset to automatically generate test cases that incorporate assumptions. This included
updating the automated translations of SCR models into SPIN, NuSMV, and PVS.
May – Aug. 2013 IBM Research , Intern in the Test Suite Automation Team

Austin, TX

I built a test automation toolset to ensue test suites automatically feed their output into RQM, IBM’s project
management tool. This toolset bridges existing automation scripts with the RQM and is in place at IBM today.
RESEARCH FUNDING
7 grants | Total amount: $9,669,000 | My share: $1,909,000
Grants Obtained at UTA[ 1 ] CAREER: Live Programming for Finite Model Finders
National Science Foundation, solo PI, $525k, June 2024 - May 2029
[ 2 ] SHF: Small: INCA: Incremental Analysis of Software Specification for Evolving Systems
National Science Foundation – CISE Core, solo PI, $490k, Oct 2022 - Sept 2025
[ 3 ] KeenEye: Enhancing Scenario Enumeration
National Science Foundation – Formal Methods in the Field, solo PI, $99k, Oct 2021 - Mar 2023
[ TRANSFER ] Alloy Analyzer Plus: An Integrated Development Environment for Alloy
National Science Foundation – Formal Methods in the Field, solo PI, $67k, July 2020 – Jan 2022
Grants Obtained at NC A&T[ 4 ] MRI: Acquisition of a Testbed of Connected Autonomous MicroTransit Vehicles
National Science Foundation Major Research Instrumentation, Co-PI, $550k (My Share: $55k), July 2020 - June 2022
[ 5 ] Secure and Safe Assured Autonomy (S2 A2 )NASA University Leadership Initiative, Co-PI, $7.9M (My Share: $635k), Aug 2020 - Aug 2024
[ 6 ] Alloy Analyzer Plus: An Integrated Development Environment for Alloy
National Science Foundation – Formal Methods in the Field, solo PI, $100k, Transferred to UTA July 2020

[ 7 ] Project Based Learning in the Teaching of Analysis of Algorithms
National Science Foundation – HBCU-UP ACE Implementation Project, solo PI, $5k, June 2019 - August 2021
REFEREED CONFERENCE PUBLICATIONS(C22) Adam Emerson, Allison Sullivan. Crucible: Graphical Test Cases for Alloy Models. In Proceedings of the 34th International
Symposium on Software Reliability Engineering (ISSRE 2023), pages 218-227, Oct. 2023. (Acceptance: 29.5%)(C21) Ana Jovanovic, Allison Sullivan. Mutation Testing for Temporal Alloy Models. In Proceedings of the ACM/IEEE 26th International Conference on Model-Driven Engineering Languages and Systems (MODELS 2023), pages 228-238, Oct. 2023.
(Acceptance: 24.6%)(C20) Allison Sullivan. Integrating Testing into the Alloy Model Development Workflow. In Proceedings of the ACM/IEEE 26th
International Conference on Model-Driven Engineering Languages and Systems (MODELS 2023), pages 117-128, Oct. 2023.
(Acceptance: 24.6%)(C19) Allison Sullivan. Live Programming for Finite Model Finders. In Proceedings of the 38th IEEE/ACM International Conference
on Automated Software Engineering - New Ideas and Emerging Results (ASE-NIER 2023), pages 1747-1752, Sept. 2023.
(Acceptance: 36%)
(C18) Jan Oliver Ringert, Allison Sullivan. Abstract Alloy Instances. In Proceedings of the 25th International Symposium on FormalMethods (FM 2023), pages 364-382, Mar. 2023.
(C17) Ana Jovanovic, Allison Sullivan. REACH: Refining Alloy Scenarios by Size. In Proceedings of the 33rd International Symposium on Software Reliability Engineering (ISSRE 2022), pages 229–238, Oct. 2022. (Acceptance: 29%)
(C16) Ana Jovanovic, Allison Sullivan. Towards Automated Input Generation for Sketching Alloy Models. In Proceedings of the10th International Conference on Formal Methods in Software Engineering (FormaliSE 2022), pages 58–68, May 2022.
(C15) Allison Sullivan. Hawkeye: User-Guided Enumeration of Scenarios. In Proceedings of the 32nd International Symposium
on Software Reliability Engineering (ISSRE 2021), pages 569–578, Oct. 2021. (Acceptance: 27.5%)(C14) Tanvir Ahmed Khan, Allison Sullivan, Kaiyuan Wang. AlloyFL: A fault localization framework for Alloy. In Proceedings ofthe 2021 ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
Demonstration Track (FSE DEMO 2021), pages 1535–1539, Aug. 2021. (Acceptance: 62.5%)(C13) Kaiyuan Wang, Allison Sullivan, Darko Marinov, Sarfraz Khurshid. Fault Localization for Declarative Models in Alloy. In
Proceedings of the 331st International Symposium on Software Reliability Engineering (ISSRE 2020), pages 391–402, Oct.
2020. (Acceptance: 26%)
(C12) George Thompson, Allison Sullivan. ProFL: A Fault Localization Framework for Prolog. In Proceedings of the 29th ACM
SIGSOFT International Symposium on Software Testing and Analysis Demonstration Track (ISSTA DEMO 2021), pages 561–564, July. 2020. (Acceptance: 33%)
(C11) Allison Sullivan, Darko Marinov, Sarfraz Khurshid. Solution Enumeration Abstraction - A Modeling Idiom to Enhance aLightweight Formal Method. In Proceedings of the 21st International Conference on Formal Engineering Methods (ICFEM2019), pages 336–352, Nov. 2019. (Acceptance: 30%)
(C10) Kaiyuan Wang, Allison Sullivan, Sarfraz Khurshid. ARepair: A Repair Framework for Alloy. In Proceedings of the 41stIEEE/ACM International Conference on Software Engineering (ICSE Demo 2019), pages 103–106, May. 2019. (Acceptance:47%)(C9)

Kaiyuan Wang, Allison Sullivan, Darko Marinov, Sarfraz Khurshid. ASketch: A Sketching Framework for Alloy. In Proceedings of the 2018 ACM Joint European Software Engineering Conference and Symposium on the Foundations of SoftwareEngineering (FSE Demo 2018), pages 916–919, Nov. 2018. (Acceptance: 45%)

(C8)

Kaiyuan Wang, Allison Sullivan, Sarfraz Khurshid. Automated Model Repair for Alloy. In Proceedings of the 33rd International Conference on Automated Software Engineering (ASE 2018), pages 577–588, Sept. 2018. (Acceptance: 20%)

(C7)

Kaiyuan Wang, Allison Sullivan, Darko Marinov, Sarfraz Khurshid. Solver-based Sketching of Alloy Models using Test
Valuations. In Proceedings of the 6th International Conference on Abstract State Machines, Alloy, B, TLA, VDM, and Z (ABZ2018), pages 121–136, June 2018. (Acceptance: 39%)

(C6)

Kaiyuan Wang, Allison Sullivan, Manos Koukoutos, Darko Marinov, Sarfraz Khurshid. Systematic Generation of Nonequivalent Expressions for Relational Algebra. In Proceedings of the 6th International Conference on Abstract State Machines, Alloy, B, TLA, VDM, and Z (ABZ 2018), pages 105–120, June 2018. (Acceptance: 39%)

(C5)

Kaiyuan Wang, Allison Sullivan, Sarfraz Khurshid. MuAlloy: A Mutation Testing Framework for Alloy. In Proceedings of the40th International Conference on Software Engineering Demonstration Track (ICSE Demo 2018), pages 29–32, May 2018.
(Acceptance: 42%)

(C4)

Allison Sullivan, Kaiyuan Wang, Sarfraz Khurshid. AUnit: A Test Automation Tool for Alloy. In Proceedings of the 11th IEEE
International Conference on Software Testing, Verification and Validation Demonstration Track (ICST Demo 2018), pages
398–403, Apr. 2018.

(C3)Allison Sullivan, Kaiyuan Wang, Razieh Nokhbeh Zaeem, Sarfraz Khurshid. Automated Test Generation and Mutation
Testing for Alloy. In Proceedings of the 10th IEEE International Conference on Software Testing, Verification and Validation(ICST 2017), pages 264–275, Mar. 2017. (Acceptance: 27%)

(C2)

Nima Dini, Allison Sullivan, Milos Gligoric, Gregg Rothermel. The Effect of Test Suite Type on Regression Test Selection.
In Proceedings of the 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016), pages 47–58,
Oct. 2016. (Acceptance: 35%)

(C1)

Allison Sullivan, Razieh Nokhbeh Zaeem, Sarfraz Khurshid, Darko Marinov. Towards a Test Automation Framework forAlloy. In Proceedings of the 2014 International Symposium on Model Checking of Software (SPIN 2014), pages 113–116, July.
2014. (Acceptance: 63%)

REFEREED WORKSHOP PUBLICATIONS(W2) Sanyogita Piya and Allison Sullivan. LLM4TDD: Best Practices for Test Driven Development Using Large Language Models. In The First International Workshop on Large Language Models for Code (LLMCODE@ICSE 2024), To Appear.
(W1) Allison Sullivan, Kaiyuan Wang, Sarfraz Khurshid, Darko Marinov. Evaluating state modeling techniques in Alloy. In Proceedings of the 6th Workshop on Software Quality Analysis, Monitoring, Improvement, and Applications (SQAMIA 2017),
pages 16:1–16:9, Sept. 2017.
THESIS AND DISSERTATION(T2)Allison Sullivan. Automated testing and sketching of Alloy models. In Texas ScholarWorks - UT Electronic Theses andDissertations, May 2017.

(T1)

Allison Sullivan. AUnit: A testing framework for Alloy. In Texas ScholarWorks - UT Electronic Theses and Dissertations, May
2014.

UNIVERSITY SERVICE
Faculty Supervisor
4 Ph.D. students, 1 undergraduate student
Passed proposal Ana Jovanovic, Focus: Software Engineering & Formal Methods
Passed compAnahita Samadi, Focus: Software Engineering & Machine Learning
Passed comp
Guanxuan Wu, Focus: Software Engineering, Formal Methods & Machine Learning
New
Mohammad Patwary, Focus: Software Engineering, Formal Methods & Human Computer Interaction
Graduated Students
3 M.S. students, 2 undergraduate student
BS. 2023
Sanyogita Piya, REU Summer 2023 - Fall 2023 – LLM4Code@ICSE 2024 first author paperFirst job: Mouser Electronics
BS. 2023
Adam Emerson, REU Summer 2022 - Spring 2023 – ISSRE 2023 first author paperFirst job: Incoming Ph.D. student at UTA under Dr. Torres
MS 2020
Jasmine Mabrey, Thesis: Automated Defect Classification using Machine Learning
First Job: Sandia National Laboratory
MS 2020
George Thompson, Thesis: Towards Automated Fault Localization for Prolog Models
First Job: Sandia National Laboratory
MS. 2018
Regina Bunch, Project: Improving GUIs for the Autistic Community,
First Job: Cigna
Former REU Students
Spring 2024 Moyses Ledezma Bernal, Project: Hint Generation for Alloy Mathematical Operators
Fall 2022
Cristian Munoz, Project: Generating Core Software Model from Scenarios
Fall 2021
Resha Adhikari, Project: Empirical Study of Program Repair Techniques for Java
Fall 2020
Joelle Banks, Project: Empirical Study of Program Repair Techniques for C
Spring 2020 William Dunston, Project: Fault Classification for CodeHints
Student Committee Member
I have had the honor of serving on the following student committees:
Ph.D.
Arjun Dahal (UTA), Shovon Pereira (UTA), Qiping Wei (UTA), Fadul Sikder (UTA), Krishna Khadka (UTA), Shreyosi
Endow (UTA), Sabrina Haque (UTA), Hanan Alshahr (Fall 2019, NC A&T)
M.S. Thesis
Derrick Leflore (Fall 2019, NC A&T), Shannon Wiggins Brown (Spring 2020, NC A&T), Yuanrui Ren (Fall 2018, The
University of Texas at Austin).
Outreach Activities
Ongoing
Faculty advisor for UTA’s Society of Women Engineers since Aug. 2020
Ongoing
UTA CSE Broadening Participation in Computing Committee Member since Aug. 2020
Ongoing
CAHSI REU Faculty Mentor since Fall 2023
2024
Hosted a workshop on STEM majors at Digital Divas2024
2024
2023
2023
2022
2022
2022
2022
2022
2021
2021
2021
2021
2020
2020
2020
2019
2019
2019

Moderator for the PhD Lightening Talks at SCRF
Hosted a workshop on program synthesis at OurCS@DFW by UTA.
Panelist for “Blazing Bright: Faculty Impact To The Computing” at UTA’s CSE’s 50th anniversary event.
Invited speaker at “Never Work in Theory” industry conference
Escorted a cohort of UTA students to the Grace Hopper Celebration.
Hosted a workshop on Z3 at OurCS@DFW by UTA.
Moderator for the “Career Paths in Computing” panel at SCRF.
Panelist for “The Graduate School Experience” panel at OurCS@DFW.
Hosted a workshop on STEM majors at Digital Divas
Hosted a workshop on Z3 at OurCS@DFW by UTA.
Panelist for the “Career Paths in Computing” panel at SCRF.
Hosted a workshop on STEM majors at Digital Divas
Staffed UTA’s virtual booths at TAPIA and CMD-IT Graduate Fair.
Organizer for the Triad Programming Contest
Faculty Mentor for NC A&T’s SAE International AutoDrive Challenge Student Team (Functional Safety Group)
Coach for NC A&T’s Advancing Minorities’ Interest in Engineering (AMIE) Design Challenge team
Organizer for the Triad Programming Contest
Coach for NC A&T’s 2019 Black Enterprise 2019 BE SMART Hackathon team. *Placed 1st*
Coach for NC A&T’s Advancing Minorities’ Interest in Engineering (AMIE) Design Challenge team

Professional Membership
Ongoing
Society of Women Engineers
Ongoing
Association for Computing Machinery2018 – 2020 NC A&T’s Center of Excellence in Cybersecurity Research, Education and Outreach
Professional Development
20242020202020192019
2018

Affinity Research Groups: Creating and Maintaining Effective Research Groups Course
Attended “Engaging Students in Online Synchronous Class Meetings”
Reviewer for Google’s “Technical Writing One/Two” courses.
Member of the NC A&T team to recruit Facebook’s Engineer in Residence Program.
NC A&T’s CS Representative to Intel’s HBCU Consortium and Intel’s HBCU Beyond 2020 grant.
NC A&T Representative to Facebook T3 “Train the Trainer” summit on Data Structures and Algorithms.

PROFESSIONAL SERVICEConferences
PC Member The 47th International Conference on Software Engineering (ICSE 2025)
PC Member 17th IEEE International Conference on Software Testing, Verification and Validation Testing Tool Track (ICST Demo2024)
PC Member The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering Tools and Demonstration Track (FSE Demo 2024)
PC Member The 12th International Conference on Formal Methods in Software Engineering (FormaliSE 2024)
PC Member The 34th IEEE International Symposium on Software Reliability Engineering (ISSRE 2023) - Fast Abstracts
PC Member The 23rd IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2023)
PC Member The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering Tools and Demonstration Track (FSE Demo 2023)
PC Member The 34th IEEE International Symposium on Software Reliability Engineering (ISSRE 2023)
PC Member 16th IEEE International Conference on Software Testing, Verification and Validation Testing Tool Track (ICST 2023)
Session Chair The 30th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)– ProgramAnalysis I Session
PC Member The 30th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2023)
PC Member The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering Tools and Demonstration Track (FSE Demo 2022)
PC Member The 33rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2022)
Session Chair The 44th International Conference on Software Engineering (ICSE) – Validation and Verification Session
PC Member The 44th International Conference on Software Engineering Tools and Demonstration Track (ICSE Demo 2022)
PC Member The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering Tools and Demonstration Track (FSE Demo 2020)
PC Member The 27th International SPIN Symposium on Model Checking of Software (SPIN 2020)
PC Member The Special Interest Group on Computer Science Education Technical Symposium (SIGCSE 2020)
PC Member The 5th International Conference on Research in Equity and Sustained Participation in Engineering, Computing,and Technology (RESPECT 2020)

Journals
Reviewer
ReviewerReviewer
ReviewerIEEE Transactions on Software Engineering (Impact Factor: 3.33)
ACM Transactions on Software Engineering and Methodology (Impact Factor: 2.674)
Software and Systems Modeling (Impact Factor: 1.910)
Software Tools for Technology Transfer (Impact Factor: 1.09)

Grant Panels
Panelist
Panelist
Panelist
Panelist
Panelist
Panelist

1 panel at the National Science Foundation (NSF), Virtual in 2024
2 panel at the National Science Foundation (NSF), Virtual in 2023
1 panel for the Department of Energy in 2022
1 panel at the National Science Foundation (NSF), Virtual in 2022
1 panel for the Department of Energy in 2021
2 panels at the National Science Foundation (NSF), Alexandria, VA in 2020

OtherSpeakerSpeaker
Co-Reviewer
Co-Reviewer
Co-Reviewer
Co-Reviewer
Co-Reviewer

NSF CISE CAREER Workshop 2024
Never Work in Theory - A Bridge Between Researchers and Practitioners.
The 27th International Symposium on Software Testing and Analysis (ISSTA 2018)
The 11th International Conference on Software Testing, Verification and Validation (ICST 2018)
The 25th International Symposium on the Foundations of Software Engineering (FSE 2017)
The 10th International Conference on Software Testing, Verification and Validation (ICST 2017)
The 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016)

CLASSROOM TEACHING
The University of Texas at Arlington
Spring 2024 CSE 4321: Software Testing & Maintenance, Undergraduate
Fall 2023 CSE 4321: Software Testing & Maintenance, Undergraduate
Fall 2022 CSE 4321: Software Testing & Maintenance, Undergraduate
Spring 2022 CSE 5320/6392: Computer-Aided Verification for Systems, Graduate
Fall 2021 CSE 4321: Software Testing & Maintenance, Undergraduate
Spring 2021 CSE 5320/6392: Computer-Aided Verification for Systems, Graduate
Fall 2020 CSE 5311: Design and Analysis of Algorithms, Graduate
North Carolina A&T State University
Spring 2020 COMP 496: Senior Design, Undergraduate
Spring 2020 COMP 681: Formal Methods, Graduate
Fall 2019 COMP 285: Analysis of Algorithms, Undergraduate
Fall 2019 COMP 611: System Testing and Evaluation, Graduate
Spring 2019 COMP 496: Senior Design, Undergraduate
Spring 2019 COMP 681: Formal Methods, Graduate
Fall 2008 COMP 285: Analysis of Algorithms, Undergraduate