DOI: 10.5281/zenodo.15161371
title: Research Software at the University of Illinois Urbana-Champaign: A Mixed Methods Survey Dataset

description: Description

The research employed a mixed methods online survey to understand better the meaning, use, and development of academic research software at the University of Illinois Urbana-Champaign. Other objectives include understanding academic research software support and training needs to make projects successful at Illinois, as well as investigating the use of generative AI tools in using and creating research software.

 

At the beginning of the survey, all participants gave informed consent. The University of Illinois Urbana-Champaign Institutional Review Board (IRB Protocol no.: Project IRB24-0989) reviewed the study and gave it an exempt determination.

 

Data collection took place from August 2024 to October 2024. Prior to data analysis, identifiable respondent details were removed during the data cleaning process.  Not Applicable and Unsure style responses were used for descriptive statistics, but these responses were excluded for inferential statistics.



Survey design

At the beginning of the online survey, a consent form was provided based on guidelines from the University of Illinois Institutional Review Board to the respondents stating the aims of the study, its benefits and risks, ethical guidelines, being a voluntary survey for participation and withdrawal, privacy and confidentiality, data security, estimated time for survey completion, and contact information of researchers for asking questions. Respondents clicked to indicate their consent. Survey questions were divided into four parts: demographic information, using software for research, creating software for research, and the protocol of citing software for research. The survey had to stop points, whereby not all questions applied to respondents, which led to different sample sizes at the stop points.  At the opening of the survey, the number of respondents was 251 with the funding demographic question being answered by all respondents, while other demographic questions had between 225 and 228 respondents answering them.  For the first stop question, using research software in their research, the total respondents was 212, and at the last stop question, respondents considering themselves to be research developers, the total number of respondents was 74.  The last question of the survey was answered by 71 respondents.  Respondents may also have left the survey for other reasons. The questions were primarily closed-type questions with single choice, multiple choice, or Likert scale, as well as a few open-ended questions. Likert scale responses were created utilizing validated scales from Vagias' (2006) Likert Type Scale Response Anchors.

 

Sampling

Survey Respondents’ Demographics

While most respondents were Tenure Track Faculty (34.7%, f=227), other key categories included Principal Investigator (22.4%, f=227) and Research Scientist (12.1%, f=227). Computer Science, Information Science, Mathematics, and Engineering fields combined for 16% (f=228) of the respondents surveyed, but it should be noted the remaining respondents were from various academic fields across campus from various arts, humanities, and social science fields (25%, f=228) to agriculture (10%, f=228), education (5%, f=228), economics (3%, f=228), medical sciences (4%, f=228), and politics and policy/law (1%, f=228).  Most respondents were likely to receive funding from various government agencies. A more detailed breakdown of the demographic information can be found in the supplemental figures.  Of the 74 respondents who answered whether they were a research software developer, most respondents did not consider themselves a research software developer, with respondents stating Not at All (39%, n=74) and Slightly (22%, n=74).  In addition, open-ended questions asked for further detail about research software titles used in research, research software developer challenges, how generative AI assisted in creating research software, and how research software is preserved (e.g., reproducibility).

 

Table 1:  Survey Respondents’ Demographics

 








Characteristics



Respondent (%)





Age

     18-24

     25-34

     35-44

     45-54

     55-64

     Over 64

     Preferred Not Answer



3%

14%

33%

27%

14%

7%

2%





Gender

     Woman

     Man

     Non-binary / non-conforming

     Prefer not to answer



49%

44%

2%

4%





Race

     Asian

     Black or African American

     Hispanic or Latino

     Middle Eastern or North African (MENA; new)

     White

     Prefer not to answer

     Other



12%

5%

6%

1%

67%

8%

1%





Highest Degree

     Bachelors

     Masters

     Professional degree (e.g., J.D.)

     Doctorate



6%

19%

5%

70%





Professional Title

     Tenure Track Faculty

     Principal Investigator

     Research Scientist

     Staff

     Research Faculty

     Other

     Teaching Faculty

     Postdoc

     Research Assistant

     Research Software Engineer



35%

22%

12%

8%

7%

4%

4%

4%

2%

2%





Academic Field

     Biological Sciences

     Other

     Agriculture

     Engineering

     Psychology

     Earth Sciences

     Physical Sciences

     Education

     Medical & Health Sciences

     Computer Science

     Library

     Chemical Sciences

     Human Society

     Economics

     Information Science

     Environment

     Veterinary

     Mathematical Sciences

     History

     Architecture

     Politics and Policy

     Law



18%

10%

10%

9%

8%

6%

6%

5%

4%3%

3%

3%

3%

3%

2%

2%

2%

2%

1%

1%

1%

0%





Years Since Last Degree

     Less than 1 Year

     1-2 Years

     3-5 Years

     6-9 Years

     10-15 Years

     More than 15 Years



4%

8%

11%

14%

24%

40%





Receive Funding

     Yes

     No



73%

27%





Funders for Research

     Other

     National Science Foundation (NSF)

     United States Department of Agriculture (USDA)

     National Institute of Health (NIH)

     Department of Energy (DOE)

     Department of Defense (DOD)

     Environmental Protection Agency (EPA)

     National Aeronautics and Space Administration (NASA)

    Bill and Melinda Gates Foundation

    Advanced Research Projects Agency -  Energy (ARPA-E)

   Institute of Education Sciences

   Alfred P. Sloan Foundation

   W.M. Keck Foundation

   Simons Foundation

   Gordon and Betty Moore Foundation

   Department of Justice (DOJ)

   National Endowment for the Humanities (NEH)

   Congressionally Directed Medical Research Programs (CDMRP)

   Andrew W. Mellon Foundation



22%

18%

18%

11%

9%

5%

4%

4%

2%

2%

1%

1%

1%

1%

1%

1%

0%

0%

0%






Table 2:  Survey Codebook








QuestionID



Variable



Variable Label



Survey Item



Response Options





1



age



Respondent’s Age



Section Header:

Demographics Thank you for your participation in this survey today! Before you begin to answer questions about academic research software, please answer a few demographic questions to better contextualize your responses to other survey questions.


What is your age?


Select one choice.

Years



1-Under 18

2-18-24

3-25-34

4-35-44

5-45-54

6-55-64

7-Over 64

8-Prefer not to answer





2



gender



Respondent’s Gender



What is your gender?


Select one choice.



1-Female

2-Male

3-Transgender

4-Non-binary / non-conforming

5-Prefer not to answer

6-Other:





3



race



Respondent’s Race



What is your race?


Select one choice.



1-American Indian or Alaska Native

2-Asian

3-Black or African American

4-Hispanic or Latino

5-Middle Eastern or North African (MENA; new)

6-Native Hawaiian or Pacific Islander

7-White

8-Prefer not to answer

9-Other:





4



highest_degree



Respondent’s Highest Degree



What is the highest degree you have completed?


Select one choice.



1-None

2-High school

3-Associate

4-Bachelor's

5-Master's

6-Professional degree (e.g., J.D.)

7-Doctorate

8-Other:





5



professional_title



Respondent’s Professional Title



What is your professional title?


Select all that apply.



1-professional_title___1

Principal Investigator

2-professional_title___2

Tenure Track Faculty

3-professional_title___3

Teaching Faculty

4-professional_title___4

Research Faculty

5-professional_title___5

Research Scientist

6-professional_title___6

Research Software Engineer

7-professional_title___7

Staff

8-professional_title___8

Postdoc

9-professional_title___9

Research Assistant

10-professional_title___10

Other:





6



academic_field



Respondent’s most strongly identified Academic Field



What is the academic field or discipline you most strongly identify with (e.g., Psychology, Computer Science)?


Select one choice.



1-Chemical sciences

2-Biological sciences

3-Medical & health sciences

4-Physical sciences

5-Mathematical sciences

6-Earth sciences

7-Agriculture

8-Veterinary

9-Environment

10-Psychology

11-Law

12-Philosophy

13-Economics

14-Human society

15-Journalism

16-Library

17-Education

18-Art & Design Management

19-Engineering

20-Language

21-History

22-Politics and policy

23-Architecture

24-Computer Science

25-Information science

26-Other:





7



years_since_last_degree



Number of years since last respondent’s last degree



How many years since the award of your last completed degree?


Select one choice.



1-Less than 1 year

2-1-2 years

3-3-5 years

4-6-9 years

5-10-15 years

6-More than 15 years





8



receive_funding_for_research



Whether respondent received funding for research



Do you receive funding for your research?



1-Yes

0-No





9



funders_for_research



Respondent’s funding sources if they answered yes in Question 8



Who funds your research or work (e.g., NIH, Gates Foundation)?


Select all that apply.



1-funders_for_research___1

United States Department of Agriculture (USDA)

2-funders_for_research___2

Department of Energy (DOE)

3-funders_for_research___3

National Science Foundation(NSF)

4-funders_for_research___4

National Aeronautics and Space Administration (NASA)

5-funders_for_research___5

National Institutes of Health(NIH)

6-funders_for_research___6

Department of Justice (DOJ)

7-funders_for_research___7

Environmental Protection Agency (EPA)

8-funders_for_research___8

Department of Defense(DOD)

9-funders_for_research___9

Institute of EducationSciences

10-funders_for_research___10

Congressionally DirectedMedical Research Programs(CDMRP)

11-funders_for_research___11

Advanced Research ProjectsAgency - Energy (ARPA-E)

12-funders_for_research___12

National Endowment for theArts (NEA)

13-funders_for_research___13

National Endowment for theHumanities (NEH)

14-funders_for_research___14

Social Science ResearchCouncil (SSRC)

15-funders_for_research___15

Alfred P. Sloan Foundation

16-funders_for_research___16

Gordon and Betty MooreFoundation

17-funders_for_research___17

Andrew W. MellonFoundation

18-funders_for_research___18

Bill & Melinda GatesFoundation

19-funders_for_research___19

Simons Foundation

20-funders_for_research___20

W. M. Keck Foundation

21-funders_for_research___21

Other:





10



how_software_is_used



How respondent uses software in research



Section Header:

Section 1: Research Software Definition


How do you use software in your research?


Select all that apply.



1-how_software_is_used___1

I do not use software in my research

2-how_software_is_used___2

To automate part of the research process

3-how_software_is_used___3

To collect data

4-how_software_is_used___4

To generate data

5-how_software_is_used___5

To simulate/model something

6-how_software_is_used___6

To clean/organize data

7-how_software_is_used___7

To analyze data

8-how_software_is_used___8

To present research results(e.g., visualize data)

9-how_software_is_used___9

Other:





11



use_research_software



Respondent Uses Research Software Based Upon Definition Provided



"Research Software includes source code files, algorithms, scripts,computational workflows and executables that were created during the research process or for a research purpose. Software components(e.g., operating systems, libraries, dependencies, packages, scripts,etc.) that are used for research but were not created during or with a clear research intent should be considered software in research and not Research Software. This differentiation may vary between disciplines. The minimal requirement for achieving computational reproducibility is that all the computational components (ResearchSoftware, software used in research, documentation and hardware)used during the research are identified, described, and made accessible to the extent that is possible."

With this definition in mind, do you use research software?



1-Yes

0-No

2-Unsure


Stop actions on

0





12



software_importanc



Research Software Importance



How important is research software to your research?


Select one choice.



1-Not at all, 2-Slightly, 3-Somewhat, 4-Moderately, 5-Extremely, 

6-Not Applicable





13



research_roles



Respondent’s Role in Research



Which of the following roles apply to you?


Select all that apply.



1-research_roles___1

I conduct my own research

2-research_roles___2

I support someone else's research

3-research_roles___3

I manage/supervise researchers or research projects

4-research_roles___4

I develop software for researchers

5-research_roles___5

Other:





14



research_software_titles



Research Software Used by Respondents in Research



Please provide the name(s) of the main research software you use(up to 5).

Separated by comma.



Open-ended Response





15



gui



Respondent’s use of graphical user interface in research



How often is software that has a graphical user interface (GUI)critical to successfully complete your research/engineering/etc.project?


"A graphical user interface (GUI) is a digital interface in which a user interacts with

graphical components such as icons, buttons, and menus. In a GUI, the visuals displayed

in the user interface convey information relevant to the user, as well as actions that they can take." (Juliver, 2023)


Select one choice.



1-Never

2-Rarely

3-Sometimes

4-Often

5-Always





16



operating_system_used



Operating System Used by Respondent for Research



What operating system(s) do you use for your research?


Select one or more of the following responses.


If other, type the operating system(s).



1-operating_system_used___1

Microsoft Windows

2-operating_system_used___2

Mac OS X

3-operating_system_used___3

Linux

4-operating_system_used___4

Other:





17



if_no_research_software



How would respondent complete research without research software



What would happen if you could no longer use research software?


Select one choice.



1-I would no longer be able to conduct my research

2-My research would require more effort, but it would still be possible

3-It would make no significant difference to my research





18



used_gen_ai_tool



Does respondent use GenAI in Research



Have you used a generative AI tool for your research?


Select one choice.



1-Yes

0-No





19



how_genai_used_in_research



Ways in which Respondent uses GenAI in Research



How have you used generative AI tools in your research?

Select all that apply.



1-how_genai_used_in_research___1

To automate part of the research process

2-how_genai_used_in_research___2

To collect data

3-how_genai_used_in_research___3

To analyze data

4-how_genai_used_in_research___4

To clean/organize data

5-how_genai_used_in_research___5

To visualize data

6-how_genai_used_in_research___6

To improve text

7-how_genai_used_in_research___7

To write new text

8-how_genai_used_in_research___8

To write code

9-

how_genai_used_in_research___9

To determine research questions

10-how_genai_used_in_research___10

To gather relevant information

11-how_genai_used_in_research___11

Other:





20



how_to_locate_software_used



Ways in which Respondent locates research software



How do you locate software to use for your research?

Select all that apply.



1-how_to_locate_software_used___1

Conversations with collaborators

2-how_to_locate_software_used___2

Reading the scholarly literature

3-how_to_locate_software_used___3

Using a search engine (e.g. Google)

4-how_to_locate_software_used___4

Using social media(e.g. Twitter/X)

5-how_to_locate_software_used___5

Asking a generativeAI tool

6-how_to_locate_software_used___6

Searching online communities (e.g.Stack Overfl ow)

7-how_to_locate_software_used___7

Searching code hosting sites (e.g.GitHub)

8-how_to_locate_software_used___8

Writing it myself

9-how_to_locate_software_used___9

Other:

10-how_to_locate_software_used___10

Not applicable





21



contribute_to_exist_software



Whether respondent contributes to existing software



Do you contribute to existing software (e.g., by reporting bugs at least)?


Select one choice.



1-Yes

0-No





22



cost_software_development



Whether respondent included software development costs in research proposal



Have you included costs for software development in a funding proposal?


Select all that apply.



1-cost_software_development___1

Yes

2-cost_software_development___2

No (I'm not involved in writing funding proposals)

3-cost_software_development___3

No (but we DID plan to write software as part of the project)

4-cost_software_development___4

No (we did NOT plan to write software as part of the project)





23



work_on_creating_software



Whether respondent creates software for research



Do you currently work on a project that involves the creation or maintenance of research software?


Select one choice.



1-Yes

0-No


Stop actions on

0





24



software_develop_knowledge



Respondent’s knowledge of research software development



How knowledgeable are you about research software development?


Select one choice.



1 - Not at all

2 - Slightly

3 - Somewhat

4 - Moderately

5 - Extremely





25



software_develop_training



Whether respondent received software development training



Have you received any training in software development?


Select one choice.



1-Yes - just through taught courses

2-Yes - just through self-taught training

3-Yes - through both taught courses and self-taught training

4-No





26



amt_software_develop_train



Respondent’s amount of software development training 



How much training have you received to develop reliable software?

Select one choice.



 1-None, 

2-Almost none, 

3-A small amount, 4-A moderate amount, 

5-A great deal





27



version_control_familar



Respondent’s familiarity with version control



Section Header:

How familiar are you with the following software development technologies?

Version Control



1 - Not at all

2 - Slightly

3 - Somewhat

4 - Moderately

5 - Extremely

6 - Not Applicable





28



cont_integration_familiar



Respondent’s familiarity with continuous integration



Continuous Integration



1 - Not at all

2 - Slightly

3 - Somewhat

4 - Moderately

5 - Extremely

6 - Not Applicable





29



version_control_use



Respondent’s frequency of version control use in research



Section Header:

How often do you use the following software development technologies? 


Select one choice.


Version Control



1-Never, 

2-Rarely, 3-Sometimes, 4-Often, 

5-Always


Show the field ONLY if:

[version_control_familar] = '2' or [version_control_familar] = '3' or [version_control_familar] = '4' or [version_control_familar] = '5'





30



cont_integration_use



Respondent’s frequency of continuous integration use in research



Continuous Integration



1 - Never

2 - Rarely

3 - Sometimes

4 - Often

5 - Always


Show the field ONLY if:

[cont_integration_familiar] = '2' or [cont_integration_familiar] = '3' or [cont_integration_familiar] = '4' or [cont_integration_familiar] = '5'





31



r_software_engineer_title



Whether respondent’s professional title is software engineer



Do you use the title research software engineer to define yourself?


Select one choice.



1-Yes

0-No





32



use_univ_support_services



Whether respondent has used university support services for research software development



Have you used university support services for your software development needs?


Select one choice.



1-Yes

0-No

2-Unsure





33



rate_univ_support_services



Respondent’s rating of university support services for research software development



How would you rate the university's current level of support for your software development needs?


Select one choice.



1-Poor, 

2-Fair, 

3-Good, 

4-Very Good, 5-Excellent


Show the field ONLY if:

[use_univ_support_services] = '1' or[use_univ_support_services] = '2'





34



software_develop_challenges



Whether respondent has had software development challenges



Have you had any specific challenges with your software development needs?



1-Yes

0-No





35



soft_develop_challenges_text



Respondent's specific software development challenges



What specific challenges have you had with university support for your software development needs?



Open-ended response





36



use_gen_ai_in_soft_develop



Whether respondent has used GenAI in research software development



Have you used generative AI in your software development?


Select one choice.



1-Yes

0-No





37



how_used_genai_in_soft_develop



Respondent’s use of GenAI in research software development



How has generative AI software assisted you in creating software?



Open-ended response





38



cont_use_gen_ai_in_soft_develop



Whether respondent still uses GenAI in research software development



Do you plan to continue using generative AI in your software development?


Select one choice.



1-Yes

0-No





39



hired_software_developer



Whether respondent has hired a software developer for research project



Have you (or someone in your group) ever hired someone specifically to develop software?


Select one choice.



1-Yes

0-No





40



preserve_research_software_text



How Respondent preserves research software



How do you preserve your research software (e.g., for reproducibility)?



Open-ended response





41



sustain_research_software



Whether respondent sustains research software based upon definition provided



The concept of 'software sustainability' means that the software continues to be available to its intended users for as long as it is likely to be useful.


With this definition in mind, do you plan to sustain your research software?


Select one choice.



1-Yes

0-No





42



how_will_sustain_research_soft

 


How do you plan to sustain your research software?


Select multiple choices.



1-how_will_sustain_research_soft___1

Grants that explicitly maintain the software

2-how_will_sustain_research_soft___2

Grants for research where some effort also will maintain the software

3-how_will_sustain_research_soft___3

Internal funding/effort

4-how_will_sustain_research_soft___4

Make the software open source

5-how_will_sustain_research_soft___5

Commercialize the software

6-how_will_sustain_research_soft___6

Other





43



research_soft_cont_in_hiring



Whether respondent believes research software contribution is valued in university hiring



Do you agree that research software contributions are valued in hiring at the University of Illinois?


Select one choice.



1 - Strongly disagree

2 - Disagree

3 - Neither agree or disagree

4 - Agree

5 - Strongly agree





44



research_soft_cont_in_promotion



Whether respondent believes research software contribution is valued in university promotions



Do you agree that research software contributions are valued in promotion at the University of Illinois?


Select one choice.



1- Strongly disagree

2 - Disagree

3 - Neither agree or disagree

4 - Agree

5 - Strongly agree





45



explicit_how_to_cite_software



Respondent explicitly states how to cite research software



Section Header:

Section 3: Sharing Your Code and Software


Do you make it explicit how to cite your software?


Select one choice.



1-Yes

0-No





46



cite_software_in_research_results



Respondent explicitly cites research software produced in research



Do you cite the software you use to produce your research results?


Select one choice.



1-Yes

0-No