NIST Project Final Report

1

Fostering Standards Literacy in General Education at an R1 Institution 

NIST Grant # - APP142637 70NANB15H339_0 CD-450

Final Summary Paper

Prepared by Gabe Ording (PI) and Claudia Vergara (CISGS Assessment Coordinator)

Project Description

Standardization processes guide businesses and organizations in implementing practices that are 

sustainable for the environment, the economy and society. They affect global challenges such as 

managing water and food resources equitably, while maintaining safety and efficiency. The 

overarching goal for this project was to incorporate standards literacy into general science 

courses at Michigan State University (MSU), allowing for increased awareness and 

understanding of standards and standardization processes and their significance in the social 

context. This work placed an emphasis on helping non-stem majors develop an enhanced degree 

of general “standards literacy”. An emphasis was placed on the way that STEM disciplines 

(science, technology, engineering and mathematics) are implicated in the development of 

standards that range from manufacturing and process control to regulation and program 

evaluation, while also stressing the way that standards development, implementation and 

compliance are shaped by contingent factors of the social context. We refer to our approach as 

“liberal arts standards education,” to emphasize its relevance to all college graduates, irrespective 

of their academic major or their projected career and future work life. To achieve this objective, 

we developed standards literacy modules in two laboratory courses in the Center for Integrative 

Studies in General Science (CISGS). The target courses offered an attractive opportunity, as 

more than one half of MSU undergraduates move through this course structure. 

A. Goals and Objectives

The goals for the project team were to develop and implement standards education instructional 

modules and hands on investigations in two laboratory courses in CISGS. These included an 

Integrative Studies in Biology (ISB) and an Integrative Studies in Physical Science (ISP) lab 

program. These lab programs are components of the University undergraduate requirements. 

The collective goal of these lab programs is to help enhance the scientific literacy of non-STEM

undergraduates. The development and implementation of technical standards was an ideal 

mechanism to help students better understand the role of science in society. 

Specific objectives:

1) Curricular Development: We developed instructional modules for each of the two target 

laboratories, aligned to the course objectives. Specifically, we want our students to 

understand:

●What a technical standard is.

●The process by which standards are created, implemented and regulated.

●The implications (economic, social, political, etc.) of standardization.

●The role of science and the scientific process in the development of standards
NIST Project Final Report

2

2) Curricular Implementation: We delivered the standard instructional modules in the 

targeted laboratory courses during Summer and Fall 2016. There were a total of 50

sections of these courses delivered during the implementation semester, ultimately 

impacting a total of over 1200 students. Critical aspects of the implementation included:

●Collection of standards instructional materials related to water and food safety.

●Training and supporting teams of graduate students from a wide range of disciplines 

who directly deliver these lab courses to students.

B. Project Evaluation

The Center for Integrative Studies in General Science assessment team conducts the assessment 

and evaluation of the project. The evaluation objective was to determine the impact of the 

program on the participating students. Specifically, to assess students’ learning gains aligned to 

the curricular objectives described in section A.1 of this report. Data sources include surveys 

(pre- and post-) and student artifacts including homework, pre-and post-lab assignments and 

other standard-related assignments. 

The objective of the survey was to better understand students’ perceptions, ideas and knowledge 

about standards and standardization processes and the role that they play in the social context 

and their everyday lives. The survey was administered in written format, at the start of the 

courses (pre-survey), before the students had had any instructional engagement with standards; 

and at the end of the courses (post-survey), after the students have been engaged in activities 

designed to increase their exposure and awareness to issues related to standards literacy.

We collected samples of student artifacts including homework, pre-and post-lab reports and 

recorded debates that were a part role playing activities that took place in the context of classes. 

The objective for this data stream was to compare between students’ self-reported survey data 

and actual students work. Triangulation between these different data sources allowed us to have 

a better understanding about the learning gains related to standards literacy.

Project Implementation

C. Curricular Development

New lab activities were developed and implemented in the context of an 

Integrative Studies in Biology (ISB) lab program and also in an Integrative Studies in Physical 

(ISP) Sciences lab program, each with the purpose of introducing students to the concept of 

technical standards and the role of science in the development and implementation of those 

standards. These case study labs additionally exposed students to the concept of technical standards 

from start to finish; including how and why they are proposed, developed, and implemented.
NIST Project Final Report

3

1. In the context of the ISB lab program, the lab activity was related to 

standardization in food production. Students conducted a hands-on investigation 

of the number of aphids allowable in whole hops, which is a key ingredient in the 

making of beer. The industry standard, according to the U.S. Food and Drug 

Administration’s Food Defect Action Level Handbook (Insects; AOAC 967.23), 

is that ‘there cannot be on average more than 2,500 aphids per 10 grams of whole 

hops’. Students were given packages of whole hops and microscopic equipment, 

and were required to identify and record the total number of aphids per gram of 

hops. This allowed student groups to determine whether or not the industry 

standard had been met. Additional objectives for this lab were to compare brands 

of hops grown in the United States versus Great Britain, and to compare 

differences between certified organic versus non-organic hops. This lab included 

a follow-up discussion and student led debate on whether the current industry 

standard for whole hops is acceptable.

2. In the context of the ISP lab program, students investigated the use of road 

salt (NaCl) and its potential ecological impacts. This is a real-life situation that is of local 

importance, especially in many states of the Midwest. Given the evidence in the 

literature that chloride concentrations often exceed the EPAs chronic 7-day standard, by 

significant amounts and durations, there is value in experimenting to determine 

appropriate standards for road salt application. Students used a bioassay of various 

chloride concentrations on the germination and radicle growth of a test organism (Butter 

crunch lettuce) to inform their proposed standard. Through stakeholder role-playing and 

balancing their own concerns about water quality, students were asked to determine and 

propose a standard to be put forward, this based on their own scientific evidence. This 

was a valuable exercise that helped to highlight the challenges associated with the 

determination of technical standards of any kind.

D. Data Collection and Analyses 

Survey: Students enrolled in ISB 208L in Summer 2016 (N=18) and Fall 2016 (N=681); 

and students enrolled in ISP 203L in Summer 2016 (N=23) and Fall 2016 (N=508) were 

asked to complete a written survey during class time. The instrument included open and 

close ended items (Appendix 1). The table below presents a summary of the survey data 

collection.
NIST Project Final Report

4

Table 1. Survey Data Collection Summary

Summer 2016 Fall 2016

ISB 208L ISP 203L ISB 208L ISP 203L

Students Enrolled 18 23 681 508

Pre-survey responses 

(response rate %) NA 23 (100%) 588 (86%) 473 (93%)

Post-survey responses 

(response rate %) NA 21 (91%) 486 (71%) 464 (91%)

Given that the surveys were administered using written responses we decided to analyze a 

random sample (n=60) of the responses for both of the target courses in Fall 2016. These 

responses included only students that completed both the pre- and the post-surveys:

● Pre- scheme: Picked a set of 30 respondents starting in the pool (all sections) of presurvey responses and for those respondents we found the corresponding post survey 

responses.

● Post- scheme: Picked a second set of 30 respondents starting in the pool (all sections) of 

post-survey responses and for those respondents we found the corresponding pre-survey 

responses.

This scheme gave us a total sample of 60 responses for students that completed both the pre- and 

the post- survey questionnaires.

Student artifacts: The objective for this data stream was to compare between students’ selfreported survey data and actual student work. We collected student artifacts related to the 

standards modules including homework and pre-and post-lab reports; students were also engaged 

on in-class debates related to determining an appropriate standard for salt application (appendix 

2). We recorded the group debates, transcribed them verbatim and analyzed the transcriptions.

Project Results

E. Survey Results:

Students were asked to report their level of confidence related to several aspects of standard 

literacy. We used a Likert 5-point agree/disagree scale (1- Strongly disagree; 2- Somewhat 

disagree; 3- Undecided; 4- Somewhat agree; 5- Strongly agree). Our data indicate a slight 

change in the mean values for the responses from undecided (3) in the pre-survey to somewhat 

agree (4) in the post survey. This trend holds for both courses in both semesters. Figure 1 shows 

the mean values for the responses for the ISP203L FS16 course, and Figure 2 shows the mean 

values for the responses for the ISB208L FS16 course.
NIST Project Final Report

5

Figure 1. Participants’ self-reported confidence about different aspects of standard literacy N=60

Figure 2. Participants’ self-reported confidence about different aspects of standard literacy N=60

...what a standard is and providing an example

...the role(s) of STEM in the development of Standards

...other factors (i.e. economics, societal history, political

lobbying) influence in the development of Standards

...and predicting the effects (economics, environmental,

social, political, etc.) of the implementation of Standards

...how Standards are created, implemented and regulated

3.950

3.847

4.034

3.983

3.931

2.915

2.780

2.625

2.596

2.579

Mean Values

ISP203L FS16 - Students' self-reported confidence in explaining...

Pre Post

...what a standard is and providing an example

...the role(s) of STEM in the development of Standards

...other factors (i.e. economics, societal history, political

lobbying) influence in the development of Standards

...and predicting the effects (economics, environmental,

social, political, etc.) of the implementation of Standards

...how Standards are created, implemented and regulated

4.400

4.133

3.933

4.085

3.900

2.500

2.467

2.433

2.373

2.500

Mean Values

ISB208L FS16 - Students' self-reported confidence in explaining...

Pre Post
NIST Project Final Report

6

The primary coding structure for the open-ended items was based on the project objectives. In 

particular we focused on aspects related to student’s awareness and knowledge about the 

development of standards, and their significance in the social context. Using iterative coding we 

characterized important aspects of the instructional experience (standards modules) and the 

impact it had on standards literacy in General Education courses. Table 2 shows the categories 

and the criteria used to include a response in a given category (rules of inclusion). Table 3 

summarizes the results from the analyses by providing exemplar responses for each of the coding 

categories.

Table 2. Coding Categories and Inclusion Criteria

Coding Categories Rules of Inclusion

Manufacturing (M) Responses related to manufacturing.

Process (P) Responses related to the standards creation process

Regulation (R) Responses related to methods for monitoring or enforcing standards

Evaluation (E) Responses related to checking and assessing the success of the standard

General Knowledge (G) Responses that indicate a general understanding of standards

Lobbying (L) Responses related to political lobbying

Stakeholders (S) Responses related to stakeholders (influence, opinions, stakes, interests) on standards

Economics (EC) Responses related to money or economic implications

STEM Connection (SC) Responses related to connections between science and standards

Other (O) Responses unrelated to the question

Table 3. Coding Exemplars

Response Quote Coding Category*

“[...] standards are proposed, thoroughly debated, 

and then have to be approved before they can be 

implemented by specific fields. They are maintained 

by consequences being enforced for not following 

them.”

Process, Regulation

“The idea behind standards is that they make the 

world a better, safer place. [...] some standards are 

set to profit companies with no regard to 

environmental health. [...] For instance, a new 

standard limiting the amount of plastic bags that 

can be produced per year would lose plastic 

manufacturers a lot of money but would hopefully 

help the environment.” 

General, Economic, 

Stakeholders

“Standards are created through research and testing, 

they are implemented by the government passing 

legislation enacting their use, and they are regulated 

by government agencies”

Process, Scientific connection, Regulation

*Please note that a given response can be assigned more than one code. The data is color-coded 

to indicate the alignment between the codes and the response.

Gains in Standard Literacy: Our analyses comparing the pre- and post- survey responses to the 

open-ended questions indicate that after participating in the standard modules students in both 

courses provided more elaborated and complete responses. For example, students were asked: 
NIST Project Final Report

7

“What role(s) do STEM disciplines play in the development of Standards? Use a specific 

example to illustrate.” For the pre-survey, 73% of the responses fell in the general or other 

categories indicating that the responses did not provide specific standard-related information and 

only conveyed general information (G) or that the response was unrelated to the question (O). 

For these same students, only 42% of the post-survey responses were assigned to the general 

category (Figure 3). The charts in Figures 3 and 4 summarize the gains in standard literacy.

Table 4 further illustrates the gains in standard literacy by providing exemplar responses from 

pre- and post-surveys. 

Figure 3. Comparison between students’ pre- and post-survey responses to the question: What 

role(s) do STEM disciplines play in the development of Standards? Use a specific example to 

illustrate.”

0

10

20

30

40

50

60

70

Pre

Post

Pre

Post

Pre

Post

FS16 ISB208L

FS16 ISP 203L

US16 ISB203L

% Responses 

FS16 ISB208L FS16 ISP 203L US16 ISB203L

Pre Post Pre Post Pre Post

General 68 42 58 25 23 8

Other 5 0 18 5 0 0

Gains in Standard Literacy

General Other
NIST Project Final Report

8

Figure 4. Comparison between students’ pre- and post-survey responses to the question: 

“Explain how Standards are created, implemented and regulated.”

Table 4. A representation of exemplar responses from pre- and post-surveys. 

What role(s) do STEM disciplines play in the development of Standards? Use a specific example to illustrate.

Pre-survey responses (codes) Post-survey responses (codes)

“[STEM] all play very specific roles in the 

development of standards, unfortunately, I do not 

know these key roles but hope to learn them in the 

near future” (O)

“[STEM] all play a vital role in the development of standards. 

With technology and science, that is how a standard is 

developed. Engineering comes into play within the firm who 

[sic] develops the standard. The firms use their research, data, 

and technology to develop these standards.” (P, SC)

“[STEM] enhance standards by proving things in 

experiments.” (G)

“[STEM] play a huge role in the development of standards. 

With science we can see the negative effects of too much road 

salt so in the future there will most likely be a standard of how 

much can be used.” (SC)

“Scientists play a role in standards because they 

are the ones sticking to the standards and 

technology helps them log all data.” (G)

“The role that science plays in placing standards is that 

scientists help research and provide ways that will help our 

environment out in keeping safe and efficient and this research 

helps create standards that will in turn save our envrionment 

[sic] and people. An example [sic] would be that scientists did 

research to see how much pollution we are putting into the air 

and provided us with evidence and how there are standards in 

place to stop us and manufactuers [sic] from creating so much 

pollution.” (SC, R)

0

10

20

30

40

50

60

Pre

Post

Pre

Post

Pre

Post

FS16 ISB208L

FS16 ISP 203L

US16 ISB203L

% Responses

FS16 ISB208L FS16 ISP 203L US16 ISB203L

Pre Post Pre Post Pre Post

General 53.12 21.58 44.82 28.22 11.62 4.98

Other 8.3 1.66 18.26 6.64 3.32 1.66

Gains in Standard Literacy

General Other
NIST Project Final Report

9

Explain how Standards are created, implemented and regulated

“They are created by higher officials to set rules 

and are implemented through the people that set 

them.” (G)

“Standards are created by officials and implemented through 

guidelines. For example, the USDA regulates food labeling of 

organic foods.” (R)

“Through each individual field. Historical 

references. Government, laws, media.” (G)

“Through this class I have learned that standards are proposed, 

thoroughly [sic] debated, and then have to be approved before 

they can be implemented by specific fields. They are 

maintained by consequences being enforced for not following 

them.” (P, S, R)

“Standards are created through policy.” (G) “They are heavily debated between all sides to ensure a 

compromise and agreed to all affected industries then have to 

implement them.” (S, P)

F. Student Artifacts Analyses

In this section we present the results from analyzing student artifacts and the triangulation 

between survey results and artifact analyses.

Our results suggest that students’ participation in the standard instructional modules contributed 

to increased standard literacy. We analyzed laboratory reports (artifacts) where students were 

asked to choose a standard, explain why they chose it, and how it would affect different 

stakeholders. The analyses indicated that students were able to integrate the scientific data that 

they generated during their laboratory work and applied it to their responses both in the postsurvey and their class assignments (laboratory reports, debates).

Table 5 shows an example of triangulation between two data sources, surveys (pre- and post) and 

student artifacts (laboratory report). Both examples exhibit increased awareness related to the 

role of STEM disciplines on standard development. This is evidenced by the complexity of the 

responses in the post survey compared to the pre-survey responses; students specifically referred 

to the salt concentration experiment that they did as part of their laboratory work, (underlined

text on the table).

Table 5. Exemplar for triangulation between data streams: Survey responses and lab reports

Survey responses (Q: STEM disciplines influence on standard 

development) Student Artifacts

Pre- Post- Lab. report entries**

“They [STEM] are the means 

to understanding the needs of 

standards and what they 

should be.” (G)*

“All of the above [STEM] are 

required to conduct research 

and experiments necessary to 

understand what needs to go 

into creating a standard. Ex: 

The amount of salt used in 

winter needs to be measured 

in order to know what levels 

“[...] Overall I believe that if 

enough evidence was 

produced, we would end up 

[with a standard] between 

230 ppm and 500 ppm 

because while it is low, it 

seems to be the peak before 

the environment is more 

drastically harmed, where 
NIST Project Final Report

10

effect the environment.” P, 

SC)

companies will still make a 

profit and be able to abide by 

the laws that would protect 

the planet that they need in 

order to own their 

companies.”

“Science , technology, and 

engineering help provide the 

facts which determine a 

standard.” (G)

Science is very important 

when setting standards. Most 

of the time, we need to use 

logic and tests when deciding 

what is "normal". For 

instance, to return for a 

moment to our road salt 

discussion on Monday, 

science must be used in order 

to find out a.) how harmful 

road salt is to the 

environment, and b.) how 

necessary it is to drivers 

safety, in order to find a 

reasonable balance and set a 

safe standard.” (P, S, SC)

“Ultimately, we determined 

that the standard for sodium 

chloride should be 1000ppm. 

The decision came form each 

group agreeing to 

compromise for the “greater 

good” [...] This will be a 

good standard because while 

road salt will still be used in 

extreme weather situations, it 

will not be permitted to be 

used excessively.”

* The letter in parenthesis indicates the code assigned to the response. Refer to table 2 for coding 

categories and inclusion criteria.

** The objective for this lab report was to define a standard for the application of road salt on the 

MSU campus considering various stakeholders perspectives.

Summary of the project’s suitability and potential for adoption in other educational 

organizations

Standards do in fact provide an ideal mechanism to help students engage in integrative learning. 

Standards as an embedded component of an undergraduate curriculum can help to foster 

scientific literacy, while at the same time engaging students in issues of real-world social 

consequence. Engaging students in the full process of standard development and 

implementation, and consideration of the diverse stakeholder perspectives, results in students 

developing enhanced analytical thinking, evidence based argumentation, and effective 

communication skills.

As is the case with any laboratory investigation, activities and case studies must be carefully 

selected and field tested. They must be engaging for students, but also consistently replicable. 

For example, the ISB exercise in which students investigated Food Defect Action Levels, Aphids, 

Hops and Standards was an extremely engaging topic for students. Students were keen to 

discuss and manufacturing process of beer, and intrigued by the notion that insects were an 

allowable element of common foods consumed by the public. The topic and the activity itself 
NIST Project Final Report

11

were highly engaging. The success of this activity however, was dependent on whether the hops 

purchased actually contained aphids. During the pilot of this activity, we did find aphids (by

looking at raw hops under a dissecting microscope) in packaged raw hops and were able to draw 

conclusions on whether a specific industry standard had been met (and calculate how many 

aphids, on average, would have been in a single can of beer). However, on two subsequent 

attempts, the same source of packaged raw hops produced zero aphids. On the upside, this 

hands-on activity was engaging and students were eager to find and calculate the number of 

aphids per beer. On the down-side, in order to produce consistent results, we were forced to

manipulate the packaged hops by adding aphids collected from the teaching greenhouses on 

campus. While this was not the preferred method by which to carry out this activity, 

manipulating the results did ensure that every student was able to identify and enumerate aphids 

from a sample of raw hops to complete the lab. If attempting to use this module, it would be 

important to be sure to pre-test the sources of packaged hops or (better) acquire prepackaged 

hops from a local supplier. 

The ISP exercise in which students are asked to propose an EPA standard for the use of road salt 

was comparably engaging for students as a real-world issue of local importance. Students are 

consistently engaging in hypothesis development and data collection and analysis on a weekly 

basis in this lab program, but the standards stakeholder roleplaying exercise made the end results 

come alive with vibrant discussion in attempts to reach a consensus on the chloride standard. 

Every student was assigned a particular stakeholder role and was tasked to represent the 

stakeholder’s interests while also taking into account other stakeholder interests as well as the 

scientific evidence from the bioassay. This activity enabled students to see the relevance of 

science and scientific data to solve real world problems. Additionally, it required students to 

incorporate multiple viewpoints and values in their decision making as well as communicating 

those decisions. The use of this technical standard activity in this lab program has enhanced 

student participation, the perceived relevance of science. This lab has become a permanent 

element of the ISP lab program and has been implemented every semester since its initial 

development.

Given that we have contextualized standards literacy as a component of general education at a 

major university, there is a need to determine whether it is helping our students to achieve our 

targeted learning outcomes. Assessment and evaluation is an essential component of 

determining the success of these curricular reform efforts. Assessment techniques and 

instruments must however be crafted and implemented in such a way as to collect valuable and 

useable data. It is important to complement open ended survey data with other data collection 

efforts such as analyses of student work to be able to determine the nature of the learning gains 

related to standards literacy. Based on this in our current project we have added qualitative data 

streams such as focus groups and are building upon the artifact analyses from this initial project

to include more deliberate assessments of student learning gains. A challenging component to 

make our assessment more scalable is the data collection platform; it is important to use on-line 

options such as Qualtrics. We have incorporated this model into our new NIST project.