1 

 

Evaluation Report: Impact of 

Northstar Assessment and Related 

Computer Skills Programming on 

Employment in CTEP Programs 

A CONSULTING GROUP OF THE MINNESOTA LITERACY COUNCIL 

11/23/2015 

Prepared by Daniel Backman, MDP 

Minnesota Literacy Council 
 

 

 

 

 

 

 

 

 

Evaluation Report: Northstar Assessment and Adult Learner Employment 

Outcomes 

Purpose of the study

The Community Technology Empowerment Project (CTEP) sought to determine the effect that its program, and 

specifically the use of Northstar Digital Literacy Assessments and related computer skills instruction, had on the 

employability of adult learners, and then contrast these results with a comparison group from the Minnesota 

Department of Employment and Economic Development. CTEP contracted with the Minnesota Literacy Council 

to design a study to evaluate whether participation in Northstar Assessments and related computer skills 

instruction (i.e., the "program") improved job placement rates for adult learners. 

CTEP has three years of outcome data indicating that its program led to job placement for unemployed 

participants, but until now there had been no third-party evaluation of the program. 

CTEP functions as a key program of its parent organization, Saint Paul Neighborhood Network (SPNN). Founded 

in 1984 as a nonprofit community media training center, SPNN works through community partnerships with 

nonprofits and governmental entities to increase digital and media literacy and to better educate the 

community. SPNN’s existing partnerships with many social service organizations provide a strong training 

network for CTEP members. 

Summary of Findings 

In total, 208 surveys were administered to adult learners who participated in Northstar-related basic computer 

skills classes at one of 28 unique CTEP host site locations, out of a total of 30 CTEP host sites. This study found 

that there was a noteworthy increase of 14.9 percentage points in the employment rate among the adult 

learners who were surveyed. Also notable is that among adult learners that were unemployed upon initial 

attendance in Northstar-related basic computer skills classes and reported that they were looking for work, 50 

percent had gained employment at the time of survey administration, compared to 29.9 percent of all 

unemployed adult learners in the survey. In addition to employment information, the study looked at reasons 

for attending these classes and the top three reasons why adult learners attended Northstar-related basic 

computer skills classes were reported as 1) for personal improvement (non-employment related), 2) help with 

job search (e.g. cover letter, resume, online applications) and 3) improve skills in Microsoft Word. 

As part of this study, the researchers obtained an employment sector comparison group from six Twin Cities 

metro-wide workforce center for the same time period of this study. Approximately 41 percent (339/818) of 

people who received at least 4 hours of assistance at a WorkForce Center in Hennepin or Ramsey County 

between July 1, 2014 and March 31, 2015 found employment 90 days after their last date of service. This 

compares to the 50 percent change in employment that was found for the comparable group of participants in 

this study of CTEP program participants. 

Technical summary is available on page five. 

1 
 

 

 

 

 

 

Background Information on CTEP and Northstar Digital Standards and Assessment 

Each year, CTEP trains around 35 AmeriCorps members to teach computer literacy skills intended to improve job 

placement and retention for low-income, low-skilled residents in the Twin Cities metropolitan area. Members 

provide training at no charge to adult learners at more than thirty community-based locations including 

libraries, public housing, social service agencies, workforce centers, youth employment programs, and disability 

inclusion organizations. Members use assessment tools and computer skills curriculum based on Northstar 

Digital Literacy Standards that were designed and developed by the St. Paul Public Library in 2012 with state and 

federal funding. Since then, these standards have been adopted as statewide digital literacy standards 

by Minnesota Adult Basic Education, and put into use in more than 225 Northstar public computer access 

locations in 27 states nationwide. 

In 2010, the St. Paul Public Library and the St. Paul Community Literacy Consortium began a community process 

to determine how best to assess and quantify digital literacy knowledge among lower-skilled adults, as well as 

for displaced workers who might lack such skills. The intent was to provide meaningful assessment that could 

lead to a certificate useful for employers and job seekers. Through an open community process, the Northstar 

Digital Literacy Standards were designed over a period of several months by a taskforce with representatives 

from non-profit community-based agencies, public and academic libraries, Minnesota Department of Education, 

DEED and workforce centers, Adult Basic Education professionals, and others. 

In 2011, the Friends of the St. Paul Public Library obtained funding through the Otto Bremer Foundation and the 

Library Services and Technology Act, from the Institute of Museum and Library Services, administered by the 

Minnesota Department of Education, State Library Services Division. These funds supported the hiring of a 

professional design team to create the online assessments. The Design Team included Jen Vanek, Project 

Liaison, Michael J. Graif, Digital Literacy Designer, and Jennifer Asp, Educational Specialist, under the 

coordination of Tom Cytron-Hysom, Project Manager. A Community Advisory Group, organized by St. Paul 

Library Director Kit Hadley, provided feedback and assistance to the Design Team. A CTEP Community 

Engagement Project assisted in designing online training for proctors at approved sponsor sites. 

In late-2011 to mid-2012, online assessment modules were designed, piloted, and implemented. A process 

through which organizations could be certified to award the Northstar Certificates was developed. A database 

and other technical infrastructure were completed. 

CTEP adopted the Northstar Digital Literacy Standards as a way to establish common criteria amongst all CTEP 

sites for what constitutes learning computer literacy skills. The standards were created by skilled consultants 

with funding from Otto Bremer Foundation and Friends of the St. Paul Public Library. All consultants had 

advanced degrees in adult basic education and professional experience creating online learning modules. 

Beginning in September 2012, all CTEP members began proctoring pre- and post- assessments around the five 

Northstar Standards: Basic Computer Use, Internet, Operating Systems, Email, and Word Processing. 

All of the assessments can be accessed at: https://www.digitalliteracyassessment.org/. 

2 
 

 

 

 

 

 

Each assessment takes roughly 30-45 minutes for a participant to complete and they are free and available on 

the internet. Each assessment has about 40 questions, and for an individual to be counted as making progress, 

they would need to score less than certification level of 85% correct on the pre-assessment, and score above the 

certification level on the post-assessment. CTEP provides a standardized curriculum connected to each standard. 

Members can teach participants skills above and beyond this list, but at least one of these standards must be 

incorporated into programming. 

The Northstar Digital Literacy online assessments are designed to be an economical means by which digital 

literacy can be evaluated for purposes of aiding instruction and for conferring recognition of digital literacy. The 

assessment tool was designed to assess behaviors in the form of simulated tasks that digitally literate people can 

be reasonably expected to be able to perform, knowledge of technology information in the form of structured 

questions that digitally literate people can be reasonably expected to answer correctly, and attitudes about 

appropriate use of digital technology in the form of structured questions in which technologically literate people 

can be reasonably expected to identify appropriate use. The assessment tool was tested for question validity by 

piloting beta versions both in groups thought not to be digitally literate and groups that are believed to be 

digitally literate. Questions in each group were identical. Results were analyzed to determine questions which 

may not be valid assessments of digital literacy within the context of these results. The assessment tool can be 

said to be reliable as it does not rely on independent observers potentially introducing observer bias but instead 

programmatically assesses user responses according to predetermined values in a consistent way. The tool was 

tested technically to ensure accurate and intended performance and thus can be expected to produce reliable 

results across use. 

Each year in the first month of service for all CTEP members, the educational consultants from the St. Paul 

Community Literacy Consortium, who created the Northstar Standards and Assessments, lead training for CTEP 

members on the Northstar standards, assessments, and technology curriculum implementation best practices. 

Study Design

CTEP contracted with the Minnesota Literacy Council to design and conduct this study. Daniel Backman, 

Strategic Initiatives Manager at the literacy council, was the lead evaluator on this project. Mr. Backman is a 

graduate of the Humphrey Schools of Public Affairs with a Master’s Degree in Development Practice (MDP) and 

holds a B.A. in Sociology from Iowa State University. The MDP degree program trains students in program 

design, research protocol, program evaluation and statistical analysis. Mr. Backman is the lead evaluator of 

internal programs at the literacy council and is the lead evaluator on multiple external evaluation contracts with 

public and nonprofit partners. 

This evaluation was designed as a quasi-experimental study with a comparison group. The evaluator developed 

the study and survey tool in conjunction with the CTEP senior staff Joel Krogstad and Lisa Peterson-de la Cueva. 

The survey instrument was designed to be administered by trained CTEP members. CTEP members were utilized 

to deliver the survey because each member and their host site has different methods and data practices for 

recording student data, including contact information, class attendance records and Northstar Assessment 

results. A committee of CTEP members were consulted on the design of the survey instrument and provided 

3 
 

 

 

 

 

 

edits at multiple stages of the survey instrument design. All CTEP members were trained in the use of the survey 

instrument on February 6, 2015. 

The survey instrument developed for this study contained a total of 23 questions. The first ten questions of each 

survey are filled out by CTEP members based on their records and data of the adult learner who was to be 

interviewed with the survey instrument. Demographic questions were intentionally eliminated to reduce length 

of the survey and CTEP senior staff did not intend to analyze results based on demographic data, except for age. 

The questionnaires were developed in English and translated into Hmong. 92.6% of the questionnaire 

respondents were interviewed in English and 7.4% were interviewed in Hmong. 

In order to create a comparison group, the Department of Employment and Economic Development (DEED) was 

contacted to assemble comparison employment data for participants who were served across six metro-wide 

WorkForce Centers. To meet the same criteria of a “4 hour minimum of service,” only those who attended 3 or 

more workshops, or who spent 240 or more minutes on a resource room computer, were included. "Service in a 

WorkForce Center” referred to those who either attended a workshop or used a resource area computer. 

Finally, only those WorkForce Centers in Hennepin and Ramsey County were included, which is the same 

geographic region where CTEP members serve. Next, the group was further filtered on whether they first 

received services between July 1, 2014 and March 31, 2015. The employment status of this group of job seekers 

was then analyzed, and only those who were unemployed during the quarter of first receiving services were 

retained. The employment status 90 days after the last date of service was then analyzed for this refined group 

to produce the final employment statistic. 

Methodology 

Research Questions 

1.	 From a representative sample of adult learners in CTEP programs, we sought to determine the 

employment rate of learners upon entering CTEP programs and the employment rate of learners after 

attending at least four hours of computer skills programming in a CTEP program. 

2.	 From a representative sample of adult learners in CTEP programs, we sought to determine the purpose 

of learners’ attendance in CTEP basic computer skills programs. 

3.	 From a representative sample of adult learners in CTEP programs, we sought to determine the 

employers and job types of adult learners who have attended at least four hours of computer skills 

programming in a CTEP program. 

4.	 From the Department of Employment and Economic Development, we sought to determine how the 

CTEP employment result compares to a metro wide employment comparison group. 

5.	 From a representative sample of adult learners in the CTEP program, we sought to determine the Return 

on Investment for the value of all new jobs received. 

A full copy of the survey instrument is located in the Appendix. 

Adult learners responding to the questionnaire were screened for eligibility by a CTEP Member based on the 

criteria below. CTEP staff established these criteria to conform to the average intervention. 

4 
 

 

 

 

 

 

 

In order to be eligible for survey, the adult learner: 

 Must have attended at least four hours of Northstar-related computer programming, 

 Must have passed at least one Northstar Assessment, 

 Must have passed their first Northstar Assessment at least four weeks prior to administration of survey 

(only assessments used as post-tests qualify), 

 Must have attended Northstar-related computer programming after May 26, 2014 

 Must be older than the age of 16 at the start of class 

Of the 4941 adult learners participating in CTEP's programming in its 2015 fiscal year (August 2014 to July 2015), 

1611 adults passed Northstar Assessments. 208 participants (n = 208) passed the screening and were 

administered questionnaires. 

CTEP senior staff worked with each CTEP AmeriCorps member to establish screening protocols, administered 

screening, schedule questionnaire interviews, and ensure effective data collection and input. All data was input 

into Survey Monkey to facilitate analysis. 

Questionnaires were created on paper and in Survey Monkey by the evaluator. CTEP AmeriCorps Members were 

trained in a one-day orientation on February 6, 2015 on consistent methods to question, record, and input 

questionnaire data. Members interviewed participants and entered responses directly into Survey Monkey, 

using a link provided by the evaluator. Between March 2 and July 7 (2015), twenty-eight CTEP members 

completed 208 surveys of adult learners who participated in computer skills programming at twenty-eight 

unique site locations. This date range was chosen to correspond to the last few months of the CTEP service year, 

which allowed enough time for the CTEP AmeriCorps Member to have an adequate number of adult learners 

who have passed the Northstar Assessment in their program year. 

CTEP sites were not able to equally provide survey data because some sites such as the public libraries have 

privacy restrictions that do not make it easy to contact participants in the future. Also, some sites did not 

administer Northstar Assessments during the survey time period. Therefore, different members had different 

quotas of how many surveys they would administer. 

Technical Summary 

In total, 208 surveys were administered to adult learners who participated in Northstar-related computer 

programming at one of 28 unique CTEP host site locations, out of a total of 30 CTEP host sites, which resulted in 

93% coverage of CTEP host sites in the survey results. 

CTEP reported that the total number of adult learners who passed at least one Northstar Assessment in the 

program year (August 2014-July 2015) was 1611. At a confidence interval of 95%, the sample size of 208 results 

has a margin of error of 6.34%. 

5 
 

 

 

 

 

 

 

 

 

 

 

 

Overall, the majority of the adult learner population surveyed reported that they were unemployed (61.9%, 

127/205) at initial attendance in basic computer skills class. Among all survey respondents, both employed and 

unemployed, over half reported that they were currently looking for work (53%, 111/208)1

. Among only those 

who reported being unemployed at initial attendance in basic computer skills classes, just over half (53.5%, 

68/127) reported looking for working since first attending basic computer skills class. Overall, the employment 

rate changed for all survey respondents from 38% (78/205) at time of first class attendance to 52.9% (108/205) 

at the date of survey administration. 

Of the 111 survey respondents who reported looking for employment, over 80% (89/111) of adult learners 

reported that attending computer skills classes helped them in looking for employment. Of all survey 

respondents who reported that they were employed at the date of survey (108/208), the average annualized 

salary was $32,466.88 based on their reported employer and job title2

. Meanwhile, of only survey respondents 

who transitioned from unemployment to employment over the course of the survey time period, their 

annualized average salary was $31,226.61. The three most common sectors of employment for all currently 

employed survey respondents were: office and administrative support, sales and related occupations and 

personal care and service occupations. 

Of the 127 survey respondents unemployed when starting computer skills programs, 68 survey respondents, or 

53.5% (68/127), reported that they have looked for work since starting a computer skills program. Of those 68 

survey respondents who have looked for work since they began program attendance, 34, or 50% (34/68), 

reported that they currently have a job (at the date of survey administration). Of those 34 survey respondents 

who reported current employment, 24 survey respondents, or 70.6% (24/34), reported that that attending 

computer class helped in getting offered their job. 

To compare the employment results of adult learners in surveyed CTEP programs, the researchers utilized 

employment data from six Twin Cities WorkForce Centers, which operate under the Department of Employment 

and Economic Development. DEED reported that 41 percent (339/818) of people who received an intervention 

of similar time duration (at least four hours of assistance) found employment 90 days after their last date of 

service during the same time period. This compares to 50 percent employment rate of comparable participants 

in the surveyed CTEP programs of this study. 

1 The discrepancy between the total amount surveyed (205 vs. 208) is that three survey respondents skipped the question which asked about 

their state of employment at the first attendance in class. The total number of 208 and 205 are used respectively where appropriate. 

2 Average annualized salary was obtained by comparing survey respondents’ reported job title and employer with corresponding job data 

from the Bureau of Labor Statistics for the Minneapolis/Saint Paul/Bloomington metropolitan area. 

http://www.bls.gov/oes/current/oes_33460.htm 

6 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Return on Investment 

To report on the return on investment of the Community Technology Empowerment Project for the program 

year 2014-2015, the following formula is proposed: 

Formula: 1611 x 16.3% x $31,226.61 x 70.6% x (+/-) 6.34% = ROI (+/-) 

Total Return on Investment: $5,789,121.78 (+/- $367,030.32) 

	 1611: Total number of adult learners that passed at least one Northstar assessment in CTEP programs 

	 16.3%: The percentage of sample who reported unemployment at initial class attendance (127/208) 

AND who reported that they looked for a new job since initial class attendance (68/208) AND who 

reported current employment at date of survey administration (34/208). 

	 $31,226.61: Average annualized salary of those who transitioned from unemployed to employed (based 

off sector and job title provided by survey respondents, salary data used from Bureau of Labor Statistics 

within the Minneapolis-St.Paul-Bloomington metropolitan statistical area. 

	 70.6%: The percentage of those who reported that they looked for jobs, went from unemployed to 

employed AND who reported that attending CTEP computer skills programming helped them look for 

jobs 

	 6.34%: The margin of error of survey sample size based on total CTEP population who passed Northstar 

assessments 

From the survey results, 16.3% of the adult learner population sample who passed at least one Northstar 

Assessment reported that they had looked for work since first attending the basic computer skills class and also 

transitioned from unemployment to employment. We can reasonably apply this percentage to the total 

population of adult learners across all CTEP programs (1611) who passed at least one Northstar Assessment to 

be representative of the percentage people in all CTEP programs that will transition from unemployment to 

employment if they are job-seeking. The average annualized salary of those who transitioned from unemployed 

to employed in the sample is $31,226.61. Of those who transitioned from job-seeking survey respondents who 

transitioned from unemployment to employment, 70.6% reported that attending CTEP computer skills 

programming helped them look for jobs. Thus, we can apply 16.3% to 1611, to assume that 262.6 adult learners 

from the CTEP adult learner population also transitioned from unemployment to employment if they were jobseeking. Of those 262.6 adult learners, we can assume that 70.6%, or roughly 185.4, of those would report that 

attending CTEP computer skills programming helped them look for jobs. Thus, you can take the 185.4 adult 

learners multiplied by the annualized average salary ($31,226.61) of the adult learner population sample who 

transitioned from unemployment to employment, for a total estimated economic impact of $5,789,121.78. The 

sample has a margin of error of 6.34%, which needs to be applied to final number. 

All data is provided in a spreadsheet titled Master Northstar Evaluation Data for verification purposes. 

Demographics of Adult Learners 

To preserve their anonymity, the 208 participants surveyed were not asked to report their race, ethnicity or 

primary language spoken. The questionnaire did capture age (see Table 1 in appendix) and language in which 

7 
 

 

 

 

 

 

 

 

 

 

 

 

the questionnaire interview was conducted (see Table 2 in appendix) to gather basic demographic data without 

compromising anonymity. 

The majority of respondents (124, or 59.9%) were between the ages of 25-54. The next two largest groups of 

respondents were in the age groups of 55-64 (39, or 18.8%) and 16-19 (22, or 10.6%), as represented in Table 1 

in the appendix. These age ranges were chosen as they are standard age ranges used by the Bureau of Labor 

Statistics3

. The vast majority of surveys were administered in English, with 7.4% administered in Hmong. 

Adult Learners, Northstar Assessments and Computer Skills Programming 

The average amount of Northstar-related programming attended by survey respondents was 22.9 hours, though 

the median amount of hours attended was 12 hours (see Table 3 in the appendix). This implies that the average 

number of hours is skewed higher by some outliers who have attended many hours of programming. The 

highest number of hours that a survey respondent reported attending Northstar-related programming was 160 

hours, and the most common reported amount of time spent attending programming was 10 hours. 

The average amount of days between the date of first attendance in Northstar-related programming and 

passing their first Northstar Assessment was 32.18 days, though the median amount of days between first 

attendance and passing an assessment was nine days (see Table 4 in the appendix). Thus, over half of all survey 

respondents have passed an assessment in less than nine days of first attending class, and on average, survey 

respondents will pass their first assessment in just over one month. 

The most common assessment passed by survey respondents was Basic Computer Skills (63.2%), followed by 

World Wide Web (43.1%), MS Word (35.8%) and Email (30.4%). On average, survey respondents passed 2.28 

assessments each (see Table 5 in the appendix). 

The majority (65.7%) of all survey respondents did receive certification for passing a Northstar Assessment in the 

form of a printed certificate (see Table 6 in the appendix). The Mozilla badges were only given to two survey 

respondents, indicating a lack of interest in adult learners, a lack of communication or marketing on part of the 

CTEPs, CTEP coordinators or the Northstar team or a system that has not yet been fully implemented. Overall, 

there are a significant number of survey respondents (34.3%) who opted not to receive any certification, which 

indicates that there may be a lack of interest in obtaining this type of credential in certain populations (see Table 

6 in the appendix). 

The top three reasons for attending Northstar-related programming are for personal improvement (42.9%), help 

with job search (41.9%) and to improve skills in MS Word (37.4%) (See Table 7 in the appendix). 

For those who indicated that they attended Northstar-related programming for educational advancement (64 

survey respondents), the two most common responses were that they applied for or are attending community, 

tech or vocational school or attending or have attended a more advanced class within ABE (Adult Basic 

Education) (See Table 8 in the appendix.) 

3 http://www.bls.gov/cps/demographics.htm#age 

8 
 

 

 

 

 

 

Employment 

The majority (62%, 127/205) of survey respondents were not employed when they began attending Northstarrelated programming (see Table 9 in the appendix). Over 54% of respondents reported that they have looked for 

work or a different job after they began Northstar-related programming (see Table 10 in the appendix). Of those 

survey respondents who noted that they are looking for work, the vast majority (80.14%, 89/111) indicated that 

going to computer class helped them look for jobs (see Table 11 in the appendix). However, only a small amount 

(34%, 36/106) of the same group indicated that they discussed the Northstar assessment or certifications in 

their job interview, on their resume or on their job application (see Table 12 in the appendix). This discrepancy 

could indicate that a larger emphasis needs to be placed on communicating to adult learners that this 

certification is valuable to employers and should be utilized as such. 

The employment rate of survey respondents improved markedly between the first date of attendance in 

Northstar-related programming and the date of survey. Employment rate changed for all survey respondents 

from 38% pre-class to 52.9% at the time of survey administration (see Table 9 and Table 13 in the appendix). Of 

the 108 survey respondents who indicated that they currently have a job, 35% (or 38 survey respondents) were 

not employed prior to beginning Northstar-related programming. Of these 38 survey respondents, 68.4% (or 26 

survey respondents) indicated that they think attending computer class helped them in getting offered their job. 

Within the same group of 38, 73.7% (or 28 survey respondents) indicated that their current job is an 

improvement over their most recent job. 

Of those employed at the time of survey, the average length of employment in their current job was just under 

three years, or 34.58 months (see Table 14 in the appendix). However, the median length of employment was 

seven months, indicating that many survey respondents were relatively new to their job. Supporting this, the 

most frequent length of employment was one month. Overall, this indicates that Northstar-related computer 

skills programming serves a wide range of adult learners in relation to employment status, with many adult 

learners looking for work or in new jobs but also long-term employees looking to learn or refresh their computer 

skills. 

Of those who reported current employment, 93 survey respondents provided their employer name and position 

title. The evaluator downloaded economic and employment data from the Bureau of Labor Statistics (BLS, May 

2014) for the Minneapolis-Saint Paul-Bloomington metropolitan area. The evaluator matched each survey 

respondent’s employment position and employer with the corresponding BLS job sector and job category, as 

accurately as possible. This data is available in an attached spreadsheet to this report. Each job category had 

mean and median salary data, as well as a defined job sector. 

Overall, the average wage for all combined survey respondents is $15.62 per hour, with a median wage of 

$15.08 per hour. The average wage annualized results in a mean average salary of $32,466 (can be seen in 

attached Excel file title Master Data). 71% (71) of survey respondents reported that they are making more 

money (51%) or the same amount of money (20%) (see Table 17 in appendix). The most common job sectors for 

survey respondents are Office and Administrative Support Occupations (e.g. receptionist, administrative 

assistant), Sales and Related Occupations (e.g. retail salesperson, cashiers), Personal Care and Service 

9 
 

Occupations (e.g. personal care aide, nursing assistants) and Food Preparation and Serving Related Occupations 

(e.g. waiters and waitresses, restaurant cook). 

END OF ANALYSIS 

10 
 

 

 

 

APPENDIX 

DATA TABLES. 

Table 1. Age of Questionnaire Respondents 

Survey Question #2: What is the age of this learner? 

Answer Options 

AGES 16-19 

AGES 20-24 

AGES 25-54 

AGES 55-64 

AGES 65+ 

Response 

Percent 

10.6% 

6.3% 

59.9% 

18.8% 

4.3% 

answered question 

skipped question 

Response 

Count 

22 

13 

124 

39 

9 

207 

1 

Table 2. Language of Survey Administration 

Survey Question #3: Will this survey be administered in a language other than English? 

Answer Options Response 

Percent 

Response 

Count 

Yes (Hmong) 7.4% 15 

No (English) 92.6% 189 

answered question 204 

skipped question 4 

Table 3. Hours of Attendance 

Survey Question #6: How many approximate hours of Northstar-related 

programming did this learner attend? Must be over four hours. 

Answer Options Response Count 

201 

answered question 201 

skipped question 7 

Median: 12 hours 

Average: 22.9 hours 

Mode: 10 hours 

Range 4 - 160 hours 

11 
 

 

 

Table 4. Time in program before passing first assessment 

Survey Question #7-8: Number of days in between the date of first 

attendance and the date of passing first assessment. 

Answer Options Response Percent Response Count 

Date / Time 203 

answered question 203 

skipped question 5 

Mean: 32.18 days 

Median: 9 days 

Mode: 4 days 

Range: 0-345 days 

Table 5. Type of Northstar Assessment passed 

Survey Question #9: Which Northstar assessments did the learner pass? 

Answer Options Response 

Percent 

Basic Computer Skills 63.2% 

World Wide Web 43.1% 

MS Word 35.8% 

Email 30.4% 

Windows 21.1% 

Excel 20.6% 

Social Media 8.3% 

Max OS X 5.4% 

answered question 

skipped question 

Average number of assessments completed per survey respondent: 

Response 

Count 

129 

88 

73 

62 

43 

42 

17 

11 

204 

4 

2.28 

Table 6. Type of certificate the learner received 

Survey Question #10: Did they receive certification for any passed assessment? If so, please 

specify. Choose all that apply. 

Answer Options Response 

Percent 

Response 

Count 

Mozilla badge 1.0% 2 

Printed certificate 65.7% 132 

None 34.3% 69 

answered question 201 

skipped question 7 

12 
 

 

 

Table 7. Reason for program attendance 

Survey Question #11: Why did you attend the computer class (Northstar-related)? Choose all that apply. 

Answer Options Response Percent 

For personal improvement (non-employment related) 42.9% 

Help with job search (e.g. cover letter, resume, online 

applications) 41.9% 

Improve skills in MS Word 37.4% 

Educational advancement (e.g. GED, training to work 

program, other training program, other classes) 31.5% 

Improve skills in Excel 31.5% 

Improve skills in using the internet 30.5% 

Improve skills in using email 25.6% 

Improve typing speed 20.7% 

Improve skills for current job 20.2% 

Other (please specify) 15.0% 

MFIP or other welfare program participant 12.3% 

Improve skills in obtaining citizenship 2.0% 

Response 

Count 

87 

85 

76 

64 

64 

62 

52 

42 

41 

15 

25 

4 

Table 8. How did attending computer skills classes improve the learner’s education? 

Survey question #12: How did attending computer skills classes improve your education? Please choose all that apply. 

Responses only include survey respondents who indicated Educational Advancement in survey question #11. 

Answer Options Response Percent Response 

Count 

Applied or attending community/tech/vocational school 31.1% 19 

Attending or have attended a more advanced class within 

Adult Basic Education 29.5% 18 

None of the above 23.0% 14 

Attending or have attended a more advanced workforce 

training system 19.7% 12 

Attending or have attended GED classes 14.8% 9 

Applied or attending four-year college or university 6.6% 4 

Obtained a GED 1.6% 1 

Other (please specify) 8 

answered question 61 

skipped question 147 

Table 9. Learner employment at first attendance in class 

Survey question #13: When you started computer class, did you have a job? 

Answer Options Response 

Percent 

Yes 38.0% 

No 62.0% 

Other (please specify) 

answered question 

skipped question 

Response 

Count 

78 

127 

3 

205 

3 

13 
 

 

 

 

 

Table 10. Recent job search rate among adult learners 

Survey question #14: Since you attended computer class, have you looked for work or a 

different job? 

Answer Options Response 

Percent 

Response 

Count 

Yes 54.1% 111 

No 45.9% 94 

Other (please specify) 3 

answered question 205 

skipped question 3 

Table 11. Of adult learners who have looked for jobs, did computer class 

help? 

Survey question #15: Did going to computer class help you look for jobs? Responses 

included only from those who responded that they have looked for work or a different job 

Answer Options Response 

Percent 

Response 

Count 

Yes 82.4% 89 

No 17.6% 19 

Other (please specify) 3 

answered question 108 

skipped question 100 

Table 12. Did adult learner discuss Northstar Assessments in their job 

application, interview or resume? 

Survey question #16. Did you talk about the Northstar Assessment in your job application, 

interview or resume? Responses only included from those who responded that they have 

looked for work or a different job. 

Answer Options Response 

Percent 

Response 

Count 

Yes 34.0% 36 

No 66.0% 70 

Other (please specify) 7 

answered question 106 

skipped question 102 

Table 13. Is the adult learner currently employed? 

Survey question #17: Do you have a job now? 

Answer Options Response 

Percent 

Yes 52.9% 

No 47.1% 

Other (please specify) 

answered question 

skipped question 

Response 

Count 

108 

96 

6 

204 

4 

14 
 

 

 

 

 

Table 14. Time spent in current job 

Survey question #20: How long have you worked at this job? Answer in Months and/or 

Years. 

Answer Options Response Count 

100 

answered question 100 

skipped question 108 

Mean 34.58 months 

Median 7 months 

Mode 1 months 

Range 0-444 months 

Table 15. Did attending computer class assist adult learners in obtaining 

employment? 

Survey question #21: Do you think attending computer class helped you in getting offered 

your job? 

Answer Options Response 

Percent 

Response 

Count 

Yes 41.0% 41 

No 59.0% 59 

answered question 100 

skipped question 108 

Table 16. Improvement in quality perception of current employment 

compared to previous employment 

Survey question #22: Is your current job an improvement over your last job? 

Answer Options Response 

Percent 

Response 

Count 

Yes 56.6% 56 

No 7.1% 7 

Same/No change 30.3% 30 

Not sure 6.1% 6 

answered question 99 

skipped question 109 

Table 17. Earning improvement of current employment compared to 

previous employment 

Survey question #23: Are you making more money in your current job than your last job? 

Answer Options Response 

Percent 

Response 

Count 

Yes 51.0% 51 

No 20.0% 20 

Same/No change 27.0% 27 

Not sure 2.0% 2 

answered question 100 

skipped question 108 

15 
 

 

Table 18. CTEP Host Sites and number of adult learners 

surveyed at each respective site 

Survey Question #4: At what organization did this learner attend Northstarrelated programming? 

Answer Options Response Count 

203 

answered question 203 

skipped question 5 

Project For Pride In Living 37 

Hubbs Center 17 

IOCP 17 

Central Library 14 

LEAP High School/YMCA Twin Cities 12 

Neighborhood House 11 

Hmong American Partnership-Arcade 10 

Emerge 8 

Hmong American Partnership, University 7 

Resource EAC 7 

Rice Street Library 7 

Saint Paul Public Library 7 

Adult Education Center Columbia Heights/Fridley 6 

MERC Alternative High School 5 

Minneapolis North WorkForce Center 5 

CLUES-MPLS and Andersen School 4 

CTV North Suburbs 3 

George Latimer Central Library 3 

St. Paul Public Housing 3 

Waite House 3 

Arlington Hills Community Center (AHCC) 9 

CLUES 2 

Rondo Library 2 

St Paul Central Library 2 

EAC-N 1 

Highland Park and AHCC 1 

St. Paul Regional Water Services 1 

16 
__________________________________ 

COPY OF SURVEY INSTRUMENT. 


 

17 

SURVEY: Northstar Assessment Adult Learner Employment Outcomes 

Name of adult learner taking this survey:

Date of survey:
 

Questions #1-10 (indicated by CTEP logo) should be completed by CTEP member prior to the interview with the adult learner. If 

the answer is unknown to the CTEP member, please verify with adult learner during the interview. Questions that are indented 

and have an , are only to be answered based on a response to a previous question. 

1. Name of CTEP member filling out this form. 

______________________________________________________________________________ 

2. What is the age of this learner? 

o	 AGES 16-19 ○ AGES 20-24 

o	 AGES 25-54 ○ AGES 55-64 

o	 AGES 65+ 

3. Will this survey be administered in a language other than English? o 

Yes 

o	 No 

4. At what organization did this learner attend Northstar-related programming? 

______________________________________________________________________________ 

5. What is the name of the Northstar-related programming this learner attended? Please list all that apply. 

______________________________________________________________________________ 

6.	 How many approximate hours of Northstar-related programming did the learner attend? Must be over four hours. 

_______________________________________________________________________________ 

7. On what approximate date did the learner BEGIN Northstar-related programming? 

__/__/__ 

8.	 On what approximate date did the learner pass their first Northstar Assessment? Must be at least four weeks prior to the 

date of this survey administration. Pre-assessments do not qualify. 

__/__/__ 

9. Which Northstar assessments did the learner pass? Please check all that apply. 

□		 Basic computer skills □ Windows 

□		 Mac OS X □ MS Word 

□		 World Wide Web □ Social Media 

□		 Email □ Excel 

10. Did they receive certification for any passed assessment? If so, please specify. Choose all that apply. 

□		 Mozilla Badge 

□		 Printed certificate 

□		 None 

Please begin interview with adult learner now. All questions beginning at Question #11 should be asked directly to the adult learner. 

11. Why did you attend the computer class? Please choose all that apply. 

□		 Educational advancement (GED, training to work program, other classes) □ Improve typing speed 

□		 Help with job search (e.g. cover letter, resume, online applications) □ Improve skills in using the internet 

□		 Impr ve skills for curre o nt job □ Improve skills in Excel 

□		 Improve skills in obtaining citizenship □ Improve skills in Word 

□		 MFIP or other welfare program participant □ Improve skills in using email 

□		 For personal improvement (non-employment-related) □ Other (please specify) ________________ 
 

 

 

 

 

 

 

 

 

 

 

 

 

____________________________________________________________________________________ 

 

____________________________________________________________________________________ 

 

____________________________________________________________________________________ 

 

 

 

 

If ‘Educational advancement’ was selected in question #10, please answer #11. If not, skip to question #12. 

12. How did attending computer skills classes improve your education? Please choose all that apply. 

□ Attending or have attended a more advanced class within Adult Basic Education (ABE) 

□ Attending or have attended a workforce training system 

□ Applied or attending community/tech/vocational school 

□ Applied or attending four-year college or university 

□ Attending or have attended GED classes 

□ Obtained a GED 

□ None of the above 

□ Other (please specify) _____________________________________________________ 

13. When you started computer class, did you have a job? 

o Yes 

o No 

14. Since you attended computer class, have you looked for work or a different job? 

o Yes 

o No 

If YES to #14, answer #15 and #16. If NO, please skip to #17. 

15. Did going to computer class help you look for
 

jobs? o Yes
 

o No 

16. Did you talk about the Northstar Assessment in your job application, interview or 

resume? o Yes 

o No 

17. Do you have a job now? 

o Yes 

o No 

If the learner responded NO to question #17, END SURVEY. If they responded YES, please continue to question #18 and finish the survey. 

18. Where do you work? 

19. What is your job title? 

20. How long have you worked at this job? Answer in months and/or years. 

21. Do you think attending computer class helped you in getting offered your job? 

o Yes 

o No 

22. Is your current job an improvement over your last job? 

o Yes ○ Same/No change 

o No ○ Not sure 

23. Are you making more money in your current job than your last job? 

o Yes ○ Same/No change 

o No ○ Not sure 

END SURVEY. Please enter results into the SurveyMonkey tool as soon as possible. https://www.surveymonkey.com/r/northstarevaluation 

18