**Docket:** ED
-2014
- ICCD
- 0071

2015 National Household Education Survey (NHES 2015) Full Scale Data Collection **Comment On:** ED
-2014
- IC CD
- 0071
- 0005 **Document:** ED
-2014
- IC CD
- 0071
- 0008 Comment on FR Doc # 2014
- 15935 **COMMENT** I'm happy to see that there is a second ch ance for those of us who hoped to comment on the first docket due in July but let it fall through on our summer to
- do lists! I hope I'm not the only academic sitting on the sidelines eagerly waiting for data from this survey, because this is a truly import ant project that will fill a massive gap in our understanding of the extent and effectiveness of this country's educational and credentialing system If successful, it will be a gold mine for sociologists and other social scientists studying a range of topi cs, from membership associations to educational achievement to inequality. However, I worry that the proposed CWS might be a missed opportunity to learn more about the characteristics of certifications themselves. The draft instrument is a very good start, but we need all the data we can get. NHES is not a great survey for learning about the demographic and labor market correlates of certification (and licensure) because it lacks the extensive and well
- developed core items found in such surveys as the SIPP and ELS. Perhaps, then, it makes sense to emphasize NHES as a means of getting a broad picture of the certification landscape and characteristics of certifications across occupations. It is difficult to prioritize in such a space
- limited environment as the NHES, but I would like to suggest a few points that would greatly enhance our understanding of this phenomenon that the NHES would be a uniquely suitable place to inquire about:
 Why did you choose this credential over others in this occupation?  How many h ours did you spend preparing for this credential?  How much does this certification cost, in dollars?  Is it a generalist certification covering most aspects of your occupation, or more specialized?  How would you rate the difficulty of earning this credentia l?  How closely related was the curriculum of the certification program related to your day
-to - day work?  Is this certification issued by a non
- profit association?  If so, are you a member of that association? Some further inspiration for items about certific ation characteristics might be found in the 2002 follow up to the 1992 Baccalaureate and Beyond, which seems to be the only major federal survey that asked about professional and trade association certifications before the current working group was establi shed. I hope that GEMEnA and/or DoEd will consider expanding on some of these themes in future revisions, but even in its current form this survey will be tremendously helpful for researchers and policymakers alike. I eagerly await the results! Respectivel y submitted, Kyle Albert Ph.D. Candidate Department of Sociology, Cornell University kwa23@cornell.edu**NCES RESPONSE**

First, we are happy to hear that the proposed data collection will be of use to researchers. We lo ok forward to seeing what researchers will learn from the survey and how this information might be used by policymakers, credentialing bodies, and others interested in education and skill development. Second, we agree with the commenter that the NHES, whil e useful in many ways, is limited in the amount of corollary information it can collect on adult’s labor market background, educational background, and other characteristics. For this reason, GEMEnA is working with both the Census Bureau and the Bureau of Labor Statistics (BLS) to add core certification and licensure questions to their key surveys. At this time, these core questions have been added to the Census Bureau’s Survey of Income and Program Participation, and are in the process of being added to BLS’s Current Population Survey. Those surveys will provide more detailed background on the characteristics of credential
- holders than NHES can, while NHES will provide more detail on the credentials obtained (getting extensive details on all of these topics would be prohibitively expensive and time
- consuming in a sample survey, but might eventually be possible in a unit
- record data collection system, which some states are starting to build). Thank you for suggesting a number of interesting additional questio ns for inclusion in the NHES data collection. As explained below, we had considered some of these questions in our development process, but decided to drop them in an effort to minimize respondent burden and maximize data quality. The proposed questions, o ur plans for future testing, and reasons for not including them in the current instrument are listed below. **Why did you choose this credential over others in this occupation?** This is a very interesting question, but “why” questions are notoriously difficul t in survey data collections. People are often unclear of their motivations, and social desirability effects tend to be strong when people are asked to explain their motivations (i.e., people tend to give the answer they think is most socially appropriate) . Because of this, we do not believe it would be prudent to add this question without first cognitively testing it. As 2015 will be the first administration of the Adult Training and Education Survey, we expect that some revisions might be needed after 201 5, and we thus hope to conduct a set of cognitive interviews in 2016. We will test this question during those interviews. **How many hours did you spend preparing for this credential? How much does this certification cost, in dollars?**  We agree that these are useful questions, and we originally tried to ask them. Unfortunately, our cognitive testing showed that these were difficult questions for people to answer, as what to count in the time and cost calculations was often unclear. For example, if an education al certificate or degree provided the training necessary to earn the credential, should that time and cost be counted? What about the cost of equipment (e.g., cosmetology supplies, mechanics tools)? Or the time spent in an internship? Recall was also an is sue, as many people earned their credential many years ago. Even asking about renewal costs was difficult, as renewals differ in the time they cover (one year, two years, etc.) and in who pays for the renewal (e.g., the adult or the employer). So we droppe d these questions from our initial instrument. But we do plan to keep thinking about how we might best get at these concepts, as the “investment costs” to credentialing are an important issue. **Is it a generalist certification covering most aspects of your occupation, or more specialized?**  This is another interesting question. We did see both types of certifications during our cognitive interviews, with adults often first getting a generalist certification and then adding specializations. But it is not clear how this question should be asked in a survey. Based on our work to
- date, we are not surethat the terms “generalist” and “specialized” will work for all respondents. So, again, we would need to

do cognitive testing before adding this to the instrument. We would also need to weigh the value of this information against other information in the survey, given burden issues. Meanwhile, we encourage researchers to investigate whether the write
- in responses on the question asking for the name of the certification can reliably provide this information. This might be one way to get this information. **How closely related was the curriculum of the certification program to your day -**  **to -** **day work?** Although this is an interesting question, it is subjective in nature and pro bably not as useful for research and policy purposes as the other questions currently on the survey (and the other questions raised for consideration here). Also, some adults will be many years out of their certification program and are likely to have reca ll issues, as well as issues related to the relevancy of programs taken 10, 15, or 20 years ago. We will mention this as a question to consider by our NCES colleagues who do longitudinal studies of postsecondary students, where the question would be more a ppropriate. **Is this certification issued by a non -** **profit association? If so, are you a member of that association?** We originally tried to capture information on the type of organization that issued the certification, but we found that adults had trouble an swering this question; they often were not clear on who the issuing agency was. We did not drop this issue entirely, because it is important to distinguish licenses (issued by the government) from certifications (issued by non
- governmental organizations). So we do ask if the issuing agency was the government. Fortunately, adults do seem better at identifying the government as an issuing agency than at identifying other issuers — although even here, there will inevitably be some measurement error. We hope that eventually all the states will have unit
- record data collection systems that include certification and licensure records, which would be the most reliable source for this information (admittedly, that solution is likely in a more distant future). With reg ards to the 2002 Baccalaureate and Beyond (B&B) survey instrument, it included a number of questions focusing on the relationship between education and licensure/certification, but these questions are more relevant for B&B (which samples recent graduates) than for NHES (which samples all adults). The more recent B&B included fewer questions on certification and licensure, but we are working with our NCES colleagues to re
-visit this section of the survey in order to increase the information collected on the certification and licensure topics. Thank you very much for your interest in our surveys. With regards, Sharon A. Boivin, Ph.D. Chair, Interagency Committee on Expanded Measures of Enrollment and Attainment (GEMEnA) National Center for Education Statistics