Evaluating education effectiveness.

I would like to clear up a misconception that readers might derive from your News Focus story “Is the education directorate headed for a failing grade?” by J. Mervis (24 Feb., p. [1092][1]). In the story, I am quoted as saying, “Maybe NSF [National Science Foundation] education programs need to be rethought.” In fact, I believe that NSF education programs have been instrumental in creating a series of outstanding curricula for school science in the United States and have served an important role in making both scientists and schools aware of the urgent need to rethink what we mean by “science education” ([1][2]). In my conversation with your reporter, I stressed the need to create a robust body of high-quality research on science education in our schools. In this context, I suggested that the NSF rethink its requirement for formal project evaluations, by which I meant that it support a broad range of methodologies aimed at producing useful and transferable knowledge. I suggested, for example, greater attention to multisite research that is designed to discover what does and does not work, and why. I also questioned an NSF tradition of discontinuing even the best programs after 5 years, with the expectation that school districts (or others) will be able to cover the expense of continuing the programs thereafter. 1. 1.[↵][3] 1. B. Alberts , Cell 123, 739 (2005). [OpenUrl][4][CrossRef][5][PubMed][6][Web of Science][7] # {#article-title-2} The news story “Is the education directorate headed for a failing grade?” by J. Mervis (24 Feb., p. [1092][1]) confounds a number of important issues pertaining to the National Research Council's 2004 study “On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations,” chaired by Professor Confrey. First, the story blurred an important distinction between quality of evaluations and quality of curricula: We did not say that none of the curricula are effective or that none of the evaluations had merit. Rather, we found evaluations that ranged in quality from highly flawed to very imaginative, but that taken together could not support broad claims—positive or negative—about the quality of the curricula. Second, the story failed to note that many of the studies we reviewed were not evaluations but progress reports, descriptive summaries, and advocacy pieces; that they included evaluations of commercially developed as well as NSF-sponsored curricula; and that the committee did not directly examine the 19 curricula in question. Third, the article misstated Professor Confrey's views about the importance of sustained and significant federal investment in curriculum evaluation. Contrary to the story's suggestion that past investments yielded little knowledge, her point was that they have informed us about what to avoid, about how to design rigorous evaluations, and about the need for greater attention to judging effectiveness through multiple evaluation methods. [1]: /lookup/doi/10.1126/science.311.5764.1092 [2]: #ref-1 [3]: #xref-ref-1-1 "View reference 1. in text" [4]: {openurl}?query=rft.jtitle%253DCell%26rft.stitle%253DCell%26rft.aulast%253DAlberts%26rft.auinit1%253DB.%26rft.volume%253D123%26rft.issue%253D5%26rft.spage%253D739%26rft.epage%253D741%26rft.atitle%253DA%2Bwakeup%2Bcall%2Bfor%2Bscience%2Bfaculty.%26rft_id%253Dinfo%253Adoi%252F10.1016%252Fj.cell.2005.11.014%26rft_id%253Dinfo%253Apmid%252F16325564%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [5]: /lookup/external-ref?access_num=10.1016/j.cell.2005.11.014&link_type=DOI [6]: /lookup/external-ref?access_num=16325564&link_type=MED&atom=%2Fsci%2F312%2F5770%2F54.atom [7]: /lookup/external-ref?access_num=000233814100003&link_type=ISI