If the Department of Education is going to raise the credibility of the research it supports, a new report concludes, it must bolster its system for outside review of grant proposals. At the direction of Congress, the department four years ago tightened guidelines governing the peer-review panels that evaluate proposals for agency funding. But the report, commissioned by an advisory board to the department, suggests that the changes didn't go far enough. The lack of experienced researchers on some of the panels is the most serious shortcoming in the department's peer-review practices, concludes the study released last month.
Technical expertise is key, the report says, because it appears to lead to better-quality reviews.
The opposite was true for panels that included researchers working in the same fields as the applicants. Part of the problem has been that reviewers with the know-how called for in the guidelines are hard to find, the report points out. Besides knowing something about the subject area of the proposed research, reviewers need to have in-depth knowledge of research methods and of educational policy and practice. And department staff members, sometimes given only three weeks to recruit reviewers, have to mix and match panel experts. One remedy to the problem, the panel concludes, might be to establish standing review panels for each of the department's five national research institutes.