Research Evaluation, 16(1), March 2007, pages 13–21
DOI: 10.3152/095820207X190647; http://www.ingentaconnect.com/content/beech/reev

Designing a proposal review process to facilitate
interdisciplinary research
Kazuyoshi Shimada, Mitsuo Akagi, Tohru Kazamaki andShinichi Kobayashi

This paper introduces a novel proposal review process to facilitate interdisciplinary research. Thismethod provides not only for competition for a grant, but also for collaboration among applicants.
Applicants can refine their research proposals through a workshop discussion held during the proposal
review process. This presents an opportunity for applicants to obtain feedback based on a broad rangeof viewpoints from people with various expertise. The method has actually been applied to the proposal
review process in science and engineering fields (Takeda Techno-Entrepreneurship Award, 2001,
2002). Its feasibility and some of the merits found are discussed.

T

HE OBJECTIVE of this paper is to find andto propose a new method to facilitate interdisciplinary research in terms of the proposal
review process.
Social needs for interdisciplinary1 research aiming
to solve joint problems in science, technology andsociety have been increasing. In 1994, Gibbons et al
found a new “mode” of knowledge production
scheme in response to the social needs in contemporary societies (Gibbons, 1994). In 2005, the National
Academies in the USA pointed out that science andengineering research continually evolves beyond theboundaries of single disciplines and offers employment opportunities that require not only depth ofknowledge, but also integration, synthesis and anarray of skills (National Academies, 2005). In 2004,
the European Union Research Advisory Board(EURAB) stated some problems in developing

Kazuyoshi Shimada, Mitsuo Akagi and Tohru Kazamaki are at
the Takeda Foundation, St. Lukes Tower 31F, 8-1 Akashicho,
Chuo-ku,Tokyo
104-6591,
Japan;
Email:
smd@
takeda-foundation.jp. Shinichi Kobayashi is at the ResearchCenter for University Studies, University of Tsukuba, 3-29-1
Ootuka, Bunkyo-ku, Tokyo 112-0012, Japan.
The authors cordially acknowledge Dr Ikuo Takeda for his
private donation to found such an innovative award system, andgratefully acknowledge Mrs Margaret D Ohto for critical proofreading.

Research Evaluation March 2007

interdisciplinary research and showed a number oflevels at which problems should be tackled (EURAB, 2004). In 2005, the Council for Science andTechnology Policy in Japan announced a promotionof R&D activities relating to the type of science andtechnology for the interdisciplinary resolution of issues (Council for Science and Technology Policy, Japan, 2005). Behind these approaches, there arestrong expectations for science and technology toaddress an increasing number of issues in society,for example international terrorist attacks; proliferation of weapons of mass destruction; large-scale accidents and natural disasters; crimes committed
through computer networks; and emerging or reemerging infectious diseases. Most of the issues
faced by society are complicated and a singledisciplinary approach to them is inadequate. Therefore, interdisciplinary research is required.
Despite many trials,2 it is difficult to facilitate interdisciplinary research because pathways forknowledge production and accumulation have beendeveloped individually in the traditional disciplines.
In order to bridge the gap between interdisciplinary
activities and conventional disciplinary activities, we
attempted to add a new device to the procedures fora conventional ‘grant review’. Here we introduce a
proposal review process that can be incorporated
into the grant review process. For the first time, wehave positioned the proposal review process as aknowledge production process, and designed a novel

0958-2029/07/010013-9 US$08.00 © Beech Tree Publishing 2007

13

Designing a proposal review processprocess that will facilitate interdisciplinary research.
In this paper, we introduce this proposal review
process and review its feasibility based on the resultsof its implementation.

that this review process can facilitate the interdisciplinarity of research appropriately.
On the other hand, compared with the conventional review process, the “interactive” process
places a greater workload on both applicants andreviewers (Table 1).

Design of the proposal review process
A review process that is open to all applicants

The roles of the proposal review are first to decide
grant allocation and second to review and critique
the research proposal objectively. During the review
and critique process, requirements from outside thediscipline can be worked into the research proposal.
The grant that will be allocated to the research provides an incentive for many participants from various disciplines. These intrinsic parts of the proposal
review process are favorable for facilitating interdisciplinary research.
By taking advantage of these merits, we designed
a concrete proposal review process. The new proposal review process was designed for the selectionof the recipients of the Takeda TechnoEntrepreneurship Award (TTEA3), an award for research proposals in the science and technology
fields. The review process for the TTEA scheme
consists of two stages. The first stage is a review ofthe funding sector4 itself; the second stage is the research proposal review for grant selection. In thefirst stage, we call for sector proposals for which
research grants should be allocated, and select six
sectors. Then we call for research proposals related
to those six sectors. The first stage is noteworthy for
examining problems from outside the discipline.
While the function of the first stage merits discussion, this article will focus on a discussion of the“second stage” of the proposal review process.
The newly designed proposal review process has
the following three main features: first it is an interactive review process; second it is a review processthat is open to all applicants; and third it is a review
process that is conducted on the World Wide Web.
Interactive review process

We decided to carry out an interactive review process for the research proposals. An interactive
review enables information exchange between reviewers and applicants during the review process. In
the interactive evaluation, reviewers can ask questions of the applicants directly and obtain information to help in their selection. They also can express
their concerns precisely about the selection criteria(concept of the award) through questions andanswers (Q&A). In this way, applicants can be madeaware of the reviewing criteria based on a series ofinteractions. They can revise their research proposals
during the reviewing process in order to impress
more effectively with the crucial advantages of their
proposals. The interactions offer opportunities to obtain new knowledge that will contribute to the future
development of their research. Therefore, we consider

14

Interactions during the review process are disclosed
not only between an applicant and a reviewer, butalso among all applicants and all reviewers. Here
“open” means the disclosure of the research proposals and all interactions in the review process to allapplicants to provide an opportunity for discussion.
It is not the disclosure of the entire process to persons who have no relationship to the review process.
We opened the review process only to authorized
persons (eg applicants and reviewers). When we regard the proposal review process as an interdisciplinary knowledge production process, disclosure of
the review process among the participants is necessary because they cannot interact effectively without
knowledge of each proposal. Interdisciplinary
research has to be developed through continuous andmutual revision of proposed ideas. (Here, we made arule that intellectual property rights had to be managed by the applicants in a responsible manner.)
Our review process raises expectations in the applicants that are not provided by the conventional
review process. In the conventional review process,applicants prepare documents for the application andpost them, and the reviewers read the documents andjudge them for grant allocation. Usually there are no
direct interactions between applicants and reviewers.
Through our process, applicants have the chance tolearn more by encountering the research proposals ofother applicants. They can discuss with other applicants who have different perspectives. They also get
the chance to build a new human network that maytrigger research collaborations. These expectations
may provide incentives to the applicants.
Disclosure of the review process increases thefairness of the final decisions, which is a merit for

Table 1. Merits and demerits of interactivity for participantsin the cyber workshop

Applicants
MeritsDemerits

Reviewers• They can revise theirresearch proposals
during the reviewing
process• They can ask applicantsquestions directly andget information to help
in their selection• They can stresseffectively the crucial
advantages of their
proposals• They can express theirconcerns precisely
about the selection
criteria (concept of theaward) through Q&A• Large workload• Large workload

Research Evaluation March 2007

Designing a proposal review process
Table 2. Merits and demerits of an open review process forparticipants in the cyber workshop

Applicants• They can discuss withother applicants who
have different
perspectives

Merits

Reviewers• There is increased
fairness of decisions• Large workload• Large workload

reviewers. It is easier for applicants to consider thedecisions transparent.
On the other hand, there are risks that other applicants may steal ideas. However, we have noticed
that it is not only the first publication of a piece of
new knowledge, but also collaboration (eg sharing
new findings produced by connecting each piece ofknowledge, discovering a new field to which thatknowledge can be applied, or building personal networks) that is critical for the promotion of research,especially in engineering research fields. Therefore,
we considered that applicants would recognize themerits of disclosure and accept the risks in orderto enjoy the benefits of the “open” review process(Table 2).
A review process on the World Wide Web

There is a proposal review held during a workshop
in which all applicants participate, and this workshop is held on a website. We call the workshop the“cyber workshop”. It consists of a bulletin board
located on the website. A poster explaining the research proposal of each applicant is placed in the
cyber workshop, and discussions among applicantsand reviewers are encouraged. Chairpersons, who
are reviewers assigned by the selection committee,5Table 3. Comparison of the cyber workshop with face-to-facecommunication

MeritsDemerits

Conventional
methodOur method✓

×Award money✓

Knowledge

×

Human network

×

✓
✓
✓

Save
Time/load• There are risks that
other applicants may
steal ideas

Cyber workshop

Researcher motivation

Get• They get a chance tobuild a new human
network
Demerits

Table 4. Comparison of researcher motivation between ourreview process and the conventional one

Face-to-facecommunication

moderate the cyber workshop. Applicants can accessthe cyber workshop from their favorite place at theirfavorite time and submit their comments. Program
officers, who are responsible for the whole award
program process, check the comments in advance inorder to avoid anything potentially libelous.
Operation on the Web can enhance the merits of
the open review process. Once a cyber workshop is
set up, participation time is not strongly restricted,
and applicants can utilize their time flexibly for interactive communication. In this way, international
and interdisciplinary communication can be realizedsmoothly. The savings realized in transportation
time and expense is another merit.
This method also has its demerits. As the communication involves words and figures on a bulletinboard, the amount of information exchanged is limited compared to face-to-face communication. Also,
this is not real-time communication, but rather
communication with some delay (Table 3).
Table 4 compares researcher motivation between
our review process and the conventional one. Our
method has merits in providing opportunities foracquiring new knowledge or building new human
networks, even though it takes more time and theworkload is greater.
Table 5 shows comparisons of features of our review process with the conventional process. In a
conventional proposal review process, the review ofcompeting applicants is hidden from applicants, andcommunication among applicants is impossible. The
Table 5. Comparisons of features of our review process withthe conventional one• Little restriction of time • Great restriction of timeand place forand place forcommunication
communication

Features• Saves transportation
time and expense• Costly transportationand time

Review process ofother applicants

Blind

Open

• The amount of
exchangeable
information is limited• The amount of
exchangeable
information is lesslimitedCommunication
among applicants

Impossible

Encouraged

Review process

Blind/unilateral

Transparent/interactive• Real-timecommunication

Revision of the
proposal

Forbidden/static

Allowed/dynamic• Communication withsome delay

Research Evaluation March 2007

Conventional
methodOur method

15

Designing a proposal review processreview process is also blind to the applicants themselves, and they are informed individually of the final decision of the reviewers.6 Revision of the
proposal is forbidden in principle.7
By contrast, our method is remarkable in that applicants can develop their research proposals
dynamically, utilizing many ideas provided by otherapplicants during the proposal review process.
Call for research proposals and the concrete
evaluation process
Here we explain how we implemented our new
method.
Call for research proposals

We called for research proposals for the Takeda
Techno-Entrepreneurship Award (TTEA) from allover the world. Figure 1 shows the top page for the
TTEA located on our website in 2002. We announced that the TTEA is a competition program forself-nominated research proposals, and that the selection process would be open in the cyber workshop.

We emphasized our award concepts “competition”
and “collaboration”. That is, the cyber workshop
would be a competitive area in which the contributions of the participants would be considered part ofthe selection process. At the same time, however, thecyber workshop would provide a forum for collaboration among applicants. Through the call for research proposals, we announced the funding sectors(they were the same as the topics in the cyber workshop and the research proposals were loosely restricted by these sectors), appointed chairpersons of
the cyber workshops (persons who had substantial
knowledge of the sectors), and the monetary value ofthe award (10 million yen: approx. US$80,0008).
Two sectors were selected in each of the three application fields (information and electronics, life
science, and world environment). To encourage
research proposals from abroad, all information was
released in English and Japanese.
We accepted research proposals only on the
Takeda Foundation website. We requested submission of electronic files of all documents for consideration by the reviewers and an A4-sized poster (anabstract of the documents) for presentation in the
cyber workshop.

Figure 1. Entrance to the TTEA cyber workshop

16

Research Evaluation March 2007

Designing a proposal review process

management of the intellectual property rights was
the responsibility of the applicants.

Cyber workshop

The cyber workshop comprised an online arena inwhich the research proposals were displayed. Discussions were carried out very much as in an Internet bulletin board discussion. Access to the cyber workshop
required an ID and password issued by the Takeda
Foundation. Only applicants, chairpersons, selection
committee members and program officers wereallowed access to the cyber workshop. All participants were required to state their name, organizational
affiliation, position and email address clearly. This
rule was intended to insure that participants in theworkshops would take full responsibility for theirown contributions. Applicants were encouraged toundertake active collaborations. All applicants took
part in the workshop, responded to questions from thechairperson about their own proposal, and discussed
the proposed research. The program officers of theTakeda Foundation checked the Q&A submitted tothe cyber workshop in order to support the chairpersons and to avoid libel. Revision of the posters was allowed during a certain period. We expected real-time
improvement in the research proposals stimulated bydirect feedback in the workshop discussions.
Applicants were requested to manage their own
intellectual property rights with regard to information contained in the research proposals. (For example, it was suggested that patent applications be filed
before publication on the workshop site.) As a lackof restrictions on confidentiality was expected topromote an open-minded atmosphere, applicants
were not contractually bound to maintain confidentiality with regard to what they saw or read during the
cyber workshop. Information was exchanged
on condition that no confidential materials were
included. We exchanged agreements that the

Screening (selection of awardees)

Chairpersons read the research proposals and asked
applicants questions during the workshop period.
Chairpersons also recommended topics for discussion
with all applicants. They attempted to navigate theircyber workshop clarifying the criteria or policies ofthe selection. The chairpersons deepened their understanding of the research proposals during the cyberworkshop and reported the abstract and features ofeach proposal to the selection committee. The selection committee chose the award winners taking into
account the comments of the chairpersons.9
The cyber workshops were held over a period of
about two months. The first-round selections and thesecond-round selections (finalist selections) weremade on the basis of the research proposals and discussions in the cyber workshop. Selection committee
interviews were held after the second-round selections. Selection committee interviews required theapplicants to make a presentation before the selection committee, and to answer their questions. A
telephone conference10 was used for this interview.
The selection of winners was then made based on
the research proposals, cyber workshop discussions,and the review meeting of the selection committee.
Results
Applicants

There were 141 applicants in 2001 and 134 applicants in 2002. Figure 2 shows the distribution of theNetherlands: 2
New Zealand: 2
Russia: 2
Singapore: 2
Canada: 1
China: 1

Croatia: 1
Denmark: 1
Germany: 1
Malaysia: 1
Switzerland: 1
Thailand: 1

Israel: 1
UK: 2

Others: 16
Finland: 3

USA: 13

India: 4
Korea: 8
Japan: 74
Japan:
74

Japan: 125
Japan:
125

USA: 29

2001

2002

Figure 2. Distribution of the nationalities of applicants in 2001 and 2002

Research Evaluation March 200717

Designing a proposal review process
Private company: 12

Private company: 15

National

National

institute: 26

institute: 29

University:
103 103
University:

University:
88 88
University:

2001

2002

Figure 3. Attributes of applicants in 2001 and 2002

nationalities of the applicants in 2001 and 2002. Due
to insufficient advertisement, most of the applicantsin 2001 were domestic (Japan). In 2002, better publicity of the TTEA resulted in an increase in thenumber of applicants from abroad.

Discussion among participants

In order to describe the features of the cyber workshop, we counted the comments on the bulletinboard and classified them into the following three
categories:

Attributes of the applicant affiliations

Figure 3 shows the attributes of applicants. We classified their affiliations as private company, university and public research institute. A university
was the most frequent affiliation. As we hoped
to promote practical research, we eagerly invitedresearchers from private companies. However,some researchers in private companies hesitatedto disclose research plans to be addressed in the
future.

1. Q&A between chairpersons and applicants
2. Q&A among applicants
3. Other.
Figure 4 shows the normalized number11 of comments in each workshop. The normalized number ofcomments indicates the mean number of comments
per participant, which indicates the activity of the
discussion.
In the workshops that did not come alive, most ofF

WorkshopE

D

C
Reviewer <-> Applicant
B
Applicant <-> Applicant
Others

A

0

2

4

6

8

10

12

14

Normalized number of comments
Figure 4. The normalized number of comments in each workshop

18

Research Evaluation March 2007

Designing a proposal review process

the discussion involved questions from the chairperson and answers from the applicant to the question.
This feature indicates that applicants did not participate in the workshop on their own initiative.
On the other hand, in the workshops that camealive, not only answers to the questions from thechairperson but also questions and answers among
applicants accounted for a quarter to a third of thecomments. This feature indicates that applicants participated and/or activated the discussion at their own
initiative.
We found that most of the comments were polite,
and no rancor was observed. We also found that a
few key persons played important roles in the active
workshops. The key persons frequently questioned
the other applicants and aggressively made comments to other applicants. As a result, other applicants were encouraged to ask questions. Accurate
and in-time comments from the chairpersons also
played an important role in activating the discussion.
The active discussions seemed to be stimulated by
topics that were recognized as common issues
among a broad range of expertise.
Here we show some examples of the comments:
I wish to ask all participants in this session about
their view on the relative roles of software andhardware measures to enhance, possibly enforce, system reliability. My impression is thatmost members of our group strongly favor
hardware measures — although, at least in myexperience, the software components of complex systems seem more prone to cause serious
problems. The problems may be due to less thansatisfactory programming practices, inadvertent
programming mistakes or deliberate humanmischief, such as hackers. (“Question to all participants on system reliability measures” NF,
2002, Session B — Dependable Computer
Systems and Secure Semiconductor Chips)
I think there are two subjects, ‘understanding’and ‘feeling’, in public acceptance, and they arevery different especially when the use of [genetically modified organisms] GMO are concerned. For example, I can explain my wife thesubstantial safety of the genetically engineeredsoybean on the market, and let her ‘understand’
it, but she may say “OK. You may be right, but
I still don't ‘feel’ like it.” Scientific efforts suggested by several applicants are of course very
important to get public acceptance at the levelof ‘understanding’, but I think the major difficulty of public acceptance lies in the ‘feeling’matter. Since the main concern of public acceptance of environmental biotechnology is ecosystem protection, the view point of ecologicalstudy is important, and it also can provide aclue to solve this problem. (“Public acceptance,
understanding and feeling” HM, 2002, Session
F — Environmental Biotechnology).

Research Evaluation March 2007

Questionnaire

To evaluate the performance of our method, we sentout questionnaires to all applicants and chairpersons
after announcing the award winner. We did not ask
specific questions but instead asked them to describe
their opinions about our trial (TTEA) freely. Positive
opinions and negative opinions were obtained. Here
we categorize the comments from applicants as positive and negative opinions.
Positive• The cyber workshop was a new experience andinteresting for me.
• I had an enough time to consider how to answer
the questions.
• It was convenient that I could participate in the
cyber workshop freely in my own time.
• It was attractive especially for young researchersthat the review process seemed to be promoted
without restriction by discipline or organization.
• It was exciting that not just the results but proposals of research could be seen.
• The selection process was very transparent.
Negative• Selection process was not transparent. You should
disclose the selection criteria in detail.
• It was painful for me that frequent access to thecyber workshop and responses to the chairperson’s questions were requested of the applicants.
• The workshop topic (sector) was too broad tomake discussions with others.
• Please stimulate discussions more.
• Management of intellectual property rights wasvery difficult.
In addition, some thoughts of the chairpersons wereas follows:• I felt great pressure because my decisions weredisclosed to all applicants.
• It was my great pleasure to receive research
proposals from all over the world.
• The cyber workshop was useful in that I could accumulate information effectively in line with myselection criteria.
Discussion
Was interdisciplinary research facilitated?

A positive function to facilitate interdisciplinary research was found in our method. As shown in theabove examples (discussion among applicants),
comments addressing topics of social problems were
frequently made, and many applicants joined in thediscussion on their own initiative. Our method seems
to be effective for encouraging discussion of social
aspects among applicants, and this characteristic isuseful for facilitating interdisciplinary research.

19

Designing a proposal review process

Our method seems to be effective forencouraging discussion of social
aspects among applicants, and thischaracteristic is useful for facilitatinginterdisciplinary research

Table 6. Adoption ratio of research grants of which monetary
value was about 10 million yen (approximately
US$80,000) per year

GrantsAdoption ratio

Fundamental Research (A)(JSPS)

Grant for Industrial Technology 24% (in 2003)
Research (NEDO)
Takeda TechnoEntrepreneurship award (TF)

Three factors were considered to have affected thecharacter of our cyber Workshop. First, discussionsof social aspects were encouraged by the selection
criteria of the TTEA, which praised entrepreneurial
research aiming to solve social problems or to create
economical benefit. Second, discussions of such topics were suitable for applicants from different disciplines because all equally shared the problems as
members of society. Third, researchers were eager todiscuss because they were interested in subsequent
information exchange and collaborative knowledge
production to solve problems.
Additional incentive

A major feature of this TTEA program is that it provides merits not only to the award winner, but also
to all applicants. In order to make use of this function, it is critical to attract applicants who will be
active participants in the discussion. For this purpose,
we have to understand what will serve as incentives
for applicants.
One positive factor to increase the merits to applicants is the monetary value of the award. If theaward is too small, the applicants will not be willingto spend time participating actively in the cyberworkshop. Another positive factor to increase incentive is the adoption ratio. If the adoption ratio is excessively low, the efforts of most of the applicants
will not be rewarded, and such an award program isunattractive.
Table 6 shows the adoption ratio of the TTEA (10million yen — approximately US$80,000 per year)
and other research grants with similar monetary values. The adoption ratio of the Fundamental Research(A) grant-in-aid for scientific research operated by theJapan Society for the Promotion of Science (JSPS)
was 22% (in 2003). The adoption rate of the Grant for
Industrial Technology Research operated by National
Energy Development Organization (NEDO) was 24%(in 2003). On the other hand, the adoption ratio of the
TTEA was 4% (in 2002) and 6% (in 2001). It is clearthat the adoption ratio of the TTEA was much lower
than for other grants of similar size. Moreover, applicants for the TTEA have to participate in discussions
in the cyber workshop during the selection process,with a workload that seemed to be high.
Based on these comparisons, the TTEA seems to
be unattractive. However, we should note that over20

22% (in 2003)

4% (in 2002) and 6% (in 2001)

100 applicants applied both years even though the
TTEA provided a low adoption rate and requested aheavy workload. Why were applicants interested in
the Takeda Techno-Entrepreneurship Award? We
think the review process of the TTEA provided another merit besides the adoption ratio and award
money.
In the cyber workshop, comments about other applicants’ research proposals and requests for comments about their own research proposals were
frequently seen. There were demands for constructive comments and opportunities for learning among
applicants. It was considered to be a new merit provided by the TTEA that applicants could satisfy thestrong desire to learn and to improve their research
proposals. Fortunately, applicants from all over theworld (Figure 2) could participate in the discussions
in the cyber workshop. These merits may be strong
tools to facilitate interdisciplinary research.
Merit of disclosure

Another important finding was that applicants took
the risk of having their ideas stolen. They might takeadvantage of the opportunity for mutual evaluation.
For the TTEA, it was the research proposals thatwere disclosed among applicants. This is fundamentally different from the disclosure of completed research at a conference. When we planned the
proposal review process, disclosure of the research
proposals was criticized as “senseless conduct” bysome researchers. The criticism originated with scientists who have intense respect for the first publication of pioneering points or originalities. However,many applicants for the TTEA supported our open
proposal review process. In the engineering field,
which is the target of the Takeda TechnoEntrepreneurship Award, solving a problem is more
important than the first publication. In such fields,
although it remains controversial, some researchers
may try to find better applications for their technological or experimental innovations by means of disclosure and discussion with other applicants withvarious points of view. We believe that the proposal
review process of the TTEA can serve as a kind ofknowledge production in the interdisciplinary
research.

Research Evaluation March 2007

Designing a proposal review process

Conclusion
We introduce a novel research proposal review
process to facilitate interdisciplinary research. Our
new method was implemented in the proposal review process to select awardees of the Takeda
Techno-Entrepreneurship Award. We provided amutual evaluation opportunity to all applicants via

Notes
1.

2.

3.

4.

5.

6.

7.

8.
9.

In this paper, we use the term “interdisciplinarity” to mean“different disciplines working together and trying to synthesize
cognitive approaches”, following the definition of EURAB report (2004).
Trials of inter- and transdisciplinary research have been performed and their importance and difficulty for assessment
have been discussed (eg Thompson Klein et al, 2001;
see also the special issue of Science and Public Policy, July
2006, edited and introduced by Maasen, Lengwiler and
Guggenheim, and the special issue of Research Evaluationon the assessment of interdisciplinary research, April 2006,edited and introduced by Laudel and Origgi.
TTEA is a program designed to praise and encourage researchers through the presentation of an award. See <http://
www.takeda-foundation.jp/en/award/tech/index.html>,
lastaccessed 10 March 2007.
“Sector” means a target issue to be addressed (eg environmental biotechnology, design and development of hardware
having flexibility and expandability of functionality andperformance, systems aiming at tissue engineering, etc.).
The selection committee is a body for the final consideration
and approval of the award winners. Chairpersons are
assigned by the selection committee and are tasked to explain the core competency of each applied proposal to theSelection committee members.
Interactive evaluations are performed in the peer-review processes of collaborative networks in Germany (so-called Sonderforschungsbereiche [since 1968] and Innovationskollegs[1994–2001] financed by Deuche Forschungsgemeinschaft[DFG]). They perform interactive evaluations to fully understand a proposal for interdisciplinary research and to makeappropriate decisions concerning grant allocation (Laudel,
2006).
For example, directed programs of the Natural Environment
Research Council (NERC) < http://www.nerc.ac.uk/research/
programmes/>, last accessed 10 March 2007 (since 1992) ora part of Research Contracts of the National Institute ofhealth (NIH) <http://ocm.od.nih.gov/contracts/contract.htm>,
last accessed 10 March 2007 performed a two-stage assessment of proposals. A reviewer assessment was sent tothe applicant in the first stage, and a modified proposal or
proposer response could be submitted. This was a kind of interactive evaluation, although the review process of other applicants was hidden and communication among applicants
was impossible.
Seven million yen (approx. US$58,300) in 2002.
Under the Foundation’s rules, the selection committee wasauthorized to select award winners. The chairpersons were

Research Evaluation March 2007

the Internet (cyber workshop). Then we selectedawardees based on the cyber workshop discussions.
This award program succeeded in attracting over
100 applicants from around the world in each year.
This suggests that the proposal review process canserve as a kind of knowledge production. Of course
it could provide a useful tool for the facilitation ofinterdisciplinary research.

navigators of the cyber workshop and were not authorized to
select the award winners.
10. Finalists sent PowerPoint presentations to the Takeda Foundation (TF) in advance of the selection committee interviews.
Program officers checked the slides and installed them in the
TF computers. Selection committee members met at the TF
in a room wired for a conference telephone session. Theneach finalist was phoned by a program officer and asked tomake a presentation via telephone. Manipulation of thePowerPoint slides was achieved by program officers during
the presentation and Q&A. The PowerPoint slides werechanged according to the finalist’s order (eg “Next please”).
11. The number of comments divided by the number of participants.

References
Council for Science and Technology Policy, Japan 2005. The
Third Science and Technology Basic Plan (FY2006-FY2010),
27 December 2005. <http://www8.cao.go.jp/cstp/english/basic/
index.html#third>, last accessed 10 March 2007.
EURAB, European Union Research Advisory Board 2004. Interdisciplinarity in Research (EURAB04.009-FINAL), April.
<http://ec.europa.eu/research/eurab/pdf/eurab_04_009_
interdisciplinarity_research_final.pdf>, last accessed 10 March2007.
Gibbons, Michael, Camille Limoges, Helga Nowotny, Simon
Schwartzman, Peter Scott and Martin Trow 1994. The New
Production of Knowledge: the Dynamics of Science and Research in Contemporary Societies. London: Sage.
Laudel, Grit 2006. Conclave in the Tower of Babel: how peers
review interdisciplinary research proposals, Research Evaluation, 15(1), April, 57–68.
Laudel, Grit and Gloria Origgi eds. 2006. Special issue on the assessment of interdisciplinary research. Research Evaluation,
15(1), April, 1–80.
Maasen, Sabine, Martin Lengwiler and Michael Guggenheim eds.
2006. Special issue on discipline and research: practices ofinter-/transdisciplinary cooperation in science. Science andPublic Policy, 33(6), July, 393–476.
National Academy of Sciences, National Academy of Engineering,and Institute of Medicine of the National Academies 2005. Facilitating Interdisciplinary Research. Washington DC: National
Academies Press.
Thompson Klein, J, W Grossenbacher-Mansuy, R Haberi, A Bill, R
W Scholz and M Welti eds. 2001. Transdisciplinarity: Joint Problem Solving among Science, Technology and Society. An Effective Way for Managing Complexity. Basel: Birkhauser Verlag.
2006. Science and Public Policy, 33(9), November.

21