ARCHIVESHISTORIQUES
DELACOMMISSION
COLLECTION RELIEEDES
DOCUMENTS "COM"
COM(80)889
Vol.1980/0278

Disclaimer
Conformément aurèglement (CEE, Euratom) n°354/83 duConseil du1erfévrier 1983
concernant l'ouverture aupublic des archives historiques delaCommunauté économique
européenne etdelaCommunauté européenne del'énergie atomique (JOL43du15.2.1983,
p.1),telque modifié parlerèglement (CE, Euratom) n°1700/2003 du22septembre 2003
(JO L243du27.9.2003, p.1),cedossier estouvert aupublic. Lecaséchéant, lesdocuments
classifiés présents dans cedossier ontétédéclassifiés conformément àl'article 5dudit
règlement.
Inaccordance with Council Regulation (EEC, Euratom) No354/83 of1February 1983
concerning theopening tothepublic ofthehistorical archives oftheEuropean Economic
Community and theEuropean Atomic Energy Community (OJ L43,15.2.1983, p.1),as
amended byRegulation (EC, Euratom) No1700/2003 of22September 2003 (OJ L243,
27.9.2003, p.1),thisfileisopen tothepublic. Where necessary, classified documents inthis
filehave been declassified inconformity withArticle 5oftheaforementioned regulation.
InÜbereinstimmung mitderVerordnung (EWG, Euratom) Nr.354/83 desRates vom 1.
Februar 1983 über die Freigabe der historischen Archive der Europäischen
Wirtschaftsgemeinschaft undderEuropäischen Atomgemeinschaft (ABI. L43vom 15.2.1983,
S.1),geändert durch dieVerordnung (EG, Euratom) Nr.1700/2003 vom 22.September 2003
(ABI. L243vom 27.9.2003, S.1),istdiese Datei derÖffentlichkeit zugänglich. Soweit
erforderlich, wurden dieVerschlusssachen indieser Datei inÜbereinstimmung mitArtikel 5
dergenannten Verordnung freigegeben.

COMMISSION OFTHE EUROPEAN COMMUNITIES
COM(80)889 final
Brussels ,31st December 1980
COMMON POLICY FOR SCIENCE AND TECHNOLOGY :
EXPLOITATION AND EVALUATION OF RESEARCH RESULTS
\
?a?Sf­
(Communication from the Commission tothe Council )
\y :,r-t
COM (80)889final

COMMON,POLICY.FORSCIENCEANDTECHNOLOGY :
EXPLOITATION ANDEVALUATION O.FRESEARCH RESULTS
(Communication fromtheCommissiontotheCouncil,)
Atitsmeetingon21October1979andinitsconclusions adoptedon
20December1979,theCounciI^stressedthattheultimateaimof •
Communityresearchmustbetoproduceresultswhichcontributeto
theattainmentoftheeconomic,socialandotherobjectives ofthe
CommunityanditsMemberStates.
Accordingly,theCouncilaskedtheCommissiontoformulatesuitable
^proposalsfor: " .
-apolicyontheutilization ofresults,
-asystemforevaluatingtheresultsofcommonprogrammes .
Utilization ofresults
1.OnlypartofCommunityresearchisdesignedtopromoteinventions
thatcanbeuseddirectly.Thisresearchinmanycasesinvolvesa
longrun-uptotechnological innovation (fusion,hydrogen)orelse
encouragesorfacilitates innovation (pureresearchandtheprovision
rofservices).. > - ■
Anappropriate policyforthe;utilization oftheresultsofthis
researchwillthereforeinvolveagooddealofinformation processing
anddissemination andalsoessential,although -morelimited,exploita­
tionofinventions onanindustrialscale.
2.Thedissemination ofresearchresultsshouldbeaimedatallpotential
users,economicoperatorsanddecision-makers ,andnotbeconfined,as
isoftenthecase,toscientificcircles.

-2-
Tothis end,greater efforts should bemade toanalyse ,process and
disseminate the information collected ,toovercome language barriers
andtomultiply thenumber ofinformation access points bycalling
upon trade associations ,chambers ofcommerce ,andregional and
local technological advisory services forassistance .
2.3 .Thetransposition ofresearch results from the invention stage tothe <
innovation stage calls for aseries ofupgrading operations ,i.e.
development ,experimentation andmarketing .Amore vigorous upgrading
policy ,given sufficient resources ,would make itpossible to :
-attain increased benefit from thetechnological achievements of
the Joint Research Centre ;
-use more widely onanindustrial scale the indirect-action results
obtained byuniversities and research institutes
-follow upmore intensively theuse made byindustry ofthe results
ofindirect action ; n
-make use ofthedirect spin-off from long term programmes .
. i '
2.4 .Inline with the Council Resolution of14January 1974 ,apolicy for
theutilization of .Community research results should progressi vely
beextended toinclude coordination andcooperative projects between
theMember States inthe field ofdissemination andexploitation .
The Commission therefore proposes that during 1981 CREST examines
andcompares national methods ofdissemination andexploitation ,
indicates those projects ofCommunity interest which itwould be
worth while toundertake byconcentrating existing resources .
3.Evaluation ofresults
:
3.1 .Astotheevaluation ofresearch results ,theCommission has ,investi­
gated evaluation methodologies developed within theMember ,States
andelsewhere inorder toexamine thepossibility ofadapting them
fortheevaluation ofCommunity R&Dprogrammes .

-3-- >
EvaluationofRSDprogrammescannotbeperformedwithoutreferenceto
theoverallenvironment oftheprogrammes andshouldinparticular be
appropriatetothenatureandspecificobjectivesofthework.The
CommissionthereforecametotheconclusionthattheCommunityshould
developitsownevaluationcriteriatosuitthespecificcharacteristics
ofCommunity R&Dprogrammes . /J
3.2.Inadditiontoevaluationactivitiesundertaken bytheresponsible
departments oftheCommission (limitedmainlytothescientific v.alue
ofthework),itappearedappropriate toprovideCommunitydecision-
makingbodieswithanassessmentcarriedoutbyindependent,external
expertsofresultsobtainedthroughCommunityresearch "programmes ./
3.3.Thisapproachisnowbeingtestedoutinanumberofcases.Thefirst
exercisehasalreadybeencompleted,butitsusefulness can,in,the
lastresort,beassessedonlywhenitisknownwhatpracticaluse
decision-makers reallymakeofit. '
3.4.Whenthetestrunshavebeencompleted,theCommissionwilldrawits
conclusions andchecktheminaninternational forum,wherethe
resultsofthetestcasescanbeconfronted andcomparedwithsimilar
activitiesconductedinandoutsidetheCommunity.
3.5.Onlyinthelightofallthisexperience wouldtheCommissionconsider
thattherewasacaseforpresentingtotheCouncilanyproposal
concerninga'systematicevaluationofCommunity R&Dprogrammes .
•A.TwodetailedCommissionreportsareattachedtothiscommunication .The
firstconcernstheexploitation ofresults(AnnexI),thesecondtheir
assessment (AnnexII).
TheCouncilisaskedtotakenoteofthiscommunication andofthe
Commission'sproposalsincludedinit,inresponsetotheCouncil's
requestof20December1979.

ANNEX1
ReportinresponsetotheCouncil'srequestof20December1979for
thedefinitionofapolicyontheexploitation ofresearchresults.
(
SCOPEOFAPOLICYONTHEEXPLOITATION OFRESEARCH RESULTS
The"Research",Councilmadeitclearatits619thsessionthatthe
ultimateobjectiveofCommunityresearchshould "beacontribution
throughitsresultstotheachievement ofeconomic,socialorother
r
goalsoftheCommunityanditsMemberStates,
TheCouncilconsequently requestedtheCommissiontodrawupsuitable
proposals,inparticulartowardsdefiningapolicyontheexploitation
ofresearchresults.
Becauseoftheresponsibility conferreduponitbytheTreatiesand
byCouncilRegulationsforcarryingoutresearchprogrammesthe
Commissionhasalwaysbeenconcernedtoensurethatresearchresults
arepassedontotheirpotentialusersinthemosteffectivemanner
possible.
TheECSCandEuratomTreatiesassigntotheCommissionthetaskof
disseminating Communityresearchresultswidely,andofensuringthe
industrialexploitation ofanyinventionsproducedinthecourseof
thatresearch.TheCommissionalsohassimilarresponsibilities for
EECresearchprogrammes,underArticle41oftheTreatyinrespect
ofagriculturalresearch,andunderCouncilregulation No2380/74of
17September 1974asregardsotherresearch.
Incarryingoutthistask,however,theCommissionisconstantlyobliged
toadaptitsactivitiestomatchdevelopments inthecontentof
researchprogrammes andthenatureoftheresultsachieved.
Followingtherevivalofacommonpolicyonscienceandtechnology by
theCouncilresolutionof14January1974andinconformitywiththe
guidelinesadoptedfortheperiod1977–80,thefollowingpointshave
emerged:
a)thoseareasofresearchwhoseresultscouldnotnormallybe
expectedtocontributetoaCommunityobjective,i.e.funda­

mental research ,have been eliminated from theprogrammes ;
b)research projects whose objective isthedevelopment oftechno­
logical innovation will inmany cases lead toindustrial exploita­
tion only inthemedium orlong term (e.g. controlled thermonuclear
fusion andhydrogen ):
ν
c)theprocess ofinnovation inindustry isstimulated andfacilitated
"by.research inother areas which may not itself produce inventions
(research into performance ortheoptimization ofnewtechnologies ,
technoeconomic studies ,newapplications forknown technologies
etc.);
d)intheenergy sector theCommunity hasresources tofinance
demonstration projects forexploitable technologies ; ^
e)theprogrammes also include service activities (e.g. testing of
materials andequipment ,data acquisition andprocessing ,etc.).
Such awide range ofactivity inevitably leads toawide variety of
results .Asuitable way ofdealing with each type ofresult must
therefore bedeveloped ifexploitation istobeeffective .
/ •
Irrespective ofwhether theresults relate toECSC ,Euratom orEEC
research ,thepolicy fortheir exploitation must nevertheless bebased on
similar principles ,since the economic and social objectives ofthe three
Communities arevirtually identical . '
Such anexploitation policy cannot bebased exclusively either onthe
dissemination oftheresults ,orexclusively ontheir "valorization "*. „
This isbecause Community programmes arenotintended exclusively for
theimprovement ofscientific andtechnical knowledge ,norexclusively
forthedevelopment ofnewproducts ,equipment orprocesses .
_Forthepurposes ofthis report theterms "valorization "and "dissemi­
nation "will beused only forconvenience .Inreality there isnosharp
boundary between theprocesses of"dissemination "andof"valorization ". •
These activities are distinct from the evaluation ofresearch results
butcontribute toitthrough theinformation they make available .
valorization :meaning theentire sequence ofsteps bywhich atechnically
promising invention isintroduced into economic use (patenting ,commercial
evaluation ,development ,licensing ,prototype production ,marketing ,etc.)

GUIDELINES FOR APOLICY ONTHE EXPLOITATION OPRESEARCH RESULTS
Onthebasis ofthese principles andusing theexpression "exploitation
ofresults "initswidest sense ,the Commission proposes for adoption the
following guidelines :
a)Dissemination ofresults . -
/
Results should "bechannelled more specifically towards thepotential
users oftheknowledge .Atpresent for certain programmes theinfor­
mation remains alltoofrequently inscientific and research
circles ,which areproducers ofknowledge .Results which canbe
directly utilized should bedescribed inarticles written ina
language which isaccessible toindustrialists andother decision -
makers who donot always have themeans toanalyse the original
published information .When user needs andhabits require it,
such articles should betranslated into the other Community
languages .Inaddition ,results would bebetter disseminated ,
and more certain toreach small and medium-sized firms and local
authorities ,byenlisting theaidoftrade associations ,chambers
ofcommerce andregional and local technical consultancy services .
b)Valorization ofresults
For the results ofresearch topass from the stage ofthe invention
tothat ofinnovation presupposes aseries ofsteps involving
development ,testing andmarketing .Ashasalready been mentioned
above ,Community research byitsvery nature does notgive rise to
alarge number ofinventions suitable for exploitation .Nonetheless
amore systematic valorization policy with increased resources
could result in:
–harnessing better theexisting technological capacity oftheJRC ;
–more extensive industrial exploitation ofresults produced
byindirect actions involving universities andresearch
institutes ;
–closer follow-up ofexploitation byindustries benefiting from
indirect actions ;
–theuseofshort –term spin–offfrom long-term programmes (which
isfrequently neglected );
–taking advantage ofeconomies ofscale made possible bythe size
ofthecommon market ,without which many innovations would not
reach thebreak –even point .

-4-
c)Links between Community andnational research
Itshould berecalled that inaccordance with the Council Resolution
of14January 1974 thecommon policy onscience andtechnology
"involves thecoordination ofnational policies andthe joint
implementation ,ofprojects ofinterest totheCommunity ".Thepolicy
ofexploitation ofresearch results available totheCommunity should
thus notbekept separate from that ofMember States fortheresults •
available tothem . >
ForEURATOM andEEC research the results ofasignificant part ofindirect
action research cannot beseparated from thenational prograjmmes .
Community finance isinfact only partial and itisfrequently the
case that supplementary finance ismade available directly orindirectly
from national orregional sources .
Inanumber ofareas –including some having absolute priority ^–
Community research represents only asmall part ofthepublicly – .
aided research "being undertaken within theCommunity .Theestablishment
ofCommunity –wide systems fordissemination andvalorization inthese
areas would prove offargreater cost–effectiveness ifthey were
established jointly bythe Community andtheMember States .They .
could also beused for the dissemination and valorization ofnational
research throughout the Community .
Apolicy for theexploitation ofCommunity research results should
therefore gradually beenhanced bycoordination andcooperation with
Member States inthe field ofdissemination and valorization .Such
initiatives have already been taken inisolated instances with ,for
example ,listings ofresearch projects including both Community
andnational projects ,conferences aimed atupdating results atboth
Community andnational levels ,andcollaboration with national valo-"
zation organizations .

-5-
DISS ETCNATION OFRESULTS ;-
> : – ^
3.1 Current methods
TheCommission hasmade considerable progress during thelast fewyears
?
indiversifying its dissemination media .
Dissemination was previously based principally onthe systematic publi­
cation ofresearch reports ,theproceedings ofasmall number ofconfe­
rences and ,inthenuclear field ,thereference periodical "Euratom
information ".
TheCommission iscurrently giving priority totheuseofexisting infor­
mation media -publication ofarticles inscientific andtechnical journals
andthepublication ,through theprivate sector ,ofworks such asconference
s.
reports ,compilations ,data collections ,atlases jetc.'
. 1 /
TheCommission policy encourages human contact .Inthe context ofindirect
actions ,meetings between contractors whose areas ofresearch are related
and complementary have metwith real success .Conferences ,seminars and
'study meetings ,have multiplied toapproximately 50Peryear where exchanges
take place not only ofresearch results but also ofassociated information
inareas covered byCommunity programmes .
The systematic printing and publication ofparticular research reports
has been discontinued .Selectivity ispractised onthebasis ofthe reports'
subject matter ,andsome arepublished intheconventional wayifthey have a
large potential readership ,whilst ,others arereproduced onmicrofiche . *
This selectivity innowayreduces theaccessibility of,theresults ,since ,
irrespective ofthemedium inwhich they arepublished ,theavailability
ofall reports ispublicized inthe information bulletin "Euro-abstracts "
with anadditional brief summary andbibliographical references .The EA.BS
computerized data base ,which contains all thenotices published inEuro-
abstracts (18000references since 1966 ),cannowbeaccessed viathe
"Euronet-DIANE "system.'

With thesame aim ofsupplying information digests rather than aconfusing
mass ofinformation ,priority hasbeen given tothepublication ofperio­
dicprogramme reports ,such asthe ^'status reports "onthe energy pro­
gramme . - ,
'Asithadbecome difficult toreach aclear assessment ofprogress in
JRCresearch through thepublished articles andreports ,half-yearly
progress reports arenow "published for each ofthe direct action research
programmes .Sothat national institutions ,researchers andindustrialists
within theCommunity have priority access tothese reports ,personal co­
pies are dispatched toseveral hundred such users . '
Inthenuclear field the system ofconfidential communications estab­
lished 20years agousing national contacts has "been maintained ,though
-its value has diminished and the Commission intends tooverhaul it.
Dissemination ofECSC research results hasbeen the subject ofathorough
rationalization .Section IIof"Euro-abstracts "isdevoted tothis research
andisparticularly appreciated byits1300 subscribers foritssubstan­
tial andinformative summaries (inanumber oflanguages )which give de­
tails not only ofavailable publications but also ofnew research projects .
Thestudy meetings attract alarge number ofparticipants .Methods of
distributing research reports have been discussed .
Theagricultural research programmes include numerous seminars whose
proceedings constitute useful works ofreference .These programmes also
provide for exchanges ofresearch personnel . 1
\
JVsfordemonstration projects ,proposals areonly selected ifthey wel­
come visitors and trainees .
3.2 Prooosed improvements ,
3.2.1 -Information présentation
Most information disseminated bythe Commission has sofar beenma
form chosen rather freely bytheresearchers themselves .

Thoughdissemination maybeacceptableinthisformwhentheinfor­
mationisintendedforscientificcirclesitfrequentlyconstitutes ..."
amajorbarriertotheflowofinformationrelatingtoexploitable
resultstowardsfirms-particularly smallandmedium-sized ones-
andtowardsotherdecision-makers .
Directingthedissemination ofresearchresultstowardstheirultimate
userssoastoencouragetheireffectiveandrapidexploitation will
entailamajoreffortinthepresentation ofcertaintypesofinformation
suchashasbeenundertakenforECSCandagricultural research.
Itwillbenecessarytoanalyseresearchresultstoidentifythose
whichrequireediting.Inthiswaypopularizedtexts,summaryreports,
manualsandmonographscouldbeproducedforcertainresearchareas,
highlighting pointswhichappeartobesuitableforexploitation inthe
shortterm.
Forthemostpartsuchdocumentswillbedesignedaroundtherequire–:
mentsoftheirbusinessreaders,particularly insmallandmedium-
sizedfirms.Whentheresultsobtainedarelikelytoaffectregulations
andstandards(e.g.intheenvironmentorenergysavings)documents
shouldbepreparedforreadingbyinstitutional decision-makers .
Documentdistribution isnotthemostreliablemethodofbringingabout
theexploitation ofresults,however.Adialogueshouldbeestablished
betweentheparticipants intheinformationtransfer,andforthat
reasoninformation seminarsforuserswouldbedesirable.Information
leafletswouldthenbepublishedfororaftersuchseminars.
Inthesamewayfinancialincentivesshouldbeavailabletoencourage
visits,secondments andstudytoursforpotentialuserswiththeresearch
teamsor,preferably ,withoperatorsofpilotplantsordemonstration
projects. 7
Participation atexhibitions andtechnicalnotes,bothofwhichare
appropriate mediaforthevalorization ofinventions andarealready
beingusedbytheCommission ,shouldbedevelopedalongthelinespro­
posedbelow.'

-8-
,2Dissemination channels
ν'
ManypotentialusersoftheresultsofCommunityresearchprogrammes
remainunawarethattheprogrammes evenexist.
DespitethestepstakenbytheCommissionduringthelastfewyears
(seeparagraph3«1above),andthoughtheremaybeimprovements in
-the-presentation andpackagingoftheinformation available ,some
businesses andadministrations remainbeyondthereachofthe•
dissemination network.Thereareinfactagreatmanywhichdonot
havethenecessarystafforfinancialresourcestokeepuptodate
withalltheinformation whichwouldbeusefultothem,butgoin
Bearchofinformation onlywhentheyhaveaspecificproblemtobe
solved.
Thisfacthasledtothecreationofanumberofintermediary infor-,
mationandtechnicalconsultancy services,theresultpartlyof
private,andpartlyofpublicinitiative.Itshouldbenotedthat
.the"ex–quota"sectionoftheFEDEftprovidesforthefinancingof,
suchservicesincertainregions, _
'' ■ \Establishing adecentralized networkmerelyforthedissemination of-
Communityresearchresultsisoutof"thequestion,.Itwouldnonetheless
behighlydesirabletouseexistingdecentralized networks,suchas
professional associations ,chambersofcommerce,regionalconsxiltancy
services,localauthoritygroupingsetc,
friththisinview,itisrecommendedthattheCommissionshould
establishfirmlinkswithsuchorganisations ,andshouldorganize
information meetingsforthem.
Meetingsbetweenrepresentatives ofthoseorganizations couldhave
,considerable impactiftheyalsofacilitated theexchangeofinfor­
mationontheresultsofresearchundertaken byMemberStatesandif
theyledtothegraduallyestablishment ofcross–frontiercollaborsr -
tion.Thiscollaboration could,atalaterstage,beconsolidated by
creatingEurope–wideassociations , ?

-9-
3.2.3 Language barriers .
The fact that itisimpossible totranslate allthe scientific and
technical texts published bythe Commission into the sixofficial
languages ofthe Community isanot insignificant barrier tothe optimal
dissemination ofresearch results .
Thegravity oftheproblem should notbeexaggerated ,however .
Among scientists ofacertain level aknowledge ofthemore common
languages isconstantly improving .Ontheother hand when the
dissemination isextended toinclude firms (most ofwhich aresmall
andmedium-sized )andgovernment bodies theabsence oftranslations
may result ininformation being rejected .
Careful selection oftexts tobetranslated isofcourse necessary to
avoid anywaste .
"Euro–abstracts "currently includes summaries inseveral languages .The
proceedings ofsome conferences are also translated into several
languages when itisboth possiblte anduseful .Occasionally ,major
texts arepublished inall sixCommunity languages .
Infuture it "Would bedesirable toproduce translations oftexts written
for awider public ,summary reports ,manuals ,monographs intended for
industrialists (particularly small andmedium-sized enterprises )and
administrative bodies (particularly regional andlocal government ).
Such aprinciple should nevertheless beapplied prudently inview of
the cost oftranslation and the time taken .Translation adds conside­
rably tothe cost ofcertain works while atthe same time restricting
themarket for each ofthevarious language versions .Atthe same
time ,delays inthe translation oftechnical texts arefrequently so
long that theinformation maywell beout ofdate bythetime it
appears inany language other than theoriginal .

-10-
Inconclusion ,theCommission would like tohave further funds (of
theorder of100000EUAto150000EUAperyear )availably forthe
translation andpublication ofselected works ,,
The Commission also proposes that a"translation fund "should beset
upwith thefollowing features .Financial aidwould "bemade available
tooneorseveral groups ofspecialist publishers whowould undertake
toaccept therisks inherent inpublishing translations ofscientific
I / .
.oftechnical texts ,reporting -theresults ofCommunity ornational
research relating tothepriority sectors ofthe common policy on
science andtechnology® Thepublishers would payroyalties tothe
fund inproportion tothenumber ofwork 'ssold .During itsfirst
year the "translation fund "should "befinanced tothelevel of
approximately 50000EUA ,sufficient for thetranslation into a'
second language ofabout twenty 200-page works .These appropriations
should beincreased gradually from year toyear ,but the first income >
should bereceived during thethird year .v ,
3.2.4 Problems relating to "contracts -
Inthecase ofEURATOM anaEECindirect action research ,clauses' in
the contracts laydown that the Commission shall have first priority "
indissemination . 1 -
Atthe same time ,when the contractor isalso inreceipt ofaidfrom
national sources the Commission isnot always able toinsist onthat
priority ,anddissemination isthen effected bynational administra­
tions orresearch institutes ,in .accordance with their own standards .
Dissemination outside thecountry oforigin may asaresult berestricted .
anddelayed .Itisthe Commission 'sintention torevise theterms of
such contracts toensure that inthecase ofjoint financing the
dissemination ofresults isundertaken intandem with the .national
authorities . ,
These problems donot ^.rise forECSC research .
\

-11-
3Budgetary problems . , ■
Except forthecases ofECSC andagricultural research ,theapprop­
riations necessary forthedissemination ofresearch results aremade
available under Chapter 36ofthe Communities' general budget .For
1980 these appropriations total EUA600000.
,Each year theCommission proposes anincrease inthese amounts totake
account ofthe increased volume ofresearch anddevelopment undertaken ,
.andtoimprove thequality ofdissemination along theguidelines which
have been defined above .
These proposals areregularly rejected bythebudget authority ,which '
only allows anannual increase of10to117&.This increase does nomore
than compensate for the effects ofinflation .
Atthe same time ,Community spending onresearch anddevelopment
increased by336$ from 70.5 million EUAin1973 to306million EUA in
1980 .Over the xsame period appropriations fordissemination increased
byonly 10Ofo.
The rationalization already described nevertheless allowed anaustere
.but satisfactory policy tobepursued until 1979 »andthe increase in
costs was contained byrecourse toprivate publishers andthepromotion
ofsales .Even so,in1979 and19^0 theCommission hashadtoresort to -
thetransfer ofappropriations inorder tomeet itsliabilities .
Receipts ,arecurrently estimated tobe100000EUA ,but cannot goon
increasing indefinitely ifthe Commission continues itsprevious policy
of -actively publicizing research results .This policy isbased onthe
widespread free distribution of •Euro–abstracts "andprogress reports on
programmes within the Community topublic institutions ,committees whose
work isconnected with the research programmes andresearchers and
industrialists participating intheprogrammes ;research reports are
«
circulated onamore selective basis .

-12-
-t :
s \
TheCommissionrealizesthatasfaras"thebudgetadoptionprocedure
isconcernedtheCouncilBudgetCommitteecanworkonlyonthebasis
offinancialconsiderations andca-nnotbeconcernedwith,therequire­
mentsofarationalpolicyforthemanagementofresearchresults.
Achangeinprocedurewould,however,besufficienttoimprovethe
situation. ' •
TheCommission'sproposalsforresearchappropriations (Chapter33of
thebudget)arebytraditionexaminedbytheBudgetCommitteeofthe
Councilandthegroupscompetentforquestionsofresearch(the"Res'earch"
groupandtheAtomicQuestionsgroup).If,whenexaminingtheassociated
costsofinformation dissemination ,theBudgetCommitteehadtheadvice
ofcompetenttechnicalgroups,iheCommission'spredicament andviews
wouldcertainlyreceivemoredetailedattention.
Itwouldbeequallyappropriateforeachresearchprogrammetoexpli–...■
citlyallowfortheorganization ofmeetings(conferences,seminars,
meetingsofcontractors).
I
Toconclude,aCouncilresolutioninsupportoftheguidelinesdescribed _
abovewouldhelptomakethenecessaryappropriations available.'

-13-
4.VALORIZATION OFRESULTS
4,1Current methods
The Commission 'svalorization programme during the last fewyears has
related principally toJRCinventions .This activity involves prospec­
ting inthefour establishments fornewly –invented products ,equipment
andprocesses ,developing andtesting them tothepoint where they can
beoffered forlicensing ,seeking suitable candidates forthegranting
oflicences andgiving licensees appropriate technical assistance .
V
Within thefield ofECSC research anumber ofpromising valorization
actions have recently been started . ~
Atpresent 80valorization actions areinprogress ,relating ,to55
patented and 25non-patented inventions .The small number ofactions
under way istheresult ofthedeliberately selective policy of
developing ,asageneral rule ,only those inventions which arelikely
toachieve anannual turnover greater than 100 000EUA within 5years
ofbeing commercialized .This evaluation isoften based onamarket
survey . „
\
Exploitable inventions aretheobject oftwenty –eight licences at
present inforce ,ofwhich anumber relate tomore than one invention .
Ofthese licences 14have reached the stage ofcommercial exploitation
with royalties being paid tothe Commission .
Ashasbeen explained above ,these modest results aremainly due tothe
nature oftheprogrammes .They could nevertheless beimproved .The
Commission has infact been making use oftheservicea ofmarketing
consultants since 1975 this isnowbeginning toproduce beneficial
effects .Inaddition ,theJRCprogramme authorized bythe Council on
13March 1980 ,includes anewfinancial provision forvalorization ,
totalling 509000EUAforfour years ,with oneresearch worker and
eight support staff .This provision will allow thedevelopment and
testing ofinventions which could nottake place under sectoral
programmes ,andconsequently attract theinterest ofindustrialists
andhelp licensees .

-14-
Asregards theindirect action research programmes ,thenumber of
inventions notified voluntarily tothe Commission bythe contractors
was solow that the Commission authorities decided tocjuestion each
contractor "byasking himtocompleite ayes/nostatement onhisinven­
tions .This procedure was adopted for thefirst 'new energy' programme ,
which hadbeen completed in1979t brought abqut thenotification
of107inventions ;ofthese ,90areorwill become thesubject of ,
patent applications whilst theother 17relate tounpatentable inven­
tions .
Thecontractors areonly obliged toexploit these inventions within a
3-year time limit laid down inthecontract .(Regulation (EEC)2380/74
of17September 1974 fixes therules for vthedissemination ofinformai -
tion relating toEECresearch programmes ).
\ . • •• /■*'
Confirmation that these 107 inventions arebeing exploited bythe con­
tractors will nottherefore beavailable inmost cases before 1982 .For
10ofthese inventions ,however ,thecontractors have requested the
'Commission 'sassistance inseeking licensees ,andpriority will be
given tothe evaluation ofthese inventions .
Asimilar survey isnowbeing carried out amongst thecontractors of
/ .
the "Radioactive waste management and storage "programme .Itwill be
.extended toinclude other programmes likely tohave produced inventions
before being completed .
.4.2 Proposed improvements " '
4.2.1 JRC inventions
Theindustrial importance ofthe current JRC programme isrecognized ,
but itincludes fewresearch areas likely toproduce inventions which
canbeexploited inthe short term .

-15-
Thebudget provisions forvalorization mentioned earlier will allow
the Commission valuable scope for action .The Commission considers
that nofurther improvements should'be proposed before the results
ofthis newline ofactivity have been evaluated .Nevertheless ,the
appropriation only covers development andfeasibility studies carried
out atthe JRC .Itisessential that funds bemade available for
further market studies ,theeffectiveness ofwhich nolonger needs
tobedemonstrated .These studies have produced anumber ofbenefits
for the Commission and itspartners :
–amore effective selection ofinventions which are worth valo-
rizing ;
–information about firms likely tobeinterested inexploiting .-
aninvention ;
–presentation ofarguments likely toconvince industrialists
ofthevalue ofaninvention ;
«---frequently decisive guidelines forresearch inprogress or
tobeundertaken .
There is,finally ,every reason tocontinue thedistribution of
technical notes andparticipation atexhibitions asinformation media .
4.2.2 Inventions resulting from research carried outunder contract
Asregards theregime applicable toinventions ,most research
programmes begun since 1974 aresubject tothe Council regulation
of17September 1974 *Only onvery rare occasions have there been
difficulties innegotiating theterms ofcontracts inline with
theRegulation . , ', -
However ,inJune 1977 ,theUnion ofEuropean Communities Industries
(UNECl ),sent totheCommission areport criticizing the
current system .The Commission 'sresponse tothis criticism was
1
asfollows :
–that itwasnotappropriate atthetime forthe Commission to
propose anyamendment tothe1974 regulation ;

–that ontheother hand anyproposals for improvement mthe
drafting ofcontracts asregards clarifying the rights and _
obligations ofboth parties under the 1974 regulation would
beacceptable , % '
The Commission therefore undertook only minor revisions tothe
drafting oftheterms ofthe contracts and itwas observed that
few contractors raised any objections during negotiation comparable
tothose ofthe UNECI .Itcantherefore beconsidered that the present
system iswidely accepted ,at'least during the contract negotiation
stage . /
Ontheother hand ,because ofthethree –year time limit for exploiter -
tion allowed tocontractors ,there has notyetbeen anyexperience
concerning the clauses relating totheobligation toexploit . s
Ashasbeen mentioned above ,systematic inquiries regarding contrac­
tors' inventions areundertaken when the scope offered byaprogramme
justifies it.Inventions reported arefirst evaluated byCommission
departments with aview toidentifying those where acheck should be
made attheendoft&ree years -that exploitation hasbegun .
Itwill therefore notbeuntil 1982 that the Commission will beina
position todecide whether ornotamendments tothe1974 regulations
should beproposed . .^
Nevertheless ,twoweaknesses have atthis stage been observed inthe
system : . v ~ . y
–the Commission hasnoexplicit right toinquire about a
contractor 'sintention toexploit aninvention before the '
expiry ofthethree –year time limit ,andafter that expiry
itmay .well transpire that opportunities forexploitation have
been lost ;
–provision ismade forthesharing ofroyalties bythecontractor
andtheCommission only ifthe Commission hasexercised its .'.
*
right togrant asub–licence when .insufficient exploitation
work hastaken place ;itwould ,however ,bemore equitable if
.such sharing were possible each time the Commission isobliged
byacpntractor 'sinaction totake responsibility forthe

-17-
valorization ofaninvention ,even ifthecontractor finally
grants the licence bymutual agreement .
These twoshortcomings could undoubtedly heovercome "byminor
amendments tothe standard clauses ofresearch contracts and
without amending the1974 regulation , ;
Inorder tocarry outaneffective check that theobligation to
exploit isbeing met ,andtooffset contractors *failure toact ,the
Commission should have appropriations available forfeasibility
studies ,market research andanynecessary ,promotional activities .
\• ■
Finally ,itshould benoted that forseveral years theclauses of
Euratom research contracts relating tothe exploitation ofinventions
have been inline with those ofthecontracts under the 1974 Regula­
tion ,sofar asispossible under theprovisions ofArticle 12ofthe
Euratom Treaty .
Revision ofthe clauses inECSC financial aid contracts would also be
desirable .This should allow aclarification ofcertain obligations of
thecontractors andensure effective exploitation . ^
4.3 Budgetary problems
Apart from theJRCvalorization fund ,appropriations intended forvalo-
rization (feasibility studies andmarket research )should inprinciple
beentered under theheading ofChapter 36oftheCommunities' general
budget .This budget does infact include item 3611 having atoken entry
andanappropriation under chapter 100 .Thetoken entry does ,however ,
allow limited transfers during theyear from other Chapter 36headings .
Sofaronly -asmall number ofmarket studies have been undertaken :about
10peryear atanaverage cost of3000EUA.The costs ofprinting and
distributing 'technical notes' andofparticipation atexhibitions are
met outofdissemination appropriations . ~
Asinthe case oftheappropriations intended for thedissemination of
research results ,itwould beappropriate forthe groups competent for
questions ofresearch toexpress anopinion ontheCommission 'sbudget
proposals (seeparagraph 3»3above ).ACouncil Resolution insupport of
the .guidelines defined above would facilitate theallocation ofthe
necessary appropriations .

-18-
5.COORDINATION OFNATIONAL POLICIES
Ashasalready been stressed several times inthis report ,noinfra *- .
structure forapolicy ofexploiting Community research results would be
really effective anduseful unless itwere also beused byMember ^States
fortheresults oftheir ownresearch programmes .
There isnodenying theobstacles which such aproject would meet ifit
were tooambitious .Thecoordination ofnational policies ondissemi­
nation andvalorization canonly bealong-term objective ,since the
policies areeven more diverse than thepolicies onresearch and
development themselves and ,insome Member States ,relatively unde­
veloped * ,
However anumber ofspecific initiatives incoordination andcooperation
would present fewpolitical orfinancial problems andwould beinthe
recognized interests ofboth the Community and theMember States .
Itshould notbeforgotten that some projects ofthis nature already
exist ,for example the listings ofnational and Community research
projects (ACREP ,andENRAP ),publications andconferences giving the
,latest information onnational and Community research ,and Commission ,
cooperation with anumber ofnational valorization bodies ,particularly
injoint participation atexhibitions .Aspart ofthe second action plan
inthefield ofscientific andtechnical information anddocumentation ,
the Community will also begiving financial assistance toaninformation
andexchange system fornon–conventional (grey )literature ,established
bybodies from several Member States andwhich should help develop and -
accelerate theexchange ofresearch results .The Commission will also be
giving support toaFrench system ofinformation onmarketable technology
which isprincipally intended forvalorizing theresults ofpublicly
financed research .
Other joint initiatives could betaken inthefollowing areas :
–cooperation between information andtechnical consultancy
service networks ;
-joint organization ofscientific andtechnical conferences ofEuropean
interest ;

- ■ ί^'-
*
-19-
–expansion ofCommunity publication activities anddata-bases onthe
results ofnational research inpriority sectors ;
–establishment ofreferral services allowing theareas ofinterest
ofresearchers engaged inCommunity ornational research andof
thepotential users ofresearch results tobeidentified ;
–joint market studies aimed atvalorizing theresults ofCommunity
andnational research ;
–joint publicity projects byvalorization organizations ,e.g. during
technology exhibitions .
Following theguidelines laid down bythe Council Resolution of14January
1974 andthe Council fsconclusions of20December 1979 »"theCommission
nowproposes that during 1981 CREST should examine andcompare thevarious
national andCommunity methods ofdissemination andvalorization andthen
decide what initiatives ofCommunity interest could usefully beundertaken
bythe concentration ofexisting resources .

20-
TH5 STIMULATION OPINNOVATION
I
Nomatter howjudicious theefforts made toensure theexploitation of
research results ,they will not succeed unless thepotential users of
research results areprepared totake advantage ofthem .
Ithas tobesaid that the existence ofanumber ofdisincentives to
innovation andthegeneral economic situation scarcely encourage industry
totake risks .
Itisnowadays recognized that innovation does notfollow spontaneously
from support forresearch anddevelopment andthat other factors have
arole toplay before economic development canbenefit fully from the
contribution ofscience andtechnology . •
Itisforthis reason that Member States areintroducing agrowing
number ofmeasures aimed atencouraging theprocess ofinnovation ,
although notonasystematic basis .Itisfo^rthe same reason that two
years agotheDirectors –General ofindustry intheMember States –who
meet periodically attheinvitation oftheCommission –setupanadhoc
group ofgovernment experts toconduct apreliminary enquiry into
innovation with theCommission .Inparticular ,theenquiry hasconsidered
the following questions :
–acomparison ofnational and Community measures which influence
innovation ; - *
–astudy ofbarriers toinnovation ;
–American andJapanese measures for the stimulation ofinnovation ;
–thefunding ofinnovation ; f .
–the influence ofpublic procurement ; «
–theaggregation ofmarkets ;
-<■therole ofinformation ,patents andlicences .
Thegeneral problem ofinnovation isnotthesubject of .this report
however ,andshould notbedealt with here .The Commission will put
itsproposals onthe subject tothe Council atamore appropriate time . •
Nevertheless theexploitation ofresearch results ismerely oneparti­
cular aspect ofafargreater problem towhich reference was therefore
appropriate . ,

-21-
7.SUMMARY OFTHEMAIN PROPOSALS
Theprincipal guidelines andproposals ofthis report canbesummarized
asfollows :
a)General guidelines
–Results should bechannelled more specifically towards their
users through documents available inalanguage accessible
tothem ,andthrough translation anddecentralization ;
–Valorization oftheresults oftheCommunities *ownresearch ,
andfollow-up oftheexploitation ofresults ofindirect
research require amodest increase infinancial resources ,
b)Dissemination ofresults -
–Special presentation ofcertain types ofinformation isnecessary
toensure that allpotential users have access toresearch results ;
forthis reason ,certain research areas should bethe subject of
information innonr-specialist 'sterms ,summaries ,manuals ,
monographs ,etc.
–Information seminars forusers andfinancial incentives forvisits ,
secondment and study tours with research teams andoperators of
pilot installations anddemonstration projects would bedesirable .
–More use should bemade ofexisting dissemination channels ,
including asamatter ofpriority intermediary information and
technical consultancy services .
–Translation facilities should beavailable fortexts ofgeneral
interest andforthis purpose thenecessary appropriations should
beincluded inthebudget ;a'translation fund' will beproposed .
–Provisions inresearch contracts should beamended toensure that in
thecase ofjoint financing results aredisseminated jointly with
national authorities .
c)Valorization ofresults
–Modest financing isnecessary forfurther market studies inconnection
with inventions arising from direct andindirect research activities ;

-Theprovisions ofresearch contracts should "beamended toensure
a)that theCommission isinformed ofintentions regarding the
exploitation ofinventions without waiting thethree years
laid down inthecontracts ;
b)that ,intheevent oftheCommission taking responsibility for
thevalorization ofacontractor 'sinvention ,Commission and
contractor share anyroyalties . : ,
Coordination ofnational policies
Following theguidelines ofthe Council 'sresolution of14January
1974 andtheCouncil 'sconclusions ot20December 1979 »"theCommission
nowproposes that during 1981CREST should examine andcompare the
various national and Community methods ofdissemination andutilization ,
anddetermine onthe "basis ofitsfindings what initiatives could
usefully "beundertaken "bytheconcentration ofexisting resources .
Implement ation
-TheCommission ,suggests that infuture itsproposals for thebudget
relating tothedissemination and valorization ofresearch results
'should beexamined not only bytheBudget Committee oftheCouncil ,
butalsothat anopinion should begiven bythegroups competent for
questions ofresearch ,asisthe case forbudget proposals relating to
research .
-Atthe appropriate time theCommission will submit tothe Council
adraft resolution bywhich theCouncil would give support tothe
guidelines set out inthis report .
-In1982 theCommission will ifnecessary submit totheCouncil
proposals fortheamendment oftheregulation of17September 1974 *

APPENDIX IT
EVALUATION OFTHE COMMISSION
RESEARCH MO DEVELOPMENT PROGRAMMES
Current and planned activities
INTRODUCTION -
1.Atitsmeeting ofthe 21st October 1979 ,theCouncil stressed
that the ultimate aim ofthe Community research must beto
produce results capable ofcontributing totheeconomic ,social
andother .objectives pfthe Community and itsMember States .
Itaccordingly requested theCommission todevelop appropriate
proposals for asystem forevaluating the results of'common
Research andDevelopment programmes (conclusions agreed atthe
619th Council Meeting of20th December 1979 ).
2.Forsome time already ,theCommission hasbeen studying ways andmeans
tobetter assess the results ofCommunity R&Dprogrammes in
scientific aswell asineconomic and social terms .
3.InJune 1978 itorganized aninternational seminar on"Research
Evaluation "inorder toanalyse existing evaluation methods and
procedures with theaim ofdetermining the most appropriate
methods tomeet the specific characteristics ofthe Community R&D
programmes .
1
A.In1979 itwasdecided toundertake afirst pilot experiment by
entrusting ateam ofindependent external experts toevaluate
twosub-programmes oftheenergy research programme inorder to
gain experience astothemethodology best suited toevaluate
Community "indirect action "research programmes .Thefinal report
ofthis first test case makes numerous practical recommendations .
Additional pilot experiments arepresently under way .

5.Settingupageneralevaluationsystemhoweverisa
<complextask.Anassessmentofthemethodandthe-
definitionofanevaluation mechanismwhichmightbeapplicableto
allECresearchprogrammeswillbemadein.thelightofthe
outcomeofthesefirstevaluations andtheirutilization .
COMMISSION ACTIONSINTHEFIELbOFRESEARCHEVALUATION
PRESENTSITUATION
6.TheCommissionhassincelong(see"CommonPolicyforScienceand
Technology"1977)recognizedthattheassessmentofthescientific/
< »
economicandsocialusefulnessofR&Dprogrammesisamatter ,of
greatimportance andurgency,especiallyatatimewhenresources
arebecomingincreasingly scarse.
7.Asafirststeptoimprovingcurrentevaluation'proceduresfor
Community R&Dprogrammes,theCommissionorganizedaseminarin
Copenhagenbetween29Juneand1stJuly1978.Theseminarwhich
gatheredover60participants representing awidespectrumof
interestsinR&Danditsevaluation,aimedatreviewingcurrent
evaluationpracticeatCommunityandnationallevelandat
suggesting waysandmeansofimprovingcurrentevaluationprocedures
appliedtoCommunity R&Dprogrammesinordertofacilitatethe
relevantdecision-making andplanningprocesses.
8.Whendiscussingresearchevaluation,threedifferentphasesshould
beborneinmind:before,duringandaftertheresearchprogramme
isimplemented .
Atitsfirstphase,theevaluationallowsfortheproperdefinition
ofthecontentandaimsofaparticular R&Dprogramme,including
themeansrequiredtoeffectively carryitout. <
Atthesecondphase,theevaluationisanessentialtoolduring
themanagement "ofaresearchprogramme.Itshouldensurethat
progressoftheresearchisinlinewiththeoriginalaimsand
objectivessothatthenecessaryadjustments"canbemadeatthe

3..Ci
appropriate time .,finally ,atitsthird phase ,theevaluation
isthe measure ofthefinal success ofagiven research in
relation tothe input ,inmoney and manpower .Itshould provide
anappreciation oftheresults interms oftheir scientific
value andtheir -economic and social impact .Although the three
evaluation phases are closely related ,they serve clear Ly
different objectives and usedifferent procedures .
9.The seminar focussed onevaluation methods and procedures which
arenecessary toensure aneffective evaluation both during the
execution ofthe programme and anex-post evaluation ofthe
results . '
10.Aswas recognized atthe seminar ,the Commission ,assisted by
the ACPM 'sandother experts ,has been carrying out the evaluation
ofits R&Dactivities asanessential and permanent task during
the implementation ofits programmes .This evaluation ensures
that the research isconstantly adapted andreoriented inthe
light ofprogress made,ofnew developments andofnew problems
encountered .
11.As ageneral conclusion the seminar confirmed that research
evaluation isimperative ,buthighlighted thedifficulty ofthe
task and therelatively undeveloped state ofthe art inevaluation
methodology .Itwas evident that noapriori evaluation system
existed which could beapplied toallevaluation activities .The
numerous methods that had and were being applied nationally had
been developed tomeet specific needs ofparticular activities .
Theevaluation ofR&D programmes ,itwasstressed ,should not be
made without reference totheoverall environment ofthe programmes
and shouLd inparticular beappropriate tothe nature ofthe work
andtheinstitution inwhich the work was being carried out.Under
these circumstances ,itwas concluded that the Commission should
develop its ownevaluation criteria tosuit thespecific characte­
ristics ofCommunity research and development programmes .

12.Itwas suggested amongst others that thepresent use ofthe peer
!
review "method could find appropriate application inth.e European
context . .
R&D EVALUATION INMEMBER STATES ,THE U.S.A. AND INTERNATIONAL ORGANIZATIONS
13,Growing awareness ofthe need for research evaluation isdeveloping
inthe .frame ofnational activities inmember countries where
research result assessment isbeing undertaken indifferent degrees
and indifferent forms .While itisnotpossiblein this context ,
X6describe indetail the current state ofthe art some examples
can becited .
Inthe Federal Republic ofGermany amulti-level evaluation scheme
hasbeen set upforevaluating the selection ,progress and results
ofGovernment supported R&Dprojects .The scientific ,technological ,
economic and administrative performance ofR&D institutions asa
whole are also evaluated externally . -
InFrance some interesting examples exist ofthe evaluation of
whole institutes bymeans ofexternal auditors (e.g. Gaz deFrance ,
Bureau National deMetrologie ,etc...). The CNRS currently evaluates
itsresearch attwo levels ,that ofcontracts and that ofresearch
teams .This iscarried out bytwoexternal bodies ,the Comite de
Direction deTravail andCommission duComite National which cover
allmajor scientific disciplines .Inaddition ,supervisory bodies have
.been set upforeach research programme ,Comites desActions Thematiques Pro­
grammes ,which areplaying anincreasing role inevaluation activities ..
Inthe United Kingdom evaluation efforts have been devoted primarily
tothe selection ofR&D areas andprojects andthe subsequent distri­
bution offunds .Typical example ofthe apriori evaluation ofresearch areas
isfound inthe context ofthe activities oftheResearch Requirement
:Boards . "
InItaly ,examples also exist ofR&Devaluation activities andefforts
arebeing devoted tothe study anduseofappropriate methodologies .
InIreland ,experience exists -intheevaluation ofresults of
research ,primarily output oriented andconcerned with applied R&D
and mainly though notexclusively inthearea ofagriculture
and food research . • .^

InBelgium emphasis isplaced ontheselection ofresearch projects
andperformance andoutput monitoring during research implementation .
Inthe Netherlands research evaluation ispresently being promoted in
thecontext ofspecific sectors ,system which isbeing extended ,through .
amore diversified -partly project oriented -flow offunds foruniver­
sity R&D andthrough thereorganization ofinstitutes asTNO.Aspecific
example being theexplicit evaluation ofthefirst Dutch satellite project
Finally the United States have since many years associated great
importance tothe evaluation ofresearch .Very large sums are devoted
annually toevaluate federally sponsored programmes .Evaluation methods
and procedures vary widely according tothe type ofresearch being
evaluated and eventual applications .From quantitative andqualitative
reviews offundamental and applied research ,toyearly evaluation of
federally sponsored institutes ,orpanel reviews ofpluriannual
programmes ,policy related research activities ,etc...
Attheinternational level ,worth mentioning isthe "evaluation of
the economic effects ofthe agreements concluded between CERN and
the Industry "and theevaluation ofeconomic spin-offs ofthe
industrial contracts of ,the European Space Agency .
The OECD has also showed the importance' itattaches
tothe subject ofresearch evaluation ,byorganizing amajor conference
on"Science and Technology Indicators "inParis inSeptember 1980 .
Similarly the U.N. Economic Commission for Europe will organize a
symposium inPrague inSeptember 1981 on"Research Evaluation "to
review theactivities ofits member countries inthis field ,analysing
the state ofthe art and drawing attention tothe need for research
result assessment .

CURRENT COMMUNITY R&D EVALUATION PRACTICE
16.Inattempting toset upacomprehensive Community evaluation mechanism
itisnecessary toconsider the current evaluation practice as
applied toCommunity R&D programmes andtoidentify areas for improve­
ment .The current evaluation procedures themselves may vary from
•programme toprogramme and inparticular they vary according tothe
different programme execution modalities such asdirect andindirect
actions . -
. •• ' - /
17.Inthe case ofthedirect action programmes ,theevaluation practices
arestill evolving .They comprise two major aspects :
-the evaluation ofthe results with respect totheobjectives during
the phase ofexecution ofthe research programme :this process
isreferred toas"Programme Evaluation ";
-the evaluation ofthe results after completion ofacertain
activity :this process isknown as"Result Evaluation ".
\
18.'The "Programme Evaluation "includes internal andexternal evaluation ,
processes .The internal evaluation inessence consists inchanneling -
information ontechnical progress ,financial status ,planning ,etc...
tomanagement levels where appropriate decisions can betaken .The
4
most important tool inthis evaluation isanadequate programme
progress reporting which permits atimely comparison ofwork
achievements andstated objectives and enables appropriate modifica­
tions tobemade whenever necessary .Other typical important inputs
are work plan flowsheets ,budgetary and manpower analyses ,etc...
Theexternal evaluation isbased on'theopinions expressed bythe
members oftheAdvisory Committees onProgramme Management andthe
General Advisory Committee .Their inputs carry substantial weight
andplay animportant role intheinternal decision making processes .
The combination ofinternal andexternal evaluation processes
highlight ifand where adjustments need tobemade interms of
objectives ,budget ,manpower andplanning .

7.'/I
19.The"ResultEvaluation"canbedescribedasavaluejudgementof
!theR&Doutput,inotherwordsanassessmentoftheextentto
whichtheR&Dresultsareusefultotheoutside"customers".
Forsuchapurposesomeindicatorsforscientificproductivity
havebeendevisedsuchas:,
-indicatorsrelatedtothetransferofknowledgeincluding
publications,patents,licences,aswellastrainingand
educational activities
-indicatorsrelatedtothedegreeofjointworkwiththeinterna­
tionalscientific community,includingworkforoutsidecustomers
andinternational collaboration .
20.TheJointResearchCentreiscontinuously collectingdataonits
outputsandonhowtheR&Dresultsarebeingputtouse.TheJRC
isalsoinvestigating differentpossibleevaluationmethodologies
inv{ewoftheirapplicationtoitsspecificneedsandisexamining
waysandmeanstobetterchanneltheevaluationresultsintothe
decision-making andplanningprocess. -.■-
21.Fortheindirectactionprogrammes,presentevaluationprocedures
maydifferfromcasetocaseaccordingtospecificcharacteristics
andrequirements .Atypicalexample.oftheevaluationofindirect
actionprogrammesconcernsanevaluationprocedurecarriedoutat
twolevels:the"evaluationoftheprogrammeasawhole"andthe
"evaluationofindividual (orgroupsof)researchprojects".
22.Theevaluationofawholeprogrammeisperformedbyexaminingthe
programmewithrespectto:
-itscorrespondence totheinitiallydefinedgoalsorobjectives
(success,failure),includingitsefficiency,
-itsgeneral(technicalorother)"fall-out",
-potentialusersoftheresearch.

8.y>
Suchanevaluationisundertaken inordertodecideonthereview,
-continuation orabandonment ofaprogramme.,
Inadditiontothecontribution oftheCommission'sstaff,the
AdvisoryCommittees onProgrammeManagement,theCRESTCommittee
andoutsideexpertscontributetothisexercise.
Theevaluationofindividualresearchprojectsincludesthe
examination ofaspectsconcerningthe:
-qualityoftechnicalorscientificresults,
-achievement ofgoalssetoutinthecontract,
-intrinsiceffects(coordination,stimulation offurtherwork,
educational impact,etc...). >
Thiskindofevaluationtakesplacepermanently andindiffering
ways.Theyincludeanappreciation oftheresearchworkbythe
Commission'sstaff,outsideexpertsacting -asprojectleaders
andACPM's^nthebasisofregularprogressreportsaswellas
indicatorsof"success"suchaspublications,patentsandlicences,,
etc.;.Inthiscontext,the"contractors meetings"deserveparticular
mention.Theseareorganizedonaregularbasisduringtheprogramme
implementation ,andallowthecontractors workinginthesamefield
notonlytodeliverpresentations oftheirownworkbuttooffer
theirworkforevaluation bytheircompetitors "aswellbythe
Commission'sstaffandprojectleaders.-
CHOICEOFANEVALUATION METHODOLOGY ''
23.Theanalysisofthecurrentevaluationpracticesshowsthattheyare
largelybasedoninternalproceduresandconcentrate ontheimple­
mentationphaseoftheprogrammes.Inthelightofthisanalysisand
inlinewiththefindingsoftheCopenhagenseminar,theCommission con­
cludedthatitscurrentevaluationprocedurescouldbeusefully -
strengthened byplacinggreateremphasisonthe.retrospective assess--
mentofprogrammes.Inordertosupplementtheinternalaspectof

9. 
present evaluation procedures ,thedecision wastaken toappeal to
external independent experts who were not involved either inthe
adoption phase oftheprogramme orinits ^implementation .This
kind of"review byexternal evaluators "isalready been followed
inmany instances both inthe Community Member States andoutside
the Community especially inthe United States .However major
adaptations need tobeintroduced because oftheparticular charac­
teristics and scope ofthe Community programmes .
This review should provide aneffective aidfor :
-the Commi ssion when defining andreorienting itsresearch strategies
andpriorities onthe basis ofidentified needs ;
-the Counci I,European Parliament ,Economic and Social Committee ,
Member States and delegated bodies (e.g. CREST )toassist them inthe
evaluation ofthe progress ,utility andcontributions ofCommunity
research ,providing them with valuable ir.puts into their decision -
making process for programme revisions and extensions .
24.Having identified the above asthe most direct and interested users
ofthe outcome ofthe Community programme evaluations ,the following
evaluation objectives were defined asofprincipal importance :
-thescientific andtechnical quality oftheresults ,
-management effectiveness and useofresources ,
-contribution ofthe programmes toCommunity sectoral policies and
objectives ,tothe socio-economic development ofthe Community and
_tothe development ofrelated R&Dwithin the Community .
TEST CASE
25.Before embarking onasystematic evaluation ofthe Community programmes ,
itwasfelt appropriate tocarry out some test cases inorder togain
experience onprocedures andcriteria applicable toCommunity R&D.

110.
Furthermoreitwasdeemednecessarytohaveameasureofthevalidity
ofsuchanevaluation methodbyassessingtheusemadeofitby
decision-makers andplanners.
Thedecisionwastakentostartwithsomeindirectactionprogrammes .
Thisdoesnothoweverexcludetheusefulness -oftestingthemethod
-subsequently ondirectandconcertedactions.
ForthefirstevaluationexercisetheEnergyConservation andSolar
Energysub-programmes oftheCommunity'sfirstEnergyR&Dindi-rect
actionprogramme (1975-1979)-wereselected.Thechoiceofthese
twosub-programmes ,wasdictatedbytheirimportanceinresources -
andpotentialimpact,thedifferentnature(longandshort,term)
oftheresearchandthetimingsincetheevaluationcouldfollow
justafterthetermination ofthesub-programmes andcouldbe
utilizedasaninputfortheprogrammerevisionenvisagedin1981.
26.Thefiveexternalexpertscarryingoutthereviewwerenominated
bytheCommission ontherecommendation oftheEuropeanCommittee
onResearchandDevelopment (CERD),theCommission'sprincipal
independent advisorybodyonallmattersrelatingtoR&D.They
« -
wereselectedonthebasisoftheirexpertiseintheirrespective
fields,theirknowledgeofevaluationprocedures andtheir
independence withregardtotheCommunitysub-programmes being
evaluated.
27.Theexternalevaluationteamweregiventhefollowingtermsof
reference :
-toassessthescientific andtechnicalqualityoftheresearch
undertaken "andoftheresultsachieved;
-toevaluatetheeffectiveness ofthemanagementoftheprogramme
andoftheresourcesutilised;
-todeterminethepracticalcontribution oftheresultsofthe
twosub-programmes totheprogressofR&Dinthesetwoareasof.
research,toCommunitysectoralpoliciesandtothesocio-economic
development oftheCommunityingeneral. •

ThepanelstarteditsworkinOctober1979,takingeightmonthsto■
completeitsfinalreport.Havinganalysedthemanyexistingevaluation
proceduresandcriteria,theteamunanimously agreed,inlinewith
theconelusionsoftheCopenhagenseminar,thatnoapriorievaluation
schemecouldbeappliedinthegivencontextanddecidedtofollow
aflexible ,andpragmaticapproach.Thepanelhasthereforedeveloped
andapplieditsowncriteriaandprocedurestomeetthespecial
characteristics ofthesub-programmes beingevaluated.
ThefinalevaluationreporthasbeensubmittedtotheCommission
andisbeingpublishedinthesixofficiallanguagesoftheCommunity
forawidedistribution (EUR6902).
Asafirstgeneralappreciation,theCommissionconsidersthatthe
evaluationreportprovidessubstantial information whichcanbe
expectedtogiveavaluableinputforthe1981revisionofthetwo
sub-programmes .Detailedassessments arepresentedoftheachievements,
programmebyprogrammeandsectorbysector,pointingoutnotonly
thescientific andtechnicalvaluebutotherimportantelementssuch
aspromotionofinternational cooperation andimpactonnational
activities.Whererelevant,recommendations aremadetorectify
identifiedshortcomings .Usefulsuggestions areincludedforthe
valorization oftheresearchresultsobtained.Attentionisalso
focussedontheprogramme'smanagerialaspectsincludingprocurement
ofresearchproposals,selectionofcontractsandmonitoringof
researchwork. .
Certainotheraspectsofthereportmaybelesscomprehensive such
astheareaofeconomicandsocialimpactassessment .Thisvery
importantobjectiveisoneofthemostproblematic sincethe
correlation betweenresearchresultsandtheirpracticalapplication
isoftenmadedifficultduetothetimelaginvolved.Inthisarea
theavailablemethodologyisclearlyinsufficient andfurtherresearch
isneededtoequiptheevaluationteamwithmoreadequateinvestiga­
tionmeans.

12.
32.This pilot experiment has enabled the Commission staff responsible
for the evaluation activities togain experience intheorganization
ofevaluation exercises ,inparticular ir)the selection ofexperts , ■
duration andtiming ofthe evaluation ,its methodology and planning .
Firstly ,itappears that thecomposition ofthe team ofevaluators
should bemore -heterogenous ,thescientific experience ofthe team
which carried out thefirst exercise was too dominant which may
account /for the reduced emphasis onthe socio-economic aspects .
Secondly ,thetiming oftheevaluation could profitably bechanged
inorder tomaximize the use made ofthe evaluation .Instead of
carrying out the evaluation immediatly after the completion of a
programme itcould betimed sothat thefinal evaluation report
isavailable fordiscussion onthe programme extention phase .
This isindicated below ,taking asreference a5-year sliding
""programme scheme . '
PROGRAMME n
1Υγ
Evaluation
i J.
Submission
ofprogramm <
toCouncilPROGRAMME n+1
I I L.
Evaluation
Counci l
Décision
l Ltt
Submi ssion
ofprogramme
toCouncilPROGRAMME n+2
i i 1
Evaluation
Counci l
Décision

13.
Such timing would allow theevaluation toentirely encompass the
previous programme andpart ofthecurrent programme with the
advantage that assessments and recommendations would also cover the
current programme and would therefore provide anupdated evaluation
report atthe time when it ,ismost needed /i.e. before the new
programme adoption phase .Furthermore ,onthe basis ofthe
experience acquired ,improved planning oftheevaluation will ensure
more efficient and speedy procedures andtheinclusion ofresearch
productivity indicators will addvaluable objective information .
Inthis context ,the Commission organized acolloque inSeptember 1980
toanalyse the methodology utilized during this first exercise .The
practical andflexible approach adopted bytheevaluation team was
widely approved bytheparticipants while stressing the need to
strengthen certain methodological aspects inparticular for*thesocio­
economic impact assessment .
FUTURE PLANS
33.Inorder toprofit tothe maximum extent from thetest case the
Commission will undertake anin-depth follow-up tothis first
evaluation .After the wide distribution ofthe evaluation report ,
itisessential toreceive feed-back from potential users asto
itsvalue ,toseetowhat extent the recommendations are applied
and how useful they have been sothat improvements can bemade
where necessary .Inother words ,effective ways ofensuring proper
' impact andapplication oftheevaluation results must beanintegral ;
part ofthe evaluation mechanism .
34.Atthe same time the decision was taken toproceed with the evaluation
ofother programmes togain further experience and knowledge of
evaluation procedures andcriteria asapplied tothediffering
characteristics ofCommunity R&D programmes .Thus thethree remaining
sub-programmes ofthe 1975-1979 Energy R&D programme ,i.e. "geothermal
energy ,hydrogen production andutilization ,system analysis :develop­
ment ofmodels ",are now undergoing anevaluation along the same lines

14.■>i.
asthefirstone.Threeofthefiveoriginalevaluatorsarepartici­
patinginthesecondexercisetoprovidesomecontinuity ;theywere
joinedbythreeotherexpertstofillparticularneedsofspecialized
competence .
35.TheCommunityBureauofReferencesistheobjectofthethird
evaluationexercise.Itscharacteristics willrequiretheapplication
ofconsiderably differentevaluationcriteriaasafunctionofits
specificobjectives .N -.
36.Atthebeginningof1981the"Management andStorageofRadioactive
Waste"programmewillbeevaluated. /
37.Adifferentexercisewillbeorganizedinthefieldoffusionresearch.
Inthisareaastrategyforfuturedevelopment hastobeestablished .
Tothisendafusionreviewpanelofindependent highrankingexperts
willelaborateareportwhichwillstartwithanexamination ofthe
existingprogrammeandformulate recommendations onthechoiceofthe
nextmajorstagesfortheEuropeanfusionprogramme,takinginto
-account currentinternational developments andpossibilities for.
international collaboration .
Furtherexperiencewillalsobegainedfromthisuniqueexercise.--'
38.Inaddition,otherprogrammeswillbescheduledforevaluationin1981
possiblyincludingconcertedanddirectactionprogrammes.
39.TheCommissionwillorganizelatein1981anewmajor"Research
Evaluation"seminar.Theseminar,partoftheCommission'sactivities
instimulating theexchangeofinformation anddevelopments in.evaluation
methodologywithintheECcountries,willpermitthecontinuedconfronta­
tionofnationalevaluation workanddrawingrecommendations for
Community R&Dapplications . ',

15.
CONCLUSION
40.The Council has asked the Commission tomake appropriate proposals
for asystem forevaluating the results ofcommon Research and
Development programmes . ,
41.The Commission hasinvestigated available evaluation methodologies
within the Member States and elsewhere inorder toascertain the
possibility oftheir adaptation totheevaluation ofCommunity R&D
programmes .
42.The Commission considers itappropriate toadopt anevaluation method
which would supplement the evaluation practice which iscurrently
.being applied inthe frame ofits R&Dactivities .This method is
based on aperiodic retrospective assessment ofits research programmes
carried out by asmall group .ofexternal independent experts with the .
major objective ofproviding anadditional input into the programme
revision andextention decision process .
Before drawing final conclusions astothevalidity ofthis method ,
if
the Commission deemed /useful togain some practical experience .A ,
first pilot experiment has been completed satisfactorily .However
the assessment ofitsutilization bypolicy makers andplanners has
still tobe .undertaken .Other test cases are considered necessary
inorder toexamine different procedures andcriteria -which will
need tobeapplied tomeet theparticular characteristics ofthe
differing Community R&D programmes .
43.Late in1981 the Commission will organize aseminar topresent the
results ofthe test cases and confront them with similar activities
conducted within andoutside the European Community .Itwill bepossible
atthis stage todraw conclusion astothepractical utilization ofthe
results oftheevaluation atthe various policy levels .

-,16.
44.OnlyintheLightoftheexperiencegainedfromtheevaluationtest,
cases,offurtherinvestigations andoflessonsdrawnfromnational
evaluationactivities,theCommission wouldconsideritjustified
tosubmittotheCouncilanyproposalconcerning asystematic
evaluationtobeprogressivelyappliedtoCommunity R&Dprogrammes .
Itwilltakeintoaccount,interalia,necessarymanpowerand
financialrequirements .