## 1 Introduction
### Reliability of the Three Metrics Used in External R&D Evaluations  
The metrics used in the ex-ante R&D evaluation process are based on bibliometric data retrieved from three different information sources: the national information system \"SIRCIS\", the international information system \"Web of Science\" and data retrieved from all research organizations in Slovenia concerning their third-party funding of projects. All of these information sources offer quite reliable data and provide--to put it frankly--great support for the external R&D evaluation process at the Slovenian Research Agency. Let us take a brief look at the three databases from which administrative and quasi-expert bodies at the Slovenian Research Agency obtain the bibliographic information. First, Web of Science, which was developed by the E. Garfield Institute for Scientific Information in Philadelphia (now owned by Thomson-Reuters), offers a very standardized source of information concerning articles in scientific journals with an impact factor as well concerning the number of citations. Although the Journal Impact Factor was originally created as a tool to help librarians identify the journals they should purchase and not as a measure of the scientific quality of research in an article, in recent times this bibliometric tool has become a dominant way of characterizing the excellence of the publication output of researchers in many countries around the world [22]. In almost all countries where bibliometry in external R&D evaluation is used, the databases of the Institute for Scientific Information (ISI) in Philadelphia are the main evaluation instrument in science. With use of the Scopius database (Elsevier), the systems for evaluating scientific output gained a new dimension. In this regard, some very preliminary steps have been taken in Slovenia. Given that in Slovenia the use of data from Web of Science is very standardized, similar types of criticism as seen in other Eastern European countries are emerging. In Slovenia, complaints are sometimes made that Web of Science is too fragile and inaccurate in some dimensions. The main criticism concerns the lack of feedback loops by finding errors in data which regard to correct names of authors and assigning them to the academic institutions they come from. The experts at ISI do not \begin{table}
\begin{tabular}{|c|c|} \hline DIMENSIONS OF SCIENTIFIC & BIBLIOMETRIC TOOLS FOR MEASUURING SCIENTIFIC \\ PERFORMANCE & PERFORMANCE \\ \hline scientific productivity & the number of publications within the 5 last years \\ \hline scientific impact & the number of citations within Web of Science in the last 10 years \\ \hline efficiency in obtaining financial means & third-party funding within the last 5 years \\ \hline \end{tabular}
\end{table} 
Table 1: Bibliometric tools for measuring scientific performances in external R&D evaluations. check for such errors. Due to this inaccuracy, difficulties arise at the national level in efforts to obtain correct data. The Slovenian Research Agency also obtains information concerning the entire publication output of Slovenian researchers (not merely information about articles in scientific journals with an impact factor) from the national database. This database is called SICRIS and it provides a unique, officially maintained system of the complete personal bibliographies of all researchers registered with the Slovenian Research Agency. An extensive typology of publication documents has been prescribed in SICRIS in order to classify scientific bibliographic items for each individual researcher [23]. The national meta-database (SICRIS) has accumulated a large volume of highly standardized bibliographic data. Most other European countries use less standardized sources of bibliographic data than Slovenia, such as researchers' CV databases, open archive systems or subject-specific databases like the ArXiv repository, _etc._ Collecting data in this way is time-consuming and it is not easy to provide comparable data even within a national context [24].