## 1 Introduction
### Validity of the Three Metrics Used in External R&D Evaluations
\begin{table}
\begin{tabular}{|c|c|c|c|c|} \hline Type of & Engineering & Social Sciences & Natural Sciences & Life Sciences \\ publication & Sciences & and Humanities & & \\ \hline articles & 31.56 & 32.73 & 66.93 & 51.02 \\ \hline proceedings & 63.98 & 26.93 & 21.93 & 37.00 \\ \hline chapters & 2.64 & 19.55 & 8.11 & 9.29 \\ \hline monographs & 0.44 & 20.77 & 3.25 & 2.20 \\ \hline patents & 1.38 & 0.03 & 0.34 & 0.50 \\ \hline \end{tabular}
\end{table} 
Table 2: Structure of different types of publications in the main scientific fields in Slovenia (in %)—period: 1990–2012 (source: SICRIS). In the Slovenian case, specific biases arise from using citations as a measure of scientific impact. Namely, the calculation of citations from Web of Science is based on a very strict understanding of what constitutes a valid citation. 6  
Footnote 6: The citations are collected from Web of Science for the period of the last 10 years. Self-citations are excluded. The main goal of excluding self-citations is to avoid the short-term effect of an author citing their own work in subsequent articles. In the procedure of “normalisation”, third-party citations received by authors from Slovenia are further divided by the average impact factor of ISI for the particular scientific field in which the article was published. Namely, only those citations of scientific articles that have a full bibliographic record in Web of Science are considered as valid citations. This means that only those citations from articles published in scientific journals directly indexed in the ISI Journal Citation Report are counted. Citations which come into the ISI citation databases indirectly (through reference lists at the end of articles indexed in the Journal Citation Report) are excluded. Such an arbitrary and restrictive 'normalization' of an indicator of scientific impact is very biased against the social sciences and humanities in Slovenia. Last but not least, it results in misleading information about the impact capacities of the social sciences and disfavors the social sciences, which publish less often in scientific articles than the natural sciences. Let us take the example of [PERSON], the most famous Slovenian philosopher and social scientist in the world. [PERSON] derives the biggest share of his scientific impact as measured through citations from monographs and chapters in monographs, _i.e._, types of publications outside scientific articles that have a full bibliographic record in Web of Science. I made a preliminary statistical analysis of his citations indicated in Web of Science. The result of my analysis shows that the ratio between all of [PERSON]'s normalized citations ('normalized' in accordance with the Slovenian Research Agency's methodology) and all of his citations in Web of Science is 1:22. Altogether, he received 11,056 citations, but only 568 citations that have a full bibliographic record in Web of Science. This ratio well demonstrates the bias emerging from the restrictive use of ISI citations as a measure of scientific impact. Without entering into a more extensive discussion of the epistemological reasons for the differences in the practices of using scientific citations, let us mention only the following fact: at the very beginning of the bibliometric evaluation of research activity, [PERSON] and [PERSON] [31] compared citations received by social science publications and by publications in natural science and technology. They found a big difference in both scientific fields. Several recent bibliometric analyses ascertained that books make up more than half of the published references in the main disciplines of the social sciences and humanities ([32], p. 482; [33], p. 5; [34], p. 365). The social sciences are also characterized by a slower pace of theoretical development, and this might also be reflected in various citation practices ([35], p. 1545; [36], p. 273). We could list many other epistemological reasons that produce differences in citation practices between the'soft' and 'hard' sciences.