  In the past, several works have investigated ways for combining quantitative
and qualitative methods in research assessment exercises. Indeed, the Italian
National Scientific Qualification (NSQ), i.e. the national assessment exercise
which aims at deciding whether a scholar can apply to professorial academic
positions as Associate Professor and Full Professor, adopts a quantitative and
qualitative evaluation process: it makes use of bibliometrics followed by a
peer-review process of candidates' CVs. The NSQ divides academic disciplines
into two categories, i.e. citation-based disciplines (CDs) and
non-citation-based disciplines (NDs), a division that affects the metrics used
for assessing the candidates of that discipline in the first part of the
process, which is based on bibliometrics. In this work, we aim at exploring
whether citation-based metrics, calculated only considering open bibliographic
and citation data, can support the human peer-review of NDs and yield insights
on how it is conducted. To understand if and what citation-based (and,
possibly, other) metrics provide relevant information, we created a series of
machine learning models to replicate the decisions of the NSQ committees. As
one of the main outcomes of our study, we noticed that the strength of the
citational relationship between the candidate and the commission in charge of
assessing his/her CV seems to play a role in the peer-review phase of the NSQ
of NDs.
