# Modeling and Simulation Analysis of Journal Impact Factor Dynamics Based on Submission and Citation Rules

Authors: Jian Zhou; Lin Feng; Ning Cai; Jie Yang
Journal: Complexity
 (2020)

---

## Abstract

The variation of the journal impact factor is affected by many statistical and sociological factors such as the size of citation window and subject difference. In this work, we develop an impact factor dynamics model based on the parallel system, which can be used to analyze the correlation between the impact factor and certain elements. The parallel model aims to simulate the submission and citation behaviors of the papers in journals belonging to a similar subject, in a distributed manner. We perform Monte Carlo simulations to show how the model parameters influence the impact factor dynamics. Through extensive simulations, we reveal the important role that certain statistics elements and behaviors play to affect impact factors. The experimental results and analysis on actual data demonstrate that the value of the JIF is comprehensively influenced by the average review time, average number of references, and aging distribution of citation.


---

## Body

## 1. Introduction

Academic impact assessment and scientific journal ranking have always been a hot topic, which plays a very key role in the process of dissemination and development of academic research [1–3]. As one of the most important evaluation indicators in SCI, journal impact factor (JIF) is calculated by the scientific division of Clarivate Analytics® and commonly used to rank and evaluate the grades of various scientific journals in the Journal Citation Report® (JCR) database [4]. Since the introduction of the JIF, a growing stream of studies have discussed the mechanism, characteristics, and applications, as well as limitations and misuses, particularly in recent years. JIF first aims at evaluating scientific journals, but it is now increasingly used to assess research and guide publishing strategies of researchers and institutions. In this respect, JIF has gradually become an important indicator to measure the quality or reputation of a journal [5, 6]. Some publishers, for example, deem impact factor values as an indirect marketing tool for selling their journals. In [6], Larivière and Gingras argued that the JIF not only reflects the “quality” of a paper but also represents the reputation of the journal in which it is published because the possibility of citation to a paper is also significantly affected by the impact factor of the journal. In addition, JIF is often regarded as an important reference that can be relied on by most authors when they are preparing for submission [7]. A major weakness of the JIF, however, is that the two-year citation cycle of the JIF is considered too short to reflect the real academic impact of publications in some “slow” disciplines [8, 9]. The concept of JIF was first introduced by Garfield in 1955 [10], and its calculation can be formulated as the following form:(1)JIFk=Ncitationsk,k−1+Ncitationsk,k−2Npapersk−1+Npapersk−2,where JIFk∈R+ denotes the JIF of the kth year; Npapersk−1∈N denotes the number of papers published in the k−1th year; and Ncitationsk,k−1∈N denotes the number of those citations received during the kth year by the papers published in the k−1th year.According to the definition of the JIF, it can be found that the value of the JIF in thekth year is computed by Npapers and Ncitations in k−1th and k−2th years. In reality, however, the variation of the JIF is affected by many statistical and sociological factors. For instance, the value of the JIF would be strongly influenced by the research field of the journal and the type of the journal (full papers, letters, and reviews) [11, 12]. JIF is calculated via the number of references; thus, it could also be improved by increasing the number of references [12–14]. In [15], Zhang and Van Poucke suggested that journals with short publication delay tend to receive higher impact factors. Yu et al. developed a transfer function model to simulate the distributed citation process [16]. However, such a transfer function model does not consider the differences of citation behavior in different disciplines. Notwithstanding some researchers pay great attention to the significant difference in the levels of impact factors over different disciplines, they often neglect the results affected by the process of submission and citation [9, 12]. Clearly, scientific publication process and JIF need to be studied as a unified system, particularly as a complex system [17–19]. Scientific community, for example, in its most typical form, can be modeled as a complex system where researchers interact with each other taking the roles of authors, journal editors, and reviewers. Computer simulations are able to reproduce certain simple behaviors, and therefore, they can be used to model and reveal some correlations that are very difficult or impossible to be studied in real life.Random theory based on probability has long been the dominating methodology in the domain of scientometrics [20–22]. As a typical method of social system simulation, Monte Carlo method is very suitable for modeling such issues. In contrast to other approaches, the key idea of the method here is primarily based on distributed social computing. In this paper, we employ parallel social systems [23–26] to simulate and analyze the correlation between the variation of the impact factor and the behavior of submission and citation, which can be used to interpret the JIF dynamics. In essence, the main objective of the parallel system is prediction for analysis and control. Simulation models are used to generate a lot of empirical data by setting various conditions and tuning related parameters, while statistical analysis is to perform mathematical statistics and analysis on existing data and information, which can also be applied for prediction. The fundamental difference between statistical analysis and simulation analysis is data volume rather than prediction. Bibliometric indicators are quantitative measures of science based on the publication and citation data. They are characterized by a quantitative approach and evaluation scales, which can be macro, meso, or micro, and reveal the scientific performance of a particular field over time. This paper aims to address these problems by using the dynamic modeling approach of the system. The demand for the model is completely different when the model is designed for prediction or theoretical analysis. For the prediction systems, various possibilities should be taken into account as much as possible, and the hyperparameters of the model should be tuned based on the dataset observed from the real world. By contrast, for theoretical analysis, the system should be as simple as possible to grasp merely those prominent features. Thus, our model only considers some basic social factors, with the idiosyncrasy of individuals embodied by random noises. To the best of our knowledge, most of the parallel social systems are composed of individuals and communities, which are agent-based and networked [27–32], with some of the analytical properties potentially acquirable by relevant theories in system science [26, 33]. It is worth nothing that the primary goal of virtual simulation systems is not for mimicking the real-world counterparts quantitatively, instead, they should be very helpful for verifying, interpreting, and enlightening the underlying factors and the possibility of some of the phenomena and mechanisms, qualitatively. Such a framework has already been widely and effectively applied for analyzing various complex social systems, e.g., [34–38]. Moreover, the virtual citation networks generated by the model here are well compatible with the most typical scale-free networks [34, 39] (please see Section 2.3 for detailed explanation). We hope our research could provide meaningful theoretical hints and enrich the relevant literature studies for better comprehending the mechanism of JIF dynamics and further aiding to facilitate enhancement in managing academic journals. The key contributions of this work can be summarized as follows:(i)
We employ parallel social system theory to interpret the mechanism of JIF dynamics(ii)
We develop an empirically driven parallel experiment framework that analyzes interactions between JIF and submission and citation rules (see Section2)(iii)
The correlation between JIF and some elements and behaviors that are usually ignored is revealed via simulation experimentsThe rest of this paper is organized as follows. Section2 introduces the main framework of the model in detail. The relations between JIF and certain elements and behaviors are revealed and analyzed in Section 3. Finally, Section 4 presents the concluding remarks.


## 2. Model Construction

In this study, we consider the paper submission process, citation process, and JIF to construct a comprehensive system. We develop a virtual citation community in which authors submit and cite papers, and journals review and publish papers. Simultaneously, the model will record those already published and cited papers and compute automatically the impact factors of the corresponding journals based on the citation distribution. The simulation model is discrete-timed, and the unit of time is months, with each iteration representing a round of submission, publication, and citation. The model is implemented in MATLAB (MATLAB and Statistics Toolbox Release 2018b, The MathWorks, Inc., Natick, Massachusetts, United States). The program code can be found inhttps://github.com/pjzj/JIF-Modeling.


### 2.1. Model Initialization

The first stage is setting the parameters of the simulation model, where general knowledge of the system is considered, namely,(1)
The number of journals (NJ).(2)
The number of issues of a journal published per year (Nissues), and each month corresponds to each issue.(3)
The number of papers published per issue (NPissues). According to Nissues and NPissues, the number of papers published per journal within one year can be computed by NPyear=NPissues×Nissues.(4)
The average review time of journals (T). This parameter denotes the length between two time periods, which are the time of a paper being submitted and the time of the paper being accepted, respectively. For instance, the submitted date of a paper (no. 0826) is December 2018; then, the accepted date of the paper is October 2019, T0826=10 (months), and T represents the average value of Ti published in the journal (see Appendix Figure 1). It should be explained that the average review time (T) is a macrostatistical indicator, which reflects an overall level of process speed, being different journal by journal. Furthermore, for simplicity, assume that publication occurs instantly as a paper is accepted in the model.(5)
The average number of references per paper in a journal (N).(6)
Here, we define and assign some relevant parameters such asα, β, and γ because each of the subsystems can be parameterized independently (see Sections 2.2 and 2.3 for more details).Figure 1
Sketch of the average review time of a type of journal (T).
(a)(b)Note that, in the current model, similar disciplines can be represented by identical settings, while different disciplines are differentiated via tuning the parameters above. In particular, the calculations of impact factors between journals and their series (for example,Nature and Nature Cell Biology) are also independent of each other. Moreover, according to the definition of the impact factor which considers the correlation between the number of papers a journal published in the previous two years and the total number of citations, we uniformly set the impact factor in the first two years to 1 that could avoid certain undesirable situations in the citation process.


### 2.2. Modeling of Submission Process

The second stage is the modeling of the submission process, which mainly consists of the following three steps:(1)
Characterization of papers:The variation in paper quality is an objective fact, which is somewhat similar to the situation of examination scores in education. However, paper quality is difficult to be measured practically. For such situations, people generally assess through scoring questionnaire manner. Existing typical instances include publication scores (https://publons.com/), which are generated by selecting a score from 1 to 10 in two fields, jointly indicating the measure of a paper’s methodology, rigor, and novelty, as well as relevance to its field. Following the routine, in our model, the intrinsic quality of the paper is scored by number q∈0,10, with 0 being the worst and 10 being the best, and the overall quality of all papers follows a skewed distribution with certain expectation and variance. According to the expectation and variance of the distribution, the model will create corresponding numbers of papers with different scores, automatically (shown in Appendix Figure 2). In reality, due to the fact that the quality of the majority of papers is mainly mediocre-leveled, those papers with extremely high or low quality would be relatively less. Therefore, we assume that the quality qii=1,2,…,n of each paper is drawn from positively skewed distribution over the interval 1,10, which is implemented by gamma function. The basis for doing this is that, as the number of papers (c) keeps on increasing, the papers with ultrahigh quality become rare, and their number holds relatively stable, with only minor increase if above a limit. To ensure that the papers with high quality remain relatively constant regardless of changes of c, a numerical integration with respect to gamma function over the interval θ,+∞ is conducted in the model, which is computed by the following form:(2)cq≥θ=c⋅∫θ+∞Γqdq,where θ (set at 0.92) is the threshold, cq≥θ is the number of papers with high quality, and Γq is the gamma function.(2)
Journal targeting process:Figure 2
Quality distribution (0–10) of all papers with expectation being 4.2 or 2.3. The number of journals is 10, the number of issues of a journal published per year is 12, the number of papers published per issue is 10, the time length is 13 years, and the total number of all papers is 10∗ 12 ∗ 10 ∗ 13 = 15,600. Red curve denotes Gaussian distribution.
(a)(b)In general, the authors would give priority to those journals with higher reputation in the field when they are preparing to select journals for submission [5, 6, 40]. Of course, although the high-quality papers covering platelet function in vitro have a great impact in this scientific subdomain, they will never be accepted or published in high-ranked journals, which can be attributed to a fact that the interest for the readers of some top journals such as Nature and Science is not comparable with the technical breakthrough at a microlevel for specific journals dealing with platelets. Therefore, for certain specific topics, the papers with higher quality are usually submitted to the journals that have the highest ranking covering a specific topic. In this work, the correlation between reputation and JIF is assumed to be linear and positive [5, 6]. That is, the reputation of a journal is rescaled by the impact factor of the journal, and the reputation of a journal is higher if the impact factor of this journal is greater. Furthermore, each paper has an initial estimation quality q^ affected by the author’s scientific level (rescaled by q). This score will determine how an author chooses a target journal. In principle, the professional scholars usually assess their work more accurately, and vice versa. In most cases, a paper with high quality also implicates the scientific level and innovation of a “competent” author behind it. Suppose that the author’s estimation quality (q^) on the paper mainly depends on the academic level (q) of the author. The relation between the intrinsic quality of a paper and the author’s estimation quality on the paper could be depicted in the following form:(3)q^i=qi⋅ξi,i=1,2,…,n,where qi is the intrinsic quality of each paper, q^i is the author’s estimation quality on the paper, and ξi∈R is the estimation noise by the author, which is a random value around 1. A paper with relatively higher q tends to have ξ that more approximates to 1. Thus, the variance of ξi indicates the magnitude of noise and should be negatively correlated to the paper quality. In the parameter settings of the model, ξi follows a truncated Gaussian distribution with expectation 1 and variance λ/qiω (so that only positive numbers can be sampled). Note that parameters λ and ω are positive numbers that jointly affect the aggregation or dispersion of the distribution curve.Next, we set a condition that is corresponding to the psychological expectations of authors comprehensively considering adventurism and conservatism: a paper should be submitted to a journal with the absolute difference betweenq^ and the average quality of the papers already published in the journal last year (qJ) being less than coefficient τ, namely, q^−qJ<τ. Taken together, the authors would give priority to the journals that meet conditions and have higher impact factors when they are submitting the paper (see Part I of Figure 3).(3)
Peer review process and publication:Figure 3
Summary of the submission process. In the example depicted, the author will select a target journal for submission based on the submission principles. Clearly, the target journal would be very different if the result of author’s estimation on the paper is different. In Part II, according to the scores and ranks of the paper, the paper will be accepted (published) or rejected by the journal, and the rejected paper will be resubmitted to other journals in the next round.Here, all journals are characterized by two state variables: a reputation value (rescaled by impact factors) and related rejection or acceptance thresholds. The journals first have an evaluation on each submitted paper, which can be regarded as a simplification form of the peer review [36]. In the current model, it is assumed that all reviewers in a discipline are peers, and therefore, the selection strategy of reviewers is a random selection process. The relation between the real quality of a paper (q) and a journal’s estimation on the paper (h^) can be represented as follows:(4)h^i=qi⋅δi,i=1,2,…,n,where q denotes the real quality of a paper, h^i denotes a journal’s assessment on the paper, and δ follows a lognormal distribution, i.e., lnδ∼Nμ,σ.Subsequently, according to the evaluation scores, the journals will rank the submitted papers in the descending order and make decision to accept or reject the papers based on their rankings. The rejection or acceptance thresholds are determined by the number of papers published per issue. In the model, each journal in each issue only accept (publish) the highest top 10 papers. Furthermore, only papers be accepted that, in each resubmission round, are in the top 10, and the rejected papers will be resubmitted to other journals in the next round or ultimately be abandoned and remain unpublished (shown in Part II of Figure3). Figure 3 illustrates the modeling of the submission process, which mainly consists of two parts.


### 2.3. Modeling of Citation Process

Here, a virtual simulation model is developed to structure citation networks, in which papers are submitted, published, and cited in sequence. The papers are generated one by one in our program. Each time a new paper is increased, the set of papers is also correspondingly added by one item, which can be viewed as a new node in the virtual citation network. The number of citations a paper has already obtained is the degree of the corresponding node. The number of new edges that grow out of a newly added node in the citation network is the number of references of this new paper. Whether or not a new node should link to any existing node is assigned randomly, through probabilityPcite. Following this way, a virtual citation network was gradually structured. Therefore, the key of calculation of the impact factor is how to assign the value to Pcite properly. In this work, enlightened by certain abstract simulation models on bibliometrics [12, 14, 36], assume that the probability of a paper to be cited in a specific discipline (Pcite) is comprehensively affected by three factors.(1)
The quality of a paper (q):As described in Section2.2, according to the submission rules, the papers with different q will be submitted and published in the corresponding journals. In this model, the correlation between Pcite and q is supposed to be linear and positive, i.e., the corresponding effect function can be defined as follows:(5)lq=q−minQmaxQ−minQ,with Q=q1,q2,…,qn, q∈1,10, and correspondingly, lq∈0,1.(2)
The paper age (t):For decades, the time dependence in the preferential attachment mechanism (PAM) [39] has always been a hot topic in citation networks. Younger papers will draw increasing attention via citations, while older papers are often overlooked by scholars. This aging effect is a universal phenomenon in growing networks. Furthermore, timeliness of research contents has a different effect on different disciplines. For instance, scholars in some experimental disciplines prefer to cite younger scientific achievements (for example, biology and AI), whereas in contrast, in certain theoretical disciplines (for example, Mathematics), those existing literature studies that have been fully validated are more likely to be cited [12, 15]. To this end, we consider employing a function to measure the relation between the effect to the probability of a paper to be cited by another paper in the same discipline and the paper age t, and the function can be expressed as the following form:(6)ft=12tanht+αβ+12,where tanh⋅ is a hyperbolic tangent function; t∈Z− indicates the paper age, with its unit being months; and ft∈0,1 is a coefficient to the probability of the paper to be cited (see Appendix Figure 4).(3)
The number of citations a paper receives (n):Figure 4
Relation between paper age (t) and weight factor to probability of a paper to be cited. (a) Sketch of paper age (t). (b) ft=1/2tanht+α/β+1/2.
(a)(b)Humans are social animals, and therefore, our opinions or selections would be strongly influenced by our peers. This is particularly true in the citation network [34, 39]. Suppose that a weight factor to the probability of the paper to be cited is higher if the current number of citations (node of the citation network) of this paper is greater. The effect of the number of times a paper has already been cited to the probability of the paper to be cited in our model is also formulated by a function, which can be written as follows:(7)gn=tanhnγ,with tanh⋅ being a hyperbolic tangent function, n∈N being the number of citations already obtained, gn∈0,1 being the weight factor to the probability of the paper to be cited, and γ∈R+ being the parameter changing the overall shape of the curve (shown in Appendix Figure 5).Figure 5
Relation between the number of citations already achieved (n) and the effect to probability of a paper to be cited. (a) Sketch of the number of citations a paper receives (n). (b) gn=tanhn/γ.
(a)(b)Finally, according to the descriptions of the citation behaviors, the probability of a paper to be cited in the model can be comprehensively defined as a coefficient combination of a series of factors, which can be represented as the following form:(8)Pcite=∏i=13Factorsi+ε0=lq⋅ft⋅gn+ε0=q−minQmaxQ−minQ⋅12tanht+αβ+12⋅tanhnγ+ε0.In Equation (8), q denotes the real quality of a paper, t denotes the paper age, n denotes the number of citations a paper has, and ε0 is an i.i.d. zero-mean Gaussian noise. Besides, it should be explained that the probability to cite Pcite is the kernel of the program, but note that what really affects the citation result is merely the relative value of Pcite compared with other papers, rather than its absolute value. This plays a very important role in the model because it is easy to compute JIF when given Pcite in equation (8). Overall, the virtual scientific system is mainly composed of three sections, which are initial setup, submission, and citation (see Figure 6).Figure 6
Description of the virtual scientific system. Key features are subsystems of submission and citation. In the submission process, authors select journals for submission based on the submission rules, and journals select papers for publication based on papers’ scores and ranks. In the citation process, the probability of published papers to be cited by submitted papers is exactly determined byPcite.By the end of this section, one should notice that the most key feature of the analysis based on parallel systems is that a parallel model stands independently, each of the subsystems can be parameterized independently, and can itself be viewed as a feasible alternative of real systems. Such a parallel system is particularly suitable for solving situations with an ultracomplex mechanism or with unavailable data. Based on our knowledge from system analysis, either qualitative or quantitative, we expect to discover certain general laws behind various phenomena.


## 2.1. Model Initialization

The first stage is setting the parameters of the simulation model, where general knowledge of the system is considered, namely,(1)
The number of journals (NJ).(2)
The number of issues of a journal published per year (Nissues), and each month corresponds to each issue.(3)
The number of papers published per issue (NPissues). According to Nissues and NPissues, the number of papers published per journal within one year can be computed by NPyear=NPissues×Nissues.(4)
The average review time of journals (T). This parameter denotes the length between two time periods, which are the time of a paper being submitted and the time of the paper being accepted, respectively. For instance, the submitted date of a paper (no. 0826) is December 2018; then, the accepted date of the paper is October 2019, T0826=10 (months), and T represents the average value of Ti published in the journal (see Appendix Figure 1). It should be explained that the average review time (T) is a macrostatistical indicator, which reflects an overall level of process speed, being different journal by journal. Furthermore, for simplicity, assume that publication occurs instantly as a paper is accepted in the model.(5)
The average number of references per paper in a journal (N).(6)
Here, we define and assign some relevant parameters such asα, β, and γ because each of the subsystems can be parameterized independently (see Sections 2.2 and 2.3 for more details).Figure 1
Sketch of the average review time of a type of journal (T).
(a)(b)Note that, in the current model, similar disciplines can be represented by identical settings, while different disciplines are differentiated via tuning the parameters above. In particular, the calculations of impact factors between journals and their series (for example,Nature and Nature Cell Biology) are also independent of each other. Moreover, according to the definition of the impact factor which considers the correlation between the number of papers a journal published in the previous two years and the total number of citations, we uniformly set the impact factor in the first two years to 1 that could avoid certain undesirable situations in the citation process.


## 2.2. Modeling of Submission Process

The second stage is the modeling of the submission process, which mainly consists of the following three steps:(1)
Characterization of papers:The variation in paper quality is an objective fact, which is somewhat similar to the situation of examination scores in education. However, paper quality is difficult to be measured practically. For such situations, people generally assess through scoring questionnaire manner. Existing typical instances include publication scores (https://publons.com/), which are generated by selecting a score from 1 to 10 in two fields, jointly indicating the measure of a paper’s methodology, rigor, and novelty, as well as relevance to its field. Following the routine, in our model, the intrinsic quality of the paper is scored by number q∈0,10, with 0 being the worst and 10 being the best, and the overall quality of all papers follows a skewed distribution with certain expectation and variance. According to the expectation and variance of the distribution, the model will create corresponding numbers of papers with different scores, automatically (shown in Appendix Figure 2). In reality, due to the fact that the quality of the majority of papers is mainly mediocre-leveled, those papers with extremely high or low quality would be relatively less. Therefore, we assume that the quality qii=1,2,…,n of each paper is drawn from positively skewed distribution over the interval 1,10, which is implemented by gamma function. The basis for doing this is that, as the number of papers (c) keeps on increasing, the papers with ultrahigh quality become rare, and their number holds relatively stable, with only minor increase if above a limit. To ensure that the papers with high quality remain relatively constant regardless of changes of c, a numerical integration with respect to gamma function over the interval θ,+∞ is conducted in the model, which is computed by the following form:(2)cq≥θ=c⋅∫θ+∞Γqdq,where θ (set at 0.92) is the threshold, cq≥θ is the number of papers with high quality, and Γq is the gamma function.(2)
Journal targeting process:Figure 2
Quality distribution (0–10) of all papers with expectation being 4.2 or 2.3. The number of journals is 10, the number of issues of a journal published per year is 12, the number of papers published per issue is 10, the time length is 13 years, and the total number of all papers is 10∗ 12 ∗ 10 ∗ 13 = 15,600. Red curve denotes Gaussian distribution.
(a)(b)In general, the authors would give priority to those journals with higher reputation in the field when they are preparing to select journals for submission [5, 6, 40]. Of course, although the high-quality papers covering platelet function in vitro have a great impact in this scientific subdomain, they will never be accepted or published in high-ranked journals, which can be attributed to a fact that the interest for the readers of some top journals such as Nature and Science is not comparable with the technical breakthrough at a microlevel for specific journals dealing with platelets. Therefore, for certain specific topics, the papers with higher quality are usually submitted to the journals that have the highest ranking covering a specific topic. In this work, the correlation between reputation and JIF is assumed to be linear and positive [5, 6]. That is, the reputation of a journal is rescaled by the impact factor of the journal, and the reputation of a journal is higher if the impact factor of this journal is greater. Furthermore, each paper has an initial estimation quality q^ affected by the author’s scientific level (rescaled by q). This score will determine how an author chooses a target journal. In principle, the professional scholars usually assess their work more accurately, and vice versa. In most cases, a paper with high quality also implicates the scientific level and innovation of a “competent” author behind it. Suppose that the author’s estimation quality (q^) on the paper mainly depends on the academic level (q) of the author. The relation between the intrinsic quality of a paper and the author’s estimation quality on the paper could be depicted in the following form:(3)q^i=qi⋅ξi,i=1,2,…,n,where qi is the intrinsic quality of each paper, q^i is the author’s estimation quality on the paper, and ξi∈R is the estimation noise by the author, which is a random value around 1. A paper with relatively higher q tends to have ξ that more approximates to 1. Thus, the variance of ξi indicates the magnitude of noise and should be negatively correlated to the paper quality. In the parameter settings of the model, ξi follows a truncated Gaussian distribution with expectation 1 and variance λ/qiω (so that only positive numbers can be sampled). Note that parameters λ and ω are positive numbers that jointly affect the aggregation or dispersion of the distribution curve.Next, we set a condition that is corresponding to the psychological expectations of authors comprehensively considering adventurism and conservatism: a paper should be submitted to a journal with the absolute difference betweenq^ and the average quality of the papers already published in the journal last year (qJ) being less than coefficient τ, namely, q^−qJ<τ. Taken together, the authors would give priority to the journals that meet conditions and have higher impact factors when they are submitting the paper (see Part I of Figure 3).(3)
Peer review process and publication:Figure 3
Summary of the submission process. In the example depicted, the author will select a target journal for submission based on the submission principles. Clearly, the target journal would be very different if the result of author’s estimation on the paper is different. In Part II, according to the scores and ranks of the paper, the paper will be accepted (published) or rejected by the journal, and the rejected paper will be resubmitted to other journals in the next round.Here, all journals are characterized by two state variables: a reputation value (rescaled by impact factors) and related rejection or acceptance thresholds. The journals first have an evaluation on each submitted paper, which can be regarded as a simplification form of the peer review [36]. In the current model, it is assumed that all reviewers in a discipline are peers, and therefore, the selection strategy of reviewers is a random selection process. The relation between the real quality of a paper (q) and a journal’s estimation on the paper (h^) can be represented as follows:(4)h^i=qi⋅δi,i=1,2,…,n,where q denotes the real quality of a paper, h^i denotes a journal’s assessment on the paper, and δ follows a lognormal distribution, i.e., lnδ∼Nμ,σ.Subsequently, according to the evaluation scores, the journals will rank the submitted papers in the descending order and make decision to accept or reject the papers based on their rankings. The rejection or acceptance thresholds are determined by the number of papers published per issue. In the model, each journal in each issue only accept (publish) the highest top 10 papers. Furthermore, only papers be accepted that, in each resubmission round, are in the top 10, and the rejected papers will be resubmitted to other journals in the next round or ultimately be abandoned and remain unpublished (shown in Part II of Figure3). Figure 3 illustrates the modeling of the submission process, which mainly consists of two parts.


## 2.3. Modeling of Citation Process

Here, a virtual simulation model is developed to structure citation networks, in which papers are submitted, published, and cited in sequence. The papers are generated one by one in our program. Each time a new paper is increased, the set of papers is also correspondingly added by one item, which can be viewed as a new node in the virtual citation network. The number of citations a paper has already obtained is the degree of the corresponding node. The number of new edges that grow out of a newly added node in the citation network is the number of references of this new paper. Whether or not a new node should link to any existing node is assigned randomly, through probabilityPcite. Following this way, a virtual citation network was gradually structured. Therefore, the key of calculation of the impact factor is how to assign the value to Pcite properly. In this work, enlightened by certain abstract simulation models on bibliometrics [12, 14, 36], assume that the probability of a paper to be cited in a specific discipline (Pcite) is comprehensively affected by three factors.(1)
The quality of a paper (q):As described in Section2.2, according to the submission rules, the papers with different q will be submitted and published in the corresponding journals. In this model, the correlation between Pcite and q is supposed to be linear and positive, i.e., the corresponding effect function can be defined as follows:(5)lq=q−minQmaxQ−minQ,with Q=q1,q2,…,qn, q∈1,10, and correspondingly, lq∈0,1.(2)
The paper age (t):For decades, the time dependence in the preferential attachment mechanism (PAM) [39] has always been a hot topic in citation networks. Younger papers will draw increasing attention via citations, while older papers are often overlooked by scholars. This aging effect is a universal phenomenon in growing networks. Furthermore, timeliness of research contents has a different effect on different disciplines. For instance, scholars in some experimental disciplines prefer to cite younger scientific achievements (for example, biology and AI), whereas in contrast, in certain theoretical disciplines (for example, Mathematics), those existing literature studies that have been fully validated are more likely to be cited [12, 15]. To this end, we consider employing a function to measure the relation between the effect to the probability of a paper to be cited by another paper in the same discipline and the paper age t, and the function can be expressed as the following form:(6)ft=12tanht+αβ+12,where tanh⋅ is a hyperbolic tangent function; t∈Z− indicates the paper age, with its unit being months; and ft∈0,1 is a coefficient to the probability of the paper to be cited (see Appendix Figure 4).(3)
The number of citations a paper receives (n):Figure 4
Relation between paper age (t) and weight factor to probability of a paper to be cited. (a) Sketch of paper age (t). (b) ft=1/2tanht+α/β+1/2.
(a)(b)Humans are social animals, and therefore, our opinions or selections would be strongly influenced by our peers. This is particularly true in the citation network [34, 39]. Suppose that a weight factor to the probability of the paper to be cited is higher if the current number of citations (node of the citation network) of this paper is greater. The effect of the number of times a paper has already been cited to the probability of the paper to be cited in our model is also formulated by a function, which can be written as follows:(7)gn=tanhnγ,with tanh⋅ being a hyperbolic tangent function, n∈N being the number of citations already obtained, gn∈0,1 being the weight factor to the probability of the paper to be cited, and γ∈R+ being the parameter changing the overall shape of the curve (shown in Appendix Figure 5).Figure 5
Relation between the number of citations already achieved (n) and the effect to probability of a paper to be cited. (a) Sketch of the number of citations a paper receives (n). (b) gn=tanhn/γ.
(a)(b)Finally, according to the descriptions of the citation behaviors, the probability of a paper to be cited in the model can be comprehensively defined as a coefficient combination of a series of factors, which can be represented as the following form:(8)Pcite=∏i=13Factorsi+ε0=lq⋅ft⋅gn+ε0=q−minQmaxQ−minQ⋅12tanht+αβ+12⋅tanhnγ+ε0.In Equation (8), q denotes the real quality of a paper, t denotes the paper age, n denotes the number of citations a paper has, and ε0 is an i.i.d. zero-mean Gaussian noise. Besides, it should be explained that the probability to cite Pcite is the kernel of the program, but note that what really affects the citation result is merely the relative value of Pcite compared with other papers, rather than its absolute value. This plays a very important role in the model because it is easy to compute JIF when given Pcite in equation (8). Overall, the virtual scientific system is mainly composed of three sections, which are initial setup, submission, and citation (see Figure 6).Figure 6
Description of the virtual scientific system. Key features are subsystems of submission and citation. In the submission process, authors select journals for submission based on the submission rules, and journals select papers for publication based on papers’ scores and ranks. In the citation process, the probability of published papers to be cited by submitted papers is exactly determined byPcite.By the end of this section, one should notice that the most key feature of the analysis based on parallel systems is that a parallel model stands independently, each of the subsystems can be parameterized independently, and can itself be viewed as a feasible alternative of real systems. Such a parallel system is particularly suitable for solving situations with an ultracomplex mechanism or with unavailable data. Based on our knowledge from system analysis, either qualitative or quantitative, we expect to discover certain general laws behind various phenomena.


## 3. Computational Experiments and Data Analysis

### 3.1. Model Implementation

In this section, we perform simulations for the JIF dynamics model proposed in the last section. We first simulate and analyze the relation between the average quality of the journals (qJ) and their impact factors within 13 years. As stated in Introduction, the main objective of the simulation system is not for comprehensively and quantitatively mimicking the real-world scenarios. Thus, the model neglects those less important factors such as the very specific situation of journal disappearing or renaming. The comparison results of average quality and impact factors of journals are shown in Figure 7.Figure 7
Comparison of average quality and impact factors of journals within 13 years. On the left, colors indicate groups of no significant pairwise differences, with greater results (qJ≥6 and JIF≥5) shown in dark red, the ones in between 4.8≤qJ<6 and 3≤JIF<5 in light red or light blue, and worse results (qJ<4.8 and JIF<3) in dark blue. On the right are the illustrations of parallelcoords and Gaussian fitting, respectively.It can be seen from Figure7 that the trends of color change are similar. In other words, the average quality of journals is consistent with the variations of the JIF. The journals with higher average quality (for example, journals 9–12 with qJ≥6) always have greater impact factors. In contrast, the impact factors of the journals with lower average quality (for example, journals 1–4 with qJ<4.8) are unsatisfactory. The above simulation result is also demonstrated by the Gaussian fitting (bottom right in Figure 7). The explanation of this correlation is that the sample of papers with a higher average quality will tend to have a higher average number of citations in the model. Thus, the strength of this correlation between average quality and average citations will depend on the variation of the quality and citation distributions.Next, we conduct experiments to verify whether or not JIF has an expected variation after it is artificially manipulated by the model. To see this, we artificially manipulate the impact factors of two different journals in a random year. The average performance in the 100 Monte Carlo simulation runs of a normal JIF and manipulated JIF is shown in Figure8. Figures 8(a) and 8(e) are the impact factors without manipulation. Figures 8(b) and 8(f) are the impact factors with one manipulation in the 8th year. Figures 8(c), 8(d), 8(g), and 8(h) are the impact factors with two manipulations in the 5th and 10th years.Figure 8
Comparison of the normal JIF and manipulated JIF in 100 Monte Carlo simulation runs. (a) Normal impact factors within 13 years. (b) Impact factors with one manipulation. (c) Impact factors with two manipulations. (d) Impact factors with two manipulations. (e) Normal impact factors within 13 years. (f) Impact factors with one manipulation. (g) Impact factors with two manipulations. (h) Impact factors with two manipulations.
(a)(b)(c)(d)(e)(f)(g)(h)We can see from Figure8 that the trends of the impact factors without manipulation do not change much within 13 years (Figures 8(a) and 8(e)), while the trends of the impact factors with manipulation are fluctuating strongly (Figures 8(b)–8(h)). In addition, we note that when the impact factors are artificially decreased or increased to a certain level, they will maintain corresponding trends in the next few years until they are manipulated again. This is mainly due to the fact that the submissions only depend on the JIF of the last year in the model, whatever happens last year will immediately determine the outcome of the next year, and that will be conditionally independent of the situation two years (or more) before. Actually, it was already concluded in an influential study [9] that “…by manipulating JIF in different ways, their JIF will increase quickly, …, as an academic evaluation indicator, JIF is able to distinguish the differences of certain academic performances such as citation and publication process.”


### 3.2. Analysis of Features Correlating with Impact Factors

The task of this section is to analyze how the JIF changes under different parameters. The baseline values, variation intervals, and variation step of parameters in the model are reported in Table1.Table 1
Model parameters, baseline values, variation intervals, and step of variation for probabilistic sensitivity analysis.
No.ParameterAveragePcite gapAverage JIF gapNotationDescriptionBaseline and variation intervalsStep of variation1αHorizontal translation of the curve in equation (5)40 [10, 100]100.4±0.158±0.502βOverall shape of the curve in equation (5)10 [5, 20]50.1±0.051±0.053γOverall shape of the curve in equation (6)7 [2, 10]10.2±0.051±0.054TAverage review time (months)10 [2, 20]20.3±0.056±0.505NAverage number of references30 [5, 60]50.3±0.056±1.00Bold values indicate best performance.From an analysis of Table1, it can be found that all else being equal, parameter α, average review time (T), and average number of references (N) increased the impact factors’ gap the most in all tests. These results are confirmed by a probabilistic sensitivity analysis, which evaluated the sensitivity of Pcite and average JIF to simultaneous changes in multiple parameter values away from their baseline values (shown in Figure 9).Figure 9
Correlation between impact factors and parametersα, T, and N: (a) βbaseline=10, γbaseline=7, Tbaseline=10, and Nbaseline=30; (b) αbaseline=40, βbaseline=10, γbaseline=7, and Nbaseline=30; (c) αbaseline=40, βbaseline=10, γbaseline=7, and Tbaseline=10.
(a)(b)(c)Additionally, we conducted 50 realizations at each of the 5 tested parameter values across a given range, and the univariate sensitivity analysis showed that the JIF in the current model is also most sensitive to parametersT and N. By comparing and analyzing the curves of Figure 10, under three different parameter settings (S1 (Figure 10(a)): αmin=10, βmin=5, and γmin=2; S2 (Figure 10(b)): αbaseline=40, βbaseline=10, and γbaseline=7; and S3 (Figure 10(c)): αmax=100, βmax=20, and γmax=10), it can be seen that the overall trends of impact factors are similar. In greater detail, the impact factor of the journal with the number of references, N=60 is much higher than the impact factor of other two types of journals with N=30 and N=10. This result seems to show that the greater the average number of references (N) of a journal, the higher the impact factor of the journal would tend to be. Moreover, we also note that the overall trends of impact factors in three figures are correspondingly decreasing with the average review time (T) becoming longer.Figure 10
Correlation between impact factors andT and N for univariate sensitivity analysis: (a) αmin=10, βmin=5, and γmin=2; (b) αbaseline=40, βbaseline=10, and γbaseline=7; (c) αmax=100, βmax=20, and γmax=10.
(a)(b)(c)Now, we analyze what happens when different parameters take values from small to large, that is, we would like to verify how the parameters such asα and β influence the dynamics of the model. To see this, we performed 100 simulation runs, each lasting 13 years (156 months) of simulated time. For each simulation run, five parameter values (α, β, γ, T, and N) were sampled from each of the maximum, baseline, and minimum given in Table 1. The simulation results are summarized as follows: Figure 11 shows the variations of the JIF influenced by minimum, baseline, and maximum of parameters α, β, and γ when Tmin=2 and Nmax=60 in the 100 Monte Carlo simulations (Figures 11(a)–11(c), respectively); Figure 12 shows the variations of the JIF influenced by minimum, baseline, and maximum of parameters α, β, and γ when Tbaseline=10 and Nbaseline=30 in the 100 Monte Carlo simulations (Figures 12(a)–12(c), respectively); and Figure 13 shows the variations of the JIF influenced by minimum, baseline, and maximum of parameters α, β, and γ when Tmax=20 and Nmin=5 in the 100 Monte Carlo simulations (Figures 13(a)–13(c), respectively). Table 2 gives the relation between the average JIF and different parameters in the 100 Monte Carlo simulations.Figure 11
Variation of the JIF within 13 years in 100 Monte Carlo simulations withTmin=2 and Nmax=60.
(a)(b)(c)Figure 12
Variation of the JIF within 13 years in 100 Monte Carlo simulations withTbaseline=10 and Nbaseline=30.
(a)(b)(c)Figure 13
Variation of the JIF within 13 years in 100 Monte Carlo simulations withTmax=20 and Nmin=5.
(a)(b)(c)Table 2
Average JIF under different parameter combination conditions.
No.Average JIFMinimum, baseline, and maximum of parametersIllustrationαβγTN114.468 (max)10 (min)5 (min)2 (min)2 (min)60 (max)Figure11(a)26.17540 (baseline)10 (baseline)7 (baseline)2 (min)60 (max)Figure11(b)32.333100 (max)20 (max)10 (max)2 (min)60 (max)Figure11(c)47.65510 (min)5 (min)2 (min)10 (baseline)30 (baseline)Figure12(a)54.224 (baseline)40 (baseline)10 (baseline)7 (baseline)10 (baseline)30 (baseline)Figure12(b)62.649100 (max)20 (max)10 (max)10 (baseline)30 (baseline)Figure12(c)71.94810 (min)5 (min)2 (min)20 (max)5 (min)Figure13(a)80.88140 (baseline)10 (baseline)7 (baseline)20 (max)5 (min)Figure13(b)90.382 (min)100 (max)20 (max)10 (max)20 (max)5 (min)Figure13(c)Bold values indicate best performance.Through extensive simulations, under three different conditions (Figure11: Tmin=2 and Nmax=60; Figure 12: Tbaseline=10 and Nbaseline=30; and Figure 13: Tmax=20 and Nmin=5), an important observation from the simulation results in Figures 11–13 is that no matter what the conditions of parameters T and N are, the average JIF would increasingly grow with the decrease of parameters α, β, and γ. It is worth remarking that the values of α and β jointly reflect the citation lifetime of papers, where a steeper one, namely, lesser β, indicates that the corresponding journals in some particular discipline tend to cite younger papers, whereas greater β signifies longer citation life cycle of papers and wider range of aging distribution of references in that discipline. This can be understood from the hyperbolic tangent function in Appendix Figure 4 that when parameters α and β are very small, the probability of those papers with older age (t) to be cited by others is almost zero.From Table2 and Figures 9–13, we see that the value of the JIF is comprehensively influenced by different factors such as average review time (T), average number of references (N), and citation distribution (α and β). The maximum average JIF is obtained from the conditions as follows: αmin=10; βmin=5; γmin=2; Tmin=2; and Nmax=60. The minimum average JIF is achieved by setting αmax=100; βmax=20; γmax=10; Tmax=20; and Nmin=5. It is clear that the value of the JIF is positively correlated to the average number of references (N), whereas it is negatively related to the other four parameters (i.e., α, β, γ, and T). The experimental results demonstrate that the younger of the reference (namely, the smaller the parameters α and β), the higher the impact factor of the journal would tend to be. Similarly, a journal would achieve a higher impact factor if the journal has a greater average number of references (N). Furthermore, a journal with a relatively shorter review time (T) tends to hold a higher impact factor and vice versa.In order to further verify the experimental results, we select 120 journals in three different fields (biology, artificial intelligence, and mathematics) from the JCR database as a reference, which can be found in Appendix Figure14 and Tables 3–5. These three different disciplines have been chosen because we can use them to represent the top, middle, and bottom of the overall impact factor level, respectively, and therefore, they should be better suited to showing general results. We recruited 20 graduate students with informatics background to record the data. Concretely speaking, we first randomly select 200 paper samples for each journal. Subsequently, according to the number of references, date of submission, and publication time displayed in each paper, the average number of references and the average review time of the corresponding journal can be easily computed. By comparing and analyzing the data from Tables 3–5, it can be found that there exists drastic difference in the levels of average JIF, average review time (T), and average number of references (N) over the three different disciplines. To be specific, we see from Appendix Figure 14 that the average JIF of biology is 2–5 times higher than AI and mathematics, particularly in Q1 and Q2 (biology: 9.59; AI: 3.89; and mathematics: 2.17). Interesting enough, the average number of references (N) of biology is also much higher than other two disciplines (biology: 46.5; AI: 43.25; and mathematics: 20.5). In addition, we note that the journals in biology have the shorter average review time (T) in comparison with AI and mathematics journals (biology: 6.3; AI: 11.6; and mathematics: 16.5). It is clear that these observations are basically consistent with our experimental results.Figure 14
Comparison of average JIF, average review time (T), and average number of references (N) in Q1–Q4 for the 120 journals belonging to biology, AI, and mathematics. (a) Histogram of different indicators in Q1–Q4. (b) Comparison of average review time (T) and average number of references (N). (c) Comparison of average review time (T) and impact factors. (d) Comparison of average number of references (N) and impact factors.
(a)(b)(c)(d)Table 3
Average JIF, average review time, and average number of references for 40 biology journals.
No.Journal titleQuantileAverage JIF (2009–2019)Average review time (months)Average number of references1Nature Reviews GeneticsQ141.474.3552Nature Reviews Molecular Cell BiologyQ135.614.6523Nature Reviews MicrobiologyQ131.854.4544CellQ131.403.9625Nature GeneticsQ127.134.4566Nature MethodsQ126.925.1497Cell Stem CellQ123.294.6528Cell MetabolismQ120.575.8559Nature Cell BiologyQ119.064.15810Trends in Cell BiologyQ118.564.06111Current Opinion in Chemical BiologyQ27.575.75212Current Opinion in Plant BiologyQ27.354.95213Current Opinion in Structural BiologyQ27.186.25314Redox BiologyQ27.135.84415Molecular Ecology ResourcesQ27.065.65016Annual Review of Animal BiosciencesQ26.787.14617Cellular and Molecular Life SciencesQ26.727.04218Current Opinion in MicrobiologyQ26.715.94419Protein & CellQ26.237.24620Critical Reviews in Plant SciencesQ26.616.64421Current Opinion in Insect ScienceQ34.175.94322Harmful AlgaeQ34.146.25123Fungal Biology ReviewsQ33.977.13924Journal of Biology RhythmsQ33.916.94825Phytochemistry ReviewQ33.887.44026Freshwater BiologyQ33.777.44527Cell CalciumQ33.726.84228GlycobiologyQ33.666.24129Journal of Systematics and EvolutionQ33.667.54030Frontier in ZoologyQ33.637.44131Connective Tissue ResearchQ42.077.34232Botanical ReviewQ42.507.83933EXCLI JournalQ42.427.04134Animal Health Research ReviewsQ42.417.34535Probiotics and Antimicrobial ProteinsQ42.357.74036Biological ResearchQ42.366.84137Journal of MicrobiologyQ42.327.43838Endangered Species ResearchQ42.318.13939International Review of HydrobiologyQ42.287.84140BryologistQ42.267.338Table 4
Average JIF, average review time, and average number of references for 40 AI journals.
No.Journal titleQuantileAverage JIF (2009–2019)Average review time (months)Average number of references1International Journal of Computer VisionQ111.549.8452IEEE Transactions on CyberneticsQ110.3910.4483IEEE Transactions on Pattern Analysis and Machine IntelligenceQ19.4510.6494IEEE Transactions on Fuzzy SystemsQ18.4210.5515IEEE Transactions on Evolutionary ComputationQ18.1211.1496IEEE Transactions on Neural Networks and Learning SystemsQ17.9810.3477Information FusionQ16.649.4398IEEE Computational Intelligence MagazineQ16.618.9399Neural NetworksQ15.229.24110International Journal of Neural SystemsQ14.589.44111IEEE Transactions on Image ProcessingQ25.079.24412IEEE Transactions on Affective ComputingQ24.599.44113Knowledge-Based SystemsQ24.409.24214Neural Computing and ApplicationsQ24.2211.43915Pattern RecognitionQ23.9613.14616Swam and Evolutionary ComputationQ23.829.63717NeurocomputingQ23.248.94218Artificial IntelligenceQ23.0313.25519Artificial Intelligence ReviewQ23.8111.86120Expert Systems with ApplicationsQ23.7710.84621Advanced Engineering InformaticsQ33.3610.24122Artificial Intelligence in MedicineQ32.8813.24423International Journal of Machine Learning and CyberneticsQ32.6912.84624Frontiers in NeuroroboticsQ32.619.83925IEEE Transactions on Human-Machine SystemsQ32.5611.84826Computer Vision and Image UnderstandingQ32.3913.24227Soft ComputingQ32.3714.23928SIAM Journal on Imaging SciencesQ32.3611.84029International Journal of Bio-Inspired ComputationQ32.2714.14630Knowledge and Information SystemsQ32.2512.04131International Journal of Computational Intelligence SystemsQ42.012.24332IET BiometricsQ41.8413.53833Genetic Programming and Evolvable MachinesQ41.4612.53834Expert SystemsQ41.4313.34635IET Image ProcessingQ41.4014.93936Machine Vision and ApplicationsQ41.3112.84237Pattern Analysis and ApplicationsQ41.2814.23838Journal of Intelligence Information SystemsQ41.1113.53639IET Computer VisionQ41.0914.44040AI MagazineQ41.0313.342Table 5
Average JIF, average review time, and average number of references for 40 mathematics journals.
No.Journal titleQuantileAverage JIF (2009–2019)Average review time (months)Average number of references1SIAM ReviewQ14.8914.5182Annals of MathematicsQ14.7715.8223Advances in Nonlinear AnalysisQ14.6716.5204Journal of The American Mathematical SocietyQ14.6316.9225Multivariate Behavioral ResearchQ13.6914.8246Communications on Pure and Applied MathematicsQ13.3914.8197Mathematical Models and Methods in Applied SciencesQ13.3218.2268Annual Review of Statistics and its ApplicationQ13.2916.5219Foundations of Computational MathematicsQ13.0614.81810Risk AnalysisQ12.916.81911American StatisticianQ22.3514.42312Bayesian AnalysisQ22.3416.02213Applied Mathematics and ComputationQ22.3015.41814Journal of The American Statistical AssociationQ22.3011.92615Stata JournalQ22.1613.32116Communications in Applied Mathematics and Computational ScienceQ22.1318.22317Annals of Applied ProbabilityQ22.1218.82318Journal of Nonlinear ScienceQ22.1115.62719PsychometrikaQ22.0913.92220International Statistical ReviewQ22.0516.22221Memoirs of the American Mathematical SocietyQ31.7414.41722Journal of Fourier Analysis and ApplicationsQ31.5314.62123Inverse Problems and ImagingQ31.4713.31124Statistical ModelingQ31.4315.91925CombinatoricaQ31.4120.02126Advances in Differential EquationsQ31.4017.22327Analysis and Mathematical PhysicsQ31.3814.92028American Journal of MathematicsQ31.3815.52629IMA Journal of Applied MathematicsQ31.3718.81930Russian Mathematical SurveysQ31.3616.01931Mathematics and Financial EconomicsQ41.0817.52132Advances in Difference EquationsQ41.0716.81433Annals of Pure and Applied LogicQ41.0019.41834Journal of Fixed Point Theory and ApplicationsQ40.9718.21835Journal of Numerical MathematicsQ40.9515.02336MetrikaQ40.9519.61637Statistical Methods and ApplicationsQ40.9317.72138Izvestiya: MathematicsQ40.9221.01739PositivityQ40.9218.51940Journal of Theoretical ProbabilityQ40.8819.422


## 3.1. Model Implementation

In this section, we perform simulations for the JIF dynamics model proposed in the last section. We first simulate and analyze the relation between the average quality of the journals (qJ) and their impact factors within 13 years. As stated in Introduction, the main objective of the simulation system is not for comprehensively and quantitatively mimicking the real-world scenarios. Thus, the model neglects those less important factors such as the very specific situation of journal disappearing or renaming. The comparison results of average quality and impact factors of journals are shown in Figure 7.Figure 7
Comparison of average quality and impact factors of journals within 13 years. On the left, colors indicate groups of no significant pairwise differences, with greater results (qJ≥6 and JIF≥5) shown in dark red, the ones in between 4.8≤qJ<6 and 3≤JIF<5 in light red or light blue, and worse results (qJ<4.8 and JIF<3) in dark blue. On the right are the illustrations of parallelcoords and Gaussian fitting, respectively.It can be seen from Figure7 that the trends of color change are similar. In other words, the average quality of journals is consistent with the variations of the JIF. The journals with higher average quality (for example, journals 9–12 with qJ≥6) always have greater impact factors. In contrast, the impact factors of the journals with lower average quality (for example, journals 1–4 with qJ<4.8) are unsatisfactory. The above simulation result is also demonstrated by the Gaussian fitting (bottom right in Figure 7). The explanation of this correlation is that the sample of papers with a higher average quality will tend to have a higher average number of citations in the model. Thus, the strength of this correlation between average quality and average citations will depend on the variation of the quality and citation distributions.Next, we conduct experiments to verify whether or not JIF has an expected variation after it is artificially manipulated by the model. To see this, we artificially manipulate the impact factors of two different journals in a random year. The average performance in the 100 Monte Carlo simulation runs of a normal JIF and manipulated JIF is shown in Figure8. Figures 8(a) and 8(e) are the impact factors without manipulation. Figures 8(b) and 8(f) are the impact factors with one manipulation in the 8th year. Figures 8(c), 8(d), 8(g), and 8(h) are the impact factors with two manipulations in the 5th and 10th years.Figure 8
Comparison of the normal JIF and manipulated JIF in 100 Monte Carlo simulation runs. (a) Normal impact factors within 13 years. (b) Impact factors with one manipulation. (c) Impact factors with two manipulations. (d) Impact factors with two manipulations. (e) Normal impact factors within 13 years. (f) Impact factors with one manipulation. (g) Impact factors with two manipulations. (h) Impact factors with two manipulations.
(a)(b)(c)(d)(e)(f)(g)(h)We can see from Figure8 that the trends of the impact factors without manipulation do not change much within 13 years (Figures 8(a) and 8(e)), while the trends of the impact factors with manipulation are fluctuating strongly (Figures 8(b)–8(h)). In addition, we note that when the impact factors are artificially decreased or increased to a certain level, they will maintain corresponding trends in the next few years until they are manipulated again. This is mainly due to the fact that the submissions only depend on the JIF of the last year in the model, whatever happens last year will immediately determine the outcome of the next year, and that will be conditionally independent of the situation two years (or more) before. Actually, it was already concluded in an influential study [9] that “…by manipulating JIF in different ways, their JIF will increase quickly, …, as an academic evaluation indicator, JIF is able to distinguish the differences of certain academic performances such as citation and publication process.”


## 3.2. Analysis of Features Correlating with Impact Factors

The task of this section is to analyze how the JIF changes under different parameters. The baseline values, variation intervals, and variation step of parameters in the model are reported in Table1.Table 1
Model parameters, baseline values, variation intervals, and step of variation for probabilistic sensitivity analysis.
No.ParameterAveragePcite gapAverage JIF gapNotationDescriptionBaseline and variation intervalsStep of variation1αHorizontal translation of the curve in equation (5)40 [10, 100]100.4±0.158±0.502βOverall shape of the curve in equation (5)10 [5, 20]50.1±0.051±0.053γOverall shape of the curve in equation (6)7 [2, 10]10.2±0.051±0.054TAverage review time (months)10 [2, 20]20.3±0.056±0.505NAverage number of references30 [5, 60]50.3±0.056±1.00Bold values indicate best performance.From an analysis of Table1, it can be found that all else being equal, parameter α, average review time (T), and average number of references (N) increased the impact factors’ gap the most in all tests. These results are confirmed by a probabilistic sensitivity analysis, which evaluated the sensitivity of Pcite and average JIF to simultaneous changes in multiple parameter values away from their baseline values (shown in Figure 9).Figure 9
Correlation between impact factors and parametersα, T, and N: (a) βbaseline=10, γbaseline=7, Tbaseline=10, and Nbaseline=30; (b) αbaseline=40, βbaseline=10, γbaseline=7, and Nbaseline=30; (c) αbaseline=40, βbaseline=10, γbaseline=7, and Tbaseline=10.
(a)(b)(c)Additionally, we conducted 50 realizations at each of the 5 tested parameter values across a given range, and the univariate sensitivity analysis showed that the JIF in the current model is also most sensitive to parametersT and N. By comparing and analyzing the curves of Figure 10, under three different parameter settings (S1 (Figure 10(a)): αmin=10, βmin=5, and γmin=2; S2 (Figure 10(b)): αbaseline=40, βbaseline=10, and γbaseline=7; and S3 (Figure 10(c)): αmax=100, βmax=20, and γmax=10), it can be seen that the overall trends of impact factors are similar. In greater detail, the impact factor of the journal with the number of references, N=60 is much higher than the impact factor of other two types of journals with N=30 and N=10. This result seems to show that the greater the average number of references (N) of a journal, the higher the impact factor of the journal would tend to be. Moreover, we also note that the overall trends of impact factors in three figures are correspondingly decreasing with the average review time (T) becoming longer.Figure 10
Correlation between impact factors andT and N for univariate sensitivity analysis: (a) αmin=10, βmin=5, and γmin=2; (b) αbaseline=40, βbaseline=10, and γbaseline=7; (c) αmax=100, βmax=20, and γmax=10.
(a)(b)(c)Now, we analyze what happens when different parameters take values from small to large, that is, we would like to verify how the parameters such asα and β influence the dynamics of the model. To see this, we performed 100 simulation runs, each lasting 13 years (156 months) of simulated time. For each simulation run, five parameter values (α, β, γ, T, and N) were sampled from each of the maximum, baseline, and minimum given in Table 1. The simulation results are summarized as follows: Figure 11 shows the variations of the JIF influenced by minimum, baseline, and maximum of parameters α, β, and γ when Tmin=2 and Nmax=60 in the 100 Monte Carlo simulations (Figures 11(a)–11(c), respectively); Figure 12 shows the variations of the JIF influenced by minimum, baseline, and maximum of parameters α, β, and γ when Tbaseline=10 and Nbaseline=30 in the 100 Monte Carlo simulations (Figures 12(a)–12(c), respectively); and Figure 13 shows the variations of the JIF influenced by minimum, baseline, and maximum of parameters α, β, and γ when Tmax=20 and Nmin=5 in the 100 Monte Carlo simulations (Figures 13(a)–13(c), respectively). Table 2 gives the relation between the average JIF and different parameters in the 100 Monte Carlo simulations.Figure 11
Variation of the JIF within 13 years in 100 Monte Carlo simulations withTmin=2 and Nmax=60.
(a)(b)(c)Figure 12
Variation of the JIF within 13 years in 100 Monte Carlo simulations withTbaseline=10 and Nbaseline=30.
(a)(b)(c)Figure 13
Variation of the JIF within 13 years in 100 Monte Carlo simulations withTmax=20 and Nmin=5.
(a)(b)(c)Table 2
Average JIF under different parameter combination conditions.
No.Average JIFMinimum, baseline, and maximum of parametersIllustrationαβγTN114.468 (max)10 (min)5 (min)2 (min)2 (min)60 (max)Figure11(a)26.17540 (baseline)10 (baseline)7 (baseline)2 (min)60 (max)Figure11(b)32.333100 (max)20 (max)10 (max)2 (min)60 (max)Figure11(c)47.65510 (min)5 (min)2 (min)10 (baseline)30 (baseline)Figure12(a)54.224 (baseline)40 (baseline)10 (baseline)7 (baseline)10 (baseline)30 (baseline)Figure12(b)62.649100 (max)20 (max)10 (max)10 (baseline)30 (baseline)Figure12(c)71.94810 (min)5 (min)2 (min)20 (max)5 (min)Figure13(a)80.88140 (baseline)10 (baseline)7 (baseline)20 (max)5 (min)Figure13(b)90.382 (min)100 (max)20 (max)10 (max)20 (max)5 (min)Figure13(c)Bold values indicate best performance.Through extensive simulations, under three different conditions (Figure11: Tmin=2 and Nmax=60; Figure 12: Tbaseline=10 and Nbaseline=30; and Figure 13: Tmax=20 and Nmin=5), an important observation from the simulation results in Figures 11–13 is that no matter what the conditions of parameters T and N are, the average JIF would increasingly grow with the decrease of parameters α, β, and γ. It is worth remarking that the values of α and β jointly reflect the citation lifetime of papers, where a steeper one, namely, lesser β, indicates that the corresponding journals in some particular discipline tend to cite younger papers, whereas greater β signifies longer citation life cycle of papers and wider range of aging distribution of references in that discipline. This can be understood from the hyperbolic tangent function in Appendix Figure 4 that when parameters α and β are very small, the probability of those papers with older age (t) to be cited by others is almost zero.From Table2 and Figures 9–13, we see that the value of the JIF is comprehensively influenced by different factors such as average review time (T), average number of references (N), and citation distribution (α and β). The maximum average JIF is obtained from the conditions as follows: αmin=10; βmin=5; γmin=2; Tmin=2; and Nmax=60. The minimum average JIF is achieved by setting αmax=100; βmax=20; γmax=10; Tmax=20; and Nmin=5. It is clear that the value of the JIF is positively correlated to the average number of references (N), whereas it is negatively related to the other four parameters (i.e., α, β, γ, and T). The experimental results demonstrate that the younger of the reference (namely, the smaller the parameters α and β), the higher the impact factor of the journal would tend to be. Similarly, a journal would achieve a higher impact factor if the journal has a greater average number of references (N). Furthermore, a journal with a relatively shorter review time (T) tends to hold a higher impact factor and vice versa.In order to further verify the experimental results, we select 120 journals in three different fields (biology, artificial intelligence, and mathematics) from the JCR database as a reference, which can be found in Appendix Figure14 and Tables 3–5. These three different disciplines have been chosen because we can use them to represent the top, middle, and bottom of the overall impact factor level, respectively, and therefore, they should be better suited to showing general results. We recruited 20 graduate students with informatics background to record the data. Concretely speaking, we first randomly select 200 paper samples for each journal. Subsequently, according to the number of references, date of submission, and publication time displayed in each paper, the average number of references and the average review time of the corresponding journal can be easily computed. By comparing and analyzing the data from Tables 3–5, it can be found that there exists drastic difference in the levels of average JIF, average review time (T), and average number of references (N) over the three different disciplines. To be specific, we see from Appendix Figure 14 that the average JIF of biology is 2–5 times higher than AI and mathematics, particularly in Q1 and Q2 (biology: 9.59; AI: 3.89; and mathematics: 2.17). Interesting enough, the average number of references (N) of biology is also much higher than other two disciplines (biology: 46.5; AI: 43.25; and mathematics: 20.5). In addition, we note that the journals in biology have the shorter average review time (T) in comparison with AI and mathematics journals (biology: 6.3; AI: 11.6; and mathematics: 16.5). It is clear that these observations are basically consistent with our experimental results.Figure 14
Comparison of average JIF, average review time (T), and average number of references (N) in Q1–Q4 for the 120 journals belonging to biology, AI, and mathematics. (a) Histogram of different indicators in Q1–Q4. (b) Comparison of average review time (T) and average number of references (N). (c) Comparison of average review time (T) and impact factors. (d) Comparison of average number of references (N) and impact factors.
(a)(b)(c)(d)Table 3
Average JIF, average review time, and average number of references for 40 biology journals.
No.Journal titleQuantileAverage JIF (2009–2019)Average review time (months)Average number of references1Nature Reviews GeneticsQ141.474.3552Nature Reviews Molecular Cell BiologyQ135.614.6523Nature Reviews MicrobiologyQ131.854.4544CellQ131.403.9625Nature GeneticsQ127.134.4566Nature MethodsQ126.925.1497Cell Stem CellQ123.294.6528Cell MetabolismQ120.575.8559Nature Cell BiologyQ119.064.15810Trends in Cell BiologyQ118.564.06111Current Opinion in Chemical BiologyQ27.575.75212Current Opinion in Plant BiologyQ27.354.95213Current Opinion in Structural BiologyQ27.186.25314Redox BiologyQ27.135.84415Molecular Ecology ResourcesQ27.065.65016Annual Review of Animal BiosciencesQ26.787.14617Cellular and Molecular Life SciencesQ26.727.04218Current Opinion in MicrobiologyQ26.715.94419Protein & CellQ26.237.24620Critical Reviews in Plant SciencesQ26.616.64421Current Opinion in Insect ScienceQ34.175.94322Harmful AlgaeQ34.146.25123Fungal Biology ReviewsQ33.977.13924Journal of Biology RhythmsQ33.916.94825Phytochemistry ReviewQ33.887.44026Freshwater BiologyQ33.777.44527Cell CalciumQ33.726.84228GlycobiologyQ33.666.24129Journal of Systematics and EvolutionQ33.667.54030Frontier in ZoologyQ33.637.44131Connective Tissue ResearchQ42.077.34232Botanical ReviewQ42.507.83933EXCLI JournalQ42.427.04134Animal Health Research ReviewsQ42.417.34535Probiotics and Antimicrobial ProteinsQ42.357.74036Biological ResearchQ42.366.84137Journal of MicrobiologyQ42.327.43838Endangered Species ResearchQ42.318.13939International Review of HydrobiologyQ42.287.84140BryologistQ42.267.338Table 4
Average JIF, average review time, and average number of references for 40 AI journals.
No.Journal titleQuantileAverage JIF (2009–2019)Average review time (months)Average number of references1International Journal of Computer VisionQ111.549.8452IEEE Transactions on CyberneticsQ110.3910.4483IEEE Transactions on Pattern Analysis and Machine IntelligenceQ19.4510.6494IEEE Transactions on Fuzzy SystemsQ18.4210.5515IEEE Transactions on Evolutionary ComputationQ18.1211.1496IEEE Transactions on Neural Networks and Learning SystemsQ17.9810.3477Information FusionQ16.649.4398IEEE Computational Intelligence MagazineQ16.618.9399Neural NetworksQ15.229.24110International Journal of Neural SystemsQ14.589.44111IEEE Transactions on Image ProcessingQ25.079.24412IEEE Transactions on Affective ComputingQ24.599.44113Knowledge-Based SystemsQ24.409.24214Neural Computing and ApplicationsQ24.2211.43915Pattern RecognitionQ23.9613.14616Swam and Evolutionary ComputationQ23.829.63717NeurocomputingQ23.248.94218Artificial IntelligenceQ23.0313.25519Artificial Intelligence ReviewQ23.8111.86120Expert Systems with ApplicationsQ23.7710.84621Advanced Engineering InformaticsQ33.3610.24122Artificial Intelligence in MedicineQ32.8813.24423International Journal of Machine Learning and CyberneticsQ32.6912.84624Frontiers in NeuroroboticsQ32.619.83925IEEE Transactions on Human-Machine SystemsQ32.5611.84826Computer Vision and Image UnderstandingQ32.3913.24227Soft ComputingQ32.3714.23928SIAM Journal on Imaging SciencesQ32.3611.84029International Journal of Bio-Inspired ComputationQ32.2714.14630Knowledge and Information SystemsQ32.2512.04131International Journal of Computational Intelligence SystemsQ42.012.24332IET BiometricsQ41.8413.53833Genetic Programming and Evolvable MachinesQ41.4612.53834Expert SystemsQ41.4313.34635IET Image ProcessingQ41.4014.93936Machine Vision and ApplicationsQ41.3112.84237Pattern Analysis and ApplicationsQ41.2814.23838Journal of Intelligence Information SystemsQ41.1113.53639IET Computer VisionQ41.0914.44040AI MagazineQ41.0313.342Table 5
Average JIF, average review time, and average number of references for 40 mathematics journals.
No.Journal titleQuantileAverage JIF (2009–2019)Average review time (months)Average number of references1SIAM ReviewQ14.8914.5182Annals of MathematicsQ14.7715.8223Advances in Nonlinear AnalysisQ14.6716.5204Journal of The American Mathematical SocietyQ14.6316.9225Multivariate Behavioral ResearchQ13.6914.8246Communications on Pure and Applied MathematicsQ13.3914.8197Mathematical Models and Methods in Applied SciencesQ13.3218.2268Annual Review of Statistics and its ApplicationQ13.2916.5219Foundations of Computational MathematicsQ13.0614.81810Risk AnalysisQ12.916.81911American StatisticianQ22.3514.42312Bayesian AnalysisQ22.3416.02213Applied Mathematics and ComputationQ22.3015.41814Journal of The American Statistical AssociationQ22.3011.92615Stata JournalQ22.1613.32116Communications in Applied Mathematics and Computational ScienceQ22.1318.22317Annals of Applied ProbabilityQ22.1218.82318Journal of Nonlinear ScienceQ22.1115.62719PsychometrikaQ22.0913.92220International Statistical ReviewQ22.0516.22221Memoirs of the American Mathematical SocietyQ31.7414.41722Journal of Fourier Analysis and ApplicationsQ31.5314.62123Inverse Problems and ImagingQ31.4713.31124Statistical ModelingQ31.4315.91925CombinatoricaQ31.4120.02126Advances in Differential EquationsQ31.4017.22327Analysis and Mathematical PhysicsQ31.3814.92028American Journal of MathematicsQ31.3815.52629IMA Journal of Applied MathematicsQ31.3718.81930Russian Mathematical SurveysQ31.3616.01931Mathematics and Financial EconomicsQ41.0817.52132Advances in Difference EquationsQ41.0716.81433Annals of Pure and Applied LogicQ41.0019.41834Journal of Fixed Point Theory and ApplicationsQ40.9718.21835Journal of Numerical MathematicsQ40.9515.02336MetrikaQ40.9519.61637Statistical Methods and ApplicationsQ40.9317.72138Izvestiya: MathematicsQ40.9221.01739PositivityQ40.9218.51940Journal of Theoretical ProbabilityQ40.8819.422


## 4. Conclusion

As a quantitative evaluation indicator of journal quality, JIF plays a very important role in the process of the dissemination and development of academic research. However, JIF is not only widely used but also misused, producing skewed and misleading results. For example, JIF is misused to assess individual papers, authors, publishers, and institutions. Clearly, it is of great theoretical and practical significance to analyze and optimize the mechanism and effectiveness of existing evaluation indicators. This paper developed a simulation model based on the mechanism of submission, review, and citation of papers, which can be used to reproduce the differentiation process of impact factors of different journals within a similar discipline. The study is dedicated to providing a novel experimental approach based on social parallel systems, which can structure virtual citation networks for a specific discipline. Relevant series of studies can provide enlightening and helpful hints for facilitating and managing academic journals in the future. It is worth noting that intuition and speculation taken for granted are often unreliable in science, until they are validated by scientific evidence and statistical inference. Objectively speaking, without the method and the assistance of simulation experiments here, it might be very difficult to present the experimental evidence that implies the statistical correlations between the JIF dynamics and certain behaviors and elements in publication. The simulation results demonstrate that the behaviors of submission and citation would be influenced and driven by the JIF. It is the interplay between the submission and citation that further clarifies the mechanism of JIF dynamics. From an analysis of the experimental and statistical results, it can be found that the impact factor of a journal is affected by many variables and latent factors, including the discipline field of the journal, the average number of references per journal, and the peer review time of the journal. These factors can be mainly summarized as three aspects. (1) Discipline difference: experimental results in some subject fields require relatively more time to mature due to the delay in verification and recognition. The difference is so significant that the bottom journal in one discipline may have an impact factor higher than the top journal in another field, for example, the average JIF of biology in Q3 is 2-3 times higher than AI and mathematics in Q2. (2) The number of references: in principle, JIF is computed via the count of references; therefore, it is possible to be increased by adding the number of references in a reasonable and scientific manner. Moreover, the number and the aging distribution of references in a discipline field not only reflect the timeliness and characteristic of research achievements but also have a cumulative effect on changing the overall level of impact factors. (3) Peer review time: the experimental results demonstrated that journals with short peer review time tend to obtain higher impact factors. Thus, the impact factor of a journal can be increased by scientific peer review and effective journal management.In the future, a series of meaningful work can be conducted subsequently; an interesting study on how the impact factor is differentiated within a similar discipline can be carried out by extending the current model. In addition to simplistic algorithms such as JIF, the advantages of various methods and the factors that can be weighted should be taken into account. It also should be a meaningful future direction to refine the configuration of the review process. For instance, what is the relationship between the reviewers and the subject of the paper and whether or not decision trees could be used to make selection or decision, as well, what is the correlation between the impact factor of a journal and the publication cycle of the journal. Furthermore, certain phenomena observed in experiments could possibly be explained in analytical or statistical manners, combining theories and approaches in scientometrics and machine learning. One may consider utilizing the technologies of PCA (principal component analysis) and Laplacian score for better analyzing the distribution and the stability of the JIF between different disciplines.



---

*Source: 3154619-2020-08-26.xml*