# The Correlation between Altmetric Attention Score and Traditional Bibliometrics in Top Nursing Journal Articles

Authors: Lingmin Chen; Mutong Yang; Nian Li; Ying He; Yonggang Zhang
Journal: Journal of Nursing Management
 (2023)

---

## Abstract

Background. Altmetric Attention Score (AAS) is a quantitative measurement of the online impact of research and has a potential correlation with traditional bibliometrics. However, the correlation for nursing journal articles is still unknown. The objective of the study was to analyze the correlation between AAS and traditional bibliometrics in the top nursing journal articles. Materials and Methods. Articles published in top nursing journals (the journals with the top 20 5-year impact factors) from 2010 to 2019 were included. The correlations between AAS and citations, AAS and Relative Citation Ratio (RCR) score, AAS and Category Normalized Citation Impact (CNCI) value, and AAS and impact factors were analyzed. Statistical analyses were performed using Stata 25.0 software. Results. A total of 15,212 journal articles were included in the study. Very weak correlations were found between AASs and citations [0.124 (95% CI, 0.108–0.14)], AASs and RCRs [0.26 (95% CI, 0.244–0.275)], and AASs and CNCIs [0.207 (95% CI, 0.192–0.223)]. The weak correlations were also found between AASs and impact factors in several journals. The weak correlations between AASs and citations, AASs and CNCIs, and AASs and RCRs were also found for most journals based on subgroup analysis. Conclusions. There is very weak correlations between AASs and traditional bibliometrics in top nursing journal articles. More studies should be conducted to assess how AAS influence bibliometrics, and how they can help manage nursing journal articles and research.


---

## Body

## 1. Introduction

Bibliometrics is the use of statistical methods to analyze publications, especially for scientific contents [1, 2]. It is used to justify the role of the researchers and the research team [3] and also to identify key partners [4] and the progress of a specific research field [5]. The most widely used bibliometrics method is based on citations [6], such as citation times, impact factors [7], H index [8], Eigenfactor score [9], and CiteScore [9]. However, these metrics have limitations. It would take time to accumulate citation times [10], and different journal articles could not be compared directly even in the same research field [11, 12]. Absolute citation times favor older journal articles and have limited utility in comparing journal articles from different fields [12, 13]. Thus, new metrics have been developed. RCR [14], developed by NIH, is defined as the total number of citations that a paper received per year divided by the average field-specific citation rate for a peer companion group [14, 15]. It allows the comparison of articles in different fields. CNCI is another metrics to assess the paper's impact and has been analyzed in different studies [16, 17], which can also be used to assess scientific articles from different research fields [16, 18].In recent years, with the development of digital technology and the use of online platforms to discuss research, alternative-level metrics (altmetrics) have been introduced in research fields [12–14]. They were used to measure the journal articles with attention, dissemination, overall influence, and impacts [14]. The main altmetrics platform is Altmetric [19, 20], which compiles the number of mentions of a paper across the most used social media platforms such as Twitter, Facebook, and LinkedIn, and public policy documents, mainstream media, online reference managers, and other online platforms, to generate a weighted score, Altmetric Attention Score (AAS) [14]. AAS is a dynamic bibliometric which captures the online impact of a paper. It has been used to assess the online impact of journal articles across different fields [7, 14, 21] and provide potential evidence for research impact or journal strategy.Nursing is an important discipline, and research helps the progress of the discipline by improving nursing practice and finally helping nursing management. Factors that influence the characteristics of nursing journal articles could further help journal articles to be quickly transformed into nursing practice and applied to nursing management. It has been reported that bibliometric analysis might help nursing management and nursing practice, and thus some studies have been performed in the field of nursing management [2, 16]. However, these studies did not analyze the important impact of online platforms on nursing management. It is still unclear whether the online impact would be associated with traditional bibliometrics and whether the online impact would help nursing paper citations. Furthermore, it is also unclear whether the nursing manager could use social media, public research platforms, or online platfroms to help nursing research impact. Although several articles have used bibliometrics methods to assess the top cited [2, 16] or the top impact nursing articles [2, 16], whether impact from social media, public research platforms, or online platforms will help higher citations is a mystery. Therefore, in the current study, we analyzed the Altmetric and bibliometric data of journal articles published in nursing journals, to assess the correlation between AASs and the citation metrics.


## 2. Methods

### 2.1. Inclusion and Exclusion Criteria for the Selection of Nursing Journal Articles

The inclusion criteria were as follows: (1) the journal articles should be published in top nursing journals. Five-year impact factors were used to identify the top nursing journals, and thus the top 20 5-year impact factor journals in the nursing category in the JCR report in 2021 were used in the current study. (2) The journals should be indexed by PubMed. Thus, the journals included in the study were as follows: Australian Critical Care, Birth-Issues in Perinatal Care, BMC Nursing, European Journal of Cardiovascular Nursing, Intensive And Critical Care Nursing, International Journal of Mental Health Nursing, International Journal of Nursing Studies, International Nursing Review, Journal of Advanced Nursing, Journal of Clinical Nursing, Journal of Nursing Management, Journal of Nursing Scholarship, Nurse Education Today, Nursing Ethics, Nursing Outlook, Research in Nursing & Health, Seminars in Oncology Nursing, Women and Birth, Worldviews on Evidence-Based Nursing, and Asian Nursing Research. (3) The paper type should be an article or review. (4) The paper could be found in Incites, iCite, Altmetric, and PubMed. The following exclusion criteria were used: (1) duplicated journal articles; (2) mismatched journal articles; (3) editorial articles, corrections, and letters.


### 2.2. Paper Searching

The journal articles were searched by the selected journal names in the three databases (Incites [16], iCite [14], and Altmetric [12]). All search results were downloaded. If any data for an article was missing, the article was further searched in PubMed or the specific journal websites.


### 2.3. Paper Screening and Data Extraction

All articles were screened according to the inclusion and exclusion criteria. Only articles which had all the required data (RCR, citation time, AAS, and CNCI) were included. The following information was extracted from each article: journal, author, impact factor, RCR score, citation time, CNCI value, AAS, etc. The RCR scores were extracted from the downloaded file from the iCite database, the CNCI values were extracted from the downloaded file from the Incites database, and the AASs were extracted from the downloaded file from the Altmetric database. The journals' impact facotrs were downloaded from the Web of Science.


### 2.4. Statistical Analysis

The characteristics of the included studies were analyzed. The correlations between AASs and citations, AASs and RCRs, AASs and CNCIs, and AASs and impact factors, were analyzed using Spearman correlation coefficients [12]. Subgroup analyses were performed according to journals, and years. All analyses were performed by SPSS 25.0 software.


## 2.1. Inclusion and Exclusion Criteria for the Selection of Nursing Journal Articles

The inclusion criteria were as follows: (1) the journal articles should be published in top nursing journals. Five-year impact factors were used to identify the top nursing journals, and thus the top 20 5-year impact factor journals in the nursing category in the JCR report in 2021 were used in the current study. (2) The journals should be indexed by PubMed. Thus, the journals included in the study were as follows: Australian Critical Care, Birth-Issues in Perinatal Care, BMC Nursing, European Journal of Cardiovascular Nursing, Intensive And Critical Care Nursing, International Journal of Mental Health Nursing, International Journal of Nursing Studies, International Nursing Review, Journal of Advanced Nursing, Journal of Clinical Nursing, Journal of Nursing Management, Journal of Nursing Scholarship, Nurse Education Today, Nursing Ethics, Nursing Outlook, Research in Nursing & Health, Seminars in Oncology Nursing, Women and Birth, Worldviews on Evidence-Based Nursing, and Asian Nursing Research. (3) The paper type should be an article or review. (4) The paper could be found in Incites, iCite, Altmetric, and PubMed. The following exclusion criteria were used: (1) duplicated journal articles; (2) mismatched journal articles; (3) editorial articles, corrections, and letters.


## 2.2. Paper Searching

The journal articles were searched by the selected journal names in the three databases (Incites [16], iCite [14], and Altmetric [12]). All search results were downloaded. If any data for an article was missing, the article was further searched in PubMed or the specific journal websites.


## 2.3. Paper Screening and Data Extraction

All articles were screened according to the inclusion and exclusion criteria. Only articles which had all the required data (RCR, citation time, AAS, and CNCI) were included. The following information was extracted from each article: journal, author, impact factor, RCR score, citation time, CNCI value, AAS, etc. The RCR scores were extracted from the downloaded file from the iCite database, the CNCI values were extracted from the downloaded file from the Incites database, and the AASs were extracted from the downloaded file from the Altmetric database. The journals' impact facotrs were downloaded from the Web of Science.


## 2.4. Statistical Analysis

The characteristics of the included studies were analyzed. The correlations between AASs and citations, AASs and RCRs, AASs and CNCIs, and AASs and impact factors, were analyzed using Spearman correlation coefficients [12]. Subgroup analyses were performed according to journals, and years. All analyses were performed by SPSS 25.0 software.


## 3. Results

### 3.1. The Main Characteristics of Included Journal Articles

The data were collected by September 9th, 2022. After screening all data of the journal articles according to the inclusion and exclusion criteria, a total of 15,212 journal articles were included. The impact factors for all the selected 20 journals from 2010 to 2019 are shown in Table 1, and the impact factors of most journals increased from 2010 to 2019. The average citation of all journal articles was 17.36 times, and the median citation was 11 times. A total of 318 journal articles were cited 0 times, 14,059 journal articles were cited 1 to 50 times, 660 were cited 51 to 100 times, and 175 journal articles were cited more than 100 times.Table 1
The impact factors of the included 20 nursing journals from 2010 to 2019.
Journal2010201120122013201420152016201720182019Asian Nursing Research0.1330.0710.440.4181.0000.8490.7680.9181.2560.988Australian Critical Care—0.9730.9531.2651.5621.4791.9071.9302.5152.214Birth-Issues in Perinatal Care1.8212.1822.9262.0481.2641.8672.5182.3292.1292.705BMC Nursing—————————1.846European Journal of Cardiovascular Nursing1.3481.7112.0421.8281.8762.4912.7632.6512.4972.296Intensive and Critical Care Nursing—————1.2141.3261.6531.6521.886International Journal of Mental Health Nursing1.4271.0711.2872.0091.9501.9431.8692.0332.4332.383International Journal of Nursing Studies2.1032.1782.0752.2482.9013.5613.7553.6563.5703.783International Nursing Review0.5881.0380.9390.7360.9481.0731.5171.4961.5622.034Journal of Advanced Nursing1.5401.4771.5271.6851.7411.9171.9982.2672.3762.561Journal of Clinical Nursing1.2281.1181.3161.2331.2551.3841.2141.6351.7571.972Journal of Nursing Management1.4521.1811.4541.1421.5001.7211.9051.9122.3862.243Journal of Nursing Scholarship1.3921.4901.6121.7721.6362.1282.3962.6622.5402.655Nurse Education Today1.1131.2411.2181.4561.3641.5912.5332.0672.4422.490Nursing Ethics1.0850.8151.2101.0931.2471.4691.7551.8761.9572.597Nursing Outlook1.6531.5222.3591.8311.5882.2872.2362.4252.5402.833Research in Nursing & Health1.7361.7082.1811.1631.2671.6381.6931.7621.6782.163Seminars in Oncology Nursing———————1.6671.4121.330Women and Birth———1.6961.5731.5252.1381.8222.0792.308Worldviews on Evidence-Based Nursing1.4291.2391.3492.3182.3811.7622.1032.1432.5001.991—: not available.


### 3.2. The Performance of Included Journal Articles

The average RCR score of included articles was 1.71, and the median RCR score was 1.14. The RCR scores of 258 articles were 0; 6,576 articles had RCR scores between 0 and 1; 8,236 journal articles had RCR scores between 1 and 10; the RCR scores of 142 journal articles were higher than 10. The average CNCI value was 1.52, and the median CNCI value was 1.05. The CNCI values of 318 journal articles were 0; 7,017 journal articles had CNCI values between 0 and 1; 7,785 journal articles had CNCI values between 1 and 10; the CNCI values of 92 journal articles were higher than 10. The average AAS was 8.92, and the median AAS was 3. The AASs of 2264 journal articles were 0; 9,950 journal articles had AASs between 0 and 10; 2,866 journal articles had AASs between 10 and 100; the AASs of 132 journal articles were higher than 100. The citations, RCRs, CNCIs, and AASs of all journal articles in the 20 journals were summarized in listed in Table2.Table 2
The citations, RCRs, CNCIs, and AASs of the included 20 nursing journals [median (range)].
JournalCitationsRCRsCategory normalized citation impact (CNCI)sAASsAsian Nursing Research9 (0–677)0.88 (0.08–34.29)0.9267 (0–26.0303)1 (0–151)Australian Critical Care8 (0–73)0.83 (0–10.92)0.6222 (0–8.4983)3 (0–137)Birth-Issues in Perinatal Care12 (0–202)1.23 (0–11.48)1.0873 (0–10.717)4 (0–773)BMC Nursing7 (0–78)1.155 (0–11.67)0.9076 (0–11.9965)2 (0–215)European Journal of Cardiovascular Nursing11 (0–676)0.93 (0–36.46)0.7728 (0–36.5083)1 (0–206)Intensive and Critical Care Nursing8 (0–66)1.07 (0–7.69)1.0534 (0–8.3769)2 (0–76)International Journal of Mental Health Nursing10 (0–172)1.09 (0–13.22)0.82735 (0–11.0035)7 (0–220)International Journal of Nursing Studies19 (0–582)1.79 (0–35.94)1.6983 (0–27.4897)3 (0–703)International Nursing Review9 (0–253)0.94 (0–23.61)0.89685 (0–20.285)2 (0–73)Journal of Advanced Nursing12 (0–415)1.15 (0–25.58)1.1492 (0–18.9222)7 (0–1462)Journal of Clinical Nursing10 (0–262)0.98 (0–21.32)0.9812 (0–20.571)2 (0–251)Journal of Nursing Management11 (0–203)1.43 (0–17.58)0.9246 (0–11.235)1 (0–228)Journal of Nursing Scholarship12 (0–301)1.19 (0–17.95)1.3087 (0–23.6331)2 (0–170)Nurse Education Today11 (0–770)1.34 (0–81.92)1.08825 (0–100.5749)3 (0–373)Nursing Ethics9 (0–132)1.15 (0–15.36)1.1756 (0–16.0068)1 (0–165)Nursing Outlook10 (0–1841.045 (0–15.45)1.0906 (0–13.1287)2 (0–670)Research in Nursing & Health11 (0–363)0.915 (0–40.90)1.1363 (0–22.0488)1 (0–244)Seminars in Oncology Nursing6 (0–230)0.51 (0–20.55)0.4514 (0–17.3025)1 (0–56)Women and Birth9 (0–234)1.07 (0–17.53)0.8593 (0–8.8129)3 (0–231)Worldviews on Evidence-Based Nursing11 (0–234)1.18 (0–21.91)1.0154 (0–20.5276)2 (0–245)


### 3.3. The Contribution of Different Sources to AAS

The sources that contributing to the AAS were individually analyzed from 2010 to 2019. For all years, Twitter was the most frequent contributor to AAS followed by news outlets and Policy (Table3).Table 3
Attention received by papers from high-impact journals from 2010 to 2019 stratified by different resources.
YearNews mentionsBlog mentionsPolicy mentionsPatent mentionsTwitter mentionsPeer review mentionsWeibo mentionsFacebook mentionsWikipedia mentionsGoogle + mentionsLinkedIn mentionsReddit mentionsPinterest mentionsF1000 mentionsQ&A mentionsVideo mentionsSyllabi mentionsNumber of mendeley readersNumber of dimensions citations201013237252499650121266000207084885362512011153482772614061011538500010709419935584201213740286213198202692520240109011741039041201319846351656165142873922401020701371764156020142946129818873903633735030101201526094258220152687624641379430847433202010140176717458582016873852484299567099760420200029019698343404201766063258031567301010291230301150189607392072018940107175552475509525473050302802064793925120198388812803992813905173390704013016278726578


### 3.4. The Correlation of AASs with Citations, RCRs, CNCIs

The correlations of AASs with citations, CNCIs, and RCRs are shown in Figures1–3. In summary, the correlation coefficient was 0.124 (95% CI, 0.108–0.14) between AASs and citations, 0.26 (95% CI, 0.244–0.275) between AASs and RCRs, and 0.207 (95% CI, 0.192–0.223) between AASs and CNCIs. The correlations for each journal are shown in Table 4.Figure 1
The correlation between AAS and citations.Figure 2
The correlation between AAS and CNCIs.Figure 3
The correlation between AAS and RCRs.Table 4
The correlations between citation and AAS, CNCI and AAS, RCR, and AAS based on journals.
JournalCitation and AASCNCI and AASRCR and AASRhoP valueRhoP valueRhoP valueAsian Nursing Research0.2840.0020.2400.0110.2340.014Australian Critical Care0.1150.0430.309<0.0010.342<0.001Birth-Issues in Perinatal Care0.234<0.0010.311<0.0010.347<0.001BMC Nursing0.286<0.0010.2440.0030.2040.014European Journal of Cardiovascular Nursing0.1070.0230.225<0.0010.233<0.001Intensive and Critical Care Nursing0.1430.0050.239<0.0010.234<0.001International Journal of Mental Health Nursing0.0120.7560.233<0.0010.315<0.001International Journal of Nursing Studies0.207<0.0010.267<0.0010.371<0.001International Nursing Review0.207<0.0010.226<0.0010.268<0.001Journal of Advanced Nursing−0.0220.2990.105<0.0010.209<0.001Journal of Clinical Nursing0.133<0.0010.230<0.0010.312<0.001Journal of Nursing Management0.158<0.0010.206<0.0010.239<0.001Journal of Nursing Scholarship0.164<0.0010.246<0.0010.209<0.001Nurse Education Today0.0860.0010.188<0.0010.269<0.001Nursing Ethics0.0250.6000.0550.2470.0080.860Nursing Outlook0.255<0.0010.327<0.0010.316<0.001Research in Nursing & Health0.1520.0060.1740.0020.200<0.001Seminars in Oncology Nursing0.2460.0010.2380.0010.295<0.001Women and Birth0.1110.0050.248<0.0010.275<0.001Worldviews on Evidence-Based Nursing0.245<0.0010.265<0.0010.295<0.001For the International Journal of Nursing Studies, which ranked first in the JCR report (2021) with an impact factor of 6.612, a total of 1,437 journal articles were included. The total citation times was 43,729 (ranging from 0 to 582 by paper, median 19), the total AAS was 12,975 (ranging from 0 to 703 by paper, median 3), and the total RCR score was 3,985.92 (ranging from 0 to 35.94 by paper, median 1.79). Very weak correlations were found: 0.207 (95% CI, 0.158–0.258) between AASs and citations, 0.371 (95% CI, 0.325–0.418) between AASs and RCRs, and 0.267 (95% CI, 0.219–0.317) between AASs and CNCIs.The Journal of Clinical Nursing ranked fourth in the JCR report (2021) with an impact factor of 4.423, from which we collected the highest number of journal articles in the current study. The total number of citations was 48,236 (ranging from 0 to 262 by paper, median 10), the total AASs was 17,842 (ranging from 0 to 251 by paper, median 2), and the total RCR score was 4,709.72 (ranging from 0 to 21.32 by paper, median 0.98). Very weak correlations were found: 0.133 (95% CI, 0.097–0.166) between AASs and citations, 0.312 (95% CI, 0.28–0.346) between AASs and RCRs, and 0.23 (95% CI, 0.199–0.264) between AASs and CNCIs.The Journal of Nursing Management was the only management journal in the field of nursing, from which a total of 783 journal articles were included. The total number of citations was 13,906 (ranging from 0 to 203 by paper, median 11), the total AASs was 3,007 (ranging from 0 to 228 by paper, median 1), and the total RCR score was 1,593.84 (ranging from 0 to 17.58 by paper, median 1.43). Very weak correlations were found: 0.158 (95% CI, 0.086–0.229) between AASs and citations, 0.239 (95% CI, 0.162–0.304) between AASs and RCRs, and 0.206 (95% CI, 0.137–0.274) between AASs and CNCIs.Nurse Education Today was the only education journal in the field of nursing, from which a total of 1,566 journal articles were analyzed. The total citation was 26,555 (ranging from 0 to 770 by paper, median 11), the total AAS was 15,837 (ranging from 0 to 373, median 3), and the total RCR score was 3,104.38 (ranging from 0 to 81.92, median 1.34). Very weak correlations were found: 0.086 (95% CI, 0.038–0.134) between AASs and citations, 0.269 (95% CI, 0.223–0.317) between AASs and RCRs, and 0.188 (95% CI, 0.139–0.234) between AASs and CNCIs.


### 3.5. The Correlation Analysis by Years

Spearman rank correlation coefficient was calculated between citations and the AASs by years (Table5). The results showed very weak correlations between AASs and citations, AASs and CNCIs, and AASs and RCRs in all years.Table 5
Spearman rank correlation coefficient between citation, CNCI, RCR and AAS from 2010 to 2019.
YearCitation and AASCNCI and AASRCR and AASRhoP valueRhoP valueRhoP value20100.293<0.0010.262<0.0010.275<0.00120110.327<0.0010.318<0.0010.306<0.00120120.238<0.0010.231<0.0010.219<0.00120130.278<0.0010.258<0.0010.262<0.00120140.301<0.0010.284<0.0010.277<0.00120150.285<0.0010.253<0.0010.271<0.00120160.259<0.0010.207<0.0010.241<0.00120170.280<0.0010.236<0.0010.268<0.00120180.275<0.0010.240<0.0010.269<0.00120190.242<0.0010.208<0.0010.246<0.001


### 3.6. The Correlation between AASs and Impact Factors

The correlations between AASs and impact factors were individually analyzed based on years. Weak correlations were found in most of the years (Table6). Spearman rank correlation coefficient was calculated between the impact factors and AASs based on journals (Table 7). Weak correlations were found for the Australian Critical Care, International Journal of Mental Health Nursing, International Journal of Nursing Studies, Journal of Advanced Nursing, and Nurse Education Today.Table 6
The correlation between impact factors and AASs from 2010 to 2019.
YearRhoP value20100.156<0.00120110.0750.02520120.125<0.00120130.186<0.00120140.165<0.00120150.190<0.00120160.0780.00120170.203<0.00120180.183<0.00120190.141<0.001Table 7
The correlation between impact factors and AASs based on journals.
JournalRhoP valueAsian Nursing Research−0.0860.371Australian Critical Care0.550<0.001Birth-Issues in Perinatal Care−0.0500.312BMC Nursing——European Journal of Cardiovascular Nursing0.208<0.001Intensive and Critical Care Nursing0.1660.003International Journal of Mental Health Nursing0.590<0.001International Journal of Nursing Studies0.307<0.001International Nursing Review0.1210.023Journal of Advanced Nursing0.526<0.001Journal of Clinical Nursing0.287<0.001Journal of Nursing Management0.142<0.001Journal of Nursing Scholarship0.1530.001Nurse Education Today0.352<0.001Nursing Ethics0.1100.021Nursing Outlook0.168<0.001Research in Nursing & Health−0.0580.299Seminars in Oncology Nursing−0.1440.108Women and Birth0.214<0.001Worldviews on Evidence-Based Nursing0.0600.265—: not available.


## 3.1. The Main Characteristics of Included Journal Articles

The data were collected by September 9th, 2022. After screening all data of the journal articles according to the inclusion and exclusion criteria, a total of 15,212 journal articles were included. The impact factors for all the selected 20 journals from 2010 to 2019 are shown in Table 1, and the impact factors of most journals increased from 2010 to 2019. The average citation of all journal articles was 17.36 times, and the median citation was 11 times. A total of 318 journal articles were cited 0 times, 14,059 journal articles were cited 1 to 50 times, 660 were cited 51 to 100 times, and 175 journal articles were cited more than 100 times.Table 1
The impact factors of the included 20 nursing journals from 2010 to 2019.
Journal2010201120122013201420152016201720182019Asian Nursing Research0.1330.0710.440.4181.0000.8490.7680.9181.2560.988Australian Critical Care—0.9730.9531.2651.5621.4791.9071.9302.5152.214Birth-Issues in Perinatal Care1.8212.1822.9262.0481.2641.8672.5182.3292.1292.705BMC Nursing—————————1.846European Journal of Cardiovascular Nursing1.3481.7112.0421.8281.8762.4912.7632.6512.4972.296Intensive and Critical Care Nursing—————1.2141.3261.6531.6521.886International Journal of Mental Health Nursing1.4271.0711.2872.0091.9501.9431.8692.0332.4332.383International Journal of Nursing Studies2.1032.1782.0752.2482.9013.5613.7553.6563.5703.783International Nursing Review0.5881.0380.9390.7360.9481.0731.5171.4961.5622.034Journal of Advanced Nursing1.5401.4771.5271.6851.7411.9171.9982.2672.3762.561Journal of Clinical Nursing1.2281.1181.3161.2331.2551.3841.2141.6351.7571.972Journal of Nursing Management1.4521.1811.4541.1421.5001.7211.9051.9122.3862.243Journal of Nursing Scholarship1.3921.4901.6121.7721.6362.1282.3962.6622.5402.655Nurse Education Today1.1131.2411.2181.4561.3641.5912.5332.0672.4422.490Nursing Ethics1.0850.8151.2101.0931.2471.4691.7551.8761.9572.597Nursing Outlook1.6531.5222.3591.8311.5882.2872.2362.4252.5402.833Research in Nursing & Health1.7361.7082.1811.1631.2671.6381.6931.7621.6782.163Seminars in Oncology Nursing———————1.6671.4121.330Women and Birth———1.6961.5731.5252.1381.8222.0792.308Worldviews on Evidence-Based Nursing1.4291.2391.3492.3182.3811.7622.1032.1432.5001.991—: not available.


## 3.2. The Performance of Included Journal Articles

The average RCR score of included articles was 1.71, and the median RCR score was 1.14. The RCR scores of 258 articles were 0; 6,576 articles had RCR scores between 0 and 1; 8,236 journal articles had RCR scores between 1 and 10; the RCR scores of 142 journal articles were higher than 10. The average CNCI value was 1.52, and the median CNCI value was 1.05. The CNCI values of 318 journal articles were 0; 7,017 journal articles had CNCI values between 0 and 1; 7,785 journal articles had CNCI values between 1 and 10; the CNCI values of 92 journal articles were higher than 10. The average AAS was 8.92, and the median AAS was 3. The AASs of 2264 journal articles were 0; 9,950 journal articles had AASs between 0 and 10; 2,866 journal articles had AASs between 10 and 100; the AASs of 132 journal articles were higher than 100. The citations, RCRs, CNCIs, and AASs of all journal articles in the 20 journals were summarized in listed in Table2.Table 2
The citations, RCRs, CNCIs, and AASs of the included 20 nursing journals [median (range)].
JournalCitationsRCRsCategory normalized citation impact (CNCI)sAASsAsian Nursing Research9 (0–677)0.88 (0.08–34.29)0.9267 (0–26.0303)1 (0–151)Australian Critical Care8 (0–73)0.83 (0–10.92)0.6222 (0–8.4983)3 (0–137)Birth-Issues in Perinatal Care12 (0–202)1.23 (0–11.48)1.0873 (0–10.717)4 (0–773)BMC Nursing7 (0–78)1.155 (0–11.67)0.9076 (0–11.9965)2 (0–215)European Journal of Cardiovascular Nursing11 (0–676)0.93 (0–36.46)0.7728 (0–36.5083)1 (0–206)Intensive and Critical Care Nursing8 (0–66)1.07 (0–7.69)1.0534 (0–8.3769)2 (0–76)International Journal of Mental Health Nursing10 (0–172)1.09 (0–13.22)0.82735 (0–11.0035)7 (0–220)International Journal of Nursing Studies19 (0–582)1.79 (0–35.94)1.6983 (0–27.4897)3 (0–703)International Nursing Review9 (0–253)0.94 (0–23.61)0.89685 (0–20.285)2 (0–73)Journal of Advanced Nursing12 (0–415)1.15 (0–25.58)1.1492 (0–18.9222)7 (0–1462)Journal of Clinical Nursing10 (0–262)0.98 (0–21.32)0.9812 (0–20.571)2 (0–251)Journal of Nursing Management11 (0–203)1.43 (0–17.58)0.9246 (0–11.235)1 (0–228)Journal of Nursing Scholarship12 (0–301)1.19 (0–17.95)1.3087 (0–23.6331)2 (0–170)Nurse Education Today11 (0–770)1.34 (0–81.92)1.08825 (0–100.5749)3 (0–373)Nursing Ethics9 (0–132)1.15 (0–15.36)1.1756 (0–16.0068)1 (0–165)Nursing Outlook10 (0–1841.045 (0–15.45)1.0906 (0–13.1287)2 (0–670)Research in Nursing & Health11 (0–363)0.915 (0–40.90)1.1363 (0–22.0488)1 (0–244)Seminars in Oncology Nursing6 (0–230)0.51 (0–20.55)0.4514 (0–17.3025)1 (0–56)Women and Birth9 (0–234)1.07 (0–17.53)0.8593 (0–8.8129)3 (0–231)Worldviews on Evidence-Based Nursing11 (0–234)1.18 (0–21.91)1.0154 (0–20.5276)2 (0–245)


## 3.3. The Contribution of Different Sources to AAS

The sources that contributing to the AAS were individually analyzed from 2010 to 2019. For all years, Twitter was the most frequent contributor to AAS followed by news outlets and Policy (Table3).Table 3
Attention received by papers from high-impact journals from 2010 to 2019 stratified by different resources.
YearNews mentionsBlog mentionsPolicy mentionsPatent mentionsTwitter mentionsPeer review mentionsWeibo mentionsFacebook mentionsWikipedia mentionsGoogle + mentionsLinkedIn mentionsReddit mentionsPinterest mentionsF1000 mentionsQ&A mentionsVideo mentionsSyllabi mentionsNumber of mendeley readersNumber of dimensions citations201013237252499650121266000207084885362512011153482772614061011538500010709419935584201213740286213198202692520240109011741039041201319846351656165142873922401020701371764156020142946129818873903633735030101201526094258220152687624641379430847433202010140176717458582016873852484299567099760420200029019698343404201766063258031567301010291230301150189607392072018940107175552475509525473050302802064793925120198388812803992813905173390704013016278726578


## 3.4. The Correlation of AASs with Citations, RCRs, CNCIs

The correlations of AASs with citations, CNCIs, and RCRs are shown in Figures1–3. In summary, the correlation coefficient was 0.124 (95% CI, 0.108–0.14) between AASs and citations, 0.26 (95% CI, 0.244–0.275) between AASs and RCRs, and 0.207 (95% CI, 0.192–0.223) between AASs and CNCIs. The correlations for each journal are shown in Table 4.Figure 1
The correlation between AAS and citations.Figure 2
The correlation between AAS and CNCIs.Figure 3
The correlation between AAS and RCRs.Table 4
The correlations between citation and AAS, CNCI and AAS, RCR, and AAS based on journals.
JournalCitation and AASCNCI and AASRCR and AASRhoP valueRhoP valueRhoP valueAsian Nursing Research0.2840.0020.2400.0110.2340.014Australian Critical Care0.1150.0430.309<0.0010.342<0.001Birth-Issues in Perinatal Care0.234<0.0010.311<0.0010.347<0.001BMC Nursing0.286<0.0010.2440.0030.2040.014European Journal of Cardiovascular Nursing0.1070.0230.225<0.0010.233<0.001Intensive and Critical Care Nursing0.1430.0050.239<0.0010.234<0.001International Journal of Mental Health Nursing0.0120.7560.233<0.0010.315<0.001International Journal of Nursing Studies0.207<0.0010.267<0.0010.371<0.001International Nursing Review0.207<0.0010.226<0.0010.268<0.001Journal of Advanced Nursing−0.0220.2990.105<0.0010.209<0.001Journal of Clinical Nursing0.133<0.0010.230<0.0010.312<0.001Journal of Nursing Management0.158<0.0010.206<0.0010.239<0.001Journal of Nursing Scholarship0.164<0.0010.246<0.0010.209<0.001Nurse Education Today0.0860.0010.188<0.0010.269<0.001Nursing Ethics0.0250.6000.0550.2470.0080.860Nursing Outlook0.255<0.0010.327<0.0010.316<0.001Research in Nursing & Health0.1520.0060.1740.0020.200<0.001Seminars in Oncology Nursing0.2460.0010.2380.0010.295<0.001Women and Birth0.1110.0050.248<0.0010.275<0.001Worldviews on Evidence-Based Nursing0.245<0.0010.265<0.0010.295<0.001For the International Journal of Nursing Studies, which ranked first in the JCR report (2021) with an impact factor of 6.612, a total of 1,437 journal articles were included. The total citation times was 43,729 (ranging from 0 to 582 by paper, median 19), the total AAS was 12,975 (ranging from 0 to 703 by paper, median 3), and the total RCR score was 3,985.92 (ranging from 0 to 35.94 by paper, median 1.79). Very weak correlations were found: 0.207 (95% CI, 0.158–0.258) between AASs and citations, 0.371 (95% CI, 0.325–0.418) between AASs and RCRs, and 0.267 (95% CI, 0.219–0.317) between AASs and CNCIs.The Journal of Clinical Nursing ranked fourth in the JCR report (2021) with an impact factor of 4.423, from which we collected the highest number of journal articles in the current study. The total number of citations was 48,236 (ranging from 0 to 262 by paper, median 10), the total AASs was 17,842 (ranging from 0 to 251 by paper, median 2), and the total RCR score was 4,709.72 (ranging from 0 to 21.32 by paper, median 0.98). Very weak correlations were found: 0.133 (95% CI, 0.097–0.166) between AASs and citations, 0.312 (95% CI, 0.28–0.346) between AASs and RCRs, and 0.23 (95% CI, 0.199–0.264) between AASs and CNCIs.The Journal of Nursing Management was the only management journal in the field of nursing, from which a total of 783 journal articles were included. The total number of citations was 13,906 (ranging from 0 to 203 by paper, median 11), the total AASs was 3,007 (ranging from 0 to 228 by paper, median 1), and the total RCR score was 1,593.84 (ranging from 0 to 17.58 by paper, median 1.43). Very weak correlations were found: 0.158 (95% CI, 0.086–0.229) between AASs and citations, 0.239 (95% CI, 0.162–0.304) between AASs and RCRs, and 0.206 (95% CI, 0.137–0.274) between AASs and CNCIs.Nurse Education Today was the only education journal in the field of nursing, from which a total of 1,566 journal articles were analyzed. The total citation was 26,555 (ranging from 0 to 770 by paper, median 11), the total AAS was 15,837 (ranging from 0 to 373, median 3), and the total RCR score was 3,104.38 (ranging from 0 to 81.92, median 1.34). Very weak correlations were found: 0.086 (95% CI, 0.038–0.134) between AASs and citations, 0.269 (95% CI, 0.223–0.317) between AASs and RCRs, and 0.188 (95% CI, 0.139–0.234) between AASs and CNCIs.


## 3.5. The Correlation Analysis by Years

Spearman rank correlation coefficient was calculated between citations and the AASs by years (Table5). The results showed very weak correlations between AASs and citations, AASs and CNCIs, and AASs and RCRs in all years.Table 5
Spearman rank correlation coefficient between citation, CNCI, RCR and AAS from 2010 to 2019.
YearCitation and AASCNCI and AASRCR and AASRhoP valueRhoP valueRhoP value20100.293<0.0010.262<0.0010.275<0.00120110.327<0.0010.318<0.0010.306<0.00120120.238<0.0010.231<0.0010.219<0.00120130.278<0.0010.258<0.0010.262<0.00120140.301<0.0010.284<0.0010.277<0.00120150.285<0.0010.253<0.0010.271<0.00120160.259<0.0010.207<0.0010.241<0.00120170.280<0.0010.236<0.0010.268<0.00120180.275<0.0010.240<0.0010.269<0.00120190.242<0.0010.208<0.0010.246<0.001


## 3.6. The Correlation between AASs and Impact Factors

The correlations between AASs and impact factors were individually analyzed based on years. Weak correlations were found in most of the years (Table6). Spearman rank correlation coefficient was calculated between the impact factors and AASs based on journals (Table 7). Weak correlations were found for the Australian Critical Care, International Journal of Mental Health Nursing, International Journal of Nursing Studies, Journal of Advanced Nursing, and Nurse Education Today.Table 6
The correlation between impact factors and AASs from 2010 to 2019.
YearRhoP value20100.156<0.00120110.0750.02520120.125<0.00120130.186<0.00120140.165<0.00120150.190<0.00120160.0780.00120170.203<0.00120180.183<0.00120190.141<0.001Table 7
The correlation between impact factors and AASs based on journals.
JournalRhoP valueAsian Nursing Research−0.0860.371Australian Critical Care0.550<0.001Birth-Issues in Perinatal Care−0.0500.312BMC Nursing——European Journal of Cardiovascular Nursing0.208<0.001Intensive and Critical Care Nursing0.1660.003International Journal of Mental Health Nursing0.590<0.001International Journal of Nursing Studies0.307<0.001International Nursing Review0.1210.023Journal of Advanced Nursing0.526<0.001Journal of Clinical Nursing0.287<0.001Journal of Nursing Management0.142<0.001Journal of Nursing Scholarship0.1530.001Nurse Education Today0.352<0.001Nursing Ethics0.1100.021Nursing Outlook0.168<0.001Research in Nursing & Health−0.0580.299Seminars in Oncology Nursing−0.1440.108Women and Birth0.214<0.001Worldviews on Evidence-Based Nursing0.0600.265—: not available.


## 4. Discussion

In the current study, the Altmetric and traditional bibliometric of journal articles published in 20 top nursing journals were analyzed. A very weak correlation between AASs and citations was found. AAS was also found to be correlated with RCR and CNCI, which were novel metrics of research influence based on citation times. The results suggest that traditional bibliometrics and AAS cannot be used interchangeably but rather complementarily when assessing the impact of journal articles in the nursing field.Further analysis by journals and publication years found that the International Journal of Nursing Studies (IJNS) had more correlation between the AASs and citations, and no correlation was found in the Nursing Ethics. This difference might be due to the difference in the breadth of journal topics since one is a comprehensive nursing journal and the other is a subspecificity nursing journal. Articles from IJNS with high online attention might also impact the scientific community and thus increase citations. However, articles from Nursing Ethics, which are likely to be controversial articles for the public to garner public interest, thereby increasing AAS, while they might not impact the scientific community in the same manner [12].With the COVID-19 pandemic [22], more training programs in the nursing practice field have been performed via the social media platforms and other online platforms. That should be of interest to nursing managers, nurses, and journals. It indicates that the social media and other online platforms would not only help the scientific community but also help the public community, especially during the COVID-19 pandemic period [23]. Implementing new research works on the online platforms will contribute to these results and further help the scientific community.The study has several strengths. First, the current study was a study with large sample size analyzing alternative metrics and bibliometrics based on articles in top nursing journals. Second, it analyzed the AASs with 3 citation-based metrics and found weak correlations between these parameters. Third, we analyzed the correlation by journals, which was a unique contribution to the nursing field. The paper also has several limitations. First, to analyze the precision correlations, we excluded journal articles that missed any data of the four variables (AASs, citations, RCRs, and CNCIs). Second, the correlations might be different when analyzing at a different time. Third, the study only analyzed the journal articles from top nursing journals, and nursing-associated journal articles published in other journals could have different kinds of correlations, especially for those articles published in low-impact factor journals. Fourth, the study did not include other bibliometrics [24, 25], such as the Eigenfactor score and CiteScore; in the future, a new study should be performed to assess these correlations.In conclusion, the study of articles published in high-impact nursing journals between 2010 to 2019, finds very weak correlations between AAS and citation-based metrics, which suggests promoting journal articles on online platforms may help research, journals, and nurses. Future studies are needed to assess the long-term correlations among these metrics for nursing journal articles.


## 5. Implication to Nursing Management

Understanding the correlation of online impact with traditional bibliometrics of research is critical to nursing practitioners, which in turn helps manage nursing journal articles and research. Nursing managers should develop targeted strategies to increase the online impact of research or nursing practice and increase research impact.



---

*Source: 2789960-2023-02-22.xml*