Artificial IntelligenceIndex Report 2022

CHAPTER 1:

Research &Development

Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT

CHAPTER 1:

Chapter Preview
Overview

3

AI Repositories

20

Chapter Highlights

4

Overview

20

By Region

21

1.1 PUBLICATIONS

5

By Geographic Area

22

Overview

5

Citations23

Total Number of AI Publications

5

By Type of Publication

6

Overview

24

By Field of Study7

By Region and Application Status

25

By Sector

8

Cross-Country Collaboration

10

By Geographic Area andApplication Status

27

Cross-Sector Collaboration

11

AI Patents

24

12

1.2 CONFERENCES

29

Overview

12

Conference Attendance

29

By Region

13

By Geographic Area

14

Women in Machine Learning (WiML)
NeurIPS Workshop

31

Citations

15

AI Journal Publications

AI Conference Publications

16

Workshop Participants

31

Demographics Breakdown

32

Overview

16

By Region

17

1.3 AI OPEN-SOURCE SOFTWARELIBRARIES

33

By Geographic Area

18

GitHub Stars

33

Citations

19
APPENDIX

35

ACCESS THE PUBLIC DATA

2

Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT

Overview
Research and development is an integral force that drives the rapid progressof artificial intelligence (AI). Every year, a wide range of academic, industry,
government, and civil society experts and organizations contribute to AI
R&D via a slew of papers, journal articles, and other AI-related publications,conferences on AI or on particular subtopics like image recognition or
natural language processing, international collaboration across borders, andthe development of open-source software libraries. These R&D efforts arediverse in focus and geographically dispersed.
Another key feature of AI R&D, making it somewhat distinct from other areasof STEM research, is its openness. Each year, thousands and thousands ofAI publications are released in the open source, whether at conferencesor on file-sharing websites. Researchers will openly share their findings at
conferences; government agencies will fund AI research that ends up in the
open source; and developers use open software libraries, freely available tothe public, to produce state-of-the-art AI applications. This openness also
contributes to the globally interdependent and interconnected nature ofmodern AI R&D.
This first chapter draws on multiple datasets to analyze key trends in the
AI research and development space in 2021. It first looks at AI publications,
including conference papers, journal articles, patents, and repositories.
It then analyzes AI conference attendance. And finally, it examines
AI open-source software libraries used in the R&D process.

Chapter 1 Preview

3Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT

CHAPTER HIGHLIGHTS• Despite rising geopolitical tensions, the United States and China had the greatest number ofcross-country collaborations in AI publications from 2010 to 2021, increasing five times since2010. The collaboration between the two countries produced 2.7 times more publications
than between the United Kingdom and China—the second highest on the list.
• In 2021, China continued to lead the world in the number of AI journal, conference, andrepository publications—63.2% higher than the United States with all three publication types
combined. In the meantime, the United States held a dominant lead among major AI powers
in the number of AI conference and repository citations.
• From 2010 to 2021, the collaboration between educational and nonprofit organizations
produced the highest number of AI publications, followed by the collaboration between
private companies and educational institutions and between educational and government
institutions.
• The number of AI patents filed in 2021 is more than 30 times higher than in 2015, showing acompound annual growth rate of 76.9%.

Chapter 1 Preview

4Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

This section draws on data from the Center for Security and Emerging Technology (CSET) at Georgetown University. CSET maintains
a merged corpus of scholarly literature that includes Digital Science’s Dimensions, Clarivate’s Web of Science, Microsoft Academic
Graph, China National Knowledge Infrastructure, arXiv, and Papers with Code. In that corpus, CSET applied a classifier to identify
English-language publications related to the development or application of AI and ML since 2010.1

1.1 PUBLICATIONS2OV E R V I E W

Total Number of AI Publications

The figures below capture the total number of Englishlanguage AI publications globally from 2010 to 2021—by
type, affiliation, cross-country collaboration, and crossindustry collaboration. The section also breaks down
publication and citation data by region for AI journal
articles, conference papers, repositories, and patents.

Figure 1.1.1 shows the number of AI publications in
the world. From 2010 to 2021, the total number of AI
publications doubled, growing from 162,444 in 2010 to334,497 in 2021.

NUMBER of AI PUBLICATIONS in the WORLD, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

Number of AI Publications (in thousands)

350

334.50

300

250

200

150

100

50
0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.1

1 See the Appendix for more information on CSET’s methodology. Due to the change in data provider and classification method, the publication trend/data may be different from past reports. For moreon the challenge of defining AI and correctly capturing relevant bibliometric data, see the AI Index team’s discussion in the paper “Measurement in AI Policy: Opportunities and Challenges.”
2 The number of AI publications in 2021 in this section may be lower than the actual count due to the lag in the collection of publication metadata by aforementioned databases.

ChapterTable of Contents
1 Preview

Chapter 1 Preview

5Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Type of Publication

Books, book chapters, theses, and unknown document
types comprised the remaining 10.1% of publications.
While journal and repository publications have grown
2.5 and 30 times, respectively, in the past 12 years, thenumber of conference papers has declined since 2018.

Figure 1.1.2 shows the variation overtime in the types ofAI publications released globally. In 2021, 51.5% of all AI
documents published were journal articles, 21.5% wereconference papers, and 17.0% were from repositories.

NUMBER of AI PUBLICATIONS by TYPE, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

172.11, Journal

180

Number of AI Publications (in thousands)

160
140
120
100
80
71.92, Conference60

56.73, Repository40
20.60, Book Chapter
10.84, Unknown
1.16, Book
1.14, Thesis

20
0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.2

Chapter 1 Preview

6Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Field of Study
Figure 1.1.3 shows that publications in pattern recognition
and machine learning have more than doubled since 2015.

Other areas strongly influenced by deep learning, suchas computer vision, data mining and natural language
processing, have shown smaller increases.

NUMBER of AI PUBLICATIONS by FIELD of STUDY (EXCLUDING OTHER AI), 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

51.69, Pattern Recognition

Number of AI Publications (in thousands)

50

39.93, Machine Learning40

30
24.80, Computer Vision20

18.30, Algorithm
15.27, Data Mining
13.43, Natural Language Processing
9.70, Human–Computer Interaction
8.45, Control Theory

10

5.78, Linguistics

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.3

Chapter 1 Preview

7Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Sector

United Kingdom (Figure 1.1.4d).3 The education sector
dominates in each of the regions. The level of company
participation is highest in the United States, then in theEuropean Union. China is the only area in which the shareof education has been rising.

This section shows the number of AI publications affiliatedwith industry, education, government, and nonprofit in
the world (Figure 1.1.4a), China (Figure 1.1.4b), the UnitedStates (Figure 1.1.4c), and the European Union plus the
AI PUBLICATIONS (% of TOTAL) by SECTOR, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report60%

59.58%, Education

AI Publications (% of Total)

50%

40%

30%

20.77%, Unknown

20%

11.27%, Nonpro t

10%

5.21%, Company
3.17%, Government

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.4a

AI PUBLICATIONS in UNITED STATES (% of TOTAL) by SECTOR, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

60%

57.63%, Education

AI Publications (% of Total)

50%

40%

30%

20%

17.63%, Unknown
12.36%, Nonpro t
9.76%, Company

10%2.62%, Government

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021

3 The categorization is adapted based on the Global Research Identifier Database (GRID). See definitions of each category here. Healthcare, including hospitals and facilities,
are included under nonprofit. Publications affiliated with state-sponsored universities are included in the education sector.

Chapter 1 Preview

Figure 1.1.4b

8

Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

AI PUBLICATIONS in CHINA (% of TOTAL) by SECTOR, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report60.24%, Education

60%

AI Publications (% of Total)

50%

40%

30%
23.74%, Unknown

20%

10%

8.47%, Nonpro t
3.93%, Company
3.62%, Government

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.4c

AI PUBLICATIONS in EUROPEAN UNION and UNITED KINGDOM (% of TOTAL) by SECTOR, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

60%
54.81%, Education

AI Publications (% of Total)

50%

40%

30%

20%

19.76%, Unknown
16.14%, Nonpro t

10%
5.68%, Company
3.61%, Government

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.4d

Chapter 1 Preview

9Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

Cross-Country Collaboration

By far, the greatest
number of collaborationsin the past 12 years
took place between the
United States and China,
increasing five times
since 2010.

Cross-border collaborations between academics,
researchers, industry experts, and others are a key
component of modern STEM development that accelerate
the dissemination of new ideas and the growth of researchteams. Figures 1.1.5a and 1.1.5b depict the top crosscountry AI collaborations from 2010 to 2021. CSET counted
cross-country collaborations as distinct pairs of countries
across authors for each publication (e.g., four U.S. andfour Chinese-affiliated authors on a single publication are
counted as one U.S.-China collaboration; two publications
between the same authors counts as two collaborations).
By far, the greatest number of collaborations in the past
12 years took place between the United States and China,
increasing five times since 2010. The next largest set ofcollaborations is between the United Kingdom and both theUnited States and China, which have increased more than
three times since 2010. In 2021, the number of collaborations
between the United States and China was 2.7 times greaterthan between the United Kingdom and China.

UNITED STATES and CHINA COLLABORATIONS in AI PUBLICATIONS, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

Number of AI Publications (in thousands)

10

9.66

8

6

4

2

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.5a

Chapter 1 Preview

10Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

CROSS-COUNTRY COLLABORATIONS in AI PUBLICATIONS (EXCLUDING U.S. and CHINA), 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

3.56, United Kingdom and China

Number of AI Publications (in thousands)

3.34, United States and United Kingdom

32.63, China and Australia
2.44, United States and Germany
2.37, United States and Canada
2.16, United States and Australia

2

1.42, United States and France

1

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.5b

Cross-Sector Collaboration
The increase in AI research outside of universities has broughtcollaborations between universities and other industries.
Figure 1.1.6 shows that in 2021, educational institutions
and nonprofits had the greatest number of collaborations(29,839), followed by companies and educational institutions(11,576), and governments and educational institutions(8,087). There were 2.5 times as many collaborations
between educational institutions and nonprofits in 2021 as
between educational institutions and companies.

CROSS-SECTOR COLLABORATIONS in AI PUBLICATIONS, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

29.84, Education and Nonpro t

Number of AI Publications (in thousands)

30

25

20

15
11.58, Company and Education

10
8.09, Education and Government

52.48, Government and Nonpro t
1.83, Company and Nonpro t
0.66, Company and Government

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.6

Chapter 1 Preview

11Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

A I J O U R N A L P U B L I CAT I O N S

since 2015 (Figure 1.1.7). As a percentage of all journal
publications, as captured in Figure 1.1.8, AI journal
publications in 2021 were about 2.5% of all publications,compared to 1.5% in 2010.

Overview
After growing only slightly from 2010 to 2015, thenumber of AI journal publications grew almost 2.5 times
NUMBER of AI JOURNAL PUBLICATIONS, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

Number of AI Journal Publications (in thousands)

172.11

150

100

50

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.7

AI JOURNAL PUBLICATIONS (% of TOTAL JOURNAL PUBLICATIONS), 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

2.53%

AI Journal Publications (% of Total Journal Publications)

2.50%

2.00%

1.50%

1.00%

0.50%

0.00%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.8

Chapter 1 Preview

12Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

In addition, South Asiaand the Middle East andNorth Africa saw the mostsignificant growth astheir number of AI journal
publications grew around12 and 7 times, respectively,in the last 12 years.

By Region4
Figure 1.1.9 shows the share of AI journal publications
by region between 2010 and 2021. In 2021, East Asia
and Pacific leads with 42.9%, followed by Europe andCentral Asia (22.7%) and North America (15.6%). In
addition, South Asia and the Middle East and North
Africa saw the most significant growth as their numberof AI journal publications grew around 12 and 7 times,
respectively, in the last 12 years.

AI JOURNAL PUBLICATIONS (% of WORLD TOTAL) by REGION, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Journal Publications (% of World Total)

50%
42.87%, East Asia and Paci c

40%

30%

22.70%, Europe and Central Asia

20%
15.63%, North America

10%7.99%, South Asia
6.18%, Middle East and North Africa
3.57%, Latin America and the Caribbean
1.06%, Sub-Saharan Africa

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.9

4 Regions in this chapter are classified according to the World Bank analytical grouping.

Chapter 1 Preview

13Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Geographic Area5

powers. China has remained the leader throughout, with
31.0% in 2021, followed by the European Union plus theUnited Kingdom at 19.1% and the United States at 13.7%

Figure 1.1.10 breaks down the share of AI journal
publications over the past 12 years by three major AI

AI JOURNAL PUBLICATIONS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Journal Publications (% of World Total)

40%

31.04%, China

30%

19.05%, European Union and United Kingdom

20%

13.67%, United States

10%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.10

5 Geographic areas in this chapter combine the number of publications between the European Union and the United Kingdom to reflect the historically strong association between them with regards toresearch collaboration.

Chapter 1 Preview

14Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

Citations

The three geographic areas
combined accounted formore than 66% of the total
citations in the world.

On the number of citations of AI journal
publications, China’s share has gradually increased
while those of the European Union plus the United
Kingdom and the United States have decreased.
The three geographic areas combined accounted formore than 66% of the total citations in the world.

AI JOURNAL CITATIONS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

30%

AI Journal Citations (% of World Total)

27.84%, China

25%

21.13%, European Union and United Kingdom

20%

17.45%, United States

15%

10%

5%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.11

Chapter 1 Preview

15Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

A I C O N F E R E N C E P U B L I CAT I O N S(Figure 1.1.12). Despite the decline in the total numbers,
however, the share of AI conference publications among
total conference publications in the world has increasedby more than five percentage points since 2010 (Figure
1.1.13).

Overview
The number of AI conference publications peaked in
2019, and fell about 19.4% below the peak in 2021
NUMBER of AI CONFERENCE PUBLICATIONS, 2010–21

Number of AI Conference Publications (in thousands)

Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

80
71.92

60

40

20

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.12

AI CONFERENCE PUBLICATIONS (% of TOTAL CONFERENCE PUBLICATIONS), 2010–21
AI Conference Publications (% of Total Conference Publications)

Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

17.83%

16%

12%

8%

4%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.13

Chapter 1 Preview

16Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Region

share represented by East Asia and Pacific continues torise since taking the lead in 2014, accounting for 40.4%
in 2021, followed by Europe and Central Asia (23.0%) andNorth America (19.0%). The percentage of AI conference
publications in South Asia saw a noticeable rise in thepast 12 years, from 4.0% in 2010 to 10.4% in 2021.

Figure 1.1.14 shows the number of AI conference
publications by region. Similar to the trend in AI journal
publication, East Asia and Pacific, Europe and Central
Asia, and North America account for the world’s highest
numbers of AI conference publications. Specifically, theAI CONFERENCE PUBLICATIONS (% of WORLD TOTAL) by REGION, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

40.39%, East Asia and Paci c

AI Conference Publications (% of World Total)

40%

30%

22.93%, Europe and Central Asia

20%

18.93%, North America

10%

10.44%, South Asia

0%3.82%, Middle East and North Africa
2.83%, Latin America and the Caribbean
0.66%, Sub-Saharan Africa

2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.14

Chapter 1 Preview

17Artificial IntelligenceIndex Report 2022

By Geographic Area
In 2021, China produced the greatest share of the world’s
AI conference publications at 27.6%, opening an evengreater lead than in 2020, while the European Union plus
the United Kingdom followed at 19.0% and the UnitedStates came in third at 16.9% (Figure 1.1.15).

AI CONFERENCE PUBLICATIONS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Conference Publications (% of World Total)

27.64%, China

25%

20%

18.95%, European Union and United Kingdom
16.90%, United States

15%

10%

5%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.15

Chapter 1 Preview

18Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

Citations
Despite China producing the most AI conference
publications in 2021, Figure 1.1.16 shows that the UnitedStates led among the major powers with respect tothe number of AI conference citations, with 29.5% in
2021, followed by the European Union plus the United
Kingdom (23.3%) and China (15.3%).

AI CONFERENCE CITATIONS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Conference Citations (% of World Total)

35%

30%

29.52%, United States25%
23.32%, European Union and United Kingdom

20%

15%

15.32%, China

10%

5%
0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.16

Chapter 1 Preview

19Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

A I R E P O S I TO R I E S

work outside traditional avenues for publications. Those
repositories allow researchers to share their findings before
submitting them to journals and conferences, which greatly
accelerates the cycle of information discovery. The numberof AI repository publications grew almost 30 times in thepast 12 years (Figure 1.1.17), now account for 15.3% of all
repository publications (Figure 1.1.18).

Overview
Publishing pre-peer-reviewed papers on repositories ofelectronic preprints (such as arXiv and SSRN) has become
a popular way among AI researchers to disseminate their
NUMBER of AI REPOSITORY PUBLICATIONS, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

56.73

Number of AI Repository Publications (in thousands)

50

20

10

5

2

1
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.17

AI REPOSITORY PUBLICATIONS (% of TOTAL REPOSITORY PUBLICATIONS), 2010–21
AI Repository Publications (% of Total Repository Publications)

Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

15.30%

15%

10%

5%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.18

Chapter 1 Preview

20Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Region

repository publications in the world since 2014 while thatof Europe and Central Asia has declined. Since 2013, theshare of East Asia and Pacific has grown significantly.

The analysis by region in Figure 1.1.19 shows that North
America has maintained a steady lead in the share of AI

AI REPOSITORY PUBLICATIONS (% of WORLD TOTAL) by REGION, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Repository Publications (% of World Total)

40%
35.89%, North America

30%
27.25%, Europe and Central Asia
26.38%, East Asia and Paci c

20%

10%
4.23%, South Asia
4.13%, Middle East and North Africa
1.67%, Latin America and the Caribbean
0.44%, Sub-Saharan Africa

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.19

Chapter 1 Preview

21Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Geographic Area
While the United States has held the lead in the percentage
of AI repository publications in the world since 2011, China
is catching up while the European Union plus the United
Kingdom’s share continues to drop (Figure 1.1.20). In 2021,

the United States accounted for 32.5% of the world’s AI
repository publications—a higher percentage comparedto journal and conference publications, followed by theEuropean Union plus the United Kingdom (23.9%) andChina (16.6%).

AI REPOSITORY PUBLICATIONS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21

AI Repository Publications (% of World Total)

Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

32.52%, United States30%

23.92%, European Union and United Kingdom

20%
16.60%, China

10%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.20

Chapter 1 Preview

22Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

Citations
On the citations of AI repository publications, Figure 1.1.21
shows that the United States tops the list with 38.6%

of overall citations in 2021, establishing a dominantlead over the European Union plus the United Kingdom(20.1%) and China (16.4%).

AI REPOSITORY CITATIONS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Repository Citations (% of World Total)

40%

38.60%, United States30%

20.13%, European Union and United Kingdom

20%

16.44%, China

10%

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.21

Chapter 1 Preview

23Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

A I PAT E N T S

The number of patents
filed in 2021 is morethan 30 times higher
than in 2015, showing
a compound annual
growth rate of 76.9%.

This section draws on data from CSET and 1790 Analytics
on patents relevant to AI development and applications—indicated by Cooperative Patent Classification (CPC)/
International Patent Classification (IPC) codes andkeywords. Patents were grouped by country and year andthen counted at the “patent family” level, before CSET
extracted year values from the most recent publication
date within a family.

Overview
Figure 1.1.22 captures the number of AI patents filed from2010 to 2021. The number of patents filed in 2021 is morethan 30 times higher than in 2015, showing a compound
annual growth rate of 76.9%.
NUMBER of AI PATENT FILINGS, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report141.24

Number of AI Patents (in thousands)

140

120

100

80

60

40

20
0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.22

Chapter 1 Preview

24Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Region and Application Status
Figure 1.1.23a breaks down AI patent filings by region.
The share of East Asia and Pacific took off in 2014 andled the rest of the world in 2021 with 62.1% of all patentapplications, followed by North America and Europe andCentral Asia. In terms of granted patents in those regions,North America leads with 57.0%, followed by East Asia
and Pacific (31.0%), and Europe and Central Asia (11.3%)(Figure 1.1.23b). The other regions combine to make up
roughly 1% of world patents (Figure 1.1.23c).

AI PATENT FILINGS (% of WORLD TOTAL) by REGION, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report70%

AI Patent Filings (% of World Total)

60%

62.14%, East Asia and Paci c

50%

40%

30%

20%

17.07%, North America

10%4.16%, Europe and Central Asia

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.23a

Chapter 1 Preview

25Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

GRANTED AI PATENTS (% of WORLD TOTAL) by REGION, 2010–21 (Part 1)
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

80%

Granted AI Patents (% of World Total)

70%

60%
56.96%, North America

50%

40%
31.09%, East Asia and Paci c

30%

20%
11.27%, Europe and Central Asia

10%
0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.23b

GRANTED AI PATENTS (% of WORLD TOTAL) by REGION, 2010–21 (Part 2)
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

Granted AI Patents (% of World Total)

1.50%

1.25%

1.00%

0.75%

0.50%
0.35%, Latin America and the Caribbean

0.25%

0.25%, South Asia
0.08%, Middle East and North Africa

0.00%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.23c

Chapter 1 Preview

26Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

By Geographic Area and Application Status
Trends revealed by the regional analysis can also be
observed in AI patent data broken down by geographic
area (Figure 1.1.24a and Figure 1.1.24b). China is now
filing over half of the world’s AI patents and being granted
about 6%, about the same as the European Union plus theUnited Kingdom. The United States, which files almost
all the patents in North America, does so at one-third therate of China. Figure 1.1.24c shows that compared to theincreasing numbers of AI patents applied and granted,
China has far greater numbers of patent applications(87,343 in 2021) than those granted (1,407 in 2021).

AI PATENT FILINGS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

AI Patent Filings (% of World Total)

60%

50%

51.69%, China

40%

30%

20%
16.92%, United States

10%
3.89%, European Union and United Kingdom

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.24a

GRANTED AI PATENTS (% of WORLD TOTAL) by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

Granted AI Patents (% of World Total)

70%

60%

50%

40%

39.59%, United States

30%

20%

10%

7.56%, European Union and United Kingdom
5.90%, China

0%
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.1.24b

Chapter 1 Preview

27Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.1 Publications

AI PATENTS by APPLICATION STATUS by GEOGRAPHIC AREA, 2010–21
Source: Center for Security and Emerging Technology, 2021 | Chart: 2022 AI Index Report

China
90
Application
Grant

Number of AI Patent Filings (in thousands)

80

European Union and United Kingdom

United States

87.34

70
60
50
40
30
19.61

20

9.45

10

4.88
1.81

1.41

0
2011

2016

2021

2011

2016

2021

2011

2016

2021
Figure 1.1.24c

Chapter 1 Preview

28Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.2 Conferences

AI conferences are key venues for researchers to publish and communicate their work, as well as to connect with peers andcollaborators. Conference attendance is an indication of broader industrial and academic interest in a scientific field. In the past
20 years, AI conferences have grown not only in size but also in number and prestige. This section presents data on the trends inattendance at major AI conferences, covering more conferences (16) than previous Index reports.

1.2 CONFERENCESC O N F E R E N C E AT T E N DA N C E

Figure 1.2.1 shows that attendance at top AI conferencesin 2021 was relatively consistent with 2020, with morethan 88,000 participants worldwide. Figure 1.2.2 andFigure 1.2.3 show the attendance data for individual
conferences, with 16 major AI conferences split into
two categories: large AI conferences with over 2,500
attendees and small AI conferences with fewer than
2,500 attendees.6

Similar to 2020, most AI conferences were offeredvirtually in 2021. Only the International Conference on
Robotics and Automation (ICRA) and the Conferenceon Empirical Methods in Natural Language Processing(EMNLP) were held using a hybrid format. Conference
organizers reported measuring the exact attendance
numbers at a virtual conference is difficult, as virtual
conferences allow for higher attendance of researchers
from all around the world.

NUMBER of ATTENDEES at SELECT AI CONFERENCES, 2010–21
Source: Conference Data, 2021 | Chart: 2022 AI Index Report

90

88.76

Number of Attendees (in thousands)

80
70
60
50
40
30
20
10
0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.2.1

6 The International Conference on Machine Learning (ICML) used the number of session visitors as a proxy for the number of conference attendees, which explains the high attendance count in 2021.
The International Conference on Intelligent Robots and Systems (IROS) extended the virtual conference to allow users to watch events for up to three months, which explains the high attendance countin 2020. For the AAMAS conference, the attendance in 2020 is based on the number of users on site reported by the platform that recorded the talks and managed the online conference, while the 2021
number is for total registrants.

ChapterTable of Contents
1 Preview

Chapter 1 Preview

29Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.2 Conferences

ATTENDANCE at LARGE AI CONFERENCES, 2010–21
Source: Conference Data, 2021 | Chart: 2022 AI Index Report30

29.54, ICML

Number of Attendees (in thousands)

25

20
17.09, NeurIPS

15

10
8.24, CVPR
6.31, ICLR
5.01, ICCV
4.09, AAAI
3.76, EMNLP
2.59, IROS

5

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.2.2

ATTENDANCE at SMALL AI CONFERENCES, 2010–21

Number of Attendees (in thousands)

Source: Conference Data, 2021 | Chart: 2022 AI Index Report3

2.10, UAI

2

1.90, IJCAI
1.35, FaccT
1.08, AAMAS

1

1.00, ICRA
0.67, KR0.51, ICAPS

0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.2.3

Chapter 1 Preview

30Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.2 Conferences

WO M E N I N M AC H I N E L E A R N I N G( W I M L ) N E U R I P S WO R KS H O PWorkshop Participants
The number of participants attending the WiML workshop
has steadily increased since it was first introduced in2006. For the 2021 edition, Figure 1.2.4 shows an estimateof 1,486 attendees over all workshop sessions, counted
as the number of unique individuals who accessed thevirtual workshop platform at neurips.cc. The 2021 WiML
Workshop at NeurIPS happened as multiple sessions
over three days, which was a change in format from 2020.
As in 2020, the workshop was held virtually due to thepandemic.

Founded in 2006, Women in Machine Learning is anorganization dedicated to supporting and increasing
the impact of women in machine learning. This section
presents data from its annual technical workshop
colocated with NeurIPS. Starting in 2020, WiML has also
been hosting the Un-Workshop, which aims to advanceresearch via collaboration and interaction among
participants from diverse backgrounds at ICML.

ATTENDANCE at NEURIPS WOMEN in MACHINE LEARNING WORKSHOP, 2010–21
Source: Women in Machine Learning, 2021 | Chart: 2022 AI Index Report

1,486

1400

Number of Attendees

1200

1000

800

600

400

200
0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.2.4

Chapter 1 Preview

31Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.2 Conferences

Demographics Breakdown

by Europe (19.9%), Asia (16.2%), and Africa (7.3%) (Figure
1.2.5). Figure 1.2.6 shows that Ph.D. students madeup almost half of the survey participants, while theshare of university faculty is around 1.2%. Researcher
scientists/engineers, data scientists/engineers, andsoftware engineers were among the most commonly held
professional positions.

This section shows the continent of residence andprofessional position breakdowns of the 2021 workshop
participants based on a survey filled by participants
who consented to have such information aggregated.
Among the survey respondents, more than half of thesurvey respondents were from North America, followed

CONTINENT of RESIDENCE of PARTICIPANTS at NEURIPS WOMEN in MACHINE LEARNING WORKSHOP, 2021
Source: Women in Machine Learning, 2021 | Chart: 2022 AI Index Report

North
America

53.40%

19.90%

Europe16.20%

Asia

7.30%

AfricaSouth
America

1.00%

Australia

1.00%

0%

10%

20%

30%

40%

50%

% of Respondents

Figure 1.2.5

PROFESSIONAL POSITIONS of PARTICIPANTS at NEURIPS WOMEN in MACHINE LEARNING WORKSHOP, 2021
Source: Women in Machine Learning, 2021 | Chart: 2022 AI Index Report

49.40%PhD Student
16.50%MSc Student

14.20%

Research Scientist / Engineer
10.20%

Undergraduate Student
7.40%

Data Scientist / Engineer

5.70%

Postdoctoral Researcher
2.30%

Software Engineer
High School Student

0.60%

Professor (Post-Tenure)

0.60%

Professor (Pre-Tenure)

0.60%

0%

10%

20%

30%
% of Respondents

Chapter 1 Preview

40%

50%
Figure 1.2.6

32

Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.3 AI Open-Source Software Libraries

A software library is a collection of computer code that is used to create applications and products. Popular AI-specific software
libraries—such as TensorFlow and PyTorch—help developers create their AI solutions quickly and efficiently. This section analyzes the
popularity of software libraries through GitHub data.

1.3 AI OPEN-SOURCESOFTWARE LIBRARIESG I T H U B S TA R S
Figures 1.3.1 and 1.3.2 reflect the number of users of
GitHub open-source AI software libraries from 2015 to
2021. TensorFlow remained by far the most popular in
2021, with around 161,000 cumulative GitHub stars—aslight increase over 2020. TensorFlow was about threetimes as popular in 2021 as the next-most-starred

GitHub open-source AI software library, OpenCV, which
was followed by Keras, PyTorch, and Scikit-learn.
Figure 1.3.2 shows library popularity for libraries withfewer than 40,000 GitHub stars—led by FaceSwap witharound 40,000 stars, followed by 100-Days-Of-ML-Code,
AiLearning, and BVLC/caffe.

NUMBER of GITHUB STARS by AI LIBRARY (OVER 40K STARS), 2014–21
Source: GitHub, 2021 | Chart: 2022 AI Index Report

160.7, TensorFlow

Number of Cumulative GitHub Stars (in thousands)

160
140
120
100
80

58.6, OpenCV
53.2, Keras
52.7, PyTorch
48.0, Scikit-learn
46.7, DeepLearning-500-questions
41.5, TenserFlow-Examples

60
40
20
0
2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.3.1

ChapterTable of Contents
1 Preview

Chapter 1 Preview

33Artificial IntelligenceIndex Report 2022

CHAPTER 1: RESEARCH & DEVELOPMENT
1.3 AI Open-Source Software Libraries

NUMBER of GITHUB STARS by AI LIBRARY (UNDER 40K STARS), 2014–21
Source: GitHub, 2021 | Chart: 2022 AI Index Report

39.88, faceswap

Number of Cumulative GitHub Stars (in thousands)

40

33.58, 100-Days-Of-ML-Code32.27, AiLearning
32.14, BVLC/ca e
32.02, Real-Time-Voice-Cloning
32.00, deeplearningbook-chinese
31.28, Deep Learning Papers Reading Roadmap
30.26, DeepFaceLab

35
30
25
20
15
10
5
0
2014

2015

2016

2017

2018

2019

2020

2021
Figure 1.3.2

ChapterTable of Contents
1 Preview

Chapter 1 Preview

34Artificial IntelligenceIndex Report 2022

APPENDIX
Chapter 1: Research & Development

APPENDIXCENTER FOR SECURITY AND
E M E R G I N G T E C H N O LO GY,
G E O R G E TOW N U N I V E R S I T Y
Prepared by Sara Abdulla and James Dunham
The Center for Security and Emerging Technology (CSET)
is a policy research organization within Georgetown
University’s Walsh School of Foreign Service that produces
data-driven research at the intersection of security andtechnology, providing nonpartisan analysis to the policy
community.

Publications from CSET Merged Corpus of
Scholarly Literature
SourceCSET’s merged corpus of scholarly literature combines
distinct publications from Digital Science’s Dimensions,
Clarivate’s Web of Science, Microsoft Academic Graph,
China National Knowledge Infrastructure, arXiv, andPapers With Code.1
Methodology
To create the merged corpus, CSET deduplicated acrossthe listed sources using publication metadata, and then
combined the metadata for linked publications. To
identify AI publications, CSET used an English-language
subset of this corpus: publications since 2010 that appear
AI-relevant.2 CSET researchers developed a classifier
for identifying AI-related publications by leveraging thearXiv repository, where authors and editors tag papers by
subject.
To provide a publication’s field of study, CSET matches
each publication in the analytic corpus with predictions
from Microsoft Academic Graph (MAG)’s field-of-study
model, which yields hierarchical labels describing the

published research field(s) of study and correspondingscores.3 CSET researchers identified the most common
fields of study in their corpus of AI-relevant publications
since 2010 and recorded publications in all other fields as“Other AI.” English-language AI-relevant publications werethen tallied by their top-scoring field and publication year.
CSET also provided year-by-year citations for AI-relevant
work associated with each country. A publication isassociated with a country if it has at least one author whoseorganizational affiliation(s) is located in that country.
Citation counts aren’t available for all publications; those
without counts weren’t included in the citation analysis.
Over 70% of English-language AI papers published between
2010 and 2020 have citation data available.
CSET counted cross-country collaborations as distinct
pairs of countries across authors for each publication.
Collaborations are only counted once: For example, if apublication has two authors from the United States andtwo authors from China, it is counted as a single UnitedStates-China collaboration.
Additionally, publication counts by year and by
publication type (e.g., academic journal articles,
conference papers) were provided where available.
These publication types were disaggregated by affiliation
country as described above.
CSET also provided publication affiliation sector(s)where, as in the country attribution analysis, sectors wereassociated with publications through authors’ affiliations.
Not all affiliations were characterized in terms of sectors;
CSET researchers relied primarily on GRID from Digital
Science for this purpose, and not all organizations canbe found in or linked to GRID.4 Where the affiliation

1 All CNKI content is furnished for CSET by East View Information Services, Minneapolis, MN, USA.
2 For more information, see James Dunham, Jennifer Melot, and Dewey Murdick, “Identifying the Development and Application of Artificial Intelligence in Scientific Text,” arXiv [cs.DL], May 28, 2020,
https://arxiv.org/abs/2002.07143.
3 These scores are based on cosine similarities between field-of-study and paper embeddings. See Zhihong Shen, Hao Ma, and Kuansan Wang, “A Web-Scale System for Scientific Knowledge
Exploration,” arXiv [cs.CL], May 30, 2018, https://arxiv.org/abs/1805.12216.
4 See https://www.grid.ac/ for more information about the GRID dataset from Digital Science.

Chapter 1 Preview

35Artificial IntelligenceIndex Report 2022

APPENDIX
Chapter 1: Research & Development

sector is available, papers were counted toward these
sectors, by year. Cross-sector collaborations on academic
publications were calculated using the same method as in
the cross-country collaborations analysis.

iperov/DeepFaceLab, Microsoft/CNTK, opencv/opencv,
pytorch/pytorch, scikit-learn/scikit-learn, scutan90/
DeepLearning-500-questions, tensorflow/tensorflow,
Theano/Theano, Torch/Torch7.

Patents from CSET’s AI Patents Dataset

NuanceThe GitHub Archive currently does not provide a way
to count when users remove a star from a repository.
Therefore, the reported data slightly overestimates thenumber of stars. A comparison with the actual numberof stars for the repositories on GitHub reveals that the
numbers are fairly close and that the trends remainunchanged.

SourceCSET’s AI patents dataset was developed by CSET
and 1790 Analytics. It includes patents relevant to thedevelopment and application of AI, as indicated by CPC/
IPC codes and keywords.
Methodology
In this analysis, patents were grouped by year andcountry, and then counted at the “patent family”
level.5 CSET extracted year values from the most recent
publication date within a family. This method has theadvantage of capturing updates within a patent family(such as amendments).
The country of origin for a patent is derived from the first
country in which a patent was filed.6

G I T H U B S TA R S
Source
GitHub: star-history (available at star history website) wasused to retrieve the data.
Methodology
The visual in the report shows the number of stars forvarious GitHub repositories over time. The repositories
include the following:
apachecn/ailearning, apache/incubator-mxnet, AvikJain/100-Days-Of-ML-Code, aymericdamien/TensorFlowExamples, BVLC/cafe, cafe2/cafe2, CorentinJ/Real-TimeVoice-Cloning, deepfakes / faceswap, dmlc/mxnet,
exacity/deeplearningbook-chinese, fchollet/keras,
floodsung/Deep-Learning-Papers-Reading-Roadmap,

5 Patents are analyzed at the “patent family” level rather than “patent document” level because patent families are a collective of patent documents all associated with a single invention and/or
innovation by the same inventors/assignees. Thus, counting at the “patent family” level mitigates artificial number inflation when there are multiple patent documents in a patent family or if a patent isfiled in multiple jurisdictions.
6 For more details on CSET’s approach and experimentation for assigning country values, see footnote 26 in “Patents and Artificial Intelligence: A Primer,” by Dewey Murdick and Patrick Thomas. (Center
for Security and Emerging Technology, September 2020), https://doi.org/10.51593/20200038.

Chapter 1 Preview

36