UCF101: A Dataset of 101 Human Actions
Classes From Videos in The Wild
Khurram Soomro, Amir Roshan Zamir and Mubarak Shah
CRCV-TR-12-01
November 2012

Keywords: Action Dataset, UCF101, UCF50, Action Recognition

Center for Research in Computer Vision
University of Central Florida
4000 Central Florida Blvd.
Orlando, FL 32816-2365 USA

UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
Khurram Soomro, Amir Roshan Zamir and Mubarak ShahCenter for Research in Computer Vision, Orlando, FL 32816, USA{ksoomro, aroshan, shah}@cs.ucf.edu
http://crcv.ucf.edu/data/UCF101.php

Abstract
We introduce UCF101 which is currently the largest
dataset of human actions. It consists of 101 action classes,
over 13k clips and 27 hours of video data. The databaseconsists of realistic user-uploaded videos containing camera motion and cluttered background. Additionally, we provide baseline action recognition results on this new dataset
using standard bag of words approach with overall performance of 43.9%. To the best of our knowledge, UCF101
is currently the most challenging dataset of actions due toits large number of classes, large number of clips and also
unconstrained nature of such clips.

Apply Eye Makeup

Playing Dhol

Baby Crawling

Haircut

Sky Diving

SurfingRafting

Cricket Shot

Shaving Beard

1. Introduction
Figure 1. Sample frames for 6 action classes of UCF101.

The majority of existing action recognition datasets suffer from two disadvantages: 1) The number of their classes
is typically very low compared to the richness of performed
actions by humans in reality, e.g. KTH [11], Weizmann [3],UCF Sports [10], IXMAS [12] datasets includes only 6, 9,9, 11 classes respectively. 2) The videos are recorded in unrealistically controlled environments. For instance, KTH,Weizmann, IXMAS are staged by actors; HOHA [7] andUCF Sports are composed of movie clips captured by professional filming crew. Recently, web videos have beenused in order to utilize unconstrained user-uploaded data to
alleviate the second issue [6, 8, 9, 5]. However, the first disadvantage remains unresolved as the largest existing dataset
does not include more than 51 actions while several works
showed that the number of classes play a crucial role in evaluating an action recognition method [4, 9]. Therefore, wehave compiled a new dataset with 101 actions and 13320
clips which is nearly twice bigger than the largest existingdataset in terms of number of actions and clips. (HMDB51[5] and UCF50 [9] are the currently the largest ones with6766 clips of 51 actions and 6681 clips of 50 actions respectively.)
The dataset is composed of web videos which arerecorded in unconstrained environments and typically in-clude camera motion, various lighting conditions, partial
occlusion, low quality frames, etc. Fig. 1 shows sample
frames of 6 action classes from UCF101.

2. Dataset Details
Action Classes: UCF101 includes total number of101 action classes which we have divided into five types:
Human-Object Interaction, Body-Motion Only, HumanHuman Interaction, Playing Musical Instruments, Sports.
UCF101 is an extension of UCF50 which included thefollowing 50 action classes: {Baseball Pitch, Basketball
Shooting, Bench Press, Biking, Billiards Shot, Breaststroke,
Clean and Jerk, Diving, Drumming, Fencing, Golf Swing,High Jump, Horse Race, Horse Riding, Hula Hoop, Javelin
Throw,, Juggling Balls, Jumping Jack, Jump Rope, Kayaking, Lunges, Military Parade, Mixing Batter, Nun chucks,
Pizza Tossing, Playing Guitar, Playing Piano, PlayingTabla, Playing Violin, Pole Vault, Pommel Horse, Pull Ups,Punch, Push Ups, Rock Climbing Indoor, Rope Climbing,Rowing, Salsa Spins, Skate Boarding, Skiing, Skijet, Soccer Juggling, Swing, TaiChi, Tennis Swing, Throw Discus,
1

Apply Lipstick Blow Dry Hair Brushing Teeth Cutting In Kitchen

Hammering

Knitting

Mixing BatterShaving Beard Skate Boarding Soccer Juggling

Writing On Board

Yo YoBaby Crawling Blowing Candles Body Weight Squats Handstand Pushups Handstand Walking

Pull ups

Push ups

Rock Climbing Indoor

Band Marching

Haircut

Playing Flute

Apply Eye Makeup

Mopping Floor

Nun Chucks

Pizza Tossing

Hula Hoop

Juggling Balls

Jumping Jack

Jump Rope

TypingLungesSwing

Tai ChiHead Massage Military Parade

Salsa Spin

Drumming

Playing CelloPlaying Daf

Playing Dhol

Playing Guitar

Playing Piano

Playing Sitar

Playing Tabla

Playing Violin

Archery

Balance Beam

Baseball Pitch

Basketball

Basketball Dunk

Bench Press

BikingBilliardBowling

Boxing-Punching Bag Boxing-Speed BagBreaststroke

Clean and Jerk

Cliff Diving

Cricket Bowling

Cricket ShotDivingFencing

Field Hockey Penalty Floor Gymnastics

Frisbee Catch

Front Crawl

Golf Swing

Hammer Throw

High Jump

Horse Race

Horse Riding

Ice Dancing

Javelin Throw

KayakingLong Jump

Parallel Bars

Pole Vault

Pommel HorsePunch

Rafting

RowingShotput

SkiingJetskiSky Diving

Soccer Penalty

Still Rings

Sumo Wrestling

Surfing

Table Tennis Shot

Tennis Swing

Throw Discus

Uneven Bars

Volleyball Spiking

Rope Climbing

Trampoline Jumping Walking with a Dog

Wall Pushups

Figure 2. 101 actions included in UCF101 shown with one sample frame. The color of frame borders specifies to which action type theybelong: Human-Object Interaction, Body-Motion Only, Human-Human Interaction, Playing Musical Instruments, Sports.

Trampoline Jumping, Volleyball Spiking, Walking with adog, Yo Yo}. The color class labels specify which predefined action type they belong to.
The following 51 new classes are introduced in UCF101:{Apply Eye Makeup, Apply Lipstick, Archery, Baby Crawling, Balance Beam, Band Marching, Basketball Dunk, Blow
Drying Hair, Blowing Candles, Body Weight Squats, Bowl-ing,Boxing-Punching Bag, Boxing-Speed Bag, Brushing
Teeth, Cliff Diving, Cricket Bowling, Cricket Shot, Cutting In Kitchen, Field Hockey Penalty, Floor Gymnastics,Frisbee Catch, Front Crawl, Hair cut, Hammering, Hammer Throw, Handstand Pushups, Handstand Walking, Head
Massage, Ice Dancing, Knitting, Long Jump, Mopping
Floor, Parallel Bars, Playing Cello, Playing Daf, Playing

180
Number of Clips

1501209060
30
ApplyEyeMakeup
ApplyLipstickArchery
BabyCrawling
BalanceBeam
BandMarching
BaseballPitch
Basketball
BasketballDunk
BenchPress
BikingBilliards
BlowDryHairBlowingCandles
BodyWeightSquats
Bowling
BoxingPunchingBag
BoxingSpeedBag
BreastStroke
BrushingTeeth
CleanAndJerk
CliffDiving
CricketBowling
CricketShot
CuttingInKitchen
DivingDrumming
Fencing
FieldHockeyPenalty
FloorGymnastics
FrisbeeCatch
FrontCrawl
GolfSwing
Haircut
Hammering
HammerThrow
HandstandPushups
HandstandWalking
HeadMassage
HighJump
HorseRace
HorseRiding
HulaHoop
IceDancing
JavelinThrow
JugglingBalls
JumpingJack
JumpRope
Kayaking
Knitting

0180> 10.0 Sec

5.0 - 10.0 Sec

2.0 - 5.0 Sec

0.0 - 2.0 Sec

Number of Clips

15012090600LongJump
LungesMilitaryParade
MixingMoppingFloor
Nunchucks
ParallelBars
PizzaTossing
PlayingCelloPlayingDaf
PlayingDhol
PlayingFlutePlayingGuitar
PlayingPiano
PlayingSitar
PlayingTabla
PlayingViolin
PoleVault
PommelHorse
PullUps
Punch
PushUps
Rafting
RockClimbi…
RopeClimbing
Rowing
SalsaSpin
ShavingBeard
Shotput
SkateBoarding
Skiing
SkijetSkyDiving
SoccerJuggling
SoccerPenalty
StillRings
SumoWrestling
Surfing
Swing
TableTennis…
TaiChi
TennisSwing
ThrowDiscus
TrampolineJ…
TypingUnevenBars
VolleyballSp…
WalkingWit…
WallPushups
WritingOnB…
YoYo30Figure 3. Number of clips per action class. The distribution of clip durations is illustrated by the colors.

Dhol, Playing Flute, Playing Sitar, Rafting, Shaving Beard,Shot put, Sky Diving, Soccer Penalty, Still Rings, SumoWrestling, Surfing, Table Tennis Shot, Typing, Uneven Bars,
Wall Pushups, Writing On Board}. Fig. 2 shows a sample
frame for each action class of UCF101.
Clip Groups: The clips of one action class are divided
into 25 groups which contain 4-7 clips each. The clips inone group share some common features, such as the background or actors.
The bar chart of Fig. 3 shows the number of clips ineach class. The colors on each bar illustrate the durationsof different clips included in that class. The chart shown inFig. 4 illustrates the average clip length (green) and total
duration of clips (blue) for each action class.
The videos are downloaded from YouTube [2] and theirrelevant ones are manually removed. All clips have fixed
frame rate and resolution of 25 FPS and 320 × 240 respectively. The videos are saved in .avi files compressed using DivX codec available in k-lite package [1]. The audio
is preserved for the clips of the new 51 actions. Table 1
summarizes the characteristics of the dataset.

Actions
Clips
Groups per Action
Clips per Group
Mean Clip Length
Total Duration
Min Clip Length
Max Clip Length
Frame Rate
Resolution
Audio

101
13320
25
4-7
7.21 sec
1600 mins
1.06 sec
71.04 sec
25 fps320×240
Yes (51 actions)

Table 1. Summary of Characteristics of UCF101

Naming Convention: The zipped file of the dataset(availableat
http://crcv.ucf.edu/data/
UCF101.php ) includes 101 folders each containing
the clips of one action class. The name of each clip has thefollowing form:v X gY cZ.avi

2500

Time (sec)2000

16Total Time
Average Clip Duration

1412101500

81000

64500

20

ApplyEyeMakeup
ApplyLipstickArchery
BabyCrawling
BalanceBeam
BandMarching
BaseballPitch
Basketball
BasketballDunk
BenchPress
BikingBilliards
BlowDryHair
BlowingCandles
BodyWeightSquats
Bowling
BoxingPunchingBag
BoxingSpeedBag
BreastStroke
BrushingTeeth
CleanAndJerk
CliffDiving
CricketBowling
CricketShot
CuttingInKitchen
DivingDrumming
Fencing
FieldHockeyPenalty
FloorGymnastics
FrisbeeCatch
FrontCrawl
GolfSwing
Haircut
Hammering
HammerThrowHandstandPushups
HandstandWalking
HeadMassage
HighJump
HorseRace
HorseRiding
HulaHoop
IceDancing
JavelinThrow
JugglingBalls
JumpingJack
JumpRope
Kayaking
Knitting

02500

Time (sec)

2000

16

Total Time
Average Clip Duration

1412101500

81000

64500

20

LongJump
LungesMilitaryParade
MixingMoppingFloor
Nunchucks
ParallelBars
PizzaTossing
PlayingCelloPlayingDaf
PlayingDhol
PlayingFlutePlayingGuitar
PlayingPiano
PlayingSitar
PlayingTabla
PlayingViolin
PoleVault
PommelHorse
PullUps
Punch
PushUps
Rafting
RockClimbingIndoor
RopeClimbing
Rowing
SalsaSpin
ShavingBeard
Shotput
SkateBoardingSkiing
SkijetSkyDiving
SoccerJuggling
SoccerPenalty
StillRings
SumoWrestling
Surfing
Swing
TableTennisShot
TaiChi
TennisSwing
ThrowDiscus
TrampolineJumping
TypingUnevenBars
VolleyballSpiking
WalkingWithDog
WallPushups
WritingOnBoard
YoYo0

Figure 4. Total time of videos for each class is illustrated using the blue bars. The average length of the clips for each action is depicted ingreen.

where X, Y and Z represent action class label,group and clip number respectively.
For instance,v ApplyEyeMakeup g03 c04.avi corresponds tothe clip 4 of group 3 of action class ApplyEyeMakeup.

3. Experimental Results
We performed an experiment using bag of words approach which is widely accepted as a standard action recognition method to provide baseline results on UCF101.
From each clip, we extracted Harris3D corners (usingthe implementation by [7]) and computed 162 dimensional
HOG/HOF descriptors for each. We clustered a randomly
selected set of 100,000 space-time interest points (STIP) using k-means to build the codebook. The size of our codebook is k=4000 which is shown to yield good results overa wide range of datasets. The descriptors were assigned
to their closest video words using nearest neighbor classifier, and each clip was represented by a 4000-dimensional
histogram of its words. Utilizing three train/test splits, aSVM was trained using the histogram vectors of the train-ing set. We employed a nonlinear multiclass SVM with histogram intersection kernel and 101 classes each representing one action. For testing, a similar histogram representation for the query video was computed and classified using
the trained SVM. This method yielded an overall accuracyof 43.9%; The confusion matrix for all 101 actions is shownin Fig. 5.
The accuracy for the predefined action types are:Sports (49.40%), Playing Musical Instrument (42.04%),
Human-Object Interaction (36.62%), Body-Motion Only(37.64%), Human-Human Interaction (42.66%). Sports actions achieve the highest accuracy since performing sports
typically requires distinctive motions which makes the classification easier. Moreover, the background in sports clips
are generally less cluttered compared to other action types.
Unlike Sports Actions, Human-Object Interaction clips typically have a highly cluttered background. Additionally, theinformative motions typically occupy a small portion of themotions in the clips which explains the low recognition accuracy of this action class.

Dataset
KTH [11]
Weizmann [3]UCF Sports [10]IXMAS [12]UCF11 [6]HOHA [7]Olympic [8]UCF50 [9]HMDB51 [5]UCF101

Number of Actions
699111112
1650
51
101

Clips
600
81182
165
1168
2517
800
6681
6766
13320

Background
StaticStaticDynamic
StaticDynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic

Camera Motion
SlightNo
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes

Release Year
200420052009200620092009
2010
2010
2011
2012

Resource
Actor Staged
Actor Staged
TV, Movies
Actor Staged
YouTubeMovies
YouTubeYouTube
Movies, YouTube, WebYouTube

Table 2. Summary of Major Action Recognition Datasets

We recommend a three train/test split (availableat:
http://crcv.ucf.edu/data/UCF101/
UCF101TrainTestSplits-RecognitionTask.
zip ) experimental setup to keep consistency of thereported tests on UCF101; the baseline results provided inthis section were computed using the same scenario. These
train/test splits have been designed in a way to keep thegroups separate, hence not sharing the clips from the samegroup in training and testing, as the clips within a groupare obtained from a single long video. Each test split has 7
different groups and their respective remaining 18 groups
are used for training.
The above experiment was also performed using a leaveone-group-out 25-fold cross validation setup, giving an
overall accuracy of 44.5%. By testing on one group andtraining on the rest, it was made sure that the clips from agroup are not divided between training and testing set.

4. Related Datasets
UCF Sports, UCF11, UCF50 and UCF101 are the fouraction datasets compiled by UCF in chronological order;
each one includes its precursor. We made two minor modifications in the portion of UCF101 which includes UCF50
videos: the number of groups is fixed to 25 for all the actions, and each group includes up to 7 clips. Table 2 showsa list of existing action recognition datasets with detailed
characteristics of each. Note that UCF101 is remarkably
larger than the rest.

5. Conclusion
We introduced UCF101 which is the most challenging dataset for action recognition compared to the existing ones. It includes 101 action classes and over 13k clips
which makes it outstandingly larger than other datasets.
UCF101 is composed of unconstrained videos downloaded
from YouTube which feature challenges such as poor lighting, cluttered background and severe camera motion. We
provided baseline action recognition results on this newdataset using standard bag of words method with overall
accuracy of 43.9%.

References[1] K-lite codec package. http://codecguide.com/. 3[2] Youtube. http://www.youtube.com/. 3[3] M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri.
Actions as space-time shapes, 2005. International Conference on Computer Vision (ICCV). 1, 5
[4] G. Johansson, S. Bergstrom, and W. Epstein. Perceiving
events and objects, 1994. Lawrence Erlbaum Associates. 1[5] H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre.
Hmdb: A large video database for human motion recognition, 2011. International Conference on Computer Vision(ICCV). 1, 5
[6] J. Liu, J. Luo, and M. Shah. Recognizing realistic actions
from videos in the wild, 2009. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1, 5
[7] M. Marszaek, I. Laptev, and C. Schmid. Actions in context,
2009. IEEE Conference on Computer Vision and Pattern
Recognition (CVPR). 1, 4, 5
[8] J. Niebles, C. Chen, and L. Fei-Fei. Modeling temporal
structure of decomposable motion segments for activity classication, 2010. European Conference on Computer Vision(ECCV). 1, 5
[9] K. Reddy and M. Shah. Recognizing 50 human action categories of web videos, 2012. Machine Vision and Applications Journal (MVAP). 1, 5
[10] M. Rodriguez, J. Ahmed, and M. Shah. Action mach: A
spatiotemporal maximum average correlation height lter foraction recognition, 2008. IEEE Conference on Computer
Vision and Pattern Recognition (CVPR). 1, 5
[11] C. Schuldt, I. Laptev, and B. Caputo. Recognizing human actions: A local svm approach, 2004. International Conference
on Pattern Recognition (ICPR). 1, 5
[12] D. Weinland, E. Boyer, and R. Ronfard. Action recognition
from arbitrary views using 3d exemplars, 2007. InternationalConference on Computer Vision (ICCV). 1, 5

Head Massage

Salsa Spins

Band Marching

Walking with a Dog

Tai ChiRope Climbing

Push Ups

LungesHandstand Walking

Body Weight Squats

Baby Crawling

Writing On BoardSoccer Juggling

Shaving Beard

Nun chucks

Mixing Batter

Jump Rope

Cutting In Kitchen

Hula Hoop

Blow Dry Hair

Apply Eye Makeup

Playing FlutePlaying Daf

Playing Violin

Drumming

Playing Piano

Uneven Bars

Tennis Swing

Surfing

Still Rings

SkyDiving

SkiingPunch

RowingPole Vault

Long Jump

Horse Riding

Javelin Throw

High Jump

Golf Swing

Frisbee CatchField Hockey Penalty

DivingCricket Bowling

Clean and Jerk

Bowling

Boxing Speed Bag

BikingBaseball Pitch

Basketball Dunk

ArcheryArchery

Balance Beam

Baseball Pitch

Basketball Shooting

Basketball Dunk

Bench Press

BikingBilliards Shot

Bowling

Boxing Punching Bag

Boxing Speed Bag

0.9

Breaststroke

Clean and Jerk

Cliff Diving

Cricket Bowling

Cricket Shot

DivingFencing

Field Hockey Penalty

Floor Gymnastics

Frisbee Catch

0.8

Front Crawl

Golf Swing

Hammer Throw

High Jump

Horse Race

Javelin Throw

Ice Dancing

Horse Riding

KayakingLong Jump

0.7

Parallel Bars

Pole Vault

Pommel HorsePunch

Rafting

RowingShotput

Skiing

SkijetSky Diving

Soccer Penalty

Still Rings

0.6

SumoWrestling

Surfing

Table Tennis Shot

Tennis Swing

Throw Discus

Uneven Bars

Volleyball Spiking

Drumming

Playing Guitar

Playing Piano

0.5

Playing Tabla

Playing Violin

Playing CelloPlaying Daf

Playing Dhol

Playing FlutePlaying Sitar

Apply Eye Makeup

Apply Lipstick

Blow Dry Hair

0.4

Brushing Teeth

Cutting In Kitchen

Hammering

Hula Hoop

Juggling Balls

Jump RopeKnitting

Mixing Batter

Mopping Floor

Nun chucks

0.3

Pizza Tossing

Shaving Beard

Skate Boarding

Soccer Juggling

TypingWriting On BoardYo YoBaby Crawling

Blowing Candles

BodyWeight Squats

0.2

Handstand Pushups

Handstand Walking

Jumping JackLunges

Pull Ups

Push Ups

Rock Climbing Indoor

Rope Climbing

Swing

Tai Chi0.1Trampoline Jumping

Walking with a Dog

Wall Pushups

Band Marching

Haircut

Head Massage

Military Parade

Salsa Spins
Military Parade

Haircut

Wall Pushups

Trampoline Jumping

Swing

Rock Climbing Indoor

Pull Ups

Jumping JackHandstand Pushups

Blowing Candles

Yo YoTypingSkate Boarding

Pizza Tossing

Knitting

Mopping Floor

Juggling Balls'Hammering

Brushing Teeth

Apply Lipstick

Playing Sitar

Playing Dhol

Playing CelloPlaying Tabla

Playing Guitar

Volleyball Spiking

Throw Discus

Table Tennis Shot

Sumo Wrestling

SkijetSoccer Penalty

Shotput

Rafting

Pommel Horse

Parallel Bars

Kayaking

Ice DancingHorse Race

Hammer Throw

Front Crawl

Floor Gymnastics

Fencing

Cliff Diving

Cricket Shot

Breaststroke

Billiards Shot

Boxing Punching Bag

Bench Press

Basketball Shooting

Balance Beam0Figure 5. Confusion table of baseline action recognition results using bag of words approach on UCF101. The drawn lines separate different
types of actions; 1-50: Sports, 51-60: Playing Musical Instrument, 61-80: Human-Object Interaction, 81-96: Body-Motion Only, 97-101:
Human-Human Interaction.