Papers On Object Detection
Gong Qiqi

Paper List
• 1. Training Region-based Object Detectors with Online Hard Example Mining• 2. R-FCN: Object Detection via Region-based Fully Convolutional Networks• 3. Feature pyramid networks for object detection (FPN)• 4. Cascade R-CNN: Delving into High Quality Object Detection• 5. You Only Look Once: Unified, Real-Time Object Detection (YOLO)• 6. SSD: Single Shot MultiBox Detector• 7. Focal Loss for Dense Object Detection

Tr a i n i n g R e g i o n - b a s e d O b j e c t D e t e c t o r s w i t hOnline Hard Example Mining(OHEM)
Author: Abhinav Shrivastava; Abhinav Gupta; Ross GirshickOnline Hard Example Mining -- OHEM
• Basic Info.:• 2016 CVPR
• Author Introduction:• Ross Girshick (RBG) : Facebook AI Research• Abhinav Shrivastava, Abhinav Gupta: CMU

Online Hard Example Mining -- OHEM
• Motivation:• Unbalanced labels for background examples and foreground examples• Overwhelming number of easy examples and a small number of hard examples• Contribution:• Removes need for several heuristics and hyperparameters• Consistent and significant boosts in mAP• Effectiveness increased• Inspiration:• Could be useful when samples of positive examples are small

Online Hard Example Mining -- OHEM
• Overview of Fast R-CNN:

hyperparameters are needed!

Online Hard Example Mining -- OHEM
• Overview of Fast R-CNN:• Foreground RoIs (fg): IoU>=0.5• Background RoIs (bg): IoU∈[0.1，0.5)• 0.1 is a threshold (hyperparameter) here
• Method proposed in this paper elinimate this parameter• Balance fg-bg RoIs:• In paper of Fast R-CNN, ratios of examples of fg to bg is SET to 1:3• Proposed method elinimate this hyperparameter ratio

Online Hard Example Mining -- OHEM
• Definition:• Hard examples mining:• for some period, the model is fixed to find new examples• for some period, model is trained on fixed training set• Hard examples: examples with high loss
• Easy examples: examples with low loss

Online Hard Example Mining -- OHEM
• Implementation:• Compute feature map first• Use all RoIs as input RoIs
• Take hard examples• Set loss of easy eg.s as 0
• Use NMS to remove highoverlapped examples

Online Hard Example Mining -- OHEM
• Implementation:
Readonly Layer (a):• Only perform forward passes
Hard RoI Sampler:• Take B hard examples for N images
Layer (b):• Use hard examples to compute forward andbackward passes• Weights are shared between (a) and (b)Online Hard Example Mining -- OHEM
• A Question:• What does Online mean?
• An explanation• 在线学习中，每次录入一条数据（而非一个batch），训练完后直接更新模型• 而离线学习是一个batch全部录入完成后，才更新模型

R-FCN: Object Detection via Region-based Fully
Convolutional NetworksAuthor: Jifeng Dai; Yi Li; Kaiming He; Jian Sun

R-FCN• Basic Info.:• 2016 NIPS
• Author Introduction:• MSRA(微软亚洲研究院)• Jifeng Dai(代季峰)• Kaiming He(何恺明) :现 Facebook AI Research• Jian Sun(孙剑)R-FCN• Background:• VGG and AlexNet were widely used in object detection• ResNet was put forward• Motivation:• Two-stage object detction algorithm can be speeded up by sharing computation• Translation invariance of classification VS Translation variance of detection

R-FCN• Contribution:• Competitive with Faster R-CNN• Speed up for 2.5-20 times than Faster R-CNN

R-FCN• Implementation:• Compute feature map for an image• RPN (Region Proposals Network):• Compute RoI with the feature map• Object Classification:• Position-sensitive score maps : k²(C+1)-d• Vote for each classification• Bound Box Regression:• 4k²-d vector

R-FCN• Implementation:• For category person (an example)

Feature pyramid networks for object detection
Author: Tsung-Yi Lin, Piotr Dollar´ , Ross Girshick, Kaiming He, Bharath
Hariharan, and Serge Belongie

FPN• Basic Info.:• 2017 CVPR

• Team intro.:• Facebook AI Research• Cornell University and Cornell Tech

• BG & Motivation:• Low-level feature - Multi-scale; High-level feature - Strong semantic info.
• Feature pyramid could help detect objects with different scales• Deep learning based detectors avoid using pyramid representations due to computeand memory intensive (impractical for real applications)FPN• A simple compare:(a) Using IMAGE pyramid to build FEATURE
pyramid : slow(b) Do prediction at the fina feature level: like SPP
Net and RCNNs(c) No upsampling pathway : perform poorly when
detecting small scale objects

FPN• Contribution:• Proposing bottom-up pathway, top-down pathway and lateral connections (横向连接) structure• Creating a feature pyramid owning strong semantics at all scales• Do not increase testing times• A module which could be used in detection network

FPN• Implementation:• Backbone: ResNet (use the output of Conv2:5, exclude Conv1: large memory)• Bottom-up pathway: 2x downsampling• Lateral connection: 1*1 conv. (reduce channel dimensions)• Top-down pathway: 2x upsampling (finally 3*3 conv to generate feature map)

FPN
• Experiments:• Work with RPN (Region Proposals Network): To produce suitable anchor boxes• Anchor boxes: A set of bounding boxes whose aspect ratios and areas (manually)

(d)

(e)

(f)

FPN
• Experiments:• Work with FCNs

• Do well when detecting SMALL objects• Competitive when detecting LARGE objects

Cascade R-CNN: Delving into High Quality Object
DetectionAuthor: Zhaowei Cai, Nuno Vasconcelos

Cascade R-CNN• Basic Info.:• 2018 CVPR

• Team intro.:• UC San Diego• BG & Motivation:• IoU threshold u is required to define positive/negative examples (training)• Trade-off: a) low u produces noisy detections; b) high u decreases performance• The output of a detector is a good distribution for training the next better detector

Cascade R-CNN• Basic Info.:• 2018 CVPR

• Team intro.:• UC San Diego• BG & Motivation:• IoU threshold u is required to define positive/negative examples (training)• Trade-off: a) low u produces noisy detections; b) high u decreases performance• The output of a detector is a good distribution for training the next better detector

Cascade R-CNN• Introduction:• Some “close” false positives (close but notcorrect bboxes) will be produced with IoU
threshold (typical u=0.5) in training stage• (c): A detector optimized at a single IoUlevel is not necessarily optimal at others（我理解的是两个红圈里的IoU的值要接近,mismatch）• (d): Simply increase IoU threshold could
degrade detection performanceCascade R-CNN• Some other work:• Iterative Bounding-box Regression(迭代回归):• Represent a candidate b-box b of image patch x as f(x,b)• A single regression step of f is suboptimal• Use a iterative process to replace it:• Problem: No benifit beyond applying f twice

Cascade R-CNN• Some other work:• Integral Loss:• Calculate loss with different IoU threshold: (分类Loss)• Problem: Quick decrease of positive samples with u (figure on Right)

Cascade R-CNN• Implementataion of Cascade R-CNN:• Different from iterative regression: a) f is a resampling procedure which changes
distribution; b) used at both training and inference stage; (c) f t is optimized for allstages• Loss:(t is the stage)Thanks for Listening~Gong Qiqi

Reference List• 1. https://zhuanlan.zhihu.com/p/59002127, 深度学习不可忽略之OHEM:Online Hard
Example Mining• 2. https://blog.csdn.net/wfei101/article/details/79284512, 目标检测：RFCN算法原理<一>• 3. https://www.zhihu.com/search?type=content&q=R-FCN, 目标检测：R-FCN（NIPS2016）• 4. https://blog.csdn.net/baidu_30594023/article/details/82623623, FPN全解-最全最详细• 5. https://zhuanlan.zhihu.com/p/42553957, Cascade R-CNN 详细解读