Lite-HRNet: A Lightweight High-Resolution
NetworkPose Estimation
评价指标：
PCK：关键点与其对应的groundtruth间的归一化距离小于设定阈值的比例。
OKS：关键点与其对应的groundtruth间的相似度度量，[0,1]。
PCKh：以头部长度(head length：) 作为归一化参考。
pckh@0.5(MPII )： 0.5表示以头部长度作为参考，如果归一化后的距离大于阈值0.5，则认为预测正确。最好计算检测正确的比例。
OKS mAP(coco )：Pose Estimation
评价指标：
PCK：关键点与其对应的groundtruth间的归一化距离小于设定阈值的比例。
OKS：关键点与其对应的groundtruth间的相似度度量，[0,1]。
PCKh：以头部长度(head length：) 作为归一化参考。
pckh@0.5(MPII )： 0.5表示以头部长度作为参考，如果归一化后的距离大于阈值0.5，则认为预测正确。最好计算检测正确的比例。
OKS mAP(coco )：Lite-HRNet••1. Shuffle Block和Small HRNet简单融合，能够得到轻量化的HRNet
2. Naive Lite-HRNet中存在大量的1x1卷积操作，中使用conditional channel weighting模块
替代卷积，以进一步提高网络的计算效率。

HRNet(High-Resoultion Net)••••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出PAMI2019

特点与优势：（1） HRNet能够保持高分辨率， HRNet之
前算法是通过：high-to-low and low-to-highframework ：将高分辨率特征图下采样至低分辨率，再从低分辨率特征图恢复至高分辨
率的思路（U-Net ，encoder-decoder ）。

HRNet(High-Resoultion Net)••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出PAMI2019

high-to-low and low-to-high framework：transposedconvolution

HRNet(High-Resoultion Net)••••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出PAMI2019

特点与优势：（1） HRNet能够保持高分辨率， HRNet之
前算法是通过：high-to-low and low-to-highframework ：将高分辨率特征图下采样至低分辨率，再从低分辨率特征图恢复至高分辨
率的思路（U-Net ，encoder-decoder ）。
（2）融合相同深度和相似级别的低分辨率
特征图来提高高分辨率的特征图的表示效果，并进行重复的多尺度融合。

HRNet(High-Resoultion Net)••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出PAMI2019

contains four stages withfour parallel subnetworks
stagesparallelsubnetworks

Stage内部没有交互，参数不共享。

parallel subnetworks内部分辨率相同HRNet(High-Resoultion Net)••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出PAMI2019

big net HRNet-W48
small net HRNet-W32
32 and 48 represent the NUM_CHANNELS of the high-resolution subnetworks
in last three stages, respectively.
The widths of other three parallel subnetworks ：64; 128; 256 for HRNet-W32,96; 192; 384 for HRNet-W48.

HRNet(High-Resoultion Net)••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出PAMI2019

HRNetV1：只使用分辨率最高的特征图，人体姿态估计。

HRNetV2•

High-Resolution Representations for Labeling Pixels and Regions，简称HRNet V2，发表于CVPR2019

HRNetV1和HRNetV2其实不是版本迭代
的过程,只是同一个网络用在不同任务上HRNetV2：将所有分辨率的特征图进行concate，主要用于语义分割和面部关键点检测HRNetV2：采用上图的融合方式，主要用于训练分类网络。

HRNet(High-Resoultion Net)••

Deep High-Resolution Representation Learning for Human Pose Estimation
微软亚洲研究院和中科大提出CVPR2019

HRNetV2p：在HRNetV2的基础上，使用了一个特征金字塔，主要用于目标检测网络Small HRNet••

https://github.com/HRNet/HRNet-Semantic-Segmentation
It simply reduces the depth and the width of the original HRNet.

Small HRNet

HRNet

ShuffleNet••

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Face++ CVPR 2017convolution

DWConv: depthwise convolution

ShuffleNetV2•••

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Face++ & 清华大学 ECCV 2018

1. channel split: 将输入的featuremaps分为两部分c’和c-c’ .

DWConv: depthwise convolution
Gconv: group convolution

for spatial down
sampling (2×)

for spatial down
sampling (2×)

ShuffleNetV2••••

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture DesignFace++ & 清华大学 ECCV 2018

1. channel split: 将输入的featuremaps分为两部分c’和c-c’ .
2. GConv 替换成Conv

DWConv: depthwise convolution
Gconv: group convolution

for spatial down
sampling (2×)

for spatial down
sampling (2×)

ShuffleNetV2••

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture DesignFace++ & 清华大学 ECCV 2018

convolution

Gconv: group convolution

ShuffleNetV2
•
•

•
•

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture DesignFace++ & 清华大学 ECCV 2018

1. channel split: 将输入的featuremaps分为两部分c’和c-c’ .
2. GConv 替换成Conv，太多的组卷积会增加内存访问成本DWConv: depthwise convolution
Gconv: group convolution

for spatial down
sampling (2×)

for spatial down
sampling (2×)

ShuffleNetV2•••••

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Face++ & 清华大学 ECCV 2018

1. channel split: 将输入的featuremaps分为两部分c’和c-c’ .
2. GConv 替换成Conv
3. channel shuffle

DWConv: depthwise convolution
Gconv: group convolution

for spatial down
sampling (2×)

for spatial down
sampling (2×)

ShuffleNetV2••ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture DesignFace++ & 清华大学 ECCV 2018

ShuffleNetV2••••••

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture DesignFace++ & 清华大学 ECCV 2018

1. channel split: 将输入的feature
maps分为两部分c’和c-c’ .
2. GConv 替换成Conv
3. channel shuffle
channel split已经分开了feature，如果channel shuffle继续使用会
丢失另一半feature。

DWConv: depthwise convolution
Gconv: group convolution

for spatial down
sampling (2×)

for spatial down
sampling (2×)

Lite-HRNet••

Naive Lite-HRNet：Shuffle Block和Small HRNet简单融合，能够得到轻量化的HRNet
Lite-HRNet ：Naive Lite-HRNet中存在大量的1x1卷积操作，中使用conditional channel
weighting模块替代卷积，以进一步提高网络的计算效率。

Naive Lite-HRNet••

We adopt the shuffle block to replace the second 3 × 3 convolution in the stem of Small HRNet ，and replace all the normal residual blocks
The normal convolutions in the multi-resolution fusion are replaced by the separable convolutions（ Xception）

Naive Lite-HRNet••

We adopt the shuffle block to replace the second 3 × 3 convolution in the stem of Small HRNet ，and replace all the normal residual blocks .
The normal convolutions in the multi-resolution fusion are replaced by the separableconvolutions （ Xception）••

1.顺序不同：depthwise separable convolution是先做channel-wise spatial convolution在再做1x1的conv，而
Xception是相反的。
2. Xception每个操作的后面都跟了ReLU非线性激活，而depthwise separable convolution是没有的。

Xception: Deep learning with depthwise separable convolutions cvpr2017

Lite-HRNet••

Naive Lite-HRNet：Shuffle Block和Small HRNet简单融合，能够得到轻量化的HRNet
Lite-HRNet ：Naive Lite-HRNet中存在大量的1x1卷积操作，中使用conditional channelweighting模块替代卷积，以进一步提高网络的计算效率。

Lite-HRNet••

Naive Lite-HRNet：Shuffle Block和Small HRNet简单融合，能够得到轻量化的HRNet
Lite-HRNet ：Naive Lite-HRNet中存在大量的1x1卷积操作，中使用conditional channelweighting模块替代卷积，以进一步提高网络的计算效率。

Lite-HRNet••

Naive Lite-HRNet：Shuffle Block和Small HRNet简单融合，能够得到轻量化的HRNet
Lite-HRNet ：Naive Lite-HRNet中存在大量的1x1卷积操作，中使用conditional channelweighting模块替代卷积，以进一步提高网络的计算效率。

Lite-HRNet••

Naive Lite-HRNet：Shuffle Block和Small HRNet简单融合，能够得到轻量化的HRNet
Lite-HRNet ：Naive Lite-HRNet中存在大量的1x1卷积操作，中使用conditional channelweighting模块替代卷积，以进一步提高网络的计算效率。

conditional channel weighting••CCW：Conv：•••w权重计算：H: Cross-resolution Weight Computation
F: Spatial Weighting Computation

Shuffle BlockCCW Blockconditional channel weighting
H: Cross-resolution Weight Computation•s-th stage has s parallel resolutions and s weight maps•—> adaptive average pooling (AAP) —>output size：catconditional channel weighting
H: Cross-resolution Weight Computation•s-th stage has s parallel resolutions and s weight maps•—> adaptive average pooling (AAP) —>output size：catconditional channel weighting
H: Cross-resolution Weight Computation•s-th stage has s parallel resolutions and s weight maps•—> adaptive average pooling (AAP) —>output size：concate

conditional channel weighting
F: Spatial Weighting Computation•

GAP(global average pooling ) + FC + ReLU +FC + sigmoid•

Gathering the spatial information from all
the positions

Shuffle Block

CCW BlockConditional Channel Weighting•

使用CCW代替卷积以减少网络的计算需求Lite-HRNet结构Thanks！