  Weakly supervised object localization (WSOL) is a challenging problem when
given image category labels but requires to learn object localization models.
Optimizing a convolutional neural network (CNN) for classification tends to
activate local discriminative regions while ignoring complete object extent,
causing the partial activation issue. In this paper, we argue that partial
activation is caused by the intrinsic characteristics of CNN, where the
convolution operations produce local receptive fields and experience difficulty
to capture long-range feature dependency among pixels. We introduce the token
semantic coupled attention map (TS-CAM) to take full advantage of the
self-attention mechanism in visual transformer for long-range dependency
extraction. TS-CAM first splits an image into a sequence of patch tokens for
spatial embedding, which produce attention maps of long-range visual dependency
to avoid partial activation. TS-CAM then re-allocates category-related
semantics for patch tokens, enabling each of them to be aware of object
categories. TS-CAM finally couples the patch tokens with the semantic-agnostic
attention map to achieve semantic-aware localization. Experiments on the
ILSVRC/CUB-200-2011 datasets show that TS-CAM outperforms its CNN-CAM
counterparts by 7.1%/27.1% for WSOL, achieving state-of-the-art performance.
