# Minimizing Maximum Feature Space Deviation for Visible-Infrared Person Re-Identification
## 1 Introduction  
Given a visible (or infrared) query image of a specified person, the goal of Visible-infrared person re-identification [1; 2; 3; 4; 5; 6] is to retrieve infrared (or visible) images of the same person from a gallery set, as shown in Figure 1. VIPR has received more and more attention due to its importance to intelligent surveillance systems and intelligent transportation systems in light-less environments. In these application scenarios, visible images are captured by color cameras and infrared images usually are captured by near-infrared [7] or thermal [8] cameras. Just as with the traditional single-modal (i.e., visible) re-identification task [9; 10; 11], VIPR meets the great challenges of pose variation, viewpoint variation, and occlusions. Furthermore, VIPR suffers from a huge modal gap between visible and infrared modalities. Therefore, VIPR is a meaningful but challenging topic that is worthy of intensive study. Existing VIPR works can be roughly divided into three types: (1) metric learning-based methods [3; 12; 13; 14], (2) feature extraction-based methods [2; 15; 16; 17; 18; 19; 20; 21], and (3) generation-based methods [22; 23; 24; 25]. Many metric learning-based methods usually design specific triplet loss functions to optimize cross-modal samples. However, metric learning-based methods focus on identity-level discrimination, underestimating threats of outliers raised by the gap between visible and infrared modalities. Feature extraction-based methods are often dedicated to designing additional special feature learning architectures (e.g., visual  
Figure 1: The schematic diagram of visible-infrared person re-identification. attention modules) to enhance general deep networks (e.g., residual networks [26]) to map different modal features into a common feature space. However, extra computations caused by additional special feature learning architectures are required. Generation-based methods use generation adversarial networks (GAN) for modal conversion to eliminate modal gaps. For example, in addition to using ResNet-50 [26], [PERSON] et al. [22] designed a GAN to produce synthetic color images from infrared and gray images. The GAN contains a generator of five convolutional layers and five convolution-transpose layers, and each layer is followed by a batch normalization layer, a leaky rectified linear unit, and a residual connection. The discriminator is a fully convolutional network, mainly consisting of a series of \(3\times 3\) convolutional layers, where each convolutional layer is followed by a max-pooling layer, and the number of channels doubles after each max-pooling layer. The generator is applied to generate synthetic color images, while the discriminator is used to discriminate an image is synthetic or real. Through the antagonism between generator and discriminator, the generator could generate realistic synthetic color images to reduce modal gaps. GANs of generators and discriminators are commonly used, such as [24; 25; 27; 28]. In [24; 25; 27], multiple GANs are applied to realize more delicate adversarial learning. Leaving aside GAN's architecture complexities, in practice, GAN's training process is complex and involves many hyper-parameters that require stage-wise training or alternate training. Therefore, GAN-based methods require a lot of skill to avoid model collapses. In this paper, we propose a minimizing maximum feature space deviation (MMFSD) method for VIPR. For modeling the modal gap between visible and infrared modalities, we define feature space deviations (FSD) based on each identity's visible and infrared feature centers. Their centers are calculated by averaging multiple samples on the same modality of the same identity. Furthermore, for reducing the modal gap between visible and infrared modalities, we design a minimizing maximum FSD loss function, which optimizes deviations from the most severe feature dimensions. The main contributions of this paper can be summarized as follows.